php-enqueue/Lobby

Andrew

@andrewmy

Rebase onto master, it was fixed recently

George

@gdsmith

@makasim PR is green (thanks @andrewmy)

George

@gdsmith

@makasim any chance of getting that merged and released?

Michal Junek

@DerpgonCz

Hey guys, I have a peculiar task to accomplish. Due to the need of E2E tests of a huge feature that uses async messaging, I need to dispatch and process the message synchronously and use it's output during said tests. Is there any way to process the message instantly?

9 replies

If not possible, a workaround would be to invoke processor by hand, but I have no idea where to start.

George

@gdsmith

@makasim @andrewmy I missed a param on my PR so quick follow up: php-enqueue/enqueue-dev#1194

George

@gdsmith

Sorry to be a pain but any chance of getting that merged @makasim?

Max Kotliar

@makasim

George

@gdsmith

:bow:

Saad89

@Saad89

Hello Guys, I can not install the enqueue via composer for my magento 1.x cause of this error - enqueue/enqueue[0.3.0, ..., 0.8.42] require ramsey/uuid ^2|^3.5 -> found ramsey/uuid[2.0.0, ..., 2.9.0, 3.5.0, ..., 3.9.4] but it conflicts with your root composer.json require (^4.1).

syntaxgirl

@syntaxgirl_twitter

Hi everyone. I am having issue with consuming messages where I have to set arguments to the queue else I get error when connecting to the queue.

I tried adding arguments like this

        options:
          queue:
            name: "test"
            arguments:
              test: "test"

But they are never used. Is there maybe another way? I am using this with symfony and shopware 6.

If I go to the declareQueue method, inside vendor directory, and set arguments there manually, that works. But how to do this without doing it inside vendor?

Though as far as I see, messenger-enqueue-transport is making issue, as it doesn't take arguments at all..

Juan Diaz

@FuriosoJack

hello,
You know how you could implement the possibility of triggering an event (here add my business logic) when the elements of a queue of a batch of elements are finished processing?

Niv Penso

@nivpenso

Hi guys,

php-enqueue is such a great library

does anyone know a good similar library for nodejs?

fxywky

@fxywky

Hey, guys ,I just
$consumer->receive();
$consumer->acknowledge($message);

and I re-consumed messages.
I don't know how to write the code to avoid re-consuming messages？？

fxywky

@fxywky

$fooQueue->setPartition(1); is useless

$consumer->acknowledge($message); is useless.

who can help me, plz?

Vladimir Rüntü

@vladsrynty

Looking for help. How to config symfony messenger failed queue with mongodb enqueue

framework:
transports:
events: 'enqueue://default'
failed: 'enqueue://failed?queue[name]=failed'

and enqueue.yaml:

enqueue:
    default:
        transport: '%env(resolve:ENQUEUE_DSN)%'
        client: null

how should I define failed transport?

default transport DSN is ok (database enqueue, collection enqueue), only name of the collection? or queue? ot topic? should be different?

Dmytro Vasylchenko

@vasylchenko

Hi there. I'm interested, why is enqueue/enqueue released with minimum-stability set to dev?

Lewis Cowles

@Lewiscowles1986

:wave: seasons greetings. Thanks for working on this library; I've never used it before, but we've been using it at work for a while.

On my last day this year, me and another engineer noticed the throughput doesn't go much north of 500 messages /sec at best (which was sadly an error case). We're seeing enqueue struggle to process 500 messages a second; with over 12 worker instances in the same region of the same data-center (AWS).

I Noticed some sleep calls in the code; and what looks like a 10-second timeout. These both rang alarm bells as things to experiment with when I return in the new year. But I thought I could also ask here if Enqueue is designed or known to be able to scale into the tens of thousands of messages per-second with PHP, and what sort of configuration and scale might be required?

SQS advertises 30k limit to publish with 120k in-flight limit. I'm happy enough to not be hitting whitepaper specs as in my experience most are worth less than toilet paper without accompanying code, including IaC for a specific supplier on a specific plan and spend; but to be so far off, not even hitting three digits has me wondering if this is the choice of language, the architecture (minor obsession with Uncle Bob "clean coding" (nee. convoluted). Or something else.

panki3a

@panki3a

Hi,
I wont to ask, if package php-enqueue/laravel-queue can support command "php artisan queue:work" arproperty "--max-time" and "--max-jobs". Now in version 2.5.0, when i used those properties, i get fatal error, because "public function onPostMessageReceived(PostMessageReceived $context): void" call $this->stopIfNecessary($this->options, $this->lastRestart, $this->job) from parent Worker class, but the parent implementation is "protected function stopIfNecessary(WorkerOptions $options, $lastRestart, $startTime = 0, $jobsProcessed = 0, $job = null)"
Thank you

Paweł Niedzielski

@Steveb-p

@vasylchenko minimum-stability does not matter for packages - it's a root configuration option, which means only your main composer.json affects this.

Herberto Graça

@hgraca

Hi all, first things first: tkx for working on this library.

Im trying to run the tests so that I can open a PR to the rdkafka package, but booting up the containers fails:

$ docker compose up
Sending build context to Docker daemon  2.425kB
Step 1/12 : ARG PHP_VERSION=7.4
Step 2/12 : FROM makasim/nginx-php-fpm:${PHP_VERSION}-all-exts
 ---> 68f295e6bd36
Step 3/12 : ARG PHP_VERSION
 ---> Using cache
 ---> 6e38268cf402
Step 4/12 : RUN set -x &&     apt-get update &&     apt-get install -y --no-install-recommends --no-install-suggests         wget         curl         openssl         ca-certificates         nano         netcat         php${PHP_VERSION}-dev         php${PHP_VERSION}-redis         php${PHP_VERSION}-pgsql         git         python         php${PHP_VERSION}-amqp         php${PHP_VERSION}-xml         php${PHP_VERSION}-mysql         php${PHP_VERSION}-curl         libgearman-dev         php-pear         make         unzip     &&     pecl channel-update pecl.php.net &&     pecl install gearman &&     if [ ! -f /etc/php/${PHP_VERSION}/cli/conf.d/20-gearman.ini ]; then         echo "extension=gearman.so" > /etc/php/${PHP_VERSION}/cli/conf.d/20-gearman.ini &&         echo "extension=gearman.so" > /etc/php/${PHP_VERSION}/fpm/conf.d/20-gearman.ini         ;     fi;
 ---> Running in dea7e0fc1624
+ apt-get update
Get:1 http://archive.ubuntu.com/ubuntu focal InRelease [265 kB]
Get:2 http://ppa.launchpad.net/ondrej/php/ubuntu focal InRelease [23.9 kB]
Get:3 http://ppa.launchpad.net/ondrej/pkg-gearman/ubuntu focal InRelease [17.5 kB]
Get:4 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB]
Get:5 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB]
Get:6 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB]
Err:2 http://ppa.launchpad.net/ondrej/php/ubuntu focal InRelease
  At least one invalid signature was encountered.
Err:1 http://archive.ubuntu.com/ubuntu focal InRelease
  At least one invalid signature was encountered.
Err:3 http://ppa.launchpad.net/ondrej/pkg-gearman/ubuntu focal InRelease
  At least one invalid signature was encountered.
Err:4 http://archive.ubuntu.com/ubuntu focal-updates InRelease
  At least one invalid signature was encountered.
Err:5 http://archive.ubuntu.com/ubuntu focal-backports InRelease
  At least one invalid signature was encountered.
Err:6 http://security.ubuntu.com/ubuntu focal-security InRelease
  At least one invalid signature was encountered.
Reading package lists...
W: GPG error: http://ppa.launchpad.net/ondrej/php/ubuntu focal InRelease: At least one invalid signature was encountered.
E: The repository 'http://ppa.launchpad.net/ondrej/php/ubuntu focal InRelease' is not signed.
W: GPG error: http://archive.ubuntu.com/ubuntu focal InRelease: At least one invalid signature was encountered.
E: The repository 'http://archive.ubuntu.com/ubuntu focal InRelease' is not signed.
W: GPG error: http://ppa.launchpad.net/ondrej/pkg-gearman/ubuntu focal InRelease: At least one invalid signature was encountered.
E: The repository 'http://ppa.launchpad.net/ondrej/pkg-gearman/ubuntu focal InRelease' is not signed.
W: GPG error: http://archive.ubuntu.com/ubuntu focal-updates InRelease: At least one invalid signature was encountered.
E: The repository 'http://archive.ubuntu.com/ubuntu focal-updates InRelease' is not signed.
W: GPG error: http://archive.ubuntu.com/ubuntu focal-backports InRelease: At least one invalid signature was encountered.
E: The repository 'http://archive.ubuntu.com/ubuntu focal-backports InRelease' is not signed.
W: GPG error: http://security.ubuntu.com/ubuntu focal-security InRelease: At least one invalid signature was encountered.
E: The repository 'http://security.ubuntu.com/ubuntu focal-security InRelease' is not signed.
...

Any suggestions on how I can get over this?

Dmytro Vasylchenko

@vasylchenko

@Steveb-p, thanks for the explanations!
@hgraca, I would start with cleaning docker build cache by docker builder prune. BTW, I really like your blog (especially The Software Architecture Chronicles), reading it for a long time. Thank you very much for it! :+1:

patrickwebvr

@patrickwebvr

I have an application in Symfony 5.4 and this application is consuming RabbitMQ messages.
Sometimes there is a fail. And after three calls then the message is removed instead to transport to a failure enqueue.

What I'm doing wrong? Thanks for feedback.

config/packages/messanger.yaml

framework:
    messenger:
        # Uncomment this (and the failed transport below) to send failed messages to this transport for later handling.
        failure_transport: failed

        transports:
            # https://symfony.com/doc/current/messenger.html#transport-configuration
            # async: '%env(MESSENGER_TRANSPORT_DSN)%'
            failed:
                dsn: '%env(resolve:ENQUEUE_DSN)%?topic[name]=fails'
            # sync: 'sync://'

        routing:
            # Route your messages to the transports
            # 'App\Message\YourMessage': async

config/packages/enqueue.yaml

enqueue:
    default:
        transport: '%env(resolve:ENQUEUE_DSN)%'
        async_events:
            enabled: true
        client:
            traceable_producer:   true
            prefix:               enqueue
            separator:            .
            app_name:             demo_service
            router_topic:         default
            router_queue:         demo_service_default
            router_processor:     null
            redelivered_delay_time: 0
            default_queue:        demo_service
        extensions:
            doctrine_ping_connection_extension: true

Andrew

@andrewmy

@makasim are you well and safe?

Max Kotliar

@makasim

more or less

Maruf Alom

@marufmax

hi, I am very new to enqueue library. any good tutorial out there for plain PHP example with Redis transport?

Maruf Alom

@marufmax

or any github example would be very helpful too

Maxwell175

@maxwell175:matrix.org

[m]

Hello, I am getting the following error when consuming a message:

PHP Fatal error:  Uncaught LogicException: The frame size is "11073" and it must divide exactly to 64 but it leaves a reminder "1". in /data/www/vendor/enqueue/fs/FsConsumer.php:120
Stack trace:
#0 [internal function]: Enqueue\Fs\FsConsumer->Enqueue\Fs\{closure}()
#1 /data/www/vendor/enqueue/fs/FsContext.php(114): call_user_func()
#2 /data/www/vendor/enqueue/fs/FsConsumer.php(148): Enqueue\Fs\FsContext->workWithFile()
#3 /data/www/vendor/enqueue/fs/FsConsumer.php(83): Enqueue\Fs\FsConsumer->receiveNoWait()
#4 /data/www/vendor/enqueue/enqueue/Consumption/FallbackSubscriptionConsumer.php(44): Enqueue\Fs\FsConsumer->receive()
#5 /data/www/vendor/enqueue/enqueue/Consumption/QueueConsumer.php(264): Enqueue\Consumption\FallbackSubscriptionConsumer->consume()
#6 /data/www/vendor/enqueue/simple-client/SimpleClient.php(209): Enqueue\Consumption\QueueConsumer->consume()
#7 /data/www/services/emailSender.php(24): Enqueue\SimpleClient\SimpleClient->consume()
#8 {main}
  thrown in /data/www/vendor/enqueue/fs/FsConsumer.php on line 120

is there a way to discard a message that is in some way broken?

or salvage it

shilpakarbal

@shilpakarbal

Hi, how to get MessageID from send message request to SqsProducer. Bcoz SqsProducer send function is void and not returning resulted data.

RusAnd

@RuslanMelnychenko

Has php-enqueue implementation in JavaScript?

Javid Aslanov

@javidaslan

hi all,

I have an interesting situation to solve. We have built new Kafka cluster on AWS MSK and as an authentication method we are using IAM Authentication which is pretty new in AWS too (ref: https://aws.amazon.com/about-aws/whats-new/2021/05/introducing-iam-access-control-amazon-msk/).
We have some services written in PHP and we would like to connect this cluster from those services using IAM Authentication, couldn't find proper solution (or solution at all) in google, so wanted to give it a shot here, maybe someone has done similar thing. Any answer is appreciated, thanks in advance 🙂

Andrew

@andrewmy

Hey @makasim how are you? Could you please make a release?

Max Kotliar

@makasim

done - https://github.com/php-enqueue/enqueue-dev/releases/tag/0.10.16

Andrew

@andrewmy

Thank you so much man!

Max Kotliar

@makasim

welcome

Ken

@nvcken_twitter

Hi, please help!! I try to connect Amazon MQ (which is configured with RabbitMQ) using port 5671
Currently, using "enqueue/amqp-ext": "~0.10.9" , PHP 8.1
Getting this error "Library error: connection closed unexpectedly - Potential login failure"

                'host' => 'host_name',
                'port' => '5671',
                'user' => 'admin',
                'pass' => 'admin',
                'vhost' => '/test',
                'ssl_on' => true,
                'ssl_verify' => false,
                'dsn' => 'amqps:'

The login info host/ user/ pass is correct as tested it work fine with php-amqplib/php-amqplib
Thanks

Ken

@nvcken_twitter

btw, For php-amqplib/php-amqplib, tested work by this example https://github.com/php-amqplib/php-amqplib/blob/master/demo/amqp_consumer_signals.php

Where communities thrive

People

Repo info

Activity