These are chat archives for arnaud-lb/php-rdkafka

13th
Nov 2017
Luís Cobucci
@lcobucci
Nov 13 2017 10:00

Hello everyone,

We're starting to use Kafka at our company and we're experiencing A LOT of delay to produce messages when having concurrent requests (dev and testing environments).
On dev, without any concurrency, the response time of the endpoint is 7ms~35ms (which is fine, considering we're using multiple docker containers - without mounted volumes for the tests). With 20 concurrent users, our test results are:

Samples count: 1405, 0.00% failures
Average times: total 1.520, latency 1.520, connect 0.000
Percentile 0.0%: 0.006
Percentile 50.0%: 1.573
Percentile 90.0%: 2.227
Percentile 95.0%: 2.401
Percentile 99.0%: 2.781
Percentile 99.9%: 3.289
Percentile 100.0%: 3.366

2.7 s for the 99.0% is REALLY unacceptable (even in dev).

Looking at the web server load, everything seems fine (no huge spike on CPU or memory).


Details:

PHP: 7.1.10
Nginx: 1.13.6
Broker version: 1.0
librdkafka version (runtime): 0.11.1
librdkafka version (build): 0.11.1.0


RdKafka\Conf:

socket.timeout.ms: 1000
queue.buffering.max.ms: 1
internal.termination.signal: SIGIO


Important note: producer polling is done AFTER calling fastcgi_finish_request().

Any idea on how to improve this? thanks!
Luís Cobucci
@lcobucci
Nov 13 2017 10:52
It seems like #42 would be the only solution for this, right @arnaud-lb?