These are chat archives for arnaud-lb/php-rdkafka

13th
Jan 2017
Craig Patrick
@cpats007
Jan 13 2017 14:44
anyone have any ideas of how to improve performance? on php-fpm my app response times more than double when writing to Kafka
Magnus Edenhill
@edenhill
Jan 13 2017 14:56
@cpats007 measure and profile to find out what is taking time. Possibly the produce+ack. Then see queue.buffering.max.ms and socket.blocking.max.ms
Craig Patrick
@cpats007
Jan 13 2017 14:57
thanks @edenhill - so I'm not doing any ack at this stage - this is simply a "fire and forget" function - I just want to dump stuff into Kafka from PHP. I have altered the socket.blocking.max.ms down to 100 and still the same, plus we can't use the SIGIO stuff because it's php-fpm. I'll look at the queue.buffering.max.ms now
it's literally logging the request information into Kafka from the app, one big JSON string, nothing complicated
Craig Patrick
@cpats007
Jan 13 2017 15:08
I have adjusted the queue.buffering.max.ms and it has made a big difference, thanks @edenhill ! - I'll monitor now and see how it goes, but really appreciate that - is there any way I can see if a message has failed to send - like the config values I'm changing wont alter the fact that it'll still send the message right? if that makes sense
Magnus Edenhill
@edenhill
Jan 13 2017 15:11
you register a delivery report callback that'll be called once for each message to tell if it succeeded delivery or failed (and why)
if you use acks=0 (which you typically dont want to do, hello message loss) then the delivery report callback will be triggered as soon as the message has been sent to the broker.
Craig Patrick
@cpats007
Jan 13 2017 15:12
I tried that - but it didn't seem to work, the method was unsupported or something unless I was using the wrong methodology
Craig Patrick
@cpats007
Jan 13 2017 15:15
setDrMsgCb is not available in the Conf() object?
Magnus Edenhill
@edenhill
Jan 13 2017 15:16
Looks like it is. maybe you have an old version
Craig Patrick
@cpats007
Jan 13 2017 15:16
hmmmm
Magnus Edenhill
@edenhill
Jan 13 2017 15:16
anyhoo, Ive never used the php client so I dont know. Not erally sure what Im doing here! ;)
Craig Patrick
@cpats007
Jan 13 2017 15:17
ha ha ha, well you've certainly given me a huge step forward!
could it be because I'm using the stubs? and maybe the stubs are outdated?
Craig Patrick
@cpats007
Jan 13 2017 15:54
Okay, so by the looks of it, I have version 0.9.2 (latest released version) but there is a version 3.0 - however it wont install