These are chat archives for arnaud-lb/php-rdkafka

31st
Oct 2018
tambach
@tambach
Oct 31 2018 13:00
after some time (same amount of data) i have issue like this :
%3|1540990702.858|FAIL|rdkafka#producer-1| [thrd:127.0.0.1:9092/bootstrap]: 127.0.0.1:9092/bootstrap: Receive failed: Disconnected
Can i change something in config file that will make it work?
How can i correct this?
tambach
@tambach
Oct 31 2018 13:10
i mean that it works with big list of tables and in the begining it works fine, but after half of the code i have this error, i made the zookeeper.connection.timeout.ms=6000000 . ten times bigger in server.properties file but it has the same result . What can i do?
Mathieu Pipet
@mpipet
Oct 31 2018 17:39
@tambach At consume time what the consumer do is polling on a socket to see if there is new messages. If there no new messages there is a timeout as you pointed out in your last message. You can't make the difference between no new message and kafka stopped responding. So you could increase the timeout, checkout the librdkafka config for the correct one. But if you increase this timeout you also increase the time before rdkafka detect a problem. For my part I totaly ignore thoses errors, I mainly detect issues by measuring lag between produced messages and consumed messages on topic partitions (Not feasable with phprkafka at this time). Sorry for the long response.