These are chat archives for arnaud-lb/php-rdkafka

23rd
Oct 2018
amnai2p
@amnai2p
Oct 23 2018 05:43
@sandvige Thanks for your help!
Paweł Niedzielski
@steveb_gitlab
Oct 23 2018 11:49
If I'm using enable.auto.commit set to true (as per default) do I need to call KafkaConsumer::commit method, or is it called internally anyway?
karavzeka
@karavzeka
Oct 23 2018 12:03
Commit just moves further an offset in the topic. If this option is enabled you don't need call commit.
Paweł Niedzielski
@steveb_gitlab
Oct 23 2018 12:18
@karavzeka thanks. I do roughly understand how consumer's offset works, but I'm unsure if calling commit is required, or just forces Kafka to update offset immediately. Asking because php-enqueue library calls commit always on each message processed, which might slow down processing when a lot of messages have to be processed without exactly-once (somewhat) guarantee.
karavzeka
@karavzeka
Oct 23 2018 12:23
If you get new messages correctly it means an offset moves, so you don't need call commit)
Paweł Niedzielski
@steveb_gitlab
Oct 23 2018 12:27
@karavzeka so what you're basically saying is that Kafka instance handling this particular Topic/Partition holds current offset in memory and saves it to __consumer_offsets at intervals defined by auto.commit.interval.ms and I should not lower it? And that consumer offset is saved even if PHP process crashes after receiving the message (connection is dropped and so on?)
karavzeka
@karavzeka
Oct 23 2018 12:34
Mmmm... I haven't thought about this option before. I left it in default value.
And yes, If php crashes, next time your consume next message.
Paweł Niedzielski
@steveb_gitlab
Oct 23 2018 12:35
@karavzeka I'm pretty sure it's ok on default, if above is true. Thanks, I owe you a :beer: :D
karavzeka
@karavzeka
Oct 23 2018 12:36
thanks :smile: