These are chat archives for arnaud-lb/php-rdkafka

21st
Nov 2016
Paul Dragoonis
@dragoonis
Nov 21 2016 09:54

@edenhill thanks for replying! appreciated - I have this offset bug that I can't figure out - maybe you can spot something?

My consumer here that i've built - using STORED offset. The bug behaviour is:

Did this make sense ?

(dunno how to make that gist block smaller on this chat, srry)
Magnus Edenhill
@edenhill
Nov 21 2016 09:57
seems like you are never committing the offsets you store (since auto commit is disabled), so whenever you restart it wont find a committed offset and thus resort to auto.offset.reset which defaults to latest
Paul Dragoonis
@dragoonis
Nov 21 2016 09:58
I am committing the offset when I get EOF (no more data)
Magnus Edenhill
@edenhill
Nov 21 2016 09:58
it is also kind of weird to only store (and I guess you also want commit) when you reach EOF. If your consumer is slower than the producer, for whatever reason, you will never reach EOF and thus never commit.
Paul Dragoonis
@dragoonis
Nov 21 2016 09:58
@edenhill should I be calling ->storeOffset() upon every entry into RD_KAFKA_RESP_ERR_NO_ERROR? If not .. where?
Magnus Edenhill
@edenhill
Nov 21 2016 09:59
no, you are storing the offset for future commit, but you are never committing
Paul Dragoonis
@dragoonis
Nov 21 2016 09:59
I see!
Magnus Edenhill
@edenhill
Nov 21 2016 09:59
offsetStore = store offset for future commit
commit = commit stored offset
Paul Dragoonis
@dragoonis
Nov 21 2016 09:59
Totally understand 100% :) thanks
Magnus Edenhill
@edenhill
Nov 21 2016 09:59
:)
the nomenclature is kind of weird
STORED, store, commit
so your confusion is warranted