These are chat archives for arnaud-lb/php-rdkafka

23rd
Aug 2017
Thomas Ploch
@tPl0ch
Aug 23 2017 13:44
Hi, since Kafka 0.10.1 the heartbeat was moved out of poll to a separate heartbeat thread, which transforms the formerly single threaded consumers to multi-threaded ones. How does php-rdkafka handle that now?
Magnus Edenhill
@edenhill
Aug 23 2017 13:56
@tPl0ch that's a Java client implementation change. librdkafka has always used background threads for all broker communications, including heartbeats
Thomas Ploch
@tPl0ch
Aug 23 2017 13:58
@edenhill thanks, we are just sitting in a confluent training and were a bit puzzled, but that makes sense :)
Magnus Edenhill
@edenhill
Aug 23 2017 14:00
have fun :)
Thomas Ploch
@tPl0ch
Aug 23 2017 14:41
@edenhill https://github.com/confluentinc/libserdes - we have built a PHP implementation of Avro serialization and schema registry, but I think for high throughput, low-latency requirements it might be better to actually write a PHP extension. Are you aware of any people/projects already working on that?
Magnus Edenhill
@edenhill
Aug 23 2017 14:43
@tPl0ch I don't believe libserdes will necessarily make your PHP app faster since you can only use the schema-registry integration, not the actual Avro Serdes. You'll want to use a native Avro serdes for PHP to make it meaningful (deserializing an avro-formatted message value to a C or C++ object won't do you any good)
Thomas Ploch
@tPl0ch
Aug 23 2017 14:45
OK, the schema-registry integration is limited by the network latency anyway.
So does not really make sense.
Thanks.
Magnus Edenhill
@edenhill
Aug 23 2017 14:48
And schemas are cached after the initial registration and lookup so performance wise it wont make a difference
Thomas Ploch
@tPl0ch
Aug 23 2017 14:49
Yeah in our PHP implementation we have various cache adapters so teams can actually choose what fits them best.
Magnus Edenhill
@edenhill
Aug 23 2017 14:50
:+1:
Thomas Ploch
@tPl0ch
Aug 23 2017 14:51
So if anybody is interested in using the Schema Registry within their PHP projects:
https://github.com/flix-tech/schema-registry-php-client
Magnus Edenhill
@edenhill
Aug 23 2017 14:55
Great stuff :)
Thomas Ploch
@tPl0ch
Aug 23 2017 14:58
Thanks, we will also release the higher level avro serialization lib containing the whole libserdes flow: serialize -> schema registry -> wire protocol | wire protocol -> schema registry -> deserialize in a few weeks when the lib passes our open source quality criteria (mainly docs).
Magnus Edenhill
@edenhill
Aug 23 2017 15:11
:100: