These are chat archives for arnaud-lb/php-rdkafka

Oct 2018
Oct 19 2018 12:36

Hi everyone
I am trying to create a topic test which should have following configurations
cleanup.policy = compact and = 86400000(milliseconds)
RdKafka\TopicConf() object wont set these for a topic (Throwing exception )
here is my code

$rk = new RdKafka\Producer();

// Set the topic configuration:
$topicConf = new RdKafka\TopicConf();
//$topicConf->set("", 1e3);
$topicConf->set("cleanup.policy", "compact");
$topicConf->set("", 86400000);

$topic = $rk->newTopic("test", $topicConf);
$message = "Meassge testing topic config";
try {
    $topic->produce(RD_KAFKA_PARTITION_UA, 0, $message);
    // We have to call the poll function to make sure everything did as it should
   // $rk->poll(-1);
} catch (\Exception $exception) {
    //something went wrong, so handle your errors as you would...
    return false;

However I can create the topic with same configuration

kafka-topics --create --zookeeper localhost:2181 \--replication-factor 1 --partitions 1 --topic test --config --config cleanup.policy=compact

Either I am doing it wrong in php or there is any other way to set these configuration
Any help will be really appreciated

Oct 19 2018 13:08
You confused kafka flags and librdkafka flags. - this is the list of possible flags for library
Oct 19 2018 13:09
Yes I have seen that list and these are not there
How can I set these flags ? from librdkafka
Oct 19 2018 13:11
I think it's impossible
Set topic properties in kafka config.
Oct 19 2018 13:14
Yeah seems like that is the only solution
alright. Thank you for your feedback!
Oct 19 2018 13:14
php-rdkafka - it's not a library for managing kafka topics.
It's sad that there isn't convenient api for managing topics. Only kafka-topics shell script.
Oct 19 2018 13:18
yeah even REST Proxy api does not have anything related to topic creation