These are chat archives for arnaud-lb/php-rdkafka

9th
Jun 2016
Vladislav Krakhalev
@vchampion
Jun 09 2016 10:47
Hello Duncan
So in CLI mode that errors will arise in stderr stream
and as variant you can stream that messages and rise exception on that cases
yes it's not best case when normail exception raised from rdkafka, but as workaround it should work while
And question for Arnauld - please explain how it is possible to use acl?
in consumer and producer
Duncan Grist
@duncangrist
Jun 09 2016 10:54
Hey @vchampion, thanks for your response.
To be clear, I'm not worried about the errors being written to stderr, I'm worried about the fact that it seems impossible to cleanly terminate a PHP process which attempts to send messages to Kafka when no brokers are available.
Vladislav Krakhalev
@vchampion
Jun 09 2016 10:57
Yes I agree with you, it's not a right way when we need find some workaraounds
Duncan Grist
@duncangrist
Jun 09 2016 10:57

I think the issue is being caused by php-rdkafka's destructor:

PHP_METHOD(RdKafka__Kafka, __destruct)
{
    kafka_object *intern;

    if (zend_parse_parameters(ZEND_NUM_ARGS() TSRMLS_CC, "") == FAILURE) {
        return;
    }

    intern = get_custom_object_zval(kafka_object, getThis());
    if (intern->rk) {
        while (rd_kafka_outq_len(intern->rk) > 0) {
            rd_kafka_poll(intern->rk, 50);
        }
        rd_kafka_destroy(intern->rk);
        intern->rk = NULL;
    }

    kafka_conf_callbacks_dtor(&intern->cbs TSRMLS_CC);
}

There doesn't seem to be a way of getting out of that while loop - or any kind of built-in timeout; it will simply loop forever, well at least until a broker becomes available again.

Vladislav Krakhalev
@vchampion
Jun 09 2016 10:59
As I remember the same behavior realized in librdkafka, but it will be good to have the right way to handle it
Duncan Grist
@duncangrist
Jun 09 2016 11:01
Yes - it is indeed librdkafka's recommended destruction sequence, but I think from a PHP context (which is primarily used for serving web pages) there needs to be a way to be more deterministic about how long it takes to cleanup.
Duncan Grist
@duncangrist
Jun 09 2016 15:40

While we await a response, I've removed the while loop from the destructor. This means that the PHP process can exit immediately if necessary whilst still allowing a return to the old behaviour (if desired) by implementing the destructor while loop in PHP code (making use of the getOutQLen()method).

I've forked the repo to make this change which we'll be depending on for now but I'd definitely prefer a properly supported way for achieving this.

Duncan Grist
@duncangrist
Jun 09 2016 15:55
Okay wow I feel stupid. The message.timeout.ms topic configuration setting seems to do exactly what I want it to!
Its default is set to 5 minutes and I never let my test script run that long.