Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Paul Dragoonis
    @dragoonis
    @edenhill should I be calling ->storeOffset() upon every entry into RD_KAFKA_RESP_ERR_NO_ERROR? If not .. where?
    Magnus Edenhill
    @edenhill
    no, you are storing the offset for future commit, but you are never committing
    Paul Dragoonis
    @dragoonis
    I see!
    Magnus Edenhill
    @edenhill
    offsetStore = store offset for future commit
    commit = commit stored offset
    Paul Dragoonis
    @dragoonis
    Totally understand 100% :) thanks
    Magnus Edenhill
    @edenhill
    :)
    the nomenclature is kind of weird
    STORED, store, commit
    so your confusion is warranted
    Srigi
    @srigi

    Hello everyone. I'm coming here from issues on GH asking before filling the issue.
    I'm having troubles running even the simplest demo of RdKafka. I'm running the macOS 10.12.1 (Sierra). My PHP is installed via homebrew. I tried both 5.6 and 7.0.

    When I install/enable the rdkafka extension (via homebrew or manually compile, tried both) and then run $rk = new RdKafka\Producer(); I get segmentation fault. I tried both Apache & cli mode for the code.
    However in phpinfo() everything looks OK. I tried homebrew's v0.9.1 and manually installed 1.0.0. v2.0.0 won't compile.
    Any ideas? Is there a way get more info from that segfault?

    Srigi
    @srigi
    OK, for PHP7, you must use rdkafka 2.0.x.
    Paul Dragoonis
    @dragoonis
    @srigi hello! I'm using PHP5.6, with Kafka 0.10. Only started few days go. Did you get your segfault fixed?
    For php-rdkafka, there's a php7 branch you can use, which will be more uptodate than the 2.0.x tag. https://github.com/arnaud-lb/php-rdkafka/tree/php7
    The readme says "git checkout php7" to get the php7 code. Hope that helps.
    Srigi
    @srigi
    Well, today I tried start with the clean-sheet. I reinstalled cli-tools (compilers) from Apple, I reinstalled PHP7 with homebrew and then I compiled as @dragoonis suggested. Cloned & switched branch to php7. phpize, ./configure, make all, make install. All good, extesnsion enabled successfully (checked in phpinfo). But still segfault on $rk = new RdKafka\Producer().
    I also tried running w/o php.ini with command php -n -d extension="/usr/local/Cellar/php70/7.0.13_6/lib/php/extensions/no-debug-non-zts-20151012/rdkafka.so" cli-producer.php
    For my project I will be forced to use Docker for development. In container everything works fine :)
    Magnus Edenhill
    @edenhill
    @srigi does it write a 'core' file? if not, run ulimit -c unlimited first. Then do something like: gdb php core and then bt in gdb to get the backtrace
    Paul Dragoonis
    @dragoonis
    @edenhill How do I get a 'core' file dumped onto disk when the segfault happens?
    Trying to debug a segfault at my end, but ain't got a 'core' file.
    This is from the PHP CLI btw, no apache/fpm here.
    Magnus Edenhill
    @edenhill
    ulimit -c unlimited
    Paul Dragoonis
    @dragoonis
    Yes, I ran that.
    Magnus Edenhill
    @edenhill
    make sure the current directory is writable and there is no core file already
    Paul Dragoonis
    @dragoonis
    done.
    Magnus Edenhill
    @edenhill
    after that: YMMV depending on OS
    Paul Dragoonis
    @dragoonis

    i done

    gdb `which php`

    Then it says: Reading symbols from /usr/bin/php...(no debugging symbols found)...done.

    I got something with strace
    Magnus Edenhill
    @edenhill
    strace typically isnt usable for tracking down segfaults, it will only show syscalls so it is too coarse grained
    what did 'bt' in gdb say?
    Paul Dragoonis
    @dragoonis
    @edenhill i figured it out. It's nothing to do with kafka.
    If you have a class constant, and you refer to it using the wrong case (like all lowercase, when it's defined as uppercase) PHP5.6 segfaults
    Paul Dragoonis
    @dragoonis
    5.6.15 segfaults but 5.6.28 does not - must have been patched :)
    Nico
    @Nico_beBee_twitter
    Hi, have a question, when I create a new TopicPartition, warnings and other advises like deprecateds, throw exceptions, but if I reset the php error handler, the problem persists. (PHP 5.6)
    Arnaud Le Blanc
    @arnaud-lb
    Hi @Nico_beBee_twitter
    Could you show the exact error messages ?
    Nico
    @Nico_beBee_twitter
    There are no error messages, the error handler change and warnings, deprecated notifications and similar messages from other parts of the code causes exception when a TopicPartition object is created.
    Nico
    @Nico_beBee_twitter
    For example:
    // This error not throw exception
    @trigger_error('', E_USER_DEPRECATED);
    $partition = new TopicPartition($topic, $partition, $offset);
    // This error now throw exception
    @trigger_error('', E_USER_DEPRECATED);
    Nastasia Saby
    @NastasiaSaby
    Hi everybody, can I ask you something? I'm trying to use the high level consumer with the example given in the documentation, but I have a problem.
    The KafkaConsumer seems to commit my messages automatically when doing "consume" even if I put the "auto.commit.enable" or the "enable.auto.commit" to 0.
    Is there something I do wrong? Could anyone help me?
    I am working with PHP7 and rdkafka 2.0.1.
    Magnus Edenhill
    @edenhill
    @NastasiaSaby that's enable.auto.commit (for the high level consumer), and it really should work.
    what librdkafka version?
    Nastasia Saby
    @NastasiaSaby
    @edenhill Hi, I tried it but it still does not work.
    @edenhill My configuration is : rdkafka : 2.0.1, librdkafka : 0.9, kafka : 0.10, PHP7. I can send you my code if you want. It is the "high level consumer" example "https://arnaud-lb.github.io/php-rdkafka/phpdoc/rdkafka.examples-high-level-consumer.html". I have only added this : $topicConf->set('enable.auto.commit', 0);
    Magnus Edenhill
    @edenhill
    @NastasiaSaby what exact librdkafka version?
    the high-level consumer uses global configuration properties ("enable.auto.commit"), not topic-level configuration ("auto.commit.enable")
    Nastasia Saby
    @NastasiaSaby

    rdkafka

    rdkafka support => enabled
    version => 2.0.1
    build date => Dec 12 2016 10:13:42
    librdkafka version (runtime) => 0.9.2-112-g7f12b7
    librdkafka version (build) => 0.9.3.0

    Magnus Edenhill
    @edenhill
    $topicConf->set("enable.auto.commit", ..) should actually fail since that is a global property
    Nastasia Saby
    @NastasiaSaby
    Ok so do I have to change my kafka config to add an "enable.auto.commit" option in the servier_properties ?
    Thank you for your quick answer by the way