by

Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Frédéric G. MARAND
    @fgm
    Found how to reproduce, although without actually debugging: the extension crashes instead of returning an error or throwing an exception if the group.id is not set in the configuration passed to the ctor.
    Frédéric G. MARAND
    @fgm
    This message was deleted
    Issue is at arnaud-lb/php-rdkafka#88
    Paul Dragoonis
    @dragoonis
    Hello everyone! :D
    Hey @edenhill great to see you here :)
    Paul Dragoonis
    @dragoonis
    For the life of me I can't get the broker address to change from 127.0.0.1 :(
        private function initialize(array $brokers)
        {
            $conf = new ConsumerConf();
            $conf->set('metadata.broker.list', implode(',', $brokers));
            $consumer = new Consumer($conf);
    
            $consumer->addBrokers(implode(',', $brokers));
            $this->setConsumer($consumer);
        }
    didn't work with just ->addBrokers(), tried the metadata.broker.list too, still tries to go to 127.0.0.1 :( :(
    Paul Dragoonis
    @dragoonis
    Nobody seems to be around - I've raised this bug - arnaud-lb/php-rdkafka#92
    Magnus Edenhill
    @edenhill
    You shouldn't use addBrokers, config is enough
    Paul Dragoonis
    @dragoonis
    let me remove addBrokers()
    Magnus Edenhill
    @edenhill
    Can you print the string you set?
    Paul Dragoonis
    @dragoonis
    No change - yes here's the string.
            $conf = new ConsumerConf();
            $conf->set('metadata.broker.list', 'xx.xx.xx.xx:9092');
            $consumer = new Consumer($conf);
    %3|1479475193.723|ERROR|rdkafka#consumer-1| 127.0.0.1:9092/1001: Connect to ipv4#127.0.0.1:9092 failed: Connection refused
    Magnus Edenhill
    @edenhill
    Ah, 127.0.0.1 is coming from the broker. Check its advertised.listener config
    Paul Dragoonis
    @dragoonis
    interesting, do you think ti's talking to remote broker
    but broker isn't listening on external IP ?
    suggestion on how I can run sniffer on remote server, on specific port ? to check incoming traffic
    I'm running kafka using this docker IMG - https://hub.docker.com/r/wurstmeister/kafka/
    aha
    docker-compose.yml was set to this .. tweaking
          KAFKA_ADVERTISED_HOST_NAME: 127.0.0.1
    Paul Dragoonis
    @dragoonis
    I'm able to connect to the broker now, that ENV var becomes advertised.host.name
    still some issues, but this is good progress - thanks @edenhill
    Magnus Edenhill
    @edenhill
    👍
    Paul Dragoonis
    @dragoonis
    I'm replacing another tech stack with a kafka solution, I need to learn kafka FAST! what's your suggestions on best places to learn from ?
    Paul Dragoonis
    @dragoonis
    @edenhill this is what's being set in server.properties
    #advertised.listeners=PLAINTEXT://your.host.name:9092
    it's commented out ..
    but these are at the end of the file
    port=9092
    create.topics=users:1:1
    advertised.host.name=xx.xx.xx.xx
    advertised.port=9092
    Magnus Edenhill
    @edenhill
    Make sure that host and port is reachable from the client
    And I believe.. Host.name was replaced by advertised.listeners
    Paul Dragoonis
    @dragoonis
    indeed, i am on kafka 0.10
    so i removed advertised.host/port, advertised.listeners=PLAINTEXT://<my ip>:9092
    however when kafka boots, it doesn't fully finish booting, just hangs there (after making the topic)
    Magnus Edenhill
    @edenhill
    Maybe it fails to reach other brokers, or zk
    Paul Dragoonis
    @dragoonis
    Only have 1 broker, this one
    it can access ZK
    Paul Dragoonis
    @dragoonis
    @edenhill which versions of kafka does php-rdkafka support? README says
    PHP-rdkafka is a thin librdkafka binding providing a working PHP 5 / PHP 7 Kafka 0.8 / 0.9 client.
    does this mean 0.10 is not supported ?
    Magnus Edenhill
    @edenhill
    It supports all >=0.8 versions
    Paul Dragoonis
    @dragoonis
    That's good to hear
    Paul Dragoonis
    @dragoonis
    @edenhill can you help me find out what version of the kafka protocol my PHP ext is using?
    also does the Producer class require access to remote zookeeper?
    $rk = new RdKafka\Producer($conf);
    Magnus Edenhill
    @edenhill
    @dragoonis ZK is not used by clients.
    @dragoonis you decide what protocol version to use, see this https://github.com/edenhill/librdkafka/wiki/Broker-version-compatibility
    Paul Dragoonis
    @dragoonis

    @edenhill thanks for replying! appreciated - I have this offset bug that I can't figure out - maybe you can spot something?

    My consumer here that i've built - using STORED offset. The bug behaviour is:

    Did this make sense ?

    (dunno how to make that gist block smaller on this chat, srry)
    Magnus Edenhill
    @edenhill
    seems like you are never committing the offsets you store (since auto commit is disabled), so whenever you restart it wont find a committed offset and thus resort to auto.offset.reset which defaults to latest
    Paul Dragoonis
    @dragoonis
    I am committing the offset when I get EOF (no more data)
    Magnus Edenhill
    @edenhill
    it is also kind of weird to only store (and I guess you also want commit) when you reach EOF. If your consumer is slower than the producer, for whatever reason, you will never reach EOF and thus never commit.