Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    tyler
    @skotchpine
    Broker tests will be a bit easier now with the extra setup
    Travis Jeffery
    @travisjeffery
    @skotchpine Thanks, Tyler! I’ll take a look tomorrow.
    Travis Jeffery
    @travisjeffery
    Some recent good changes and contributions:
    • Added prometheus endpoint to get metrics, just exposes number of requests handle so far
    • Refactored the CLIs to have a similar UI to Hashicorp’s CLIs
    • Refactored the protocol pkg’s errors
    • Fixed bug in metadata responses
    • Updated to use dep rather than govendor
    • Added Dockerfile and docker-compose.yml
    ryanstinson
    @ryanstinson
    @travisjeffery testing Broker.startReplica, the current code will allow a dupe partition to be added. Would you like me to fix with the tests or create an issue?
    Jack Tuck
    @jacktuck

    hey @travisjeffery, I'm really excited about this project.

    I like kafka and CQRS but not the burden of jvm / zookeeper (zetcd could help here, but still.) .

    I have 2 questions:
    1 - I'd like to contribute if i find some time - are there any issues that are low-hanging fruit that i may jump on.
    2 - Is there any ETA to be able to use as a drop-in for kafka? / What are the biggest blockers atm.

    Thanks for your work!

    Also - in the spirit of cqrs - is something like kafka streaming on the table
    Travis Jeffery
    @travisjeffery
    @ryanstinson yeah adding a fix and testing it would be great!
    ryanstinson
    @ryanstinson
    👍🏻
    Travis Jeffery
    @travisjeffery
    @jacktuck two pieces of low hanging fruit are tests and api versioning: travisjeffery/jocko#48. especially if you have experience with sarama, since i want api versioning to work similar to sarama implemented it.
    @jacktuck haven’t looked into kafka streams yet, but it’s a client library so might be doable.
    Stevo Slavić
    @sslavic
    There must be server side bit to Kafka streams too, rocksdb dependency in Kafka server tar suggests so
    Travis Jeffery
    @travisjeffery
    @sslavic yeah there’s a server component, i’m talking more about how the client interops with the server. there’s a kafka stream port in golang so java shouldn’t be a hang up.
    Stevo Slavić
    @sslavic
    ran dep ensure on current master, Gopkg.lock got modified, mostly formatting changes (packages in new line), but also reference to github.com/hashicorp/go-memdb got added
    I'm using dep 0.4.1
    Stevo Slavić
    @sslavic
    it seems PR travisjeffery/jocko#91 where this was added, didn't include change to Gopkg.lock
    Stevo Slavić
    @sslavic
    @travisjeffery would it make sense to enable https://github.com/probot/dco#how-it-works on the repo?
    Jean Mertz
    @JeanMertz
    Anyone ever tried to run Jocko with https://github.com/confluentinc/confluent-kafka-go as the client? I'm looking into using Jocko as a lightweight test server (for starters), but can't seem to get it to work with confluent-kafka. I've copy/pasted the Sarama test, which are working as expected, but switching to confluent-kafka, I keep hitting localhost:9092/bootstrap: Connect to ipv6#[::1]:9092 failed: Connection refused, without a clear cause of the problem.
    The Jocko server debug logs do show a request coming in and a response going out, so I'm wondering if it's just that Jocko doesn't support the latests APIs required by librdkafka (which is the C library that the Go library uses underneath)?
    Stevo Slavić
    @sslavic
    haven't tried it
    I was trying to start jocko cluster by https://github.com/travisjeffery/jocko/tree/master/examples/cluster but it's outdated, PR still in the making travisjeffery/jocko#98 getting serf configuration to work remains to be done, seems there was recent refactoring, and it's not complete
    @JeanMertz are you starting single Jocko instance?
    Jean Mertz
    @JeanMertz
    Yes, and not with the CLI, but straight from the tests. As I mentioned, the Sarama test actually passes, so things are working on the server-side, and also on the Sarama client side, just not the confluent-kafka one.
    Actually, thinking about it, I see that the Sarama test uses the v0.10 API version, Kafka already went to v0.11, and since recently v1.0, so that might actually be it, I bet if I put the Sarama version on anything higher than 0.10, it will break as well.
    Stevo Slavić
    @sslavic
    well, by the docs, Jocko is targeting 0.10 compatibility for now
    Jean Mertz
    @JeanMertz
    Ah right, I missed that documentation, I see it mentioned now at travisjeffery/jocko#1
    Stevo Slavić
    @sslavic
    I guess rationale is that it has new clients, streaming are all supported in 0.10 - once all is working with 0.10, supporting newer protocol would be possible
    Jean Mertz
    @JeanMertz

    Hm yeah. There is this: https://github.com/edenhill/librdkafka/wiki/Broker-version-compatibility#detailed-description

    But it looks like that isn't working as expected either, I still get a Connection closed when setting the proper API version (0.10.0.0).

    Stevo Slavić
    @sslavic
    I'd debug server
    monetashi-io
    @monetashi-io
    Hai Travis and everybody, this is a very promising project! Is jocko ready to be used in a production environment? Anybody running this under any kind of load, and what are your experiences?
    yctn
    @yctn
    anyone already using this?
    Jack Tuck
    @jacktuck
    soon!
    replication and consumer groups are WIP if i understand correctly :)
    Stevo Slavić
    @sslavic
    @travisjeffery would there be difference at all if Jocko targeted Kafka 2.0 (without Scala high-level + simple consumer APIs, and sync+async producer) instead of 0.10 protocol compatibility?
    markfarnan
    @markfarnan
    Hello: I'm sure this has been asked a dozen times, (I can't see any real answers to it), BUT, any feel for how close this is to Production-Ready ? I need a nice, compact, FAST, Single node queue, and Kafka compatibility is great !. (Eventually will use it for Multi Server, / data center but first goal is single machine deployments.
    stick me in the 'despise zookeeper' bucket for why looking at this, And all our new stuff is in GO
    markfarnan
    @markfarnan
    ANyone even here ?
    Ilia Choly
    @icholy
    It's not production ready
    Daniel Nishimura
    @dnishimura
    Been keeping an eye on this a while too. Time to find a successor or fork it?
    Ilia Choly
    @icholy
    redis streams are pretty dope
    I've developed an in-house solution for my use-case using leveldb
    Daniel Nishimura
    @dnishimura
    been hearing a lot about redis streams recently. I should look into.
    loved using redis at my prev startup. It just works.
    yctn
    @yctn
    @travisjeffery maby you could provide us an update on this project
    jmwilkinson
    @jmwilkinson
    It looks like this project is mostly dead/unmaintained?