Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    tyler
    @skotchpine
    Also, happy Sunday.
    Travis Jeffery
    @travisjeffery
    @skotchpine yeah i’d probably say broker tests have higher priority, but serf tests are fine too
    @skotchpine I gave some feedback to Marc on changes to make for the API versioning, waiting for updates from him. If he doesn’t do it, i’ll do it by the end of this week
    Stevo Slavić
    @sslavic
    wondering how likely is it that at some point in the future we'll see Confluent stand behind the project, even if all of the project goals are met
    tyler
    @skotchpine
    @travisjeffery I some of my code merged. :D
    I've got a PR dangling still, though. Have you seen, should I continue, checking in
    Broker tests will be a bit easier now with the extra setup
    Travis Jeffery
    @travisjeffery
    @skotchpine Thanks, Tyler! I’ll take a look tomorrow.
    Travis Jeffery
    @travisjeffery
    Some recent good changes and contributions:
    • Added prometheus endpoint to get metrics, just exposes number of requests handle so far
    • Refactored the CLIs to have a similar UI to Hashicorp’s CLIs
    • Refactored the protocol pkg’s errors
    • Fixed bug in metadata responses
    • Updated to use dep rather than govendor
    • Added Dockerfile and docker-compose.yml
    ryanstinson
    @ryanstinson
    @travisjeffery testing Broker.startReplica, the current code will allow a dupe partition to be added. Would you like me to fix with the tests or create an issue?
    Jack Tuck
    @jacktuck

    hey @travisjeffery, I'm really excited about this project.

    I like kafka and CQRS but not the burden of jvm / zookeeper (zetcd could help here, but still.) .

    I have 2 questions:
    1 - I'd like to contribute if i find some time - are there any issues that are low-hanging fruit that i may jump on.
    2 - Is there any ETA to be able to use as a drop-in for kafka? / What are the biggest blockers atm.

    Thanks for your work!

    Also - in the spirit of cqrs - is something like kafka streaming on the table
    Travis Jeffery
    @travisjeffery
    @ryanstinson yeah adding a fix and testing it would be great!
    ryanstinson
    @ryanstinson
    👍🏻
    Travis Jeffery
    @travisjeffery
    @jacktuck two pieces of low hanging fruit are tests and api versioning: travisjeffery/jocko#48. especially if you have experience with sarama, since i want api versioning to work similar to sarama implemented it.
    @jacktuck haven’t looked into kafka streams yet, but it’s a client library so might be doable.
    Stevo Slavić
    @sslavic
    There must be server side bit to Kafka streams too, rocksdb dependency in Kafka server tar suggests so
    Travis Jeffery
    @travisjeffery
    @sslavic yeah there’s a server component, i’m talking more about how the client interops with the server. there’s a kafka stream port in golang so java shouldn’t be a hang up.
    Stevo Slavić
    @sslavic
    ran dep ensure on current master, Gopkg.lock got modified, mostly formatting changes (packages in new line), but also reference to github.com/hashicorp/go-memdb got added
    I'm using dep 0.4.1
    Stevo Slavić
    @sslavic
    it seems PR travisjeffery/jocko#91 where this was added, didn't include change to Gopkg.lock
    Stevo Slavić
    @sslavic
    @travisjeffery would it make sense to enable https://github.com/probot/dco#how-it-works on the repo?
    Jean Mertz
    @JeanMertz
    Anyone ever tried to run Jocko with https://github.com/confluentinc/confluent-kafka-go as the client? I'm looking into using Jocko as a lightweight test server (for starters), but can't seem to get it to work with confluent-kafka. I've copy/pasted the Sarama test, which are working as expected, but switching to confluent-kafka, I keep hitting localhost:9092/bootstrap: Connect to ipv6#[::1]:9092 failed: Connection refused, without a clear cause of the problem.
    The Jocko server debug logs do show a request coming in and a response going out, so I'm wondering if it's just that Jocko doesn't support the latests APIs required by librdkafka (which is the C library that the Go library uses underneath)?
    Stevo Slavić
    @sslavic
    haven't tried it
    I was trying to start jocko cluster by https://github.com/travisjeffery/jocko/tree/master/examples/cluster but it's outdated, PR still in the making travisjeffery/jocko#98 getting serf configuration to work remains to be done, seems there was recent refactoring, and it's not complete
    @JeanMertz are you starting single Jocko instance?
    Jean Mertz
    @JeanMertz
    Yes, and not with the CLI, but straight from the tests. As I mentioned, the Sarama test actually passes, so things are working on the server-side, and also on the Sarama client side, just not the confluent-kafka one.
    Actually, thinking about it, I see that the Sarama test uses the v0.10 API version, Kafka already went to v0.11, and since recently v1.0, so that might actually be it, I bet if I put the Sarama version on anything higher than 0.10, it will break as well.
    Stevo Slavić
    @sslavic
    well, by the docs, Jocko is targeting 0.10 compatibility for now
    Jean Mertz
    @JeanMertz
    Ah right, I missed that documentation, I see it mentioned now at travisjeffery/jocko#1
    Stevo Slavić
    @sslavic
    I guess rationale is that it has new clients, streaming are all supported in 0.10 - once all is working with 0.10, supporting newer protocol would be possible
    Jean Mertz
    @JeanMertz

    Hm yeah. There is this: https://github.com/edenhill/librdkafka/wiki/Broker-version-compatibility#detailed-description

    But it looks like that isn't working as expected either, I still get a Connection closed when setting the proper API version (0.10.0.0).

    Stevo Slavić
    @sslavic
    I'd debug server
    monetashi-io
    @monetashi-io
    Hai Travis and everybody, this is a very promising project! Is jocko ready to be used in a production environment? Anybody running this under any kind of load, and what are your experiences?
    yctn
    @yctn
    anyone already using this?
    Jack Tuck
    @jacktuck
    soon!
    replication and consumer groups are WIP if i understand correctly :)
    Stevo Slavić
    @sslavic
    @travisjeffery would there be difference at all if Jocko targeted Kafka 2.0 (without Scala high-level + simple consumer APIs, and sync+async producer) instead of 0.10 protocol compatibility?
    markfarnan
    @markfarnan
    Hello: I'm sure this has been asked a dozen times, (I can't see any real answers to it), BUT, any feel for how close this is to Production-Ready ? I need a nice, compact, FAST, Single node queue, and Kafka compatibility is great !. (Eventually will use it for Multi Server, / data center but first goal is single machine deployments.
    stick me in the 'despise zookeeper' bucket for why looking at this, And all our new stuff is in GO
    markfarnan
    @markfarnan
    ANyone even here ?
    Ilia Choly
    @icholy
    It's not production ready
    Daniel Nishimura
    @dnishimura
    Been keeping an eye on this a while too. Time to find a successor or fork it?
    Ilia Choly
    @icholy
    redis streams are pretty dope
    I've developed an in-house solution for my use-case using leveldb
    Daniel Nishimura
    @dnishimura
    been hearing a lot about redis streams recently. I should look into.
    loved using redis at my prev startup. It just works.
    yctn
    @yctn
    @travisjeffery maby you could provide us an update on this project
    jmwilkinson
    @jmwilkinson
    It looks like this project is mostly dead/unmaintained?