by

Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Emanuele Blanco
    @manub
    the latest version that supports Scalatest 2.2 is 0.7.1 for scalatest-embedded-kafka
    @ArchDev
    Marcin Kacperek
    @kalondar

    Hi all
    I need your suggestions.

    I've just added additional consumer and producer configuration, and @manub merged it to master, but I unable to use it for Scala 2.10.
    We still using Spark 1.6 on Kafka 0.9 and I need that configuration.

    We just replaced Kafka client by exclusion, but cannot handle with scala version

    Is it possible to build that project for Scala 2.10, or it's impossible for Kafka 0.10.2 ?
    Marcin Kacperek
    @kalondar
    or create additional branch for Kafka 0.9
    I can maintain it
    Marcin Kacperek
    @kalondar
    but.... how to version artifacts from 0.9 and master?
    Emanuele Blanco
    @manub
    hey Marcin, sorry I just read this - I've been quite busy with my day work!
    I can't build the project for Scala 2.10 as the libraries that I use they're not backwards compatible anymore
    I suggest you branch out kafka 0.8 branch and you update to kafka 0.9 that branch
    at that time I was still supporting Scala 2.10
    that would be the preferred approach. you can either build it on your sonatype repo or create a new branch and then release artifacts as 0.8.0-kafka-09 for example
    @kalondar ^^
    Arpan Chaudhury
    @arpanchaudhury
    Hi, is there a release for kafka 0.9.0.0?
    Emanuele Blanco
    @manub
    hi @arpanchaudhury - 0.5.0 works with kafka 0.9.0.1
    Salar Rahmanian
    @softinio
    can u use this to test the sasl authentication of kafka?
    Emanuele Blanco
    @manub
    @softinio I have never tried. however the Kafka that you get running is the real Kafka, so I guess if you configure it it could work
    but again - not 100% sure. it's been a while I haven't used kafka
    Salar Rahmanian
    @softinio
    ok thx
    Bartosz BÄ…bol
    @BBartosz
    hey which version supports Apache Kafka 0.11.0.1
    ?
    Emanuele Blanco
    @manub
    @BBartosz 0.16.0
    Arjun Chhabra
    @arjunchhabra
    Hey - is there a way to check a consumer groups' committed offset with Embedded Kafka?
    @manub if you get a chance to take a look :)
    Arjun Chhabra
    @arjunchhabra
    Scratch that - I was able to retrieve the groups' offset using AdminClient.listGroupOffsets(groupId)! If there is a cleaner solution feel free to send it my way :)
    Emanuele Blanco
    @manub
    I think that's a good enough solution @arjunchhabra :)
    Arjun Chhabra
    @arjunchhabra
    :thumbsup: thanks for a great library!
    Ravinder Matte
    @rmatte01
    I get this error while running basic test. An exception or error caused a run to abort: kafka.server.KafkaServer$.$lessinit$greater$default$2()Lorg/apache/kafka/common/utils/Time; java.lang.NoSuchMethodError: kafka.server.KafkaServer$.$lessinit$greater$default$2()Lorg/apache/kafka/common/utils/Time; at net.manub.embeddedkafka.EmbeddedKafkaSupport$class.net$manub$embeddedkafka$EmbeddedKafkaSupport$$startKafka(EmbeddedKafka.scala:619) at net.manub.embeddedkafka.EmbeddedKafkaSupport$$anonfun$withRunningKafka$1$$anonfun$apply$1.apply(EmbeddedKafka.scala:154)
    I am using <dependency> <groupId>net.manub</groupId> <artifactId>scalatest-embedded-kafka_2.11</artifactId> <version>1.1.0</version> <scope>test</scope> </dependency>
    Emanuele Blanco
    @manub
    @rmatte01 which version of Kafka are you using?
    Dmitriy Zakomirnyi
    @dmi3zkm
    Can I bootstrap kafka cluster programmatically using embedded kafka ?
    Emanuele Blanco
    @manub
    @dmi3zkm I haven't tried. check with the api if you can set the appropriate options
    in case a github issue will be appreciated :)
    Dmitriy Zakomirnyi
    @dmi3zkm
    @manub thanks, I will try
    Josh McDade
    @joshm1
    Hey guys. I know there hasn't been any activity . here since May but hoping somebody will see this and can help. Just starting to use kafka/http4s/fs2. When I use withRunningKafka the kafka server shuts down before I analyze the response body (which is an fs2.Stream). It works if I just use EmbeddedKafka.start
        implicit val config = EmbeddedKafkaConfig(kafkaPort = 12345)
        withRunningKafka {
          for {
            req <- Request[IO](method = Method.POST).withBody(CreateCompanyInput("foo", "Foo"))
            task <- httpService(req).value
            _ <- task.fold(IO(fail("Empty response")) *> IO.unit) { response =>
              {
                IO {
                  // if I use withRunningKafka, the kafka server is shutdown before I even run the line below
                  // if I use EmbeddedKafak.start, it works
                  val json = new String(response.body.compile.toVector.unsafeRunSync().toArray)
                  // want to run assertions on string here
                  json
                } *> IO(response.status should be(Status.Ok))
              }
            }
          } yield ()
        }
    Josh McDade
    @joshm1
    Ok. I was able to solve the above issue by running .unsafeRunSync on the IO[Unit] that returned inside withRunningKafka. Is there a better way to do this test?
    Emanuele Blanco
    @manub
    have you tried using EmbeddedKafka.start and .stop?
    Josh McDade
    @joshm1
    @manub yes, that works. i was j/w why the above wasn't working (until I later added .unsafeRunSync to the IO)
    Emanuele Blanco
    @manub
    will have a look at this in the next few days but please open an issue - I think it's better as more people will look at this as well!
    Benni
    @OberstK
    Hi everyone! First: awesome library! Works really good out off the box and provides many nice functionalities!
    I currently want to run a lot of tests on a kafka streams app handling multiple output topics. Problem is that I see sporadic failure of tests with such errors Timed out waiting for connection while in state: CONNECTING kafka.zookeeper.ZooKeeperClientTimeoutException: Timed out waiting for connection while in state: CONNECTING
    It feels like some of the tests fail if the streams app needs longer than the 6 second standard zookeeper timeout. Is this epxected? Should I group more tests together to not create so many "instances" of kafka? As the tests are run sequentially I thought this might not be a problem
    Emanuele Blanco
    @manub
    I think you can override the timeout setting IIRC
    Mohanraj Nagasamy
    @mohanraj-nagasamy
    Hi I'm trying to use the scalatest-embedded-schema-registry library. I couldn't find the version "2.0.0" in maven central, but i could find scalatest-embedded-kafka with 2.0.0. Is the publish job failed or something?
    Ram Janovski
    @rjanovski
    hey @manub, I'm trying to get started with the new repo, but sbt doesn't recognise it: "io.github.embeddedkafka" %% "embedded-kafka" % "2.1.0" % "test" (from the readme). is it still not published?
    santanu mohanty
    @km_santanu_twitter
    can some one gives end to end demo code
    "org.apache.kafka" % "kafka-clients" % "2.0.0",
    "net.manub" %% "scalatest-embedded-kafka" % "2.0.0" % "test"
    getting berror as
    20/05/19 20:19:29 WARN NetworkClient: [Producer clientId=producer-1] Error while fetching metadata with correlation id 1 : {test-topic=LEADER_NOT_AVAILABLE}
    20/05/19 20:19:28 WARN BrokerMetadataCheckpoint: No meta.properties file under dir C:\Users\