Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Filippo De Luca
    @filosganga
    cool thanks!
    Ben Plommer
    @bplommer
    Oh btw @filosganga I remember some time back we were talking about re-implementing functionality from the Java lib so Vulcan wouldn't depend on it - I have a couple of WIP/experimental PRs up for aspects of that (one for representing schemas, one for scodec-based codecs)
    Filippo De Luca
    @filosganga
    It would be great
    Ben Plommer
    @bplommer
    This one is a separate module that replaces the encoding/decoding functionality while keeping the existing vulcan API (so uses the java schema representation) - fd4s/vulcan#289
    I think even if it doesn't become the preferred implementation, it would be good to have it for cross-validating the implementations against each other and clarifying our understanding
    I finally made some proper progress by starting from what we had rather than starting from the ground up with trying to produce a fully statically validated implementation
    Fabio Labella
    @SystemFw
    oh, one quick win
    the schema registry client has a lot of synchronized for what's essentially a concurrent map and a couple of rest calls that need to be cached
    Ben Plommer
    @bplommer
    yeah, we have a ticket in fs2-kafka to reimplement that
    I feel like some of that fs2-kafka code should move to vulcan though - the thing of wiring up the serdes with the codecs to get the right schema resolution seems much more an avro thing than a kafka thing
    Filippo De Luca
    @filosganga
    Uh, I am not sure about that. If the vulcona scope is just Avro, it has nothing to do with Kafka serde, but can be used in any other context. We use it with DynamoDb for example
    Ben Plommer
    @bplommer
    What do people think about adding reader behaviour that's optionally non-compliant with the Avro spec? I'm thinking in particular about allowing a union field in a record to be decoded as a default value when it's present but not a type the reader recognises. So e.g. if the writer schema is [Foo, Bar, Baz] I can use [null, Foo] as a reader schema and decode as Option[Foo]. It really annoys me that Avro doesn't allow that.
    Keir Lawson
    @keirlawson
    Is it possible to use Vulcan to go from a GenericRecord to a specific implementation? As opposed to deserialising directly from bytes?
    Liu Yi
    @kinglywork

    Hey there, I got a question about the codec, say if I have a model like this:

    sealed abstract class Parent
    
    final case class Child1(property: AnyType) extends Parent
    final case class Child2() extends Parent

    there is no fields in Child2, how can I write codec for Child2 ??

    Keir Lawson
    @keirlawson
    Why does Codec[A].encode return an Either? What are the possible failure modes?
    Viktor Rudebeck
    @vlovgr
    @keirlawson it basically stems from this issue: fd4s/vulcan#271
    Ben Plommer
    @bplommer
    @vlovgr speaking of, what are your thoughts on throwing when a valid schema can’t be instantiated?
    Viktor Rudebeck
    @vlovgr
    I think it might be okay until we can validate at compile-time.
    Nishant Vishwakarma
    @nishantv12
    I recently updated my version of vulcan-generic and I see that the schema getting generated for my sealed trait has changed. The change is in the order of records. This also affects the references of nested records. Is there a way to control this or prevent that change?
    5 replies
    Keir Lawson
    @keirlawson
    Is there a reason why there is a LocalDate codec included with Vulcan, but not a LocalTime codec? is this because there are two possible logical encodings, millis vs micros?
    Ben Plommer
    @bplommer
    No reason. Feel free to open an issue or a PR - probably best to have localTimeMillis and localTimeMicros.
    Keir Lawson
    @keirlawson
    :+1:
    Keir Lawson
    @keirlawson
    Would it be possible to cut a Vulcan release? Keen to port my code over to the new time codecs...
    Ben Plommer
    @bplommer
    Hi Keir - sorry, I missed that. But I’ve just released 1.7.0
    Keir Lawson
    @keirlawson
    great, thanks!
    catostrophe
    @catostrophe
    @bplommer after switching to 1.7.0 I get "Error decoding Option: Error decoding union: Missing alternative array in union" in derived codecs when I decode GenericData.Record. But it happens ONLY after ser/de via GenericDatumWriter/GenericDatumReader
    Keir Lawson
    @keirlawson
    Not sure if its really a bug but as an FYI came across an issue where the LocalDate produces weird results where LocalDate.toEpochDay returns a value greater than the max int (due to internal use of Long.toInt), obviously its a fairly unlikely scenario in the real world, but just thought I'd mention it!
    Ben Plommer
    @bplommer
    @catostrophe eek, thanks for the heads up - would you mind opening a GitHub issue?
    catostrophe
    @catostrophe

    @bplommer after switching to 1.7.0 I get "Error decoding Option: Error decoding union: Missing alternative array in union" in derived codecs when I decode GenericData.Record. But it happens ONLY after ser/de via GenericDatumWriter/GenericDatumReader

    Which change might have broken the codec?

    @catostrophe eek, thanks for the heads up - would you mind opening a GitHub issue?

    Sure

    Ben Plommer
    @bplommer
    @keirlawson yeah I’d say that’s a bug - we should raise an error if the date is out of range
    Keir Lawson
    @keirlawson
    ok, will add a report on github
    Ben Plommer
    @bplommer
    @catostrophe I'd guess it's to do with this PR https://github.com/fd4s/vulcan/pull/334/files
    catostrophe
    @catostrophe
    Ben Plommer
    @bplommer
    Thanks, I'll look into it :+1:
    Keir Lawson
    @keirlawson
    I'm getting the same issue with 1.7.0
    Ben Plommer
    @bplommer
    Yep, I found the problem - pushing up a fix now
    Keir Lawson
    @keirlawson
    cheers!
    Ben Plommer
    @bplommer
    catostrophe
    @catostrophe
    @bplommer thanx!
    Ben Plommer
    @bplommer
    Thanks for the report! 🙂
    That was an interesting one to debug - the serde'd record used a collection type for arrays that's also a subtype of GenericData, which we wrongly assumed was only used for named types (record, enum, fixed)
    Ben Plommer
    @bplommer
    1.7.1 is out
    Keir Lawson
    @keirlawson
    Issue resolved for me :)
    Ben Plommer
    @bplommer
    Great
    Igor Markiewicz
    @imarkiew
    Hello
    Does vulcan-generic support renaming fields e.g. camel case to snake case? Something like circe-generic does?
    Viktor Rudebeck
    @vlovgr
    @imarkiew It does not, but you can use manually defined codecs if you need to do renames.
    marcinkrykowski
    @marcinkrykowski
    Hello, how can I write a custom implicit codec for tagged values?
    marcinkrykowski
    @marcinkrykowski
    Issue solved. Eg.
    implicit def taggedStringCodec[T]: Codec[String @@ T] = Codec[String].imap[String @@ T](_.taggedWith[T](_.toString)