This one is a separate module that replaces the encoding/decoding functionality while keeping the existing vulcan API (so uses the java schema representation) - fd4s/vulcan#289
I think even if it doesn't become the preferred implementation, it would be good to have it for cross-validating the implementations against each other and clarifying our understanding
I finally made some proper progress by starting from what we had rather than starting from the ground up with trying to produce a fully statically validated implementation
the schema registry client has a lot of
synchronized for what's essentially a concurrent map and a couple of rest calls that need to be cached
I feel like some of that fs2-kafka code should move to vulcan though - the thing of wiring up the serdes with the codecs to get the right schema resolution seems much more an avro thing than a kafka thing
What do people think about adding reader behaviour that's optionally non-compliant with the Avro spec? I'm thinking in particular about allowing a union field in a record to be decoded as a default value when it's present but not a type the reader recognises. So e.g. if the writer schema is
[Foo, Bar, Baz] I can use [null, Foo] as a reader schema and decode as Option[Foo]. It really annoys me that Avro doesn't allow that.
I recently updated my version of
vulcan-generic and I see that the schema getting generated for my sealed trait has changed. The change is in the order of records. This also affects the references of nested records. Is there a way to control this or prevent that change?
5 replies
Not sure if its really a bug but as an FYI came across an issue where the
LocalDate produces weird results where LocalDate.toEpochDay returns a value greater than the max int (due to internal use of Long.toInt), obviously its a fairly unlikely scenario in the real world, but just thought I'd mention it!
@bplommer after switching to 1.7.0 I get "Error decoding Option: Error decoding union: Missing alternative array in union" in derived codecs when I decode
GenericData.Record. But it happens ONLY after ser/de via GenericDatumWriter/GenericDatumReader
Which change might have broken the codec?
@catostrophe eek, thanks for the heads up - would you mind opening a GitHub issue?
Sure
That was an interesting one to debug - the serde'd record used a collection type for arrays that's also a subtype of
GenericData, which we wrongly assumed was only used for named types (record, enum, fixed)
Hello guys
And thank you Ben for the good library
I've started using it and having trouble decoding the case class value from the array[byte]
Having
And thank you Ben for the good library
I've started using it and having trouble decoding the case class value from the array[byte]
Having
bytes: Array[Byte]
// Can I do?
Codec.decode(bytes)
case class FooBar(foo: String, bar: String)
implicit val foobarCodec: Codec[FooBar] =
Codec.record[FooBar](
name = "foo",
namespace = "foo.bar"
) { field =>
(
field("foo", _.foo),
field("bar", _.bar)
).mapN(FooBar)
}