Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
Benjamin
@bdparrish
none of that works with Java
but i get your point
P. Oscar Boykin
@johnynek
why can’t you just pass the Class<T> in?
Benjamin
@bdparrish
i can pass that in
P. Oscar Boykin
@johnynek
ok.
Benjamin
@bdparrish
but i can't get out the right object
P. Oscar Boykin
@johnynek
what do you mean?
what is the right object?
Benjamin
@bdparrish
I am still getting a ClassTag<T> from ClassTag$.MODULE$.apply(clazz);
which I cannot pass into SpecificAvroCodecs.toBinary(classTag);
same original issue
P. Oscar Boykin
@johnynek
what is the type of clazz?
Benjamin
@bdparrish
Class<T>
P. Oscar Boykin
@johnynek
okay, so there are two issues here:
1) having the right class at runtime.
2) having the types line up at compile time.
Benjamin
@bdparrish
yep
P. Oscar Boykin
@johnynek
if you don’t have both working it fails. We got #2, but not #1.
so, your attempt here will fix #1 but not #2.
#2 can be fixed by casting if we don’t break #1
before we used casting, but actually were breaking #1
so, you can just do: (ClassTag<T extends SpecificRecordBase>)ClassTag$.MODULE$.apply(clazz) and I think it will work.
(I don’t do much java these days… :(
if that cast does not work, one will...
Benjamin
@bdparrish
cannot cast like that
P. Oscar Boykin
@johnynek
like, you could just do: (ClassTag<SpecificRecordBase>)
Benjamin
@bdparrish
the extends is already defined in the class
P. Oscar Boykin
@johnynek
as long as you keep the clazz being passed in.
Benjamin
@bdparrish
using ClassTag<SpecificRecordBase> got me the other error...
Exception in thread "main" java.lang.InstantiationException
    at sun.reflect.InstantiationExceptionConstructorAccessorImpl.newInstance(InstantiationExceptionConstructorAccessorImpl.java:48)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at java.lang.Class.newInstance(Class.java:442)
    at com.twitter.bijection.avro.SpecificAvroCodecs$.toBinary(AvroCodecs.scala:106)
    at com.twitter.bijection.avro.SpecificAvroCodecs.toBinary(AvroCodecs.scala)
Michael Hsueh
@mkhsueh
Howdy- nice work on Bijection! Currently using it for Avro handling; has anyone gotten this error deserializing Joda DateTime? java.lang.Long cannot be cast to org.joda.time.DateTime
was able to serialize a record with this value: {"name": "eventTime", "type": {"type": "long", "logicalType": "timestamp-millis"}}, but calling injection.invert(the serialized byte[]).get() throws that error
Michael Hsueh
@mkhsueh
this is my injection object private static Injection<MyModel, byte[]> injection = SpecificAvroCodecs.toBinary(scala.reflect.ClassTag$.MODULE$.apply(MyModel.class)); Thanks for any advice!
Michael Hsueh
@mkhsueh
dug into the codebase...( DateBijectionLaws.scala ) looks like round trips were expected using 'String'. Changing the underlying Avro schema to type "string" worked. Seems like it should've had dual support given the logical type?
P. Oscar Boykin
@johnynek
@mkhsueh I have not seen this error, but please file an issue and maybe we can find a fix.
Marius Soutier
@mariussoutier
@mkhsueh Might this be related to the fact that bijection uses Avro 1.7 and you are using features from Avro 1.8?
Tim Chan
@timchan-lumoslabs
I'd like to make a bijection between two case classes that I have. Do I just extend AbstractBijection?
P. Oscar Boykin
@johnynek
@timchan-lumoslabs yes. That's one way. There are also macros that can do this for you for case classes. On the phone now, but should not be too hard to find.
Tim Chan
@timchan-lumoslabs
If I stick with extending AbstractBijection, how do I use Conversion.asMethod? I'm getting a 'could not find implicit value for parameter conv' error
oh nm. i figured it out!
Sam Ritchie
@sritchie
@timchan-lumoslabs yeah, you just need to make it available in implicit scope
Tim Chan
@timchan-lumoslabs
val foo = Foo("abc", 123)
implicit val inj = SpecificAvroCodecs[Foo]
val bytes = Injection[Foo, Array[Byte]](foo)
val attempt = Injection.invert[Foo, Array[Byte]](bytes)
attempt.get
gives me
attempt: scala.util.Try[com.foo.Foo] = Failure(com.twitter.bijection.InversionFailure: Failed to invert: [B@39675182)
com.twitter.bijection.InversionFailure: Failed to invert: [B@39675182
Caused by: org.apache.avro.AvroRuntimeException: Not a Specific class: class com.foo.Foo
    at org.apache.avro.specific.SpecificData.createSchema(SpecificData.java:248)
    at org.apache.avro.specific.SpecificData.getSchema(SpecificData.java:189)
I'm using avrohugger to generate SpecificRecord case classes.
anand79
@anand79
hi I am reading from my own kafka topic and getting error - org.apache.avro.AvroRuntimeException: Malformed data. Length is negative: -128
Any idea what could be the issue
P. Oscar Boykin
@johnynek
are you using Bijection?
anand79
@anand79
yes
P. Oscar Boykin
@johnynek
what is likely the case is that the data is not in the format you expect.
it was reading, and found a negative length