julianpeeters/avrohugger

Generate Scala case class definitions from Avro schemas

Thomas

@thomasschoeftner

hi folks - quick question:
We need to split Avro models across different sbt projects.
Is it possible with sbt & avro-hugger to use a model from an .avsc in one sbt sub-project, in an .avsc file in another sbt sub-project?
With default settings, avro-hugger keeps complaining that the required type from the other project is not found.
Thx a lot!

Alexander Khotyanov

@aksharp

In case this was already discussed, is there a code example of what avro schema (avsc file contents) should look like to generate Scala ADT with sealed trait when running avroScalaGenerateSpecific?

Anshul Bansal

@anshulbansal2

Hi Folks ,
I need to add sbt plugin tht brings avrohugger core and other dependencies with 2.11 scala version instead of 2.12, currently addSbtPlugin("com.julianpeeters" % "sbt-avrohugger" % "2.0.0-RC22") is downloading 2.12 versions of libs

Thomas

@thomasschoeftner

sry, I think this channel is dead... I haven't seen a response in months :(

nicu marasoiu

@nmarasoiu

Hi guys, I am wondering how it would be possible, using IDL, avsc or other format, to reference type definitions from other types & make sure they are recognized... thanks a lot.. https://stackoverflow.com/questions/62535519/how-do-i-use-an-avro-type-in-another-avro-type-without-defining-it-again

nicolaemarasoiu

@nicolaemarasoiu

Hi, we have a Kafka topic with values of Avro type union of two Avro types (any message is of one or another type). When we have union types for fields, we know how to work with them in the AvroHugger context, but when the union is top level, is there a way to define an ad hoc type that is simply the union of the two explicit types? Thank you

PatrickEifler

@PatrickEifler

Hi, is there a way of generating case classes that are not final using SpecificRecord? Would like to use them to extend other case classes.

John Halbert

@johnhalbert

Hi all! I'm using avrohugger-maven-plugin, and for some reason fields that should be type uuid are typed as String in generated scala code. Does anyone know how to resolve this so it's properly java.util.UUID?

2 replies

Yuming Wang

@wangyum

Hi, It seems protocol generated code can not compatible with Avro 1.8.2. Do we have a way to workaround this issue?

This is because Spark only support Avro 1.8.2.

Stefano Perazzolo

@beetlecrunch

Hi, I am trying to generate an ADT from this schema using ScalaADT

protocol Foo {
    record Bar {
        string a;
    }

    record Baz {
        string b;
    }
}

but what I get is

/** MACHINE-GENERATED FROM AVRO SCHEMA. DO NOT EDIT DIRECTLY */
final case class Bar(a: String)

final case class Baz(b: String)

I would have expected this

/** MACHINE-GENERATED FROM AVRO SCHEMA. DO NOT EDIT DIRECTLY */
sealed trait Foo

final case class Bar(a: String) extends Foo

final case class Baz(b: String) extends Foo

Inside the build.sbt file I have inserted

avroScalaSpecificCustomTypes in Compile := {
  avrohugger.format.Standard.defaultTypes.copy(
    protocol = avrohugger.types.ScalaADT)
}

sourceGenerators in Compile += (avroScalaGenerate in Compile).taskValue

Did I omit something?
Is the behavior I have described to you actually what one should expect?
I apologize if it is a known problem and if it is just my misunderstanding, and I hope not to waste your time unnecessarily.

Thank you

Dmitriy Zakomirnyi

@dmi3zkm

Hey!
I'm considering to try the lib in my project and it makes me wonder why no stable release since Dec 2017.

Joan Goyeau

@joan38

Hi!

I just created https://github.com/joan38/mill-avro

The plugin for the Mill build tool

And I'm having the first issue:

new MemberData(, , , , , , , , , , )

Do you know why it's generating those , ?
It's not compilable code

Joan Goyeau

@joan38

I found out that's when we have "default": {}

It should have generated new MemberData() instead

Jeff Martin

@custommonkey

Hi, hoping someone might be able to help me setup sbt-avrohugger to use schema files from a jar. I've got sbt to download and unpack the schema files, but can't get avrohugger to see the schema files until I run compile for a second time.

Jeff Martin

@custommonkey

Worked it out, needed to use Def.sequential to set the order in which the tasks ran https://gist.github.com/custommonkey/b13e165a7404def84cd85ea3c5369157

monksy

@monksy

I've got my build.sbt file setup as:

import Dependencies._
import sbt._

ThisBuild / scalaVersion     := "2.13.2"
ThisBuild / version          := "0.0.1"

lazy val root = (project in file("."))
  .settings(
    name := "ziostreams",
    libraryDependencies ++= Seq(
      zio, zioKafka, zioStreams, avro
    )
  ).settings(
//  watchSources ++= ((avroSourceDirectory in Compile).value ** "*.avdl").get
//  ,
  sourceGenerators in Compile += (avroScalaGenerate in Compile).taskValue
)

And the plugin.sbt file set to use 2.0.0-RC4

When I do a sbt compile I'm getting error not found value avroScalaSourceDirectory and avroScalaGenerate (other line is uncommented)

What do I need to import to get it to see that I'm using the avrohugger

monksy

@monksy

The issue I had was that the plugins.sbt file was not in the right place . it never knew about the plugin

Alexey Vorobyov

@vorobuev

Hi! When to expect new version (1.7) of avrohugger-maven-plugin in maven central repository? Can you please make one?

Alexey Vorobyov

@vorobuev

I'm having this issue julianpeeters/avrohugger#85

Error message: "Unions beyond nullable fields are not supported"

It was resolved in new version of avrohugger

Ayal Kroub

@AyalKroub_twitter

@julianpeeters hey, is there a way for schemas in a certain module to access schemas in other modules in the repo? could that be possible using some sbt manipulations?
I tried to follow your example here: https://github.com/julianpeeters/sbt-avrohugger/tree/master/src/sbt-test/avrohugger/filesorter/src/main/avro
however, I'm still getting undefined type error, because each namespace I define is only reachable in its own module

Where communities thrive

People

Repo info

Activity