null
instead of []
I also use some default value = None, so it serialize as nothing. My problem is that I have a Writer
for a case class that I can't modify, I thus can't put None
as default value and so it serialize as null
while I want to drop them.
hello, I'm hitting an issue on CI (github action) when testing on Windows, due to the infamous \r
thing. Here's below a small example I did and the error I got on CI:
val raw = s"""
{
"test" : "yop",
"hola" : 1
}"""
val json = read[ujson.Value](raw)
ujson.ParseException: expected json value got "\r" at index 0
Did anyone already hit that issue?
$type
keys map to the same case class in sealed trait serialization. Use case is migration of $type keys which contain package name and when package name changes it breaks existing serialization which is already in the DB. The problem is to migrate Foo, A, B
to a new package barpkg
but be able to still deserialize A with both $type = "foopkg.A"
and $type = "barpkg.A"
, serialization should go always be with $type = "barpkg.A"
package foopkg
object Foo {
implicit val rw = macroRW[Foo]
}
sealed trait Foo
case class A(a: Int) extends Foo
case class B(b: int) extends Foo
case class Item(prefix: String, name: String, href: String)
implicit val ItemRW = upickle.default.macroRW[Item]
case class MultiItem(service: List[Item])
implicit val MultiItem RW = upickle.default.macroRW[MultiItem]
case class OneItem(service: Item)
implicit val OneItemRW = upickle.default.macroRW[OneItem]
case class Content(services: Either[MultiItem, OneItem], title: String)
implicit val ContentRW = upickle.default.macroRW[Content]
[error] upickle.core.Abort: expected sequence got dictionary
{
"dataflows": [
{
"name": "test",
"sources": [
{
"name": "person_inputs",
"path": "/data/input/events/person/*",
"format": "JSON"
}
],
"transformations": [
{
"name": "validation",
"type": "validate_fields",
"params": {
"input": "person_inputs",
"validations": [
{
"field": "office",
"validations": [
"notEmpty"
]
},
{
"field": "age",
"validations": [
"notNull"
]
}
]
}
},
{
"name": "ok_with_date",
"type": "add_fields",
"params": {
"input": "validation_ok",
"addFields": [
{
"name": "dt",
"function": "current_timestamp"
}
]
}
}
],
"sinks": [
{
"input": "ok_with_date",
"name": "raw-ok",
"paths": [
"/data/output/events/person"
],
"format": "JSON",
"saveMode": "OVERWRITE"
},
{
"input": "validation_ko",
"name": "raw-ko",
"paths": [
"/data/output/discards/person"
],
"format": "JSON",
"saveMode": "OVERWRITE"
}
]
}
]
}
val j = os.read(os.pwd/RelPath("src/main/scala/metadata.json"))
val jsonData = ujson.read(j)
ujson.Value$InvalidData: Expected ujson.Arr (data: {"dataflows":[{"name":"test","sources":[{"name":"person_inputs","path":"/data/input/events/person/*","format":"JSON"}],"transformations":[{"name":"validation","type":"validate_fields","params":{"input":"person_inputs","validations":[{"field":"office","validations":["notEmpty"]},{"field":"age","validations":["notNull"]}]}},{"name":"ok_with_date","type":"add_fields","params":{"input":"validation_ok","addFields":[{"name":"dt","function":"current_timestamp"}]}}],"sinks":[{"input":"ok_with_date","name":"raw-ok","paths":["/data/output/events/person"],"format":"JSON","saveMode":"OVERWRITE"},{"input":"validation_ko","name":"raw-ko","paths":["/data/output/discards/person"],"format":"JSON","saveMode":"OVERWRITE"}]}]})
match may not be exhaustive. It would fail on the following input: (x: Int forSome x not in 0)