Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Thomas Droxler
    @tdroxler
    Hello, is there an equivalent to the circe.Printer(dropNullValues = true)? I see that in the write we can already pass an escapeUnicode arg, would it make sense and is it feasible to add a dropNullValues there?
    Thomas Droxler
    @tdroxler

    I found in this gitter a quickfix to have

    implicit class JsonExt(val json: ujson.Value) extends AnyVal {
        def dropNullValues: Option[ujson.Value] = {
       ...

    but it sad to have to retraverse the all Value to drop the null values

    Andriy Plokhotnyuk
    @plokhotnyuk
    @tdroxler Do you need solution for Writer[Option[T]]?
    Thomas Droxler
    @tdroxler
    nope, I already use the one proposed in the examples, that serialize to null instead of [] I also use some default value = None, so it serialize as nothing. My problem is that I have a Writer for a case class that I can't modify, I thus can't put None as default value and so it serialize as null while I want to drop them.
    Doron Somech
    @somdoron
    Question about error handling, what is the best way to fail deserialization, should we just throw Abort? What is the best way to catch deserialization error, is it catching AbortException?
    Sakib Hadžiavdić
    @sake92
    is it possible to skip a key when writing?
    for example when it's value is None
    Doron Somech
    @somdoron
    @sake92 yes, by using a default value, upickle will drop the field if the value eqauls the default value.
    Sakib Hadžiavdić
    @sake92
    @somdoron IDK why libraries do that
    it makes no sense at all
    Thomas Droxler
    @tdroxler

    hello, I'm hitting an issue on CI (github action) when testing on Windows, due to the infamous \r thing. Here's below a small example I did and the error I got on CI:

    val raw = s"""
      {
        "test" : "yop",
        "hola" : 1
      }"""
    val json = read[ujson.Value](raw)
    ujson.ParseException: expected json value got "\r" at index 0

    Did anyone already hit that issue?

    Thomas Droxler
    @tdroxler
    I could do the following dirty hack in my custom upickle object https://github.com/alephium/alephium/blob/master/json/src/main/scala/org/alephium/json/Json.scala#L29-L39 but this bug sounds like something that could be tackled in upickle's core no?
    jodersky
    @jodersky
    Does this also happen outside of the CI environment? I can't remember the details, but I've run into strange locale issues in some CI environments in the past (I'm not saying that this isn't a bug in upickle, but it's good to narrow down potential causes).
    Piotr Buszka
    @pbuszka
    is there a way to have two $type keys map to the same case class in sealed trait serialization. Use case is migration of $type keys which contain package name and when package name changes it breaks existing serialization which is already in the DB. The problem is to migrate Foo, A, B to a new package barpkg but be able to still deserialize A with both $type = "foopkg.A" and $type = "barpkg.A", serialization should go always be with $type = "barpkg.A"
    package foopkg
    
    object Foo {
      implicit val rw = macroRW[Foo]
    }
    sealed trait Foo
    case class A(a: Int) extends Foo
    case class B(b: int) extends Foo
    Li Haoyi
    @lihaoyi
    I'm not sure we support that, but worst come to worst you can use upickle.default.readwriter[ujson.Value].bimap[Foo] to write this serializer/deserializer manually
    Jean-Jerome
    @jeanjerome
    Hi! I'm dealing with upickle.core.Abort: expected sequence got dictionarywhen upickle parses a JSON list which has no bracket when it contains one item.
    How to handle that? Anybody managed such case??
    I've tried custom pickers with no luck...
    Jean-Jerome
    @jeanjerome
    I've tried the simplest to handle valid JSON list with one item only (that is a dictionary not a list):
      case class Item(prefix: String, name: String, href: String)
      implicit val ItemRW = upickle.default.macroRW[Item]
    
      case class MultiItem(service: List[Item])
      implicit val MultiItem RW = upickle.default.macroRW[MultiItem]
      case class OneItem(service: Item)
      implicit val OneItemRW = upickle.default.macroRW[OneItem]
    
      case class Content(services: Either[MultiItem, OneItem], title: String)
      implicit val ContentRW = upickle.default.macroRW[Content]
    And still get [error] upickle.core.Abort: expected sequence got dictionary
    If this library can't handle simply valid JSON, I don't see its interest ??
    Piotr Buszka
    @pbuszka
    @lihaoyi thx for the hint, looks like a viable path for my problem.
    Simon Parten
    @Quafadas
    @jeanjerome I had this the other day, the library can do it, but it sometimes get's it's error messaging wrong was my conclusion. Have you tried adding types to the implicit RW and making sure they are in companion objects?
    Jean-Jerome
    @jeanjerome
    @Quafadas Thanks for your response. I finally tried traversing JSON as explain in https://www.lihaoyi.com/post/HowtoworkwithJSONinScala.html#traversing-json
    Siddhant Sanyam
    @siddhant3s
    case class Foo(bar: String) { 
      val baz = "inner"
      def dbaz = "definner"
    }
    Is there any away to support getting baz and dbaz serialized along with Foo(bar)?
    Li Haoyi
    @lihaoyi
    you'd have to write a custom serializer using upickle.readwriter[ujson.Value].bimap[Foo](..., ...)
    Siddhant Sanyam
    @siddhant3s
    Okay. Thanks I'll give it a try.
    MrMuppet-cut
    @MrMuppet-cut
    Hi! I am trying to read a json string using ujson:
    This is the string:
    MrMuppet-cut
    @MrMuppet-cut
    {
      "dataflows": [
        {
          "name": "test",
          "sources": [
           {
              "name": "person_inputs",
              "path": "/data/input/events/person/*",
              "format": "JSON"
            }
          ],
          "transformations": [
            {
              "name": "validation",
              "type": "validate_fields",
              "params": {
                "input": "person_inputs",
                "validations": [
                  {
                    "field": "office",
                    "validations": [
                     "notEmpty"
                    ]
                  },
                  {
                    "field": "age",
                    "validations": [
                      "notNull"
                    ]
                  }
                ]
              }
            },
            {
              "name": "ok_with_date",
              "type": "add_fields",
              "params": {
                "input": "validation_ok",
                "addFields": [
                  {
                    "name": "dt",
                    "function": "current_timestamp"
    
                  }
                ]
              }
            }
          ],
          "sinks": [
            {
              "input": "ok_with_date",
              "name": "raw-ok",
              "paths": [
                "/data/output/events/person"
              ],
              "format": "JSON",
              "saveMode": "OVERWRITE"
            },
            {
              "input": "validation_ko",
              "name": "raw-ko",
              "paths": [
                "/data/output/discards/person"
              ],
              "format": "JSON",
              "saveMode": "OVERWRITE"
            }
          ]
        }
      ]
    }
    which is in a file called "metadata.json"
    and I read just by doing:
    val j = os.read(os.pwd/RelPath("src/main/scala/metadata.json"))
    val jsonData = ujson.read(j)
    however, when I try
    jsonData(0)
    I get
    ujson.Value$InvalidData: Expected ujson.Arr (data: {"dataflows":[{"name":"test","sources":[{"name":"person_inputs","path":"/data/input/events/person/*","format":"JSON"}],"transformations":[{"name":"validation","type":"validate_fields","params":{"input":"person_inputs","validations":[{"field":"office","validations":["notEmpty"]},{"field":"age","validations":["notNull"]}]}},{"name":"ok_with_date","type":"add_fields","params":{"input":"validation_ok","addFields":[{"name":"dt","function":"current_timestamp"}]}}],"sinks":[{"input":"ok_with_date","name":"raw-ok","paths":["/data/output/events/person"],"format":"JSON","saveMode":"OVERWRITE"},{"input":"validation_ko","name":"raw-ko","paths":["/data/output/discards/person"],"format":"JSON","saveMode":"OVERWRITE"}]}]})
    MrMuppet-cut
    @MrMuppet-cut
    Any idea why this happens?
    Any help would be greatly appreciated. Thanks!!
    Cheng Wang
    @polarker
    Hi, is there a way to fail on excess JSON fields ? This feature is useful when working with optional fields
    Doug Roper
    @htmldoug
    Not easily. The most straightforward way probably requires modifying the macro.
    Next best would be to either convert to Value and count the fields or write a Visitor.Delegate to wrap the Reader and count them that way.
    Igor Romanov
    @pharod

    I have in my case class:

    @key("stage_name") stageName: Option[String] = None,

    and when this property is missing from JSON I am getting an error message:

    Caused by: upickle.core.AbortException: missing keys in dictionary: stage_name, state_metadata at index ...

    What is wrong?

    Igor Romanov
    @pharod
    @lihaoyi It seems it works ok only when I'm not using @key annotation for mapping. Looks like a bug in uPickle or a limitation?
    Li Haoyi
    @lihaoyi
    huh seems like a bug
    can you minimize a repro and open a ticket
    Igor Romanov
    @pharod
    @lihaoyi I've filed issue com-lihaoyi/upickle#360
    objektwerks
    @objektwerks
    Using Scala 2.13.6 and uPickle 1.4.0, I assume everyone is familiar with this macroRW error:
    match may not be exhaustive. It would fail on the following input: (x: Int forSome x not in 0)
    Currently I use an @nowarn annotation to ignore all such errors. Is there an expected fix to this in a forthcoming version of Scala and/or uPickle? Or is this with us for a long time to come?