Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Sakib Hadžiavdić
    @sake92
    is it possible to skip a key when writing?
    for example when it's value is None
    Doron Somech
    @somdoron
    @sake92 yes, by using a default value, upickle will drop the field if the value eqauls the default value.
    Sakib Hadžiavdić
    @sake92
    @somdoron IDK why libraries do that
    it makes no sense at all
    Thomas Droxler
    @tdroxler

    hello, I'm hitting an issue on CI (github action) when testing on Windows, due to the infamous \r thing. Here's below a small example I did and the error I got on CI:

    val raw = s"""
      {
        "test" : "yop",
        "hola" : 1
      }"""
    val json = read[ujson.Value](raw)
    ujson.ParseException: expected json value got "\r" at index 0

    Did anyone already hit that issue?

    Thomas Droxler
    @tdroxler
    I could do the following dirty hack in my custom upickle object https://github.com/alephium/alephium/blob/master/json/src/main/scala/org/alephium/json/Json.scala#L29-L39 but this bug sounds like something that could be tackled in upickle's core no?
    jodersky
    @jodersky
    Does this also happen outside of the CI environment? I can't remember the details, but I've run into strange locale issues in some CI environments in the past (I'm not saying that this isn't a bug in upickle, but it's good to narrow down potential causes).
    Piotr Buszka
    @pbuszka
    is there a way to have two $type keys map to the same case class in sealed trait serialization. Use case is migration of $type keys which contain package name and when package name changes it breaks existing serialization which is already in the DB. The problem is to migrate Foo, A, B to a new package barpkg but be able to still deserialize A with both $type = "foopkg.A" and $type = "barpkg.A", serialization should go always be with $type = "barpkg.A"
    package foopkg
    
    object Foo {
      implicit val rw = macroRW[Foo]
    }
    sealed trait Foo
    case class A(a: Int) extends Foo
    case class B(b: int) extends Foo
    Li Haoyi
    @lihaoyi
    I'm not sure we support that, but worst come to worst you can use upickle.default.readwriter[ujson.Value].bimap[Foo] to write this serializer/deserializer manually
    Jean-Jerome
    @jeanjerome
    Hi! I'm dealing with upickle.core.Abort: expected sequence got dictionarywhen upickle parses a JSON list which has no bracket when it contains one item.
    How to handle that? Anybody managed such case??
    I've tried custom pickers with no luck...
    Jean-Jerome
    @jeanjerome
    I've tried the simplest to handle valid JSON list with one item only (that is a dictionary not a list):
      case class Item(prefix: String, name: String, href: String)
      implicit val ItemRW = upickle.default.macroRW[Item]
    
      case class MultiItem(service: List[Item])
      implicit val MultiItem RW = upickle.default.macroRW[MultiItem]
      case class OneItem(service: Item)
      implicit val OneItemRW = upickle.default.macroRW[OneItem]
    
      case class Content(services: Either[MultiItem, OneItem], title: String)
      implicit val ContentRW = upickle.default.macroRW[Content]
    And still get [error] upickle.core.Abort: expected sequence got dictionary
    If this library can't handle simply valid JSON, I don't see its interest ??
    Piotr Buszka
    @pbuszka
    @lihaoyi thx for the hint, looks like a viable path for my problem.
    Simon Parten
    @Quafadas
    @jeanjerome I had this the other day, the library can do it, but it sometimes get's it's error messaging wrong was my conclusion. Have you tried adding types to the implicit RW and making sure they are in companion objects?
    Jean-Jerome
    @jeanjerome
    @Quafadas Thanks for your response. I finally tried traversing JSON as explain in https://www.lihaoyi.com/post/HowtoworkwithJSONinScala.html#traversing-json
    Siddhant Sanyam
    @siddhant3s
    case class Foo(bar: String) { 
      val baz = "inner"
      def dbaz = "definner"
    }
    Is there any away to support getting baz and dbaz serialized along with Foo(bar)?
    Li Haoyi
    @lihaoyi
    you'd have to write a custom serializer using upickle.readwriter[ujson.Value].bimap[Foo](..., ...)
    Siddhant Sanyam
    @siddhant3s
    Okay. Thanks I'll give it a try.
    MrMuppet-cut
    @MrMuppet-cut
    Hi! I am trying to read a json string using ujson:
    This is the string:
    MrMuppet-cut
    @MrMuppet-cut
    {
      "dataflows": [
        {
          "name": "test",
          "sources": [
           {
              "name": "person_inputs",
              "path": "/data/input/events/person/*",
              "format": "JSON"
            }
          ],
          "transformations": [
            {
              "name": "validation",
              "type": "validate_fields",
              "params": {
                "input": "person_inputs",
                "validations": [
                  {
                    "field": "office",
                    "validations": [
                     "notEmpty"
                    ]
                  },
                  {
                    "field": "age",
                    "validations": [
                      "notNull"
                    ]
                  }
                ]
              }
            },
            {
              "name": "ok_with_date",
              "type": "add_fields",
              "params": {
                "input": "validation_ok",
                "addFields": [
                  {
                    "name": "dt",
                    "function": "current_timestamp"
    
                  }
                ]
              }
            }
          ],
          "sinks": [
            {
              "input": "ok_with_date",
              "name": "raw-ok",
              "paths": [
                "/data/output/events/person"
              ],
              "format": "JSON",
              "saveMode": "OVERWRITE"
            },
            {
              "input": "validation_ko",
              "name": "raw-ko",
              "paths": [
                "/data/output/discards/person"
              ],
              "format": "JSON",
              "saveMode": "OVERWRITE"
            }
          ]
        }
      ]
    }
    which is in a file called "metadata.json"
    and I read just by doing:
    val j = os.read(os.pwd/RelPath("src/main/scala/metadata.json"))
    val jsonData = ujson.read(j)
    however, when I try
    jsonData(0)
    I get
    ujson.Value$InvalidData: Expected ujson.Arr (data: {"dataflows":[{"name":"test","sources":[{"name":"person_inputs","path":"/data/input/events/person/*","format":"JSON"}],"transformations":[{"name":"validation","type":"validate_fields","params":{"input":"person_inputs","validations":[{"field":"office","validations":["notEmpty"]},{"field":"age","validations":["notNull"]}]}},{"name":"ok_with_date","type":"add_fields","params":{"input":"validation_ok","addFields":[{"name":"dt","function":"current_timestamp"}]}}],"sinks":[{"input":"ok_with_date","name":"raw-ok","paths":["/data/output/events/person"],"format":"JSON","saveMode":"OVERWRITE"},{"input":"validation_ko","name":"raw-ko","paths":["/data/output/discards/person"],"format":"JSON","saveMode":"OVERWRITE"}]}]})
    MrMuppet-cut
    @MrMuppet-cut
    Any idea why this happens?
    Any help would be greatly appreciated. Thanks!!
    Cheng Wang
    @polarker
    Hi, is there a way to fail on excess JSON fields ? This feature is useful when working with optional fields
    Doug Roper
    @htmldoug
    Not easily. The most straightforward way probably requires modifying the macro.
    Next best would be to either convert to Value and count the fields or write a Visitor.Delegate to wrap the Reader and count them that way.
    Igor Romanov
    @pharod

    I have in my case class:

    @key("stage_name") stageName: Option[String] = None,

    and when this property is missing from JSON I am getting an error message:

    Caused by: upickle.core.AbortException: missing keys in dictionary: stage_name, state_metadata at index ...

    What is wrong?

    Igor Romanov
    @pharod
    @lihaoyi It seems it works ok only when I'm not using @key annotation for mapping. Looks like a bug in uPickle or a limitation?
    Li Haoyi
    @lihaoyi
    huh seems like a bug
    can you minimize a repro and open a ticket
    Igor Romanov
    @pharod
    @lihaoyi I've filed issue com-lihaoyi/upickle#360
    objektwerks
    @objektwerks
    Using Scala 2.13.6 and uPickle 1.4.0, I assume everyone is familiar with this macroRW error:
    match may not be exhaustive. It would fail on the following input: (x: Int forSome x not in 0)
    Currently I use an @nowarn annotation to ignore all such errors. Is there an expected fix to this in a forthcoming version of Scala and/or uPickle? Or is this with us for a long time to come?
    Doug Roper
    @htmldoug
    Should be fixed in https://github.com/com-lihaoyi/upickle/compare/1.4.0...master. Just needs a release.
    objektwerks
    @objektwerks
    @htmldoug Thanks for the update, Doug!
    Lorenzo Gabriele
    @lolgab

    Hi! :wave:
    Does anyone want to review this PR I opened to fix the CI (it wasn't running the tests): com-lihaoyi/upickle#359 ?

    Thanks!

    Then we can trigger a release @htmldoug @objektwerks

    objektwerks
    @objektwerks
    @lolgab Thanks for the update, Lorenzo. Looking forward to the new release.:)
    Lorenzo Gabriele
    @lolgab
    @objektwerks @htmldoug UPickle 1.4.1 is now going to Maven Central: https://github.com/com-lihaoyi/upickle/runs/3514651632