Hi! :wave:
Does anyone want to review this PR I opened to fix the CI (it wasn't running the tests): com-lihaoyi/upickle#359 ?
Thanks!
Then we can trigger a release @htmldoug @objektwerks
upickle to extract my case classs schema. The visitor that uPickle provide is great. I would like to have a visitor on the type if you know what I mean:- I have user defined
case class Foo(..) - I want a JSON deserializer for that class. That's easy, I can just use
macroRW - I also want to codegen Foo into a Python class. For that, it would be nice if I could re-use any of uPickle's machinery to have a Visitor on the schema of Foo. That way I'll be able to generate the Python code string easily
@lihaoyi I've been going through your wonderful tutorial on Visitor pattern yet again. I want to modify it slightly to include a cache based early stop for recursive Data structure. Think of an ADT which is self referencing. With the pattern in the tutorial, there is no way for visitor to communicate back to dispatcher to stop going further.
So I came up with this approach, could you please comment on what you think about it:
trait Visitor[T] {
def visitPrimitive(tpe: Type): T
def visitOption(innerVisited: T): T
def visitArrayType(innerVisited: T): T
def visitMapType(keyVisited: T, valueVisited: T): T
def visitStructType(tpe: Type): StructVisitor[T]
def stopValue: Option[T] = None // If defined, we stop visiting and simply return this value
}
trait StructVisitor[T] {
def visitFieldKey(name: String): Visitor[T]
def visitFiledType(name: String, visitedValue: T): Unit
def done: T
def stopValue: Option[T] = None // If defined, we stop visiting and simply return this value
}This is a visitor to visit Type from scala.reflect
java.lang.IncompatibleClassChangeError: class ilsop.models.RiskDecompositionSearchResult$$anon$10$$anon$11 has interface upickle.core.Types$CaseR$CaseObjectContext as super class
1 reply
I was looking at one of the open bugs for com-lihaoyi/upickle#356 and think I'm struggling a bit to figure out exactly where my understanding is breaking down. I've never used macros before, so that might be part of it.
The way it appears to me that things are happening right now is that the Sum types for the enumeration are being derived correctly when we go to write the values, but the implicit classtag that is carried through is the classtag for the enumeration supertype.
package upickle
import scala.language.implicitConversions
import utest._
import upickle.default._
object EnumTests extends TestSuite {
enum SimpleEnum derives ReadWriter:
case A, B
val tests = Tests {
test("simple") {
test("enum write") {
val parsed = write(SimpleEnum.B)
val expected = """{"$type":"upickle.EnumTests.SimpleEnum.B"}"""
assert(parsed == expected)
}
}
}
}I added some printlns to the code temporarily, but see the line beggining with Instance Check
----------------------- Running Tests upickle.EnumTests -----------------------
>>>>> SUM READER TRAVERSING [upickle.EnumTests$SimpleEnum$@1b73be9f]
>>>>> PRODUCT READER []
>>>>> PRODUCT READER []
>>>>> SUM WRITER TRAVERSING [upickle.EnumTests$SimpleEnum$@1b73be9f]
CASE CLASS WRITER defaults[]
>>>>> PRODUCT WRITER FULL CLASS NAME: upickle.EnumTests.SimpleEnum.A
CASE CLASS WRITER defaults[]
>>>>> PRODUCT WRITER FULL CLASS NAME: upickle.EnumTests.SimpleEnum.B
Instance Check upickle.EnumTests.SimpleEnum.A [B <: upickle.EnumTests$SimpleEnum]: true
RETURNING (upickle.EnumTests.SimpleEnum.A,upickle.implicits.CaseClassWriterPiece$CaseClassWriter@42721fe) for input B
>>>>> PRODUCT WRITER []
>>>>> PRODUCT WRITER []
X upickle.EnumTests.simple.enum write 30ms
utest.AssertionError: parsed == expected
parsed: String = {"$type":"upickle.EnumTests.SimpleEnum.A"}
expected: String = {"$type":"upickle.EnumTests.SimpleEnum.B"}
utest.asserts.Asserts$.assertImpl(Asserts.scala:30)
upickle.EnumTests$.$init$$$anonfun$3$$anonfun$2$$anonfun$2(EnumTests.scala:13)
1 targets failed
upickle.jvm[3.0.2].test.test 1 tests failed:
upickle.EnumTests upickle.EnumTests.simple.enum writeInstance Check line comes from TaggedReader.Leaf.findWriter, which I've modified like this (just enough to get some debugging information):class Leaf[T](c: ClassTag[_], tag: String, r: CaseW[T]) extends TaggedWriter[T]{
def findWriter(v: Any) = {
val isInstance = c.runtimeClass.isInstance(v)
println(s"Instance Check $tag [$v <: ${c.runtimeClass.getName}]: ${isInstance}")
if (isInstance) (tag -> r)
else null
}
}Ah, a simpler answer. The macros are more specific than the classtag. So even though the macro is giving the correct type, the classtag that is implicitly carried is not specific to the enum instance
CLASS NAME: upickle.EnumTests.SimpleEnum.B Classtag: upickle.EnumTests$SimpleEnum
I'm trying to write a custom pickler to handle java.util.Date by converting to/from an ISO8601 date string. What I have is this:
case class JSONDate(date: Date) extends AnyVal
object JSONDate {
implicit def readWrite: ReadWriter[JSONDate] = readwriter[ujson.Value].bimap[JSONDate](
(date: JSONDate) => DateTimeFormatter.ISO_INSTANT.format(date.date.toInstant),
(json: ujson.Value) => JSONDate(Date.from(OffsetDateTime.parse(json.str).toInstant))
)
}But whilst this works:
val d: JSONDate = JSONDate(comp.getTimeCreated)
write(d)This doesn't:
write(ujson.Obj(
"Name" -> "myName",
"Created" -> d
))For some reason the implicit isn't being found in the ujson.Obj case. Any suggestions?
The release notes for 0.5.1 say:
ujson.write now takes an optional `sortKeys` flag, if you want the JSON dictionaries to rendered in a standardized orderBut that doesn't seem to exist any more. How do I ensure that Object keys are sorted? This is to make it easier to compare versions of a JSON file in a diff tool...
I see ujson.Obj uses a LinkedHashMap so it will preserve insertion order. Rather than sorting into a new Obj when the contents have all been added I could do a two-step process but it's still a bit clunky as it requires a Map copy:
val sortedMap = mutable.SortedMap[String, ujson.Value]()
addSomeStuffWithUnpredictableKeyOrdering(sortedMap)
val obj = ujson.Obj.from(sortedMap)What would be nice is if Obj could be configured to use either mutable.LinkedHashMap as at present, or mutable.SortedMap - then you could choose to either preserve insertion order, or have ordering by key.
ujson.Obj could hold a reference of a mutable.Map[String, ujson.Value] instead of a mutable.LinkedHashMap[String, ujson.Value].Then the default implementation would continue to use
mutable.LinkedHashMap[String, ujson.Value] and you could pass a different type of map by doing:val myMap = mutable.SortedMap[String, ujson.Value]()
val obj = new ujson.Obj(myMap)@lolgab well, you shouldn't really be depending on map key ordering in the first place I suppose, but you already get that with the current implementation ;-)
I'd settle for being able to specify ordering during write, again I know you shouldn't do that with JSON but it does make it easier to process with existing diff tools...