Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Dec 04 2020 19:11
    scala-steward closed #142
  • Dec 04 2020 19:11
    scala-steward opened #144
  • Nov 15 2020 17:01
    scala-steward opened #143
  • Nov 03 2020 15:26
    scala-steward opened #142
  • Oct 15 2020 11:25

    mathieuleclaire on master

    Update sbt-scalajs, scalajs-com… Merge pull request #141 from sc… (compare)

  • Oct 15 2020 11:25
    mathieuleclaire closed #141
  • Oct 15 2020 11:15
    scala-steward opened #141
  • Sep 28 2020 18:17
    ochrons closed #140
  • Sep 28 2020 18:17
    ochrons commented #140
  • Sep 22 2020 13:24
    manosntoulias edited #140
  • Sep 22 2020 13:24
    manosntoulias opened #140
  • Sep 08 2020 23:55
    scala-steward opened #139
  • Aug 30 2020 06:44
    scala-steward opened #138
  • Jul 02 2020 02:00
    scala-steward opened #137
  • Jun 28 2020 01:08
    scala-steward opened #136
  • Jun 26 2020 01:22
    phb opened #135
  • Jun 09 2020 00:48
    japgolly commented #127
  • Jun 08 2020 20:58
    scala-steward opened #134
  • Jun 08 2020 20:58
    scala-steward opened #133
  • Jun 08 2020 20:58
    scala-steward opened #132
Otto Chrons
@ochrons
yea, it's a long standing issue, but not something that can be easily resolved
sometimes you just need to supply the pickler manually
Nils Kilden-Pedersen
@nilskp
Is it also an issue with TreeMap, due to the Ordering going along? It cannot use a simple collection.Map pickler for that, as the ordering would be lost.
Otto Chrons
@ochrons
could be, can you make a scalafiddle.io sample of the problem?
jhegedus42
@jhegedus42
can boopickle serialize a Map[String,SomeCaseClass[Any]] ?
Andriy Plokhotnyuk
@plokhotnyuk
Is Any for scala.Any?
jhegedus42
@jhegedus42
hmmm
good question
let's say
more like Map[String,ReferencedValue[_]]
I am not sure what is the difference betwen the two
but basically
actuall
yes
but you know what
i think i just store Map[String,JSON]
and then problem is solved :)
and use circe
Andriy Plokhotnyuk
@plokhotnyuk
Is it required to work with scala.js?
jhegedus42
@jhegedus42
will
well
yeah
ideally :)
Otto Chrons
@ochrons
same as all other serialization libraries, it needs to know the types at compile time and you need to provide picklers for them
Dermot Haughey
@hderms
anyone have any idea how much faster boopickle would be than circe?
especially if boopickle was using binary codecs
we are trying to serialize like 200MB of data
Andriy Plokhotnyuk
@plokhotnyuk
it depends a lot on payload and environment... as an example jsoniter-scala can serialize with speed up to 3Gb/sec... please see the full picture here
Dermot Haughey
@hderms
@plokhotnyuk thanks I'll give it a look
Otto Chrons
@ochrons
for text, Boopickle is not faster, but for something like numbers, especially floating point, it can be much much faster than any JSON codec
Boopickle comes with a performance test suite that you can easily modify to match your data and see the differences
Nils Kilden-Pedersen
@nilskp
@ochrons I'm surprised you write that, as JSON text needs to be escaped, unlike (presumably) Boopickle. At the very least that would lead to per-char checking, which might be cheap, but not free, and any object field will have a String name, which doesn't even exist in a binary codec.
Am I missing something?
Otto Chrons
@ochrons
not sure what you're asking :)
Nils Kilden-Pedersen
@nilskp
@ochrons I'm confused as to why you'd write that "for text, Boopickle is not faster".
It should be faster, since JSON de/serialization requires escaping, which Boopickle, being binary, presumably does not.
Otto Chrons
@ochrons
maybe on JVM the difference is not that big, but on JS the platform has highly optimized JSON.stringify function which handles strings quite much faster than doing UTF8ish encoding in JS code
Nils Kilden-Pedersen
@nilskp
So what you're saying is, that when serializing to a JS environment, Boopickle is not faster for text, because the Boopickle's JS implementation of String encoding is slower than JSON.stringify?
Otto Chrons
@ochrons
yeah, and I'm not too sure about JVM perf either, would need to be benchmarked
at least on JVM the string encoding perf is on the same level with Google protobuf :)
Nils Kilden-Pedersen
@nilskp
my point was re. JVM <-> JVM communication, and I would expect Boopickle to be faster than any JVM <-> JVM JSON implementation, because no escaping is needed.
Otto Chrons
@ochrons
could be faster, although typically not much needs to be escaped in JSON strings
Nils Kilden-Pedersen
@nilskp
Correct, but every single char still needs to be range/char checked.
might be fast, but cannot be free.
Otto Chrons
@ochrons
true, the perf statements are mostly taken in the JS context, as that was the driving factor when I was developing Boopickle
Nils Kilden-Pedersen
@nilskp
gotcha. Thanks for clarifying.
Otto Chrons
@ochrons
:+1:
moritz bust
@busti

I am trying to pickle a custom collection type using a transformPickler.
The collection is a strictly alternating sequence of two types and is implemented like this: class Alternating[Even, Odd](val first: Even, val rest: Seq[(Odd, Even)])
My Pickling code looks like this:

  implicit def alternatingPickler[Even, Odd] =
    transformPickler[Alternating[Even, Odd], (Even, Seq[(Odd, Even)])] {
      case (first, rest) => Alternating(first, rest)
    } (a => (a.first, a.rest))

I am getting an error message when compiling:

[error] C:\Users\mbust\Documents\projects\delta-t\modules\core\shared\src\main\scala\io\delta_t\collection\serializers\Boopickle.scala:14:7: exception during macro expansion:
[error] java.lang.RuntimeException: Enclosure: source-C:\Users\mbust\Documents\projects\delta-t\modules\core\shared\src\main\scala\io\delta_t\collection\serializers\Boopickle.scala,line-14,offset=432, type = Even
[error]         at boopickle.PicklerMaterializersImpl$.materializePickler(PicklerMaterializersImpl.scala:106)
[error]     } (a => (a.first, a.rest))
[error]       ^
Does transformPickler support generics? Or is there something else I am doing wrong?
Otto Chrons
@ochrons
you need to provide implicit picklers for Even and Odd
moritz bust
@busti
Like this implicit def alternatingPickler[Even : Pickler, Odd : Pickler]?