Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Nov 15 17:01
    scala-steward opened #143
  • Nov 03 15:26
    scala-steward opened #142
  • Oct 15 11:25

    mathieuleclaire on master

    Update sbt-scalajs, scalajs-com… Merge pull request #141 from sc… (compare)

  • Oct 15 11:25
    mathieuleclaire closed #141
  • Oct 15 11:15
    scala-steward opened #141
  • Sep 28 18:17
    ochrons closed #140
  • Sep 28 18:17
    ochrons commented #140
  • Sep 22 13:24
    manosntoulias edited #140
  • Sep 22 13:24
    manosntoulias opened #140
  • Sep 08 23:55
    scala-steward opened #139
  • Aug 30 06:44
    scala-steward opened #138
  • Jul 02 02:00
    scala-steward opened #137
  • Jun 28 01:08
    scala-steward opened #136
  • Jun 26 01:22
    phb opened #135
  • Jun 09 00:48
    japgolly commented #127
  • Jun 08 20:58
    scala-steward opened #134
  • Jun 08 20:58
    scala-steward opened #133
  • Jun 08 20:58
    scala-steward opened #132
  • Jun 08 20:58
    scala-steward opened #131
  • Jun 08 20:58
    scala-steward opened #130
Dermot Haughey
@hderms
we are trying to serialize like 200MB of data
Andriy Plokhotnyuk
@plokhotnyuk
it depends a lot on payload and environment... as an example jsoniter-scala can serialize with speed up to 3Gb/sec... please see the full picture here
Dermot Haughey
@hderms
@plokhotnyuk thanks I'll give it a look
Otto Chrons
@ochrons
for text, Boopickle is not faster, but for something like numbers, especially floating point, it can be much much faster than any JSON codec
Boopickle comes with a performance test suite that you can easily modify to match your data and see the differences
Nils Kilden-Pedersen
@nilskp
@ochrons I'm surprised you write that, as JSON text needs to be escaped, unlike (presumably) Boopickle. At the very least that would lead to per-char checking, which might be cheap, but not free, and any object field will have a String name, which doesn't even exist in a binary codec.
Am I missing something?
Otto Chrons
@ochrons
not sure what you're asking :)
Nils Kilden-Pedersen
@nilskp
@ochrons I'm confused as to why you'd write that "for text, Boopickle is not faster".
It should be faster, since JSON de/serialization requires escaping, which Boopickle, being binary, presumably does not.
Otto Chrons
@ochrons
maybe on JVM the difference is not that big, but on JS the platform has highly optimized JSON.stringify function which handles strings quite much faster than doing UTF8ish encoding in JS code
Nils Kilden-Pedersen
@nilskp
So what you're saying is, that when serializing to a JS environment, Boopickle is not faster for text, because the Boopickle's JS implementation of String encoding is slower than JSON.stringify?
Otto Chrons
@ochrons
yeah, and I'm not too sure about JVM perf either, would need to be benchmarked
at least on JVM the string encoding perf is on the same level with Google protobuf :)
Nils Kilden-Pedersen
@nilskp
my point was re. JVM <-> JVM communication, and I would expect Boopickle to be faster than any JVM <-> JVM JSON implementation, because no escaping is needed.
Otto Chrons
@ochrons
could be faster, although typically not much needs to be escaped in JSON strings
Nils Kilden-Pedersen
@nilskp
Correct, but every single char still needs to be range/char checked.
might be fast, but cannot be free.
Otto Chrons
@ochrons
true, the perf statements are mostly taken in the JS context, as that was the driving factor when I was developing Boopickle
Nils Kilden-Pedersen
@nilskp
gotcha. Thanks for clarifying.
Otto Chrons
@ochrons
:+1:
moritz bust
@busti

I am trying to pickle a custom collection type using a transformPickler.
The collection is a strictly alternating sequence of two types and is implemented like this: class Alternating[Even, Odd](val first: Even, val rest: Seq[(Odd, Even)])
My Pickling code looks like this:

  implicit def alternatingPickler[Even, Odd] =
    transformPickler[Alternating[Even, Odd], (Even, Seq[(Odd, Even)])] {
      case (first, rest) => Alternating(first, rest)
    } (a => (a.first, a.rest))

I am getting an error message when compiling:

[error] C:\Users\mbust\Documents\projects\delta-t\modules\core\shared\src\main\scala\io\delta_t\collection\serializers\Boopickle.scala:14:7: exception during macro expansion:
[error] java.lang.RuntimeException: Enclosure: source-C:\Users\mbust\Documents\projects\delta-t\modules\core\shared\src\main\scala\io\delta_t\collection\serializers\Boopickle.scala,line-14,offset=432, type = Even
[error]         at boopickle.PicklerMaterializersImpl$.materializePickler(PicklerMaterializersImpl.scala:106)
[error]     } (a => (a.first, a.rest))
[error]       ^
Does transformPickler support generics? Or is there something else I am doing wrong?
Otto Chrons
@ochrons
you need to provide implicit picklers for Even and Odd
moritz bust
@busti
Like this implicit def alternatingPickler[Even : Pickler, Odd : Pickler]?
Otto Chrons
@ochrons
yea
moritz bust
@busti
Neat, thanks :)
moritz bust
@busti
The interface of transformPickler compared to compositePickler.addTransform is a bit odd.
In the former it is (transformFrom)(transformTo) and in the latter it is (transformTo, transformFrom).
Is there a reason for that?
Otto Chrons
@ochrons
the curried version allows using the types from the first argument in the second argument
with addTransform that is not needed IIRC
so often you don't need to specify the types manually, like in the example
implicit val datePickler = transformPickler((t: Long) => new java.util.Date(t))(_.getTime)
moritz bust
@busti
@ochrons Thanks :)
One more thing...
Can I serialize partially applied functions?
case class FooModel(number: Int, text: String)

def setNumber(to: Int)(state: FooModel): FooModel =
  state.copy(number = to)

val foo = setNumber(5) // is it possible to serialize this ?
Otto Chrons
@ochrons
sure, if you provide a Pickler ;)
but in reality, probably not
because your pickler has no way of figuring out what arguments have already been supplied
you would have to describe the arguments some other way, like with a case class
or by building a chain of classes with apply functions returning the next class
moritz bust
@busti
Yeah, I am using case classes right now to do this, but it makes things really verbose.
@ochrons Thanks again :)
moritz bust
@busti
To give a bit of context, my environment is very similar to DIODE, but I have a hard requirement on being able to serialize actions. The idea is to have actions defined as def (a: Int, b: String) = Action { state => state.copy(foo = a, bar = b) }.
I guess that I could make this work with a macro, which generates a serialisation method and adds it to the Action object.
Otto Chrons
@ochrons
use Free Monad :wink:
moritz bust
@busti
@ochrons I have read the docs of cats Free[_] for the past hour or so.
I have looked at them before, but I am not quite sure what you intend. Is your suggestion to implement Actions as an algebra and have two separate compilers, one for serialization and one for state mutation?
Otto Chrons
@ochrons
well, Free is able to capture the Actions and then you can serialize it easily
since it's just a bunch of case classes
Rok Kralj
@strelec
I have two questions:
  1. Have there been any changes that could affect the binary format since 1.3.0?
  1. Do we consider scala.js 1.0.0 stable enough to push a boopickle release for it?
Dennis
@dennis4b_twitter
Hi! Is there a clever way to find out which types are automatically derived the most times (ie duplications that would benefit from explicitly creating the pickler in a central place so that its macro is generated only once). Because I am using sloth for client-server communications I need to use auto-derivation as the sloth macros generate case classes that I could otherwise not serialize.
Dennis
@dennis4b_twitter
aha its in the docs, sorry, thought I did read everything .. great!
Otto Chrons
@ochrons
should be no obstacles publishing Boopickle for 1.0.0
Tushar Mathur
@tusharmath
case class Point(x: Int, y: Int)

val p = Point(5, 10)
val points = Vector(p, p, p, p)
val bb = Pickle.intoBytes(points)
.
.
val newPoints = Unpickle[Vector[Point]].fromBytes(bb)
this example from gitbook doesn't work