Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Jan 26 21:07
    scala-steward synchronize #139
  • Jan 26 14:33

    mpilquist on v1.11.0

    (compare)

  • Jan 26 14:32

    mpilquist on 1.11.x

    Setting version to 1.11.0 Updated version to 1.11.1-SNAPS… (compare)

  • Jan 25 10:28
    billpcs starred scodec/scodec
  • Jan 22 22:01
    benhanna starred scodec/scodec
  • Jan 21 20:36
    scala-steward opened #139
  • Jan 21 14:51
    mpilquist closed #130
  • Jan 21 14:48
    mpilquist closed #119
  • Jan 21 14:48
    mpilquist commented #119
  • Jan 21 14:48

    mpilquist on 1.11.x

    Scala 2.13.0-M4 Merge branch 'series/1.10.x' in… Upgraded to latest scala-collec… and 2 more (compare)

  • Jan 21 14:48
    mpilquist closed #138
  • Jan 21 14:40
    mpilquist opened #138
  • Jan 21 14:37

    mpilquist on 1.11.x

    Created 1.11 branch (compare)

  • Jan 21 14:31

    mpilquist on xuwei-k

    Upgraded to 2.13.0-M5 (compare)

  • Jan 21 14:19

    mpilquist on xuwei-k

    Scala 2.13.0-M4 Merge branch 'series/1.10.x' in… Upgraded to latest scala-collec… (compare)

  • Jan 21 14:03

    mpilquist on 1.10.x

    Update sbt-scalajs, scalajs-com… Merge pull request #129 from sc… (compare)

  • Jan 21 14:03
    mpilquist closed #129
  • Jan 21 14:03
    mpilquist closed #126
  • Jan 21 14:03

    mpilquist on 1.10.x

    Pad-left on pbcd codec. Fixes #… Using padLeft in pbcd instead o… Merge pull request #133 from lJ… (compare)

  • Jan 21 14:03
    mpilquist closed #133
Pavel Khamutou
@pkhamutou
Thank you for helping me @lJoublanc !
Unfortunately, I don't have specs, I just have a source code that manually writes/read ByteBuffers which I found not easy to deal with so I decided to use scodec.
It looks like I know at compile time how request/response looks like. that's what i have right now https://github.com/pkhamutou/ACISpark/blob/add-read-message/modules/core/src/main/scala/alchemist/net/codecs/package.scala#L16
ruslan
@unoexperto

@mpilquist Could you please suggest what I could be missing in coproduct codec ? Following code compiles in Scala 2.12.8

  implicit val AffiliationCodec: Codec[Affiliation] = {
    implicit val AffiliationParsedDiscriminator: Discriminator[Affiliation, AffiliationParsed, Int] = Discriminator(1)
    implicit val AffiliationUnparsedDiscriminator: Discriminator[Affiliation, AffiliationUnparsed, Int] = Discriminator(2)
    Codec.coproduct[Affiliation].discriminatedBy(uint8).auto
  }

where Affiliation is trait and AffiliationParsed and AffiliationUnparsed are case classes.

But in Scala 2.13.0 I get following compilation error

Error:(111, 20) could not find implicit value for parameter auto: scodec.codecs.CoproductBuilderAuto[co.kernelnetworks.casejournals.model.source.Affiliation]
    Codec.coproduct[Affiliation].discriminatedBy(uint8).auto
Error:(111, 20) not enough arguments for method coproduct: (implicit auto: scodec.codecs.CoproductBuilderAuto[co.kernelnetworks.casejournals.model.source.Affiliation])auto.Out.
Unspecified value parameter auto.
    Codec.coproduct[Affiliation].discriminatedBy(uint8).auto
ruslan
@unoexperto

Well, I found culprit. It's two things
1) use of Seq in my case classes
2) use of java enum of my case classes

Neither had explicit Codec[] for them. How come it was working on 2.12 but stopped in 2.13 ?

Luciano
@lJoublanc
Hmm I know that Seq changed from mutable.Seq to immutable.Seq between those versions @unoexperto
ruslan
@unoexperto
@lJoublanc Yep, that's what I thought too but I tried using both and it doesn't have Codec for neither apparently.
ruslan
@unoexperto
@lJoublanc Also would it be bad style if I encode java enums like this ?
implicit def forJavaEnum[T <: java.lang.Enum[T]](implicit ctag: reflect.ClassTag[T]): Codec[T] = {
  val clazz: Class[T] = ctag.runtimeClass.asInstanceOf[Class[T]]
  cstring.xmap(str ⇒ java.lang.Enum.valueOf(clazz, str), _.name())
}
Luciano
@lJoublanc
I think you want to return a T.Value
Not the best example, but have a look here
@unoexperto
Bastien Teinturier
@t-bast

Hey guys,
I was wondering if it's somehow possible when using a DiscriminatorCodec[A, B] to have access to the discriminator (B) during encoding and decoding.
My usecase is that I want to encode a list of tag-length-value records that doesn't allow duplicate tags and enforces ordering by tags.
I'm currently using the following codec (simplified for demo purposes) combined with the built-in list codec:

discriminated[MyTrait].by(uint8)
    .typecase(1, myType1Codec)
    .typecase(2, myType2Codec)
    .typecase(3, myType3Codec)

I'm combining this with exmap to enforce the non-duplicate tags and strict ordering, but that currently requires a bit of wasteful code to extract the tag value from the records (I'm currently forced to re-encode each record to read the first byte and interpret it as the tag value, which is wasteful).
Is there another way to get the tag value from a discriminated record or am I out of luck?

Luciano
@lJoublanc
@t-bast could you just wrap your myTypeNCodecs with fixedSizeBytes combinator, since you know the tag, and therefore the length?
Bastien Teinturier
@t-bast
@lJoublanc I don't understand how that would solve my problem, could you please detail?
My issue is that I would like to somehow have access to the mapping Type <-> Tag when using a DiscriminatorCodec[A, B], whereas it seems that scodec hides that internal information from the application layer.
If someone is interested in the exact code, here it is: https://github.com/ACINQ/eclair/blob/c1a7b4fe509dd6ec0e02d1342ab50ccf346064ab/eclair-core/src/main/scala/fr/acinq/eclair/wire/TlvCodecs.scala#L99
In that specific codec I'd like to get rid of my tag function and instead extract that from the DiscriminatorCodec itself.
Luciano
@lJoublanc

Hi, sorry been travelling @t-bast . I just meant that if you say

My usecase is that I want to encode a list of tag-length-value records ...

Does that mean that the tag == length of the record? in this case, you could just do e.g. discrimnated[MyTrait].by(uint8).typecase(1,fixedSizeBytes(1, myType1Codec))

If that is not what you mean, but instead 'concatenation of tag, size and value', then look at the scaladoc for DiscriminatorCodec:

Often, the values are size-delimited -- that is, there is a size field after the tag field and before the value field. To support this, use the framing method to provide a transformation to each value codec. For example, framing(new CodecTransformation { def apply[X](c: Codec[X]) = variableSizeBytes(uint8, c) }).

Bastien Teinturier
@t-bast

Thanks for your answer, I think my question is hard to formulate clearly :)
My issue is not with implementing a tag-size-value codec, that part was easy to do.
My issue is generally with DiscriminatorCodec: I'd like to understand if I can somehow enrich my scala type with the discriminator used.
Let's imagine that I have the following codec:

case class MyType1(something: Long) extends MyTrait
val myType1Codec: Codec[MyType1] = ("something" | int64).as[MyType1]
val myTypesCodec = discriminated[MyTrait].by(uint8)
    .typecase(1, myType1Codec)
    .typecase(2, myType2Codec)
    .typecase(3, myType3Codec)

Is there a way for MyType1 to know that it is associated with the value 1 at encoding/decoding?
Or is there a way when using myTypesCodec inside another codec to get 1 when what I'm encoding/decoding is a MyType1?
Let me know if that's still not very clear, it's not an easy issue :)

Luciano
@lJoublanc
I think I know what you're saying. I don't think there is a way to do this in the API.
However, you can create an intermediate data structure. I've done this in several projects. I create a data structure with an uninhabited type, and that is the tag. It works nicely with simple tags and literal types in scala 2.13.
Look for example here. PartialCodec[A, W] has the value as A, and the tag as W.
This isn't part of scodec per-se. Btw my laptop battery is dying, so won't be able to answer til later :(
Bastien Teinturier
@t-bast
Oh that's interesting, I think this is a good direction to explore. Thanks for this, I'll try it out!
Tristan Lohman
@gatorcse
In scodec-stream, the scaladoc for tryOnce mentions repeat, which seems like a way repeatedly run a codec until it fails, and allow continuation. I can’t seem to find repeat anywhere though, where does this come from?
Tristan Lohman
@gatorcse
nm, I saw that with the cats interop we can use untilM, nice!
Joesan
@joesan
Guys, I came across this awesome scodec library just today as I'm in the process of implementing a parser for a binary protocol (OPC UA Pub / Sub) which is a pretty hefty protocol and deals with lots of rules on how to parse the data... Until now I just wrote a version of the parser that can do this, but I guess it is not as functional as I wanted it to be. So I'm thinking to rewrite it with scodec, but not sure how to get started.
Here for example., is the sample out of the binary data that I get: 145 132 8 45
where the first byte 145 means the following:
Bit 0-3 - Protocol version
Bit 4 - Boolean
Bit 5 - Boolean
Bit 6 - Boolean
Bit 7 - Boolean
And the second byte 132 means the following:
Bit range 0-2 - if 000 means a certain value, 001 means a certain value
Bit 3, Bit 4, Bit 5, Bit 6, Bit 7 - Boolean
It goes on like this... Any pointers on how to get started? The examples I see talk about dealing with bytes but not sure if I came across anything that looks into the Bits in a Byte
Joesan
@joesan
I ended up doing it imperatively using scodec as below:
// ********** ********** Byte 0 ********** ********** //
val byte0AsBits = BitVector(byteVector.head)
// Bit range 0-3: Version of the UADP NetworkMessage
val uadpVersion = byte0AsBits.takeRight(3).toInt(signed = false)
// Bit range 4-7 contains the flags
val publisherIdEnabled = byte0AsBits(4)
val groupHeaderEnabled = byte0AsBits(5)
val payloadHeaderEnabled = byte0AsBits(6)
val extendedFlags1Enabled = byte0AsBits(7)
Can I make this better using HLists?
Michael Pilquist
@mpilquist
@joesan You could do something like this instead:
case class Message(version: Int, publisherIdEnabled: Boolean, groupHeaderEnabled: Boolean, payloadHeaderEnabled: Boolean, extendedFlags1Enabled: Boolean)
val codec = (uint4 :: bool :: bool :: bool :: bool).as[Message]
Joesan
@joesan
@mpilquist Thanks for the suggestion! Which modules do I need? I currently have only the scodec-bits pulled in...
Michael Pilquist
@mpilquist
Ah, that explains it then — grab scodec-core from https://github.com/scodec/scodec
Nima Eskandary
@nimaeskandary
Hi all, how are classes encoded when attributes are optional? In my use case I may have several class attributes that would be represented as uints in the middle of my bitstring if they exist, otherwise the bitstring would be that much shorter if they do not exist
Luciano
@lJoublanc
There are several combinators that return Codec[Option[_]] in scodec.codecs._. That's a starting point. Check the API doc.
Nima Eskandary
@nimaeskandary
Seeing them now, thanks @lJoublanc
Luciano
@lJoublanc
encoding a scala.Enum I have this, which works:
  def enum[E <: Enumeration](
      implicit E: ValueOf[E]
  ): Codec[E.value.Value] =
      int32
        .emap(i => scodec.Attempt fromTry scala.util.Try(E.value(i)))
        .contramap[E.value.Value](_.id)
        .fuse
But I can't figure out how to derive this implicitly.
The problem is that I don't know how to infer the parent/super class from the Value.
I've tried this but it doesn't work (doesn't compile)
   implicit def implicitEnum[
      V <: Enumeration#Value,
      E <: Enumeration { type Value = V }
    ](
        implicit E: ValueOf[E],
    ): Codec[V] = 
          int32.xmap(i => E.value(i).asInstanceOf[V], _.id)
This compiles, but I don't have a way of lifting the int into an Enum, without a witness:
   implicit def implicitEnum[V <: Enumeration#Value]: Codec[V] = 
      Codec[V](int32.xmap(???,_.id))
Luciano
@lJoublanc
@mpilquist you ever tried to do something like this?
Mateusz Błażejewski
@mblaze
@mpilquist What is the state of scodec-stream? Is it possible to help get the 2.0.0 released? I started writing an SSH client using fs2 and kind of reinvented what you already did in scodec-stream but with StateT. Now I want to use your library but I see it is stuck between releases. I wanted to use the new version but its not published yet (I'm assuming it is not finished).