Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • 14:31

    plokhotnyuk on master

    Code clean up (compare)

  • 14:00

    plokhotnyuk on master

    More efficient parsing of doubl… (compare)

  • 11:12

    plokhotnyuk on master

    Clean up of code (compare)

  • 10:16

    plokhotnyuk on master

    Clean up of code (compare)

  • 10:03

    plokhotnyuk on master

    Clean up of code (compare)

  • 09:50

    plokhotnyuk on master

    Clean up of code (compare)

  • 09:17

    plokhotnyuk on master

    Clean up of code (compare)

  • May 15 08:49

    plokhotnyuk on master

    Update jackson-module-scala to … (compare)

  • May 15 08:49
    plokhotnyuk closed #899
  • May 15 07:32
    scala-steward opened #899
  • May 15 03:01

    plokhotnyuk on master

    Update jackson-module-afterburn… (compare)

  • May 15 03:01
    plokhotnyuk closed #898
  • May 15 03:01

    plokhotnyuk on master

    Update jackson-datatype-jdk8 to… (compare)

  • May 15 03:01
    plokhotnyuk closed #897
  • May 14 22:45
    scala-steward opened #898
  • May 14 22:45
    scala-steward opened #897
  • May 14 15:49

    plokhotnyuk on gh-pages

    Update JVM results for v2.13.22 (compare)

  • May 13 14:06

    plokhotnyuk on v2.13.22

    (compare)

  • May 13 14:06

    plokhotnyuk on master

    Setting version to 2.13.22 Setting version to 2.13.23-SNAP… (compare)

  • May 12 18:04

    plokhotnyuk on master

    More efficient parsing of strin… (compare)

Andriy Plokhotnyuk
@plokhotnyuk
@objektwerks could you please provide steps how to setup and run the router test? Probably imports of codecs should be defined more closely for used places or codec params should be passed explicitly.
Objektwerks
@objektwerks
@plokhotnyuk Andriy, if you feel the fromSumTypeCodec method is solid, I'll play around with the router/test imports and the like. My apologies for having taken up so much of your time. FWIW, this all worked fine with Circe and uPickle. Jsoniter just need a bit more work. Thanks, again, for your help!
Andriy Plokhotnyuk
@plokhotnyuk
@objektwerks It may be wrong but I think that using of separated routes for each Entity sub-class is overkill. It looks that routing and storage can be greatly simplified using one entities route which can work polymorphically using the type key values from payload.
Objektwerks
@objektwerks
@plokhotnyuk Yes, no doubt, there's multiple ways to build Akka-Http routes. Paths vs Pattern Matching, for instance. In Scala, there's no wrong way, just many ways. :)
Nils Kilden-Pedersen
@nilskp
Once again I have a parsing failure, but not sure where it's from since stack traces are disabled by default.
First, how do I enable those again? And second, any easy way to enable this wholesale across the entire runtime, or have the default changed?
Looks like it's a very cumbersome default to override, as it's specific all the way to the individual read methods. And not implicit either. So I have to manually reference a ReaderConfig everywhere, manually. I guess that's less than optimal.
Andriy Plokhotnyuk
@plokhotnyuk
@nilskp As a w/a you can define a delegating call with an implicit parameter for the config that have your custom default settings: def read[A](buf: Array[Byte])(implicit codec: JsonValueCodec[A], config: ReaderConfig = CustomReaderConfig) = readFromArray(buf, config)(codec)
Nils Kilden-Pedersen
@nilskp
@plokhotnyuk That should work. Thanks.
b-gyula
@b-gyula
Is there an (implicit) generic transitive codec for collections like in play-json or if I want to generate json for an Array[User] then I need to create a codec implicit val codec: JsonValueCodec[Array[User]] = make regardless if I already have a codec for the User class?
Andriy Plokhotnyuk
@plokhotnyuk
Hi, @b-gyula ! Usually you need to call make only for the top level type of your data structure as described in the README example. All subsequent codecs will be automatically generated and inlined to the top level one. Defining of type class instances for other types of messages by calling make(cfg: CodecMakerConfig) could be needed if the derivation configuration should be different from the default or custom configuration used for the top level type.
Nils Kilden-Pedersen
@nilskp
Could be because arrays are reified?
quanganhtran
@quanganhtran
I can serialize case object extending sealed trait as a string using withDiscriminatorFieldName=None but that string is in PascalCase. How can I make it kebab case?
I tried withFieldNameMapper but I'm still getting PascalCase
JsonCodecMaker.make(
  CodecMakerConfig
    .withDiscriminatorFieldName(None)
    .withFieldNameMapper(JsonCodecMaker.`enforce-kebab-case`)
)
Andriy Plokhotnyuk
@plokhotnyuk
Hi, @quanganhtran ! Have you tried adtLeafClassNameMapper yet?
quanganhtran
@quanganhtran
just now! it works!
.withAdtLeafClassNameMapper(c => JsonCodecMaker.`enforce-kebab-case`(JsonCodecMaker.simpleClassName(c)))
thanks
phlamenco
@phlamenco
Hi, I use jsoniter-scala to searialze a case class which has a byte field of default value to 0, using writeToString for a case class that the byte field being assigned to 0 gives a json string that missing this byte field, is there a way to output the byte field and it's value?
Andriy Plokhotnyuk
@plokhotnyuk
Hi, @phlamenco ! Please try the following config option during codec derivation:
JsonCodecMaker.make[YourDatatypeWithAByteField] {
    CodecMakerConfig
      .withTransientDefault(false)
  }
phlamenco
@phlamenco
tested ok!, thanks
Guillaume Vauvert
@gvauvert
Hello @plokhotnyuk, first, thanks for jsoniter-scala, performance is great !
I have created plokhotnyuk/jsoniter-scala#836 in order to be able to provide better human readable and structured response in case of invalid input.
(I forked/cloned the repo to have a first look into the code, but IntelliJ is failing to compile the code, the same using sbt with command line. First issue seems to be a missing Jackson dependency.)
Paulius Imbrasas
@CremboC
@plokhotnyuk hey, is there any update on Scala 3 support? I see plokhotnyuk/jsoniter-scala#828 is merged but there hasn't been any releases
Andriy Plokhotnyuk
@plokhotnyuk
@CremboC Hi, Paulius! Remaining works before first release with Scala 3 are checking of performance and minimization of difference with Scala 2 version
Paulius Imbrasas
@CremboC
Got it. When you say minimization of differences, do you mean is it encoding/decoding something incorrectly right now or some other minor differences?
Andriy Plokhotnyuk
@plokhotnyuk
It is mostly about differences in Macros API for derivation and non-functional requirements: CPU cycles, allocations, etc.
Dominik Dorn
@domdorn
Hi! Is there a way to modify the value of the discriminator-field ? currently its using the case class name, but I want to customize that ?
Andriy Plokhotnyuk
@plokhotnyuk
@domdorn Hi, Dominik! Have you tried adtLeafClassNameMapper yet?
Dominik Dorn
@domdorn
thanks @plokhotnyuk, that was it!
Andriy Plokhotnyuk
@plokhotnyuk

@/all

Macros API of jsoniter-scala got Scala 3 support! Binaries are available on the Maven Central

It is source compatible with Scala 2.x versions so migration to Scala 3 or adding Scala 3 cross build should be smooth for most usages.

All credits to @rssh for his amazing work!!!

Dominik Dorn
@domdorn

I'm trying to migrate a play-json codec to jsoniter... the play codec looks like

  implicit val attributeWrites: Writes[WebCreateAssetAttribute[_]] = Writes {
    case x: WebCreateDataTableAssetAttribute => dataTableAttributeWrites.writes(x)
    case x: WebCreateStringAssetAttribute    => stringAttributeWrites.writes(x)
    case _                                   => throw new IllegalStateException("unknown type")
  }

  implicit val attributeReads: Reads[WebCreateAssetAttribute[_]] = Reads { jsValue =>
    (jsValue \ "type").as[String] match {
      case x if x.startsWith("datatable") => dataTableAttributeReads.reads(jsValue)
      case _                              => stringAttributeReads.reads(jsValue)
    }
  }

I can do the encode with a custom codec, but how can I do the decode ? e.g. how do I peek into the json object to read the type and then delegate to the corresponding codec?

e.g. like this..
  implicit val stringAssetAttributeCodec: JsonValueCodec[WebCreateStringAssetAttribute] = JsonCodecMaker.make[WebCreateStringAssetAttribute](CodecMakerConfig.withTransientNone(false).withTransientEmpty(false).withTransientDefault(false))
  implicit val dataTableAssetAttributeCodec: JsonValueCodec[WebCreateDataTableAssetAttribute] = JsonCodecMaker.make[WebCreateDataTableAssetAttribute](CodecMakerConfig.withTransientNone(false).withTransientEmpty(false).withTransientDefault(false))

  implicit val assetAttributeCodec: JsonValueCodec[WebCreateAssetAttribute[_]] = new JsonValueCodec[WebCreateAssetAttribute[_]]{
    override def decodeValue(in: JsonReader, default: WebCreateAssetAttribute[_]): WebCreateAssetAttribute[_] = {
      // how can I peek into the `type` of the json object and then decide how to decode it?

      val typeField: String = ???
      typeField match {
        case "string" => stringAssetAttributeCodec.decodeValue(in, default.asInstanceOf[WebCreateStringAssetAttribute])
        case x if x.startsWith("dataTable") => dataTableAssetAttributeCodec.decodeValue(in, default.asInstanceOf[WebCreateDataTableAssetAttribute])
        case _ => ???
      }
    }

    override def encodeValue(x: WebCreateAssetAttribute[_], out: JsonWriter): Unit = x match {
      case v: WebCreateStringAssetAttribute => stringAssetAttributeCodec.encodeValue(v, out)
      case v: WebCreateDataTableAssetAttribute => dataTableAssetAttributeCodec.encodeValue(v, out)
    }

    override def nullValue: WebCreateAssetAttribute[_] = null.asInstanceOf[WebCreateAssetAttribute[_]]
  }
Dominik Dorn
@domdorn
ok, managed to get it working with setMark rollbackToMark and skipToKey
  implicit val assetAttributeCodec: JsonValueCodec[WebCreateAssetAttribute] = new JsonValueCodec[WebCreateAssetAttribute]{
    override def decodeValue(in: JsonReader, default: WebCreateAssetAttribute): WebCreateAssetAttribute = {
      in.setMark()

      in.nextToken()
      in.skipToKey("type")
      val discriminator = in.readString("")
      discriminator match {
        case "string" =>
          in.rollbackToMark()
          stringAssetAttributeCodec.decodeValue(in, default.asInstanceOf[WebCreateStringAssetAttribute])
        case x if x.startsWith("datatable") =>
          in.rollbackToMark()
          dataTableAssetAttributeCodec.decodeValue(in, default.asInstanceOf[WebCreateDataTableAssetAttribute])
        case _ => in.decodeError(s"invalid type ${discriminator} found")
      }
    }

    override def encodeValue(x: WebCreateAssetAttribute, out: JsonWriter): Unit = x match {
      case v: WebCreateStringAssetAttribute => stringAssetAttributeCodec.encodeValue(v, out)
      case v: WebCreateDataTableAssetAttribute => dataTableAssetAttributeCodec.encodeValue(v, out)
    }

    override def nullValue: WebCreateAssetAttribute = null.asInstanceOf[WebCreateAssetAttribute]
  }
Ryan Tomczik
@Tomczik76
Hello is it possible to parse either a number or a string with this library? Do I need to create a custom codec?
Ryan Tomczik
@Tomczik76
like this?
  implicit val codec: JsonValueCodec[Either[String, Long]] =  new JsonValueCodec[Either[String, Long]] {
    override def decodeValue(in: JsonReader, default: Either[String, Long]): Either[String, Long] = {
      if(in.isNextToken('"')) Left(in.readString(""))
      else Right(in.readLong())
    }

    override def encodeValue(x: Either[String, Long], out: JsonWriter): Unit =
      x match {
        case Left(value) => out.writeVal(value)
        case Right(value) => out.writeVal(value)
      }

    override def nullValue: Either[String, Long] = null
  }
Ryan Tomczik
@Tomczik76
I settled on this
  implicit val eitherCodec: JsonValueCodec[Either[String, Long]] =
    new JsonValueCodec[Either[String, Long]] {
      override def decodeValue(
          in: JsonReader,
          default: Either[String, Long]
      ): Either[String, Long] = {
        val rawString = new String(in.readRawValAsBytes())
        if (rawString.head == '"') Left(rawString.drop(1).dropRight(1))
        else
          Right(
            rawString.toLongOption match {
              case Some(value) => value
              case None =>
                throw new Exception(
                  s"Parse error: $rawString is not a string or number"
                )
            }
          )
      }

      override def encodeValue(x: Either[String, Long], out: JsonWriter): Unit =
        x match {
          case Left(value)  => out.writeVal(value)
          case Right(value) => out.writeVal(value)
        }

      override def nullValue: Either[String, Long] = null
    }
Andriy Plokhotnyuk
@plokhotnyuk
@Tomczik76 Hi, Ryan! Here is a more efficient version for parsing either a number or a string: https://github.com/plokhotnyuk/jsoniter-scala/blob/e38654eb90f0a1507f5594cb21962a0421abf770/jsoniter-scala-macros/shared/src/test/scala/com/github/plokhotnyuk/jsoniter_scala/macros/JsonCodecMakerSpec.scala#L693-L708 You can replace Int by Long in that snippet to have a codec for Either[String, Long].
Andriy Plokhotnyuk
@plokhotnyuk
@Tomczik76 BTW, new String(in.readRawValAsBytes()) will work properly only for ASCII strings without escaped characters.
Søren Valentin Silkjær
@SoerenSilkjaer

Hello! I have this scenario where i am parsing a json with a datetime field that jsoniter cannot parse with the default codec. What is the most ergonomic way to parse that field with a specific datetimeformatter?

I have a case class that looks something like this:

case class Foo(
  bar: String,
  time: Option[LocalDateTime]
)

The json looks something like this:

{
  "bar": "foo",
  "time": "02/02/2022 11:22"
}

Jsoniter throws this error with the default codec: Caused by: com.github.plokhotnyuk.jsoniter_scala.core.JsonReaderException: expected digit, offset: 0x00000fb8, buf:

Thanks!

Andriy Plokhotnyuk
@plokhotnyuk
Hi, Søren! I've added an example of how to define a custom codec that parses and serializes LocalDateTime from/to the provided format and how to inject it using implicit val for derivation of a codec that parses an array of such values: plokhotnyuk/jsoniter-scala@748e149
For more efficient implementation you can parse byte by byte without String instantiation and reparsing through DateTimeFormatter.
Søren Valentin Silkjær
@SoerenSilkjaer

Thanks, that is very helpful. I have a follow up question. How do i scope that codec to only be used within a specific case class, or even better only on a specific field? I am parsing a structure of several nested case classes, and several of them have fields of the type LocalDateTime, but I only want to use this codec on one of them.

For example:

case class FruitStore(
  bar: String,
  opensAt: Option[LocalDateTime], // Uses default codec
  closesAt: Option[LocalDateTime], // Uses default codec
  fruits: Seq[Fruit]
)

case class Fruit(
  bar: String,
  expiresAt: Option[LocalDateTime] // This field needs to use the special codec
)
Andriy Plokhotnyuk
@plokhotnyuk
You can define custom codecs for types only. Currently, there are no possibility to define a custom codec for an arbitrary field. For that case w/a would be defining some unique type and use it for the field or defining a custom codec for a whole data structure that encloses the field.
Søren Valentin Silkjær
@SoerenSilkjaer
Ok I thought so aswell. Thanks!
Andriy Plokhotnyuk
@plokhotnyuk
Here is an update of the test to show how a custom codec can be injected for the specific type only: plokhotnyuk/jsoniter-scala@ac7b4a7
Søren Valentin Silkjær
@SoerenSilkjaer
Thanks! I have the custom codec working now, which is awesome. I cannot make your scoped example work though. If I position the custom codec in the inner scope, i get a compilation error that the implicit val is unused, so i needed to move it into the outer scope.
  val customCodecOfLocalDateTime: JsonValueCodec[LocalDateTime] =
    new JsonValueCodec[LocalDateTime] {
      private val formatter =
        DateTimeFormatter.ofPattern("dd/MM/uuuu HH:mm")

      val nullValue: LocalDateTime = null

      def decodeValue(in: JsonReader, default: LocalDateTime): LocalDateTime =
        LocalDateTime.parse(in.readString(null), formatter)

      def encodeValue(x: LocalDateTime, out: JsonWriter): _root_.scala.Unit =
        out.writeNonEscapedAsciiVal(formatter.format(x))
    }
  // this scope works
  implicit val overrideCodecOfLocalDateTime      = customCodecOfLocalDateTime 
  implicit val fruitCodec: JsonValueCodec[Fruit] = {
    // this scope gives a compiler error: "local val overrideCodecOfLocalDateTime in value fruitCodec is never used"
    // implicit val overrideCodecOfLocalDateTime = customCodecOfLocalDateTime
    JsonCodecMaker.make[Fruit]
  }
Andriy Plokhotnyuk
@plokhotnyuk
Probably you can add some @nowarn annotation or some Fruit object definition like here: plokhotnyuk/jsoniter-scala@68b56db
Søren Valentin Silkjær
@SoerenSilkjaer
Hi again. I am consuming a json api with play wsclient and am wondering what is the most efficient way to feed the response into jsoniter, as a string, bytearray or inputstream? Is there any example of integration between these two components?
Andriy Plokhotnyuk
@plokhotnyuk
Hi, Søren! The fastest input to parse is Array[Byte]. Use ReaderConfig.withPreferredBufSize(...) to tune an internal buffer size for most of your messages and reduce possible allocations when they are greater than default 16K.
Søren Valentin Silkjær
@SoerenSilkjaer
Thanks! I will try that.