Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • 06:13

    plokhotnyuk on master

    Add an example of parsing and s… (compare)

  • Jul 04 09:36
    plokhotnyuk synchronize #911
  • Jul 04 09:36

    plokhotnyuk on update-smithy4s-json-to-0.14.0

    Update GitHub Actions dependenc… Add a know issue for missing su… Clean up README.md and 5 more (compare)

  • Jul 04 06:58

    plokhotnyuk on master

    Add Scala.js benchmarks for pla… (compare)

  • Jul 03 05:39

    plokhotnyuk on master

    Update sbt to 1.7.0-RC2 (compare)

  • Jul 03 05:36

    plokhotnyuk on master

    Clean up of build configuration Update sbt to 1.7.0-RC2 (compare)

  • Jul 03 05:36
    plokhotnyuk closed #912
  • Jul 03 04:50

    plokhotnyuk on master

    (compare)

  • Jul 03 02:06
    scala-steward opened #912
  • Jul 02 12:38

    plokhotnyuk on master

    Clean up of build configuration (compare)

  • Jul 01 13:02

    plokhotnyuk on master

    Code clean up (compare)

  • Jul 01 09:42

    plokhotnyuk on master

    Update badges (compare)

  • Jul 01 06:34

    plokhotnyuk on master

    Clean up README.md (compare)

  • Jul 01 06:28

    plokhotnyuk on master

    Add a know issue for missing su… (compare)

  • Jul 01 06:05

    plokhotnyuk on master

    Add a know issue for missing su… (compare)

  • Jul 01 05:22

    plokhotnyuk on master

    Update GitHub Actions dependenc… (compare)

  • Jun 30 08:02
    plokhotnyuk synchronize #911
  • Jun 30 08:02

    plokhotnyuk on update-smithy4s-json-to-0.14.0

    Update smithy4s-json to 0.14.0 (compare)

  • Jun 30 07:50
    plokhotnyuk opened #911
  • Jun 30 07:49

    plokhotnyuk on update-smithy4s-json-to-0.14.0

    Update smithy4s-json to 0.14.0 (compare)

Glen Marchesani
@fizzy33
Message and QueryRequest | QueryResponse are distinct layers so one cannot marshal them into instances at the same time.
fwiw I did some prelim testing on the JsonStr works well enough for now
I get better performance all around and probly can tweak it a bit later
Andriy Plokhotnyuk
@plokhotnyuk
Just wondering why Array[Byte] cannot be used directly but safely like in the RawVal type here instead of using intermediate Chunk[Byte]?
Glen Marchesani
@fizzy33
Array[Byte] can be used directly
the issue is I have offsets
so not in my example out.writeRawVal(x.rawBytes.toArray)
ideally if I could have
def writeRawVal(array: Array[Byte], offset: Int, len: Int)
then I can achieve zero copy (well one copy) on writes
on the read side if there was a read that gave back the same triplet as in (array: Array[Byte], offset: Int, len: Int) then I can get zero copy on my side
Andriy Plokhotnyuk
@plokhotnyuk
Do you mean def readRawValAsBytes(array: Array[Byte], offset: Int): Int = ??? // returns the length of a read chunk?
Could be the java.nio.ByteBuffer type a better option with adding following methods: def writeRawVal(bytes: java.nio.ByteBuffer): Unit and def readRawValAsByteBuffer(bytes: java.nio.ByteBuffer): Unit?
Glen Marchesani
@fizzy33
definitely java.nio.ByteBuffer would work quite well
In the internals I am in I have little choice to be squeemish about mutation ;-)
one layer up it is all pretty clean and immutable
well lets say at the low layer it is immutable when everyone plays nicely together :-)
seems like some extends AnyVal type around an Array[Byte] to give you the data of the array but not have it be mutable would be really nice. I wonder if someone has done the work on such a thing
and thanks for sharing your thoughts on this.
Marcin Szałomski
@baldram

Currently I'm going to check if it will be possible to minimize derivation configuration by adding the support for Enumeratum in some additional sub-project or even in the jsoniter-scala-macros sub-project with an optionally scoped dependency on Enumeratum.
While EnumEntry from Enumeratum is supported [...]

@plokhotnyuk Hi Andriy! Have you had a chance to evaluate creating the Enumeratum support related jsoniter-scala-macros sub-project? Is it something in your current work scope and doable based on the existing work from the past (https://github.com/lloydmeta/enumeratum/pull/176/files)?

Andriy Plokhotnyuk
@plokhotnyuk
@baldram Hi, Marcin! Thank for your question and support! I think that support of value enums of Enumeratum can be added with test-only scoped dependency on Enumeratum. Just need to add the CodecMakerConfig.useEnumeratumEnumValueId flag that turns on using of value from value fields as ids for parsing and serialization of objects which extends sealed traits as it was done for Scala enums here: plokhotnyuk/jsoniter-scala@f37dde6 Also, it can bring maximum performance because Enumeratum API will not be used at all.
Andriy Plokhotnyuk
@plokhotnyuk
I've added tests for Enumeratum enums: plokhotnyuk/jsoniter-scala@73ce8aa
Marcin Szałomski
@baldram
Oh, wow! What an answer :) Just seeing commits. Thank you.
This will be a part of 2.8.2 I guess. Is it already scheduled? I would try out a new feature.
Marcin Szałomski
@baldram

@plokhotnyuk I have one more question Andriy. Is it possible to add to generated code a SuppressWarnings? Maybe as an option?
The macro causes a lot of Wartremover errors. So we need to ignore it like below.

@SuppressWarnings(Array("org.wartremover.warts.All"))

The problem is that it doesn't make sense to create a single file, let's say Codecs.scala, put all codecs there, and add wartremoverExcluded for the whole file from build.sbt.
In this case, we would have to import codecs manually, losing the advantage of implicit codec.
If we have a codec inside a companion object, this imports automatically and also, other features relying on this codec are working automatically.

So everywhere in code we add this linter ignore for JsonCodecMaker.make. This is in each place we use Jsoniter.

object SomeFooBarEvent {
  @SuppressWarnings(Array("org.wartremover.warts.All")) // silencing generated code false-positive issues
  implicit val codec: JsonValueCodec[SomeFooBarEvent] = JsonCodecMaker.make
}

final case class SomeFooBarEvent(
  metadata: FooBarMetadata,
  request: FooBarRequest
)

Maybe such a SuppressWarnings could be already generated or with some option?
Or there is another solution?

Andriy Plokhotnyuk
@plokhotnyuk
@baldram Could you please share the code and commands to reproduce your warnings? Probably they worth to be fixed immediately in the macro that generate implementations of codecs. In the v2.7.1 release I fixed the "match may not be exhaustive" warning and it is possible that others are easy to fix too.
Marcin Szałomski
@baldram
The above code is enough. This already causes warnings if there is no @SuppressWarnings. Any use of JsonCodecMaker.make for any hello world case class caused tons of errors, when used with Wartremover wartremoverErrors ++= Warts.all.
Then enough is to dosbt clean compile from the console and that's it. Shall I provide some example warnings?
Marcin Szałomski
@baldram

I'm not sure it's easy to fix those warnings as "null" is a thing which is prohibited or lack of triple equals and many things.
Here is example:

object FooBar {
  implicit val codec: JsonValueCodec[FooBar] = JsonCodecMaker.make
}
final case class FooBar(foo: Int, bar: Option[String])

...causes:

[warn] /mnt/foobar/model/FooBar.scala:11:63: [wartremover:Equals] != is disabled - use =/= or equivalent instead
[warn]   implicit val codec: JsonValueCodec[FooBar] = JsonCodecMaker.make
Now I will disable errors one by one. Eg. @SuppressWarnings(Array("org.wartremover.warts.Equals")). Then next will come, and next, and next...
So just after I get:
[warn] /mnt/foobar/model/FooBar.scala:12:63: [wartremover:Null] null is disabled
Marcin Szałomski
@baldram

So now I do:
@SuppressWarnings(Array("org.wartremover.warts.Equals", "org.wartremover.warts.Null"))
... and I get:

[warn] /mnt/foobar/model/FooBar.scala:12:63: [wartremover:OptionPartial] Option#get is disabled - use Option#fold instead

It looks like never ending story, so I end up in adding @SuppressWarnings(Array("org.wartremover.warts.All")).
That's why it would be great to already generate this SuppressWarnings, as this is kind of not nice thing we need to add in a code and excuse ;) for it in a comment.

If not generating Supress Warnings, I was thinking also how about implementing some custom annotation alias for Jsoniter like @SupressJsoniterWarts which would have this suppress for All errors under the hood.
Maybe also a solution. So adding this short version of Supress Warnings (@SupressJsoniterWarts) would be at least a bit nicer.
The best way would be to have it already generated, or have some configuration in build.sbt or whatever while importing a library that this will generate those @SuppressWarnings(Array("org.wartremover.warts.All")). Not sure this is possible.

Marcin Szałomski
@baldram
Is my description of a problem clear enough?
Andriy Plokhotnyuk
@plokhotnyuk
Thank you, Marcin! It was clear and helpful! Here is a commit with the proposed solution: plokhotnyuk/jsoniter-scala@a550237
Could you please clone the repo with git clone --depth 1000 --branch master git@github.com:plokhotnyuk/jsoniter-scala.git, build and publish locally a snapshot version with sbt clean publishLocal and then test the 2.8.2-SNAPSHOT version if it works for you?
Marcin Szałomski
@baldram
All right. I will do that in the evening, is it ok?
Andriy Plokhotnyuk
@plokhotnyuk
Ok, I've force pushed a better solution that passes internal tests with polluted namespaces: plokhotnyuk/jsoniter-scala@fa8b53f
Marcin Szałomski
@baldram
I've compiled a project with 2.8.2-SNAPSHOT with all SuppressWarnings removed and no linter errors :tada:
I will try in the other project to double check.
Marcin Szałomski
@baldram
@plokhotnyuk I've tried this in another project. What is important, after I bumped a version, all works without changes. Next, when removed linter ignores, I see no warnings.
All tests are passing as well, btw.
I also checked what happens if I make an issue next to codec definition and it is handled correctly by Wartremover.
Thank you Andriy!
PS: I haven't played yet with Enumeratum, but I think I will try to modify some code in a project for testing purposes tomorrow.
Andriy Plokhotnyuk
@plokhotnyuk
@baldram Thank you a lot for reporting and testing the issue with Wartremover! I've published the fix in v2.8.2 release.
Marcin Szałomski
@baldram
Thank you Andriy!
Marcin Szałomski
@baldram
@plokhotnyuk I have one more question Andriy, regarding commits about Scala 3, like "Update Scala 3.x to 3.0.0".
Is there a plan to make jsoniter-scala Scala 3 compatible? I thought it's not doable because of macros, and there is no hope for that.
However, seeing commits treating Scala 3... That would be so fabulous.
Andriy Plokhotnyuk
@plokhotnyuk
I've tried to publish jsoniter-scala-core with 3.0.0-RC3 and 3.0.0 versions but failed both attempts due some classpath problems with the sonatypeBundleRelease command: xerial/sbt-sonatype#230
Marcin Szałomski
@baldram
Sounds exciting! Does it mean that even though all those Scala 3 macro related difficulties, those moves are about trying to use Jsoniter with Scala 3, and it's close anyway?
Chris Kipp
@ckipp:matrix.org
[m]
I'm not fully sure how to actually phrase this questions, but I'm working on migrating the Build Server Protocol off of circe to jsoniter and we have some case classes that represent requests in the spec that we made codecs for, but some of these requests allow for the value that is returned to be null. However in that scenario (at least I think that's what happening) jsoniter will freak out because I believe it wants valid json, but is instead receiving null. And then you're met with this
[info]   org.eclipse.lsp4j.jsonrpc.ResponseErrorException: com.github.plokhotnyuk.jsoniter_scala.core.JsonReaderException: expected '{', offset: 0x00000000, buf:
[info] +----------+-------------------------------------------------+------------------+
[info] |          |  0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f | 0123456789abcdef |
[info] +----------+-------------------------------------------------+------------------+
[info] | 00000000 | 6e 75 6c 6c                                     | null             |
[info] +----------+-------------------------------------------------+------------------+

the way we are doing these requests are like

case class Reload()

object Reload {
  implicit val codec: JsonValueCodec[Reload] =
    JsonCodecMaker.make
}

But is there another way to do this that would allow for null?

Andriy Plokhotnyuk
@plokhotnyuk
@ckipp:matrix.org Hi Chris! Great news! A good option would be using of Option[Reload] instead, so null JSON values will be parsed to the None object reference. But if it is not an option for your than some delegating custom codec can be created to wrap derived one by the standard make:
  def wrapByNullAllowingCodec[A](codec: JsonValueCodec[A]): JsonValueCodec[A] = new JsonValueCodec[A] {
    override def decodeValue(in: JsonReader, default: A): A =
      if (in.isNextToken('n')) {
        in.readNullOrError[String]("x", "expected value or null")
        nullValue
      } else {
        in.rollbackToken()
        codec.decodeValue(in, default)
      }

    override def encodeValue(x: A, out: JsonWriter): Unit =
      if (x == null) out.writeNull()
      else codec.encodeValue(x, out)

    override val nullValue: A = null.asInstanceOf[A]
  }
Chris Kipp
@ckipp:matrix.org
[m]
thanks for the response! Ah, good to see your wrapByNullAllowingCodec I was actually just going down that path so it's good to see that I was headed in the right direction