Jawn is one of most slowers. Please see results of benchmarks which compares Circe (that uses Jawn) and other JSON parsers for Scala: https://plokhotnyuk.github.io/jsoniter-scala/
In my experience Jawn is competitive with Jackson as a parsing backend for Circe, and it's generally faster than spray-json's parser.
Here is a PR with direct comparison of Jawn vs jsoniter-scala for parsing to Jawn's AST: plokhotnyuk/jsoniter-scala#424
I would be a bit more careful making sure to use the libraries correctly before coming into their channels and taking a shit on their work.
Also, with my PR you can reproduce DoS/DoW vulnerability of
JawnFacade and pick a solution of using java.util.LinkedHashMap instead of scala.collection.mutable.HashMap that was shamelessly copied from Circe. To reproduce, please, clone jsoniter-scala repo, checkout the jawn-ast branch and run the following command: sbt -no-colors 'jsoniter-scala-benchmark/jmh:run -i 1 -wi 1 -p size=1,10,100,1000,10000,100000 ExtractFieldsReading.jawn'
And, you should get result like this:
[info] REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
[info] why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
[info] experiments, perform baseline and negative tests that provide experimental control, make sure
[info] the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
[info] Do not assume the numbers tell you what you want them to tell.
[info] Benchmark (size) Mode Cnt Score Error Units
[info] ExtractFieldsReading.jawnByteBufferParser 1 thrpt 1811139.569 ops/s
[info] ExtractFieldsReading.jawnByteBufferParser 10 thrpt 260677.346 ops/s
[info] ExtractFieldsReading.jawnByteBufferParser 100 thrpt 19623.857 ops/s
[info] ExtractFieldsReading.jawnByteBufferParser 1000 thrpt 301.378 ops/s
[info] ExtractFieldsReading.jawnByteBufferParser 10000 thrpt 1.608 ops/s
[info] ExtractFieldsReading.jawnByteBufferParser 100000 thrpt 0.006 ops/s
[info] ExtractFieldsReading.jawnJsoniterScala 1 thrpt 2865937.664 ops/s
[info] ExtractFieldsReading.jawnJsoniterScala 10 thrpt 330815.496 ops/s
[info] ExtractFieldsReading.jawnJsoniterScala 100 thrpt 26956.753 ops/s
[info] ExtractFieldsReading.jawnJsoniterScala 1000 thrpt 2279.925 ops/s
[info] ExtractFieldsReading.jawnJsoniterScala 10000 thrpt 203.146 ops/s
[info] ExtractFieldsReading.jawnJsoniterScala 100000 thrpt 16.296 ops/s
[info] ExtractFieldsReading.jawnStringParser 1 thrpt 2266465.614 ops/s
[info] ExtractFieldsReading.jawnStringParser 10 thrpt 358482.177 ops/s
[info] ExtractFieldsReading.jawnStringParser 100 thrpt 24793.306 ops/s
[info] ExtractFieldsReading.jawnStringParser 1000 thrpt 352.264 ops/s
[info] ExtractFieldsReading.jawnStringParser 10000 thrpt 1.630 ops/s
[info] ExtractFieldsReading.jawnStringParser 100000 thrpt 0.006 ops/s
So the 1Mb request is able to burn 4GHz CPU core for 3 minutes... I hope this (and any other non-direct usage of Scala's
HashMap/HashSet) will be fixed before 1.0.0 release
Parsing of JSON is a minefield... Most of AST-based parsers are vulnerable under attacks which exploit using of recursion for parsing: https://github.com/lovasoa/bad_json_parsers
would adding support to limit for example big-ints via config to a certain amount of digits be viable solution?
It was introduced in typelevel/jawn#102 to maintain the
Facade interface, but in my view it doesn't serve any real purpose, it's badly named, and if we're about to commit ourselves to a long-term 1.0.0 now is the time to get rid of it.
Also, if anyone has any objection to Scalafmt-ing the Jawn repo, please let us know asap: typelevel/jawn#210
This week is our last chance to get changes into 1.0.0: https://github.com/typelevel/jawn/issues/193#issuecomment-573684699
I’ve been playing around with adding SJS support to Jawn. Obviously things like File/Channel aren’t going to work, but outside of that, one of the things I’ve stumbled into is a couple places where the project relies on IndexOutOfBoundsException/StringIndexOutOfBoundsException (for charAt). Both of those exceptions by default are undefined behavior in SJS. You can get around the IndexOutOfBoundsException with a setting, but I believe StringIndexOutOfBoundsException will still be undefined.
My ultimate goal was to add SJS support to circe-fs2.
Anyway, I can add bounds checks to get around this problem (only when running in JS) but I only want to do this where the caller doesn’t already do bounds checks.
However, I’m seeing a couple cases where the caller indicates it does bounds checks but is still failing. For example, parseNumSlow is supposedly slower than parseNum because it does bounds checks. Yet I’m seeing failures in JS where that doesn’t appear to be true. For example, one of the tests tries to access index 5 in string “1.1e+”. Is this expected?
Is your goal in a scala.js port speed, incrementality, or compatibility with other Jawn-dependent solutions? Maybe those answers guide how much code gets to remain in shared and how much becomes platform specific.
My ultimate goal is getting circe-fs2 working as I wanted to support having my client consume a streaming JSON response from the server. Another approach would be to just not use jawn at all in circe-fs2 and try to use some JS-based, streaming JSON library but I haven’t found any great fits.
I’ve patched the various places that use charAt/Array.apply to do bounds checking (only on JS side) but would rather omit that if caller is already doing the checking.
How do changes make it from github to maven central? I’d like to pull in this change: https://github.com/typelevel/jawn/pull/269/files, and am wondering if the process is automatic and I should just wait, or if it can take a while and I should patch locally.
Syncing to Central now.
The withDottyCompat causes problems downstream. And two of the three support libraries that are left have no Dotty release.
We can stop publishing Dotty builds of those and nudge Play along.
Or we can get out of the support module business entirely.
This channel is pretty quiet, so I'll open a PR, too.
