Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Jun 17 07:48
    sjrd commented #2325
  • Jun 17 01:39
    ekrich opened #2325
  • Jun 16 16:03
    sjrd commented #2324
  • Jun 16 14:56
    tgodzik commented #2324
  • Jun 16 14:50
    ekrich opened #2324
  • Jun 14 16:47
    WojciechMazur synchronize #2323
  • Jun 14 16:47

    WojciechMazur on add-support-for-volatile-fields

    Fix typos in comments (compare)

  • Jun 14 16:28
    WojciechMazur commented #2323
  • Jun 14 16:22
    WojciechMazur opened #2323
  • Jun 14 16:16

    WojciechMazur on add-support-for-volatile-fields

    Add support for @volatile annot… (compare)

  • Jun 14 16:13

    WojciechMazur on add-support-for-volatile-fields

    Add support for @volatile annot… (compare)

  • Jun 14 00:21
    shadaj synchronize #1571
  • Jun 13 23:27
    shadaj synchronize #1571
  • Jun 13 22:36
    shadaj synchronize #1571
  • Jun 13 22:10
    shadaj edited #1571
  • Jun 13 22:10
    shadaj edited #1571
  • Jun 13 22:08
    shadaj synchronize #1571
  • Jun 12 20:42
    LeeTibbert commented #1571
  • Jun 12 20:42
    LeeTibbert commented #1571
  • Jun 12 20:39
    LeeTibbert commented #1571
David Bouyssié
@david-bouyssie
@WojciechMazur @ekrich I solved the path problem. I'll update the github issue.
Next error is:
[error] java.lang.RuntimeException: Failed to compile .\xml4s\tests\native\target\scala-2.13\native\native-code-nativelib_native0.4_2.13-0.4.0-4\scala-native\gc\boehm\gc.c
[error]         at scala.sys.package$.error(package.scala:30)
[error]         at scala.scalanative.build.LLVM$.$anonfun$compile$1(LLVM.scala:61)
David Bouyssié
@david-bouyssie
I think I'm now reaching expected problems, many required C file headers are missing. I can provide the list if it is useful but I guess it's not.
Since I have it, I put it here: gc.h, pthread.h, sys/mman.h, sys/time.h, unistd.h, zlib.h
Eric K Richardson
@ekrich
Yeah those would be the Posix ones not available and Boehm and libz too.
Eric K Richardson
@ekrich
@sjrd and @WojciechMazur I made some good progress on this for review - scala-native/scala-native#2315 If you are ok, I can go ahead and format and fix the ] // comment sites so then we could have one final spot check.
David Bouyssié
@david-bouyssie
Hi there.

I tried to benchmark very quickly the brand new xml4s library, on a 5 GB XML document (with a no-op basic startElement callback):

  • on the JVM with the default SAX parser (Xerces I think) it takes 18s on my laptop
  • on the JVM with my KXML fork it takes 38s
  • on SN 0.4.0 with KXML it takes 105s
  • on SN 0.4.1-SNAPSHOT with KXML it takes 50s

Used SN settings: LTO thin, immix, release-fast

So we are far from the fastest XML parser on the planet, but it should be good enough for small XML documents processing.
I plan to add support for the C Expat library on the long term.
I have not worked yet on the Scala.js support but I think it should not be that complicated.
Eric K Richardson
@ekrich
David, please try commix and you can also try release-full to see if does better than release-fast.
Tomasz Godzik
@tgodzik
@ekrich We worked around some issue in regexes in Scalafmt, though I turns out we need to fix them a bit now. I will try to work on it next week, I need to catch up on some other work in the meantime.
Eric K Richardson
@ekrich
If it is possible, could you capture the troublesome ones in an Issue along with some sample text if it is not too hard?
Tomasz Godzik
@tgodzik
Soo, these are the original regexes: https://github.com/tgodzik/scalafmt/blob/fe3c7cef2565ef11a4bff6b033f1166a44995887/scalafmt-core/jvm/src/main/scala/org/scalafmt/internal/PlatformCompat.scala
and these are native ones with workarounds: https://github.com/tgodzik/scalafmt/blob/fe3c7cef2565ef11a4bff6b033f1166a44995887/scalafmt-core/native/src/main/scala/org/scalafmt/internal/PlatformCompat.scala
These seem to to be able to match scaladoc comments.
or maybe the do match, but wrongly replace everything with empty
waait, I think the actual issue might be that replacing with a group doesn't work
Eric K Richardson
@ekrich
ok, I can add that as an issue.
Tomasz Godzik
@tgodzik
I am currently trying to run the tests for scalafmt in order to interate over the changes easier, but it seems no tests are detected if I run testsNative/test - any idea what might be the cause? it does compile everything and starts to run, but no acrual tests are run in the end
just
sbt:scalafmtRoot> testsNative/testOnly org.scalafmt.FormatTests
[info] compiling 1 Scala source to /home/tgodzik/Documents/scalafmt/scalafmt-core/native/target/scala-2.13/classes ...
[info] compiling 1 Scala source to /home/tgodzik/Documents/scalafmt/scalafmt-dynamic/native/target/scala-2.13/classes ...
[info] compiling 1 Scala source to /home/tgodzik/Documents/scalafmt/scalafmt-tests/native/target/scala-2.13/classes ...
[info] compiling 40 Scala sources to /home/tgodzik/Documents/scalafmt/scalafmt-tests/native/target/scala-2.13/test-classes ...
[info] Linking (2313 ms)
[info] Discovered 1505 classes and 10737 methods
[info] Optimizing (debug mode) (3609 ms)
[info] Generating intermediate code (4163 ms)
[info] Produced 8 files
[info] Compiling to native code (764 ms)
[info] Linking native code (immix gc, none lto) (178 ms)
[info] Total (11191 ms)
[info] Starting process '/home/tgodzik/Documents/scalafmt/scalafmt-tests/native/target/scala-2.13/tests-out' on port '32857'.
[info] Passed: Total 0, Failed 0, Errors 0, Passed 0
[success] Total time: 15 s, completed Jun 16, 2021 6:02:28 PM
we are using munit the same as in scalameta, where there seems to be no issues at all
testsJVM/test works without any issues
Tomasz Godzik
@tgodzik
nvm, it was a munit dependency that was declared globally with %
Eric K Richardson
@ekrich
Glad you found it. Those darn %!
Tomasz Godzik
@tgodzik
Good news is:
FormatTests:87 Total explored: 532914
[error] Failed: Total 3399, Failed 75, Errors 0, Passed 3324, Ignored 7
[error] Failed tests:
[error]         org.scalafmt.FormatTests
[error] (testsNative / Test / testOnly) sbt.TestsFailedException: Tests unsuccessful
[error] Total time: 130 s (02:10), completed Jun 16, 2021 6:21:08 PM
only 75 tests failed, at least from FromatTests
Eric K Richardson
@ekrich
Do you have tests to check config at all. I don't think sconfig is used.
Tomasz Godzik
@tgodzik
I haven't yet checked all tests, why do you think sconfig is not used?
Eric K Richardson
@ekrich
Usages are pretty small?? - https://mvnrepository.com/artifact/org.ekrich/sconfig
Eric K Richardson
@ekrich
@lolgab Does Scala Native using Alpine/Musl work now?
Lorenzo Gabriele
@lolgab
@ekrich Last time I tried the -static linking was making the stacktraces not work ( all the frames informations were gone )
With dynamic linking it was working on alpine after building on ubuntu/debian after copying the ubuntu glibc in the right places.
It may be interesting to try to fix the static linking with Musl which has some problems ( probably because of the bundled libunwind )
I guess you are interested to ship a static linked version of scalafmt, right?
It would be a nice milestone!
Eric K Richardson
@ekrich
Yes, the deal is that we use C++ exception somewhere but then we expect our vendored unwind code to be used. What happens on macOS anyway is that the C++ linked libunwind is used but we are reading from the vendored one. See scala-native/scala-native#1918
Last couple of convos.
Eric K Richardson
@ekrich
Here I think - https://github.com/scala-native/scala-native/blob/master/nativelib/src/main/resources/scala-native/eh.cpp
It could be just structure diffs on macOS vs Linux but the core seems to show otherwise.
Eric K Richardson
@ekrich
Starting here https://github.com/scala-native/scala-native/issues/1918#issuecomment-849248754 with @shadaj 's help I think I get to the bottom of the problem. And given that the problem occurs in the exception test ...
It might just mean we need some more linker options on macOS so that we get our libunwind code linked and not the supplied one.
Lorenzo Gabriele
@lolgab
Very very interesting! I will read more about it, thank you! I wonder if this problem is related with the problem I faced with exceptions compiling Scala Native with Zig.. If it is it would be really great, Zig would allow us to do great stuff with cross-compilation
Eric K Richardson
@ekrich
@lolgab Let me know what you think. I certainly see the appeal to Alpine images - really great for any sort of cloud deploy.
Carefully, read from that linked issue comment.
I ran the tests single threaded and found the offender.
Once I commented out the code in exception and then ran the sanitizer than it shows up.
I am not sure why it seems to show up in tests but ...
Lorenzo Gabriele
@lolgab
Did some tests... With Zig as clang substitute..
First of all, it is a different issue than the one I was facing,
The one I'm facing happens when I compile libunwind with zig instead of clang.
If I try sandbox/run with zig the program can't catch exceptions..
libc++abi: terminating with uncaught exception of type scalanative::ExceptionWrapper: std::exception
If I then delete the libunwind folder from target/scala-2.12 and switch back to clang to compile, then switch back again to zig it can catch the exceptions
So something strange happens when compiling libunwind with zig.
The fun part is that Zig itself ships with libunwind, but I couldn't make it work to make SN use Zig's libunwind...
Eric K Richardson
@ekrich
We probably need to get to the bottom of this and come up with a plan. You are talking about the ziglang compiler or something else?