Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • 19:22
    avdv commented #2128
  • 18:10
    WojciechMazur edited #2264
  • 18:04
    WojciechMazur synchronize #2264
  • 17:59
    WojciechMazur opened #2264
  • 17:48
    LeeTibbert opened #2263
  • 17:46
    WojciechMazur review_requested #2262
  • 16:31
    LeeTibbert review_requested #2258
  • 16:31
    LeeTibbert review_requested #2258
  • 16:24
    LeeTibbert commented #2235
  • 16:23
    LeeTibbert closed #2237
  • 16:23
    LeeTibbert commented #2237
  • 16:20
    LeeTibbert closed #2128
  • 16:20
    LeeTibbert commented #2128
  • 15:46
    LeeTibbert synchronize #2258
  • 15:10
    LeeTibbert synchronize #2258
  • 15:02
    LeeTibbert commented #2258
  • 14:30
    WojciechMazur opened #2262
  • 11:40
    WojciechMazur synchronize #2162
  • 10:12

    WojciechMazur on master

    Support GC shared code and add … (compare)

  • 10:12
    WojciechMazur closed #2231
Eric K Richardson
@ekrich
If it doesn't do that with Scala JVM 2.11 then that also narrows things down. Put in an issue as needed.
Lorenzo Gabriele
@lolgab
@LeeTibbert You should check the approach in this PR: scala-native/scala-native#1102 to add atomics using the llvm ir. It should be easier and nicer.
Eric K Richardson
@ekrich
That was part of Denys' meta issue here. I think we have location data now too. scala-native/scala-native#1287
Wojciech Mazur
@WojciechMazur
@david-bouyssie I'm currently working on proper windows (you can find it in this repo https://github.com/WojciechMazur/scala-native/tree/feature/initial_windows_support) Some things might be missing, since I've have not committed some things yet. Overall it allows to run unit-tests, and most of them are passing. This week I'll be trying to finalize this part and publish a proper PR (probably it would be splitted into chunks for easier review)
Wojciech Mazur
@WojciechMazur
@LeeTibbert After finishing Windows I would like to focus on multithreading. I've tried to rebase once multithreading branch, but it took me a bit longer than a few lazy hours of the weekend, and finally, I've not finished it. Especially, since a few days later, I think @catap, published his results with working multithreaidng.
David Bouyssié
@david-bouyssie
@WojciechMazur wow! Amazing news!!!
Native windows support is going to be a huge win for SN adoption, for sure!
Denys Shabalin
@densh
@WojciechMazur Exciting stuff! Let me know if you need any help on how to integrate multi threading with our garbage collectors.
It's quite tricky, especially without regressing runtime performance.
@catap I'm not sure if zones are the best abstraction to tie this into.
Conceptually you want some form of explicit memory management that can even be used from C code.
On the other hand if zones were more general to handle any kind of abstract resource (smth that has acquire & release operation) it could compose automatically without them having to know about each other.
Eric K Richardson
@ekrich

@WojciechMazur I don't know if have had a chance to look at these PRs - there is some weird thing related to scripted tests on this first one I haven't figured out otherwise everything else is working. scala-native/scala-native#2234

There is also this one for the GC paths that is quite simple. scala-native/scala-native#2231 These make it quite simple to filter or select components of different libraries and could be a complement to the Windows build.

The good thing is that they separate concerns and allow different projects to filter C code differently. Right now only nativelib gets filtered but one filter is for GC and the other is for javalib optional component so you can see how this could help.

Eric K Richardson
@ekrich
@densh I was thinking that perhaps we should have an MTCommix or CommixMT so that we can make a direct comparison and then once everything was smoothed out that could become Commix. This way we don't affect anything current going on.
Kirill A. Korinsky
@catap
@densh I'm see this feature as extention of zones. Do you have better naming? :)
LeeTibbert
@LeeTibbert

@WojciechMazur re: After finishing Windows I would like to focus on multithreading

Thank you for the helpful & welcome information. You saved me person-months of effort on something that would have been not only wasted but counter-productive.

I may/will be doing some private studies in the mean time until that turn of the wheel comes around.

May I make two suggestions for when you are doing order-of-magnitude estimates of required effort?

1) From extensively studying PR #1068, I believe that the memory handling for atomics there was suitable for the
purpose of that PR, fair enough, but is not suitable for production code. I have tried a number of ways of finagling
the memory handling for calling C and/or LLVM Atomic routines. They have all lead me to the belief that a
proper implementation will need SN compiler support, with the time, effort, and almost certainly NIR change that entails. A different verse of but same song as volatile annotation PR #1102.

Yes, C Atomic & volatile are different issues. The former can sometimes be used to workaround the latter.

I think here are some improper workarounds that can be used in the meantime. More upon request, at the
fullness of time.

2) With respect for its original purpose & context, PR #1068 is huge. Since Multithread almost surely needs Atomics,
I suggest that the latter be spun off as a work effort before the former.

It is impossible for me to accurately forecast my future availability, and sometimes adding people to a project slows it down (Thank you, Fred Brooks). If you would like help when the time comes to address Atomics, Multithreading, and
friends, please feel free to ask.

Eric K Richardson
@ekrich
@sjrd Would it be possible to get permission to rerun github action jobs for me and @LeeTibbert ? Many times we just see spurious failures and I hate to rerun everything for one job.
Sébastien Doeraene
@sjrd
@ekrich Even we can't rerun single jobs. It's simply not supported by GitHub Actions. :(
Eric K Richardson
@ekrich
Oh gosh, I only have one job on my repositories so it is not obvious. That’s a bummer.
Eric K Richardson
@ekrich
@WojciechMazur This is still a bit of a WIP but it should show now where I am headed. I would like the move some of the nativelib unpacking and such to NativeLib from LLVM as it makes more sense. We could read properties so the Posix library could just be ignored completely on Windows and it gets compiled separate now. This is much better. scala-native/scala-native#2234
Eric K Richardson
@ekrich
Interested - https://openjdk.java.net/jeps/411
LeeTibbert
@LeeTibbert

re: re-running single jobs & wasted CI cycles. Perhaps it is time to re-think & change run-tests.yml lines (plural)
fail-fast: false to fail-fast: true. I believe that would not solve problems in the compilation stage, but would
be a relatively easy think to try to take a wack off wasted time running tests after the first failed. I do not know if it
fails jobs in the matrix.

I know, and have been part of too many times in my life. both sides of the fail-fast, true & false. There are times when
"Do what I want/mean" means one and probably an equal number of times where it means the other.

As with most things, the 'proper' solution is to make it configurable, but that is almost certainly more work than anybody, including me, wants to take on and manage over time.

The other obvious but less than helpful solution is the historical & classical answer: Buy more memory. Unfortunately,
waste by design choices increases at some rate greater than double exponential whilst free resources, such as GitHub CI are lucky to
increase at all (reference the demise of free Travis).
LeeTibbert
@LeeTibbert
Another possible approach is to change the workflow and add a middle stage, after check-lint, etc. That middle stage would have a single Scala Version (and OS), say 2.13.5 (meaning "Highest supported"). That stage would have to complete
before the rest of the matrix was fired off. Yes, that would probably increase total duration in the success case but it would, I think, decrease total time in failure cases.
LeeTibbert
@LeeTibbert
To say something good about GitHub Actions & things that go bump in the night. I now seem to be getting
the email I desire when an Action completes. For a long while, I was astonished that such was not happening,
even though I had enabled it. Web page notifications happened as I expected from the GH documentation.
At least this bump when in a good direction.
LeeTibbert
@LeeTibbert
Folks. I raise the hue & cry with some trepidation. I believe that the SN GitHub build of new PRs is hard broken
and has been for at least 5 days. The presenting problem is that sbt can not obtain its exclusive lock file due
LeeTibbert
@LeeTibbert

to a permission failure. Log files for PRs #2246 and #2259 give the full traceback and show the exact error.
My PR #2258 is known & expected to fail because it requires a previous, pending PR.

I hope this heads up saves other people time & frustration. I just spent two days of precious time trying to figure out how
adding a blank line to build.sbt worked on my system but failed in CI. Is there an emoticon for "unhappy camper" (well, I am happy to discover that I did not introduce a problem in the change I was studying.)

Eric K Richardson
@ekrich
I just pushed this last night and CI worked fine. scala-native/scala-native#2234
LeeTibbert
@LeeTibbert
Since my PR #2268 from 1 day ago, Friday, failed in the way I expected, having obtained the .sbt lock, the defect in question may have to do with the changes being in build.sbt.
@ekrich You mave have given a good clue. It looks to me like your successful PR #2234 does not touch build.sbt.
Eric K Richardson
@ekrich
Also, the lint was failing too. Did you rebase? On mobile so hard to check.
LeeTibbert
@LeeTibbert
After 20 or so iterations, I gave up caring about the lint, a mote when I was being gored by oxen. It is your PR #2246 which was the first failing with the lock file, that I could discover.

For grins, 

AthaB{lee49}$ git fetch upstream
AthaB{lee49}$ git rebase -v upstream/master
Current branch PR_build_Prepare4_Sbt150_2021-04-16 is up to date.

Both you and I probably have zero time to chase this today and for the next several days. Either the defect is not there and I am a goat for raising the hue & cry, or it is and other people will confirm that. Wish I had more time to chase this...

LeeTibbert
@LeeTibbert
@ekrich thank you, as always, for the suggestions.
Eric K Richardson
@ekrich
@LeeTibbert That was an issue by @catap and was closed as you can see why in the issue.
LeeTibbert
@LeeTibbert
Sorry, I typo'd the number of your failing PR: #2256. "Demonstrate conditional dependsOn" Looks to me like it is failing for
exactly the reason that I am reporting: 
Error:  java.io.IOException: failed to create lock file /home/scala-native/.ivy2/.sbt.ivy.lock
Error:      at xsbt.boot.Locks$.liftedTree1$1(Locks.scala:35)
Error:      at xsbt.boot.Locks$.apply0(Locks.scala:34)
Error:      at xsbt.boot.Locks$.apply(Locks.scala:28)
Error:      at sbt.internal.librarymanagement.IvySbt.withDefaultLogger(Ivy.scala:87)
Error:      at sbt.internal.librarymanagement.IvySbt.withIvy(Ivy.scala:209)
Error:      at sbt.internal.librarymanagement.IvySbt.withIvy(Ivy.scala:206)
Error:      at sbt.internal.librarymanagement.IvySbt$Module.withModule(Ivy.scala:250)
Error:      at sbt.internal.librarymanagement.IvySbt$Module.moduleDescriptor(Ivy.scala:254)
Error:      at sbt.internal.librarymanagement.IvySbt$Module.dependencyMapping(Ivy.scala:256)
Error:      at sbt.Classpaths$.$anonfun$depMap$5(Defaults.scala:3705)
Error:      at scala.collection.immutable.List.map(List.scala:293)
Error:      at sbt.Classpaths$.$anonfun$depMap$4(Defaults.scala:3705)
Error:      at scala.Function1.$anonfun$compose$1(Function1.scala:49)
Error:      at sbt.internal.util.$tilde$greater.$anonfun$$u2219$1(TypeFunctions.scala:62)
Error:      at sbt.std.Transform$$anon$4.work(Transform.scala:68)
Error:      at sbt.Execute.$anonfun$submit$2(Execute.scala:282)
Error:      at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:23)
Error:      at sbt.Execute.work(Execute.scala:291)
Error:      at sbt.Execute.$anonfun$submit$1(Execute.scala:282)
Error:      at sbt.ConcurrentRestrictions$$anon$4.$anonfun$submitValid$1(ConcurrentRestrictions.scala:265)
Error:      at sbt.CompletionService$$anon$2.call(CompletionService.scala:64)
Error:      at java.util.concurrent.FutureTask.run(FutureTask.java:266)
Error:      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
Error:      at java.util.concurrent.FutureTask.run(FutureTask.java:266)
Error:      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
Error:      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
Error:      at java.lang.Thread.run(Thread.java:748)
Error:  Caused by: java.io.IOException: Permission denied
Error:      at java.io.UnixFileSystem.createFileExclusively(Native Method)
Error:      at java.io.File.createNewFile(File.java:1014)
Error:      at xsbt.boot.Locks$.liftedTree1$1(Locks.scala:34)
Error:      at xsbt.boot.Locks$.apply0(Locks.scala:34)
Error:      at xsbt.boot.Locks$.apply(Locks.scala:28)
Error:      at sbt.internal.librarymanagement.IvySbt.withDefaultLogger(Ivy.scala:87)
Error:      at sbt.internal.librarymanagement.IvySbt.withIvy(Ivy.scala:209)
Error:      at sbt.internal.librarymanagement.IvySbt.withIvy(Ivy.scala:206)
Error:      at sbt.internal.librarymanagement.IvySbt$Module.withModule(Ivy.scala:250)
Error:      at sbt.internal.librarymanagement.IvySbt$Module.moduleDescriptor(Ivy.scala:254)
Error:      at sbt.internal.librarymanagement.IvySbt$Module.dependencyMapping(Ivy.scala:256)
Error:      at sbt.Classpaths$.$anonfun$depMap$5(Defaults.scala:3705)
Error:      at scala.collection.immutable.List.map(List.scala:293)
And so it goes for miles/kilometers. The above is the useful part.
I got the number of my PR which shows the same problem right: #2259 "WIP: Prepare build.sbt for eventual use by sbt 1.5.n."
I also corrected the number in the Issue I created: #2260. Sorry that haste on my part created wasted time on yours.
LeeTibbert
@LeeTibbert
@WojciechMazur Thank you for all the merges & activity today.
LeeTibbert
@LeeTibbert
@WojciechMazur, Folks.

FYI, FilesTest decided to begin intermittently fail, starting today. Arrgh! Looks like a rounding error to me. I have a number of demands in Real Life on my time, but I will chase this as time becomes available. I know of no recent changes which
may have caused this, so I guess that I am in for a surprise/education.

If anybody else gets there first, please let me know and I will stand down.

Error:  Test java.nio.file.FilesTest.filesCopyshouldCopyAttributes failed: java.lang.AssertionError: creationTime expected:<FileTime(18738, 59065000000000)> but was:<FileTime(18738, 59066000000000)>, took 0.005 sec

Never a dull moment....

Wojciech Mazur
@WojciechMazur
If I remember correctly from my work on Windows it fails mostly on access time. I guess we should either increase the expected tolerance for difference for this case or not test this case at all.
LeeTibbert
@LeeTibbert
I am astonished and aghast to report that there is at least one long standing conceptual error in the attribute handling.
There appears to be confusion of java "creation time" and Linux (Windows) "ctime" (change time, most emphatically not creation time). I think I know a reasonably proper solution but need to do some design studies before submitting a PR.
None of us like the confusion introduced by hack & whack PRs which show even more lack of understanding make the problem worse.
LeeTibbert
@LeeTibbert

@WojciechMazur Thank you for the information about Windows. The access time test may have its own problems
and need a reduction of precision. Since I do not have a Windows SN environment, not much I can do about that until
later. Currently, FileTimes are days & nanoseconds_in_day. SN currently uses millisecond (and sometimes? microsecond)
times and the (nanosecond) remainder can cause comparison problems.

There used to be a PR to have SN use nanosecond file time precision on Linux, but that PR got overcome by churn. Alas, poor Yorick!

In general, I dislike bypassing tests. Even broken ones are probably telling us something (which we do not want to hear.)
Yes, if we can convince ourselves that they are totally bogus, they should be removed to reduce complexity and frustration.
Almost all the time, they should be fixed or crafted to do something right. Personal opinion & probably preaching to the choir.
Out for the rest of the day, my Day Job has arrived.