Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Dec 06 02:38
    ekrich commented #1064
  • Nov 27 17:16
    ekrich synchronize #1637
  • Nov 27 01:26
    ekrich commented #1637
  • Nov 27 01:24
    ekrich synchronize #1637
  • Nov 26 20:16
    LeeTibbert commented #1635
  • Nov 26 19:35
    ekrich commented #1635
  • Nov 26 19:29
    LeeTibbert commented #1635
  • Nov 26 19:21
    LeeTibbert commented #1635
  • Nov 26 18:28
    LeeTibbert commented #1635
  • Nov 26 15:58
    ekrich commented #1635
  • Nov 26 09:13
    avdv commented #1635
  • Nov 25 21:42
    avdv commented #1635
  • Nov 25 21:41
    avdv commented #1635
  • Nov 25 20:52
    avdv commented #1635
  • Nov 25 20:37
    avdv commented #1635
  • Nov 25 20:36
    avdv commented #1635
  • Nov 25 20:35
    avdv commented #1635
  • Nov 25 18:12
    ekrich commented #1635
  • Nov 24 22:14
    LeeTibbert commented #1635
  • Nov 24 21:21
    LeeTibbert commented #1023
Eric K Richardson
@ekrich
@sjrd Ok, good to know - we only really need one but scala-js-java-time is working fine for me. I was thinking that because everytime we looked into it for Scala Native there was a blocker given his library was used on JVM pre version 8 when java.time was introduced.
Adam Fraser
@adamgfraser
Yes. ZIO moved from scala-js-time to -scala-java-time because of the lack of support in scala-js-java-time for some of the data types we were using, particularly ZoneId if I remember correctly.
So I think at least for ZIO this is a blocker in providing support for Scala Native. We have the minimal support thanks to the fantastic work of @rwhaling but if users actually want to do anything with it they are going to need a java-time dependency that works on Scala Native which it sounds like we don't have right now.
Eric K Richardson
@ekrich
Thanks for the info.
Corey O'Connor
@coreyoconnor
Is there a small task (dev or review) I can contribute to? I checkout out: https://github.com/scala-native/scala-native/issues?q=is%3Aopen+is%3Aissue+label%3A%22good+first+issue%22 tho that seems out of date :)
Eric K Richardson
@ekrich
Yeah, that one is still in limbo and some of the methods could be added to Scala.js as well. I put a comment on that one.
Corey O'Connor
@coreyoconnor
@ekrich OK. I see the reference to scala-native/scala-native#1653
I'll see about getting that working :) I'm still somehwat out of commission so no promises tho :(
Eric K Richardson
@ekrich
There are probably more fun things to work on - you had just mentioned that PR and when I looked I saw that there were no references to the work I started. I just had other things I was working on so hadn't gone back to it.
Corey O'Connor
@coreyoconnor
Looks to be: Analyze and measure the load method. In particular on allocation behavior. Optimise (add tests as required). Correct?
that's pretty fun for me ;)
Eric K Richardson
@ekrich
Also, stensorflow got an update but since it requires native C code - it would have to wait for - scala-native/scala-native#1637
Lorenzo Gabriele
@lolgab
Good stuff Eric :)
Do you plan to write some Value classes interface around sblas?
Eric K Richardson
@ekrich
I would like to but it seems it would be a pretty big project and I would need to study what has been done in that space too.
LeeTibbert
@LeeTibbert
re: cquiroz's scala-java-time is more complete At the top of the parabola, I had the core of @cquiroz 's scala-native-time working with scala-native in a private sandbox. As I spent more time to implement the missing functions in the outstanding Issue,
LeeTibbert
@LeeTibbert

ran up against the fact that, IMHO, any version of code which would pass review would require Java default methods in traits. Those are implemented in Scala 2.12 JVM but not in SN. Scalatest was pending, but not released, for SN then. Scalatest might be
available now.

It looked to me like some SN code would be needed to access the Olson time files directly, where available (in libmumble.so). From
a "not having done a trial implementation" view from afar, it looked like a Small Matter of Programming.

As I noted in the Issue, these days the issue is on the back burner for me.

More like, in the refrigerator than on the stove at all.
Eric K Richardson
@ekrich
@LeeTibbert The concern I have is that today only a JS and Native version is needed not one for the JVM for Java 6 and Java 7 so we really need one geared towards only those 2 platforms. I don't see a logical solution without a fork or an enhancement of the Scala.js version.
LeeTibbert
@LeeTibbert

@ekrich I had also put a considerable amount of sandbox work into providing some of the chunks understandably missing from
scala-js: Clock.scala and, especially, clocks than can be set for testing.

Folks of good will can read the same evidence and come up with differing conclusions. When I stepped back to see how I could
avoid duplicating and triplicating work, it looked to me that the least effort solution was to port cquiroz's scala-java-time. Clock.scala, etc. were a factor but the large body of well exercised testing code was the deciding factor.

Not to encourage fragmentation, but I think a third party (or cousin) solution is necessary/wise to deal with the thin/comprehensive
or light/heavy code weight discussion. Time handling has lots of corner case. The difference between minimal-acceptable and
comprehensive is huge, and most people only need/want minimally acceptable. IMHO, time support should be designed
with a first consideration to its own needs with a strong second consideration of how locale is handles. Agreed that locale
is over-the-horizon, but it is huge enough we can hear it slouching towards us, even on the far side of the hill.

I do not remember running into any Java 6 or Java 7 issues in any of my scala-java-time sandbox work. Perhaps I did not get to the point of triggering them. What did block me was the need for default methods in traits, which is a scala version issue.

LeeTibbert
@LeeTibbert

To try to advance the Native time discussion beyond putting one's dollar/euro on red or black:

1) @ekrich Do you know if scala-js handles java default method implementations in traits? We may not need them but it is nice
to know where the edge of the world is.

2) If, for various reasons, a comprehensive solution is in the out years, what do you see, at a first cut, that we need for a
near term (over the next year or so) solution. I offer that we need/could_use Clock.scala support. A second want, for
SN at least, would be some minimal 24 hour timezone support, even if only UTC+N (or UTC-N). An SN addon might
be able to tie into existing Olson timezone file, if present (yeah, the location of those file on various operating systems
is a mess.

3) What are your thoughts about a possible light/heavy plan, where the heavy is a hypothetical comprehensive third
party solution? Do other people have thoughts or need a better explication of the problem? Basically, I am suggesting
that a well defined set of simple things should be "batteries included" and complicated things be in code which can be
added in by devos who need them. Relatively easy to describe at that level, hard to define a minimal-set. Discussion
has to start somewhere.

Carlos Quiroz
@cquiroz
If you need to refactor scala-java-time to avoid default methods that’s ok by me
As long as it doesn’t break the API
The Internals can vary
Eric K Richardson
@ekrich
@cquiroz and @LeeTibbert I think what we want is a just a Scala Native and Scala.js solution that some of the implementation can vary as needed based on the underlying platform. I can’t see a need for the JVM version going forward. Your library is the most complete but it may be easier to start a new repo for the purpose to start from a clean state. We could also just focus on Scala.js support for 1.0.0-RC1 and Native 0.4.0-M2. We could initially leave some implementation in Scala Native that exists there now but the goal would be to pull that out of Native back to the external library. This is just my opinion right now with little skin in the game and both of you know more.
Corey O'Connor
@coreyoconnor

I'm able to reproduce the failure in #1653 . First question stream.mark(Int.MaxValue) The implementation has this logic:

      if (buf.size < readLimit)                                                                                                                                                                                                                               
        buf = new Array[Byte](readLimit)

Correct me if I'm wrong: That would attempt to allocate an Array[Byte] of Int.MaxValue size?

If I change to use stream.mark(32768) Which is an arbitrary size less than max heap. Then the test passes for all GCs.
If that is correct, then the issue is actually in BufferedInputStream's handling of mark. The difference in GCs would then be differences in overcommit behavior?
Corey O'Connor
@coreyoconnor
Ha: Gotta love the docs on mark: "an IOException might be thrown" emphasis added. Might?
Eric K Richardson
@ekrich
Nice :worried:
Eric K Richardson
@ekrich
@coreyoconnor A lot of times we refer to the Scala.js code to see what is done there.
Corey O'Connor
@coreyoconnor
@ekrich I don't see a BufferedInputStream in scala-js. this looks to be equivalent code to scala-js's BufferedReader: https://github.com/scala-js/scala-js/blob/master/javalib/src/main/scala/java/io/BufferedReader.scala#L38
which has the same logic
however, javascript seems to have no issue with overcommit of those array.
I tried allocating a max int sized array in javascript (firefox) and it didn't (AFAICT) actually allocate that much memory. hmmmm
Eric K Richardson
@ekrich
Wow, ok. The only other suggestion would be to look at the Apache Harmony code. https://github.com/apache/harmony/tree/trunk/classlib/modules/luni/src/main/java/java/io
Corey O'Connor
@coreyoconnor
Definitely seems like this is the issue. That source appears to have some interesting logic to delay the allocation of a marklimit sized array to only if the default buffer is too small. In which case it uses the marklimit as a max buffer size but not the immediately used size
is the logic I'm referring to. Note newlength is only increased by 2x the current buffer length. Not immediately set to marklimit.
anyways if that all sounds right then I'll start with changing BufferedInputStream to have similar logic. Adding some tests first, of course.
I'll submit that as a separate PR
Eric K Richardson
@ekrich
I hate to see after the second cup of coffee :smirk:
Corey O'Connor
@coreyoconnor
haha it's exactly the realization: "I should work on my actual job..." be back later :)
Alex Henning Johannessen
@ahjohannessen
a lot of PRs and radio silence. What is going on @densh ?
Eric K Richardson
@ekrich
Update on tensorflow - @rwhaling and @lolgab, remember when I was asking about a Scala Native function getting called to release the memory. I looked at the code and all is good. Tensorflow takes you tensor, copies it so it can manage it, and then calls your function to deallocate your memory it the function that creates the TF_Tensor. I finally looked at the code :smile:
Richard Whaling
@rwhaling
oh cool!
nice find @ekrich
Eric K Richardson
@ekrich
I was surprised but glad nothing is wrong.