Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Apr 07 04:00

    shadaj on master

    Update sbt-scala-native-crosspr… (compare)

  • Apr 07 04:00
    shadaj closed #67
  • Apr 07 03:59

    shadaj on master

    Update sbt-pgp to 2.0.1 (#66) (compare)

  • Apr 07 03:59
    shadaj closed #66
  • Apr 07 03:59

    shadaj on master

    Update sbt-sonatype to 3.9.2 (#… (compare)

  • Apr 07 03:59
    shadaj closed #68
  • Apr 07 03:59

    shadaj on master

    Update sbt-dynver to 4.0.0 (#65) (compare)

  • Apr 07 03:59
    shadaj closed #65
  • Apr 07 03:40
    scala-steward opened #68
  • Apr 07 03:40
    scala-steward opened #67
  • Apr 07 03:40
    scala-steward opened #66
  • Apr 07 03:40
    scala-steward opened #65
  • Apr 07 03:23

    shadaj on master

    Update nscplugin, sbt-scala-nat… (compare)

  • Apr 07 03:23
    shadaj closed #57
  • Apr 07 03:23

    shadaj on master

    Update scalatest to 3.1.1 (#60)… (compare)

  • Apr 07 03:23
    shadaj closed #60
  • Apr 07 02:55
    shadaj synchronize #60
  • Apr 07 02:54
    shadaj synchronize #57
  • Apr 07 02:53

    shadaj on master

    Update sbt-dynver to 2.1.0 (#53) (compare)

  • Apr 07 02:53
    shadaj closed #53
Maxime Kjaer
@MaximeKjaer
For sure, I can ping you too in this channel once I open source it.
Eric K Richardson
@ekrich
I guess we are not there yet, and I'm hoping Denys will come back to Scala Native at least in a part time way. Which Python API are your referring too? If I could build a high level API then perhaps you type-level code could be crafted on top.
Maxime Kjaer
@MaximeKjaer
I'm referring to TensorFlow Core for Python, which calls the TensorFlow library (which is C++). My stack is Dotty (on the JVM) -> Python -> libtensorflow.
The thing is, I can only run these fancy type features from the JVM right now, because they're Dotty-specific. Being able to have both strong type safety and native calls to TF would require Scala Native to support Scala 3 (or at least TASTy), which afaik is not planned for now
Eric K Richardson
@ekrich
Ok, I see thanks for the info. That makes sense.
jhegedus42
@jhegedus42
hi !
I get the following error :
image.png
here is the code:
any idea how can I fix this ?
Jozsefs-MacBook-Pro:scalapytest joco$ sbt
[info] Loading global plugins from /Users/joco/.sbt/1.0/plugins
[info] Loading project definition from /Users/joco/dev/ml/scalapy/test01/github/scalapytest/project
[info] Loading settings for project root from build.sbt ...
[info] Set current project to ScalaPyTest (in build file:/Users/joco/dev/ml/scalapy/test01/github/scalapytest/)
[info] sbt server started at local:///Users/joco/.sbt/1.0/server/3f6c4d00bf97ae0f90ba/sock
sbt:ScalaPyTest> run
[info] running (fork) example.Hello
[info] code runs
[error] Exception in thread "main" java.lang.Error: java.lang.UnsatisfiedLinkError: jep.MainInterpreter.initializePython([Ljava/lang/String;)V
[error]     at jep.MainInterpreter.initialize(MainInterpreter.java:183)
[error]     at jep.MainInterpreter.getMainInterpreter(MainInterpreter.java:97)
[error]     at jep.Jep.<init>(Jep.java:232)
[error]     at jep.Jep.<init>(Jep.java:228)
[error]     at me.shadaj.scalapy.py.JepInterpreter.<init>(JepInterpreter.scala:9)
[error]     at me.shadaj.scalapy.py.Platform$.newInterpreter(Platform.scala:5)
[error]     at me.shadaj.scalapy.py.package$.interpreter(package.scala:11)
[error]     at me.shadaj.scalapy.py.Writer$$anon$11.write(Writer.scala:75)
[error]     at me.shadaj.scalapy.py.Any$.from(Any.scala:28)
[error]     at example.Hello$.delayedEndpoint$example$Hello$1(Hello.scala:6)
[error]     at example.Hello$delayedInit$body.apply(Hello.scala:3)
[error]     at scala.Function0.apply$mcV$sp(Function0.scala:39)
[error]     at scala.Function0.apply$mcV$sp$(Function0.scala:39)
[error]     at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:17)
[error]     at scala.App.$anonfun$main$1$adapted(App.scala:80)
[error]     at scala.collection.immutable.List.foreach(List.scala:392)
[error]     at scala.App.main(App.scala:80)
[error]     at scala.App.main$(App.scala:78)
[error]     at example.Hello$.main(Hello.scala:3)
[error]     at example.Hello.main(Hello.scala)
[error] Caused by: java.lang.UnsatisfiedLinkError: jep.MainInterpreter.initializePython([Ljava/lang/String;)V
[error]     at jep.MainInterpreter.initializePython(Native Method)
[error]     at jep.MainInterpreter.access$100(MainInterpreter.java:64)
[error]     at jep.MainInterpreter$1.run(MainInterpreter.java:140)
[error] Nonzero exit code returned from runner: 1
[error] (Compile / run) Nonzero exit code returned from runner: 1
[error] Total time: 1 s, completed Dec 24, 2019 11:51:02 PM
sbt:ScalaPyTest>
Eric K Richardson
@ekrich
Seems like something is wrong with the python install or something.
Maxime Kjaer
@MaximeKjaer
@jhegedus42 what version of Jep have you installed? You can try again with pip install jep==3.8.2
Eric K Richardson
@ekrich
I haven't tried honestly, but there is some discussion above that could help.
Shadaj Laddad
@shadaj
looks like a version mismatch as @MaximeKjaer noted, since some shared library is being loaded but there's a JNI definition that's missing
jhegedus42
@jhegedus42
thanks !
nice
this works
with 3.8.2 jep
thanks
image.png
how about jupyter ?
image.png
Maxime Kjaer
@MaximeKjaer
@jhegedus42 This looks like Ammonite doesn't fetch from the right resolver? I can't see anything else than 0.2.0 on Maven https://search.maven.org/artifact/me.shadaj/scalapy_2.12.
Ah, it could just be that you should be loading scalapy-core:0.3.0, which is on Maven https://search.maven.org/artifact/me.shadaj/scalapy-core_2.12
Btw, @shadaj, I think deployments have been broken since this build https://travis-ci.com/shadaj/scalapy/builds/142068062. It complains about:
[error] Multiple repositories are found:
[error] [meshadaj-1361] status:open, profile:me.shadaj(efba3c89dc27) description: Implicitly created (auto staging).
[error] [meshadaj-1362] status:open, profile:me.shadaj(efba3c89dc27) description: Implicitly created (auto staging).
[error] [meshadaj-1363] status:open, profile:me.shadaj(efba3c89dc27) description: Implicitly created (auto staging).
[error] [meshadaj-1364] status:open, profile:me.shadaj(efba3c89dc27) description: Implicitly created (auto staging).
[error] [meshadaj-1365] status:open, profile:me.shadaj(efba3c89dc27) description: Implicitly created (auto staging).
[error] [meshadaj-1366] status:open, profile:me.shadaj(efba3c89dc27) description: Implicitly created (auto staging).
[error] [meshadaj-1367] status:open, profile:me.shadaj(efba3c89dc27) description: Implicitly created (auto staging).
[error] [meshadaj-1368] status:open, profile:me.shadaj(efba3c89dc27) description: Implicitly created (auto staging).
[error] [meshadaj-1369] status:open, profile:me.shadaj(efba3c89dc27) description: Implicitly created (auto staging).
[error] [meshadaj-1370] status:open, profile:me.shadaj(efba3c89dc27) description: Implicitly created (auto staging).
[error] [meshadaj-1371] status:open, profile:me.shadaj(efba3c89dc27) description: Implicitly created (auto staging).
[error] [meshadaj-1372] status:open, profile:me.shadaj(efba3c89dc27) description: Implicitly created (auto staging).
[error] [meshadaj-1373] status:open, profile:me.shadaj(efba3c89dc27) description: Implicitly created (auto staging).
[error] Specify one of the repository ids in the command line
[error] java.lang.IllegalStateException: Found multiple staging repositories
[error] Use 'last' for the full log.
Maxime Kjaer
@MaximeKjaer
This means that nothing has been published since 0.3.0+2. I'd like to submit a PR adding cross-building to Scala 2.13 to scalapy-numpy, but this is a blocker for that.
Maxime Kjaer
@MaximeKjaer
I have very little experience or knowledge of how publishing Scala projects works, but from what I can gather it seems (from this issue xerial/sbt-sonatype#78) that updating sbt-sonatype could help
jhegedus42
@jhegedus42
@MaximeKjaer hmmm, thanks, i guess for the time being I will be just playing around in colab... google, that is pretty neat ... and easy ...
Shadaj Laddad
@shadaj
oh thanks @MaximeKjaer for catching that, I'll clear out the Sonatype staging and re-publish for now
Maxime Kjaer
@MaximeKjaer
@shadaj Could you perhaps publish ScalaPy manually for now? I have scalapy-numpy cross building to 2.13 in this commit MaximeKjaer/scalapy-numpy@d8805dc but would need version 0.3.0+13-0c255e53 of ScalaPy on Maven in order to make builds reproducible
Shadaj Laddad
@shadaj
@MaximeKjaer cleared out staging repositories and restarted the build, it should be out soon!
Shadaj Laddad
@shadaj
I think I identified the source of the publish failures, just added a new commit that's publishing now
Shadaj Laddad
@shadaj
0.3.0+15-598682f0 has been released!
Maxime Kjaer
@MaximeKjaer
That's fantastic, thank you so much! I'm submitting a PR to scalapy-numpy now :)
nova
@nova999
Hi, does scalapy work with anaconda environments?
Shadaj Laddad
@shadaj
@nova999 yep, Anaconda correctly overrides the python3-config utility so as long as you use the javaOptions += s"-Djna.library.path=${"python3-config --prefix".!!.trim}/lib" config it should pick up the current environment's Python
nova
@nova999
Thank you :)
nova
@nova999
So I got a simple pyopencl program working in scalapy. With typings it could be a good alternative to the seemingly abandoned ScalaCL.
Jorge
@jvican
nice! consider working on those typings then, i'm sure they'd be well received ;)
nova
@nova999
I did intend to. I was studying the numpy facade. Some things are still not quite clear to me. But I suppose I could lay a groundwork to begin with.
Maxime Kjaer
@MaximeKjaer
Based on previous discussion in this channel, I said I'd ping both @shadaj and @ekrich once I open sourced the shape-safe TensorFlow library for Dotty. So with a little bit of a delay, here's a link! https://github.com/MaximeKjaer/tf-dotty
Maxime Kjaer
@MaximeKjaer
@shadaj I copied the code of scalapy-tensorflow into a scalapy-tensorflow module. This is just the easiest way that I found to augment the existing typings while working on tf-dotty. I would be happy to contribute them back to the main scalapy-tensorflow project, once I've typed a full TF module
Shadaj Laddad
@shadaj
@MaximeKjaer this is very exciting! Took a quick glance over the website just now, looks very nice! Also was cool to see the thought process that went into the implementation
Mark Melton
@melton1968
I'm having trouble getting Intellij to work correctly for the cross build. It is very grumpy about CPythonAPI and Platform when I am visiting any files in the shared sources. Anyone else have / solved this issue?
Shadaj Laddad
@shadaj
@melton1968 one strategy could be to just exclude either the JVM or Native sources so that IntelliJ doesn't get confused about the duplicate definitions. I've been using Metals with VS Code these days, which works pretty well for cross builds, though it doesn't have as many of the refactoring abilities as IntelliJ yet.
Piotr Gawryś
@Avasil

Which jep version should be used for 0.3.0+17-2bfe86de ? I tried both 3.8.2 and 3.9.0 and both result in following error:

[error] Native library (darwin/libpython3.7m.dylib) not found in resource path (/var/folders/lr/x_gb25v56fj3_80kyyk17lhc0000gn/T/sbt_dd5aa2c1/job-2/target/90ed79f3/562ee7c8/scala-python-example_2.12-1.0.jar:/var/folders/lr/x_gb25v56fj3_80kyyk17lhc0000gn/T/sbt_dd5aa2c1/target/567bf6c2/9e2a823f/jep-3.9.0.jar:/var/folders/lr/x_gb25v56fj3_80kyyk17lhc0000gn/T/sbt_dd5aa2c1/target/bc8bc145/464fe2e1/scala-library.jar:/var/folders/lr/x_gb25v56fj3_80kyyk17lhc0000gn/T/sbt_dd5aa2c1/target/aa0ecde6/28d0208b/scalapy-macros_2.12-0.3.0%2B17-2bfe86de.jar:/var/folders/lr/x_gb25v56fj3_80kyyk17lhc0000gn/T/sbt_dd5aa2c1/target/1a583173/abb78bc4/jna-5.4.0.jar:/var/folders/lr/x_gb25v56fj3_80kyyk17lhc0000gn/T/sbt_dd5aa2c1/target/26ba9c6e/34d55877/scalapy-core_2.12-0.3.0%2B17-2bfe86de.jar:/var/folders/lr/x_gb25v56fj3_80kyyk17lhc0000gn/T/sbt_dd5aa2c1/target/2ce76994/5a36165d/scala-reflect.jar)
[error]  at com.sun.jna.NativeLibrary.loadLibrary(NativeLibrary.java:302)
  [error]  at com.sun.jna.NativeLibrary.getInstance(NativeLibrary.java:455)
  [error]  at com.sun.jna.Native.register(Native.java:1722)
  [error]  at com.sun.jna.Native.register(Native.java:1443)
  [error]  at me.shadaj.scalapy.py.CPythonAPIInterface.<init>(CPythonAPI.scala:6)
  [error]  at me.shadaj.scalapy.py.CPythonAPI$.<init>(CPythonAPI.scala:74)
  [error]  at me.shadaj.scalapy.py.CPythonAPI$.<clinit>(CPythonAPI.scala)
  [error]  at me.shadaj.scalapy.py.CPythonInterpreter$.<init>(CPythonInterpreter.scala:4)
  [error]  at me.shadaj.scalapy.py.CPythonInterpreter$.<clinit>(CPythonInterpreter.scala)
  [error]  at me.shadaj.scalapy.py.package$.<init>(package.scala:12)
[error]  at me.shadaj.scalapy.py.package$.<clinit>(package.scala)
[error]  at com.sushant.example.PiotrekExample$.delayedEndpoint$com$sushant$example$PiotrekExample$1(PiotrekExample.scala:13)
  [error]  at com.sushant.example.PiotrekExample$delayedInit$body.apply(PiotrekExample.scala:8)
  [error]  at scala.Function0.apply$mcV$sp(Function0.scala:39)
  [error]  at scala.Function0.apply$mcV$sp$(Function0.scala:39)
  [error]  at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:17)
  [error]  at scala.App.$anonfun$main$1$adapted(App.scala:80)
  [error]  at scala.collection.immutable.List.foreach(List.scala:392)
  [error]  at scala.App.main(App.scala:80)
  [error]  at scala.App.main$(App.scala:78)
  [error]  at com.sushant.example.PiotrekExample$.main(PiotrekExample.scala:8)
  [error]  at com.sushant.example.PiotrekExample.main(PiotrekExample.scala)
  [error]  Suppressed: java.lang.UnsatisfiedLinkError: dlopen(libpython3.7m.dylib, 9): image not found
  [error]   at com.sun.jna.Native.open(Native Method)
  [error]   at com.sun.jna.NativeLibrary.loadLibrary(NativeLibrary.java:191)
  [error]   ... 21 more

The same code works with scalapy 0.3.0. Any idea what could be missing from my setup?
It fails on val tf = py.module("tensorflow")

Shadaj Laddad
@shadaj
@Avasil the new ScalaPy release doesn't need Jep anymore, it uses your existing Python installation directly. You can add your installation to the path with javaOptions in Test += s"-Djna.library.path=${"python3-config --prefix".!!.trim}/lib" in your SBT settings
Piotr Gawryś
@Avasil
Thank you, it's working fine now!