by

Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Aug 07 23:27
    dependabot[bot] labeled #92
  • Aug 07 23:27
    dependabot[bot] opened #92
  • Aug 07 23:27

    dependabot[bot] on npm_and_yarn

    Bump prismjs from 1.17.1 to 1.2… (compare)

  • Aug 07 19:15
    scala-steward opened #91
  • Aug 07 06:28
    scala-steward opened #90
  • Jul 23 21:04
    fsauer65 opened #89
  • Jul 18 01:30
    dependabot[bot] labeled #88
  • Jul 18 01:30
    dependabot[bot] opened #88
  • Jul 18 01:30

    dependabot[bot] on npm_and_yarn

    Bump lodash from 4.17.15 to 4.1… (compare)

  • Jul 14 18:32
    scala-steward opened #87
  • Jul 10 16:07
    shadaj closed #86
  • Jul 10 16:07
    shadaj commented #86
  • Jul 06 16:55
    PawelLipski opened #86
  • Jul 03 19:24
    scala-steward opened #85
  • Jun 30 20:58
    scala-steward opened #84
  • Jun 29 14:59
    scala-steward opened #83
  • Jun 28 00:55
    scala-steward opened #82
  • Jun 26 19:15

    shadaj on master

    Update sbt-mdoc to 2.2.3 (#81) (compare)

  • Jun 26 19:15
    shadaj closed #81
  • Jun 25 16:58
    scala-steward opened #81
Eric K Richardson
@ekrich
Ok, I see thanks for the info. That makes sense.
jhegedus42
@jhegedus42
hi !
I get the following error :
image.png
here is the code:
any idea how can I fix this ?
Jozsefs-MacBook-Pro:scalapytest joco$ sbt
[info] Loading global plugins from /Users/joco/.sbt/1.0/plugins
[info] Loading project definition from /Users/joco/dev/ml/scalapy/test01/github/scalapytest/project
[info] Loading settings for project root from build.sbt ...
[info] Set current project to ScalaPyTest (in build file:/Users/joco/dev/ml/scalapy/test01/github/scalapytest/)
[info] sbt server started at local:///Users/joco/.sbt/1.0/server/3f6c4d00bf97ae0f90ba/sock
sbt:ScalaPyTest> run
[info] running (fork) example.Hello
[info] code runs
[error] Exception in thread "main" java.lang.Error: java.lang.UnsatisfiedLinkError: jep.MainInterpreter.initializePython([Ljava/lang/String;)V
[error]     at jep.MainInterpreter.initialize(MainInterpreter.java:183)
[error]     at jep.MainInterpreter.getMainInterpreter(MainInterpreter.java:97)
[error]     at jep.Jep.<init>(Jep.java:232)
[error]     at jep.Jep.<init>(Jep.java:228)
[error]     at me.shadaj.scalapy.py.JepInterpreter.<init>(JepInterpreter.scala:9)
[error]     at me.shadaj.scalapy.py.Platform$.newInterpreter(Platform.scala:5)
[error]     at me.shadaj.scalapy.py.package$.interpreter(package.scala:11)
[error]     at me.shadaj.scalapy.py.Writer$$anon$11.write(Writer.scala:75)
[error]     at me.shadaj.scalapy.py.Any$.from(Any.scala:28)
[error]     at example.Hello$.delayedEndpoint$example$Hello$1(Hello.scala:6)
[error]     at example.Hello$delayedInit$body.apply(Hello.scala:3)
[error]     at scala.Function0.apply$mcV$sp(Function0.scala:39)
[error]     at scala.Function0.apply$mcV$sp$(Function0.scala:39)
[error]     at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:17)
[error]     at scala.App.$anonfun$main$1$adapted(App.scala:80)
[error]     at scala.collection.immutable.List.foreach(List.scala:392)
[error]     at scala.App.main(App.scala:80)
[error]     at scala.App.main$(App.scala:78)
[error]     at example.Hello$.main(Hello.scala:3)
[error]     at example.Hello.main(Hello.scala)
[error] Caused by: java.lang.UnsatisfiedLinkError: jep.MainInterpreter.initializePython([Ljava/lang/String;)V
[error]     at jep.MainInterpreter.initializePython(Native Method)
[error]     at jep.MainInterpreter.access$100(MainInterpreter.java:64)
[error]     at jep.MainInterpreter$1.run(MainInterpreter.java:140)
[error] Nonzero exit code returned from runner: 1
[error] (Compile / run) Nonzero exit code returned from runner: 1
[error] Total time: 1 s, completed Dec 24, 2019 11:51:02 PM
sbt:ScalaPyTest>
Eric K Richardson
@ekrich
Seems like something is wrong with the python install or something.
Maxime Kjaer
@MaximeKjaer
@jhegedus42 what version of Jep have you installed? You can try again with pip install jep==3.8.2
Eric K Richardson
@ekrich
I haven't tried honestly, but there is some discussion above that could help.
Shadaj Laddad
@shadaj
looks like a version mismatch as @MaximeKjaer noted, since some shared library is being loaded but there's a JNI definition that's missing
jhegedus42
@jhegedus42
thanks !
nice
this works
with 3.8.2 jep
thanks
image.png
how about jupyter ?
image.png
Maxime Kjaer
@MaximeKjaer
@jhegedus42 This looks like Ammonite doesn't fetch from the right resolver? I can't see anything else than 0.2.0 on Maven https://search.maven.org/artifact/me.shadaj/scalapy_2.12.
Ah, it could just be that you should be loading scalapy-core:0.3.0, which is on Maven https://search.maven.org/artifact/me.shadaj/scalapy-core_2.12
Btw, @shadaj, I think deployments have been broken since this build https://travis-ci.com/shadaj/scalapy/builds/142068062. It complains about:
[error] Multiple repositories are found:
[error] [meshadaj-1361] status:open, profile:me.shadaj(efba3c89dc27) description: Implicitly created (auto staging).
[error] [meshadaj-1362] status:open, profile:me.shadaj(efba3c89dc27) description: Implicitly created (auto staging).
[error] [meshadaj-1363] status:open, profile:me.shadaj(efba3c89dc27) description: Implicitly created (auto staging).
[error] [meshadaj-1364] status:open, profile:me.shadaj(efba3c89dc27) description: Implicitly created (auto staging).
[error] [meshadaj-1365] status:open, profile:me.shadaj(efba3c89dc27) description: Implicitly created (auto staging).
[error] [meshadaj-1366] status:open, profile:me.shadaj(efba3c89dc27) description: Implicitly created (auto staging).
[error] [meshadaj-1367] status:open, profile:me.shadaj(efba3c89dc27) description: Implicitly created (auto staging).
[error] [meshadaj-1368] status:open, profile:me.shadaj(efba3c89dc27) description: Implicitly created (auto staging).
[error] [meshadaj-1369] status:open, profile:me.shadaj(efba3c89dc27) description: Implicitly created (auto staging).
[error] [meshadaj-1370] status:open, profile:me.shadaj(efba3c89dc27) description: Implicitly created (auto staging).
[error] [meshadaj-1371] status:open, profile:me.shadaj(efba3c89dc27) description: Implicitly created (auto staging).
[error] [meshadaj-1372] status:open, profile:me.shadaj(efba3c89dc27) description: Implicitly created (auto staging).
[error] [meshadaj-1373] status:open, profile:me.shadaj(efba3c89dc27) description: Implicitly created (auto staging).
[error] Specify one of the repository ids in the command line
[error] java.lang.IllegalStateException: Found multiple staging repositories
[error] Use 'last' for the full log.
Maxime Kjaer
@MaximeKjaer
This means that nothing has been published since 0.3.0+2. I'd like to submit a PR adding cross-building to Scala 2.13 to scalapy-numpy, but this is a blocker for that.
Maxime Kjaer
@MaximeKjaer
I have very little experience or knowledge of how publishing Scala projects works, but from what I can gather it seems (from this issue xerial/sbt-sonatype#78) that updating sbt-sonatype could help
jhegedus42
@jhegedus42
@MaximeKjaer hmmm, thanks, i guess for the time being I will be just playing around in colab... google, that is pretty neat ... and easy ...
Shadaj Laddad
@shadaj
oh thanks @MaximeKjaer for catching that, I'll clear out the Sonatype staging and re-publish for now
Maxime Kjaer
@MaximeKjaer
@shadaj Could you perhaps publish ScalaPy manually for now? I have scalapy-numpy cross building to 2.13 in this commit MaximeKjaer/scalapy-numpy@d8805dc but would need version 0.3.0+13-0c255e53 of ScalaPy on Maven in order to make builds reproducible
Shadaj Laddad
@shadaj
@MaximeKjaer cleared out staging repositories and restarted the build, it should be out soon!
Shadaj Laddad
@shadaj
I think I identified the source of the publish failures, just added a new commit that's publishing now
Shadaj Laddad
@shadaj
0.3.0+15-598682f0 has been released!
Maxime Kjaer
@MaximeKjaer
That's fantastic, thank you so much! I'm submitting a PR to scalapy-numpy now :)
Rohan Sircar
@rohan-sircar
Hi, does scalapy work with anaconda environments?
Shadaj Laddad
@shadaj
@nova999 yep, Anaconda correctly overrides the python3-config utility so as long as you use the javaOptions += s"-Djna.library.path=${"python3-config --prefix".!!.trim}/lib" config it should pick up the current environment's Python
Rohan Sircar
@rohan-sircar
Thank you :)
Rohan Sircar
@rohan-sircar
So I got a simple pyopencl program working in scalapy. With typings it could be a good alternative to the seemingly abandoned ScalaCL.
Jorge
@jvican
nice! consider working on those typings then, i'm sure they'd be well received ;)
Rohan Sircar
@rohan-sircar
I did intend to. I was studying the numpy facade. Some things are still not quite clear to me. But I suppose I could lay a groundwork to begin with.
Maxime Kjaer
@MaximeKjaer
Based on previous discussion in this channel, I said I'd ping both @shadaj and @ekrich once I open sourced the shape-safe TensorFlow library for Dotty. So with a little bit of a delay, here's a link! https://github.com/MaximeKjaer/tf-dotty
Maxime Kjaer
@MaximeKjaer
@shadaj I copied the code of scalapy-tensorflow into a scalapy-tensorflow module. This is just the easiest way that I found to augment the existing typings while working on tf-dotty. I would be happy to contribute them back to the main scalapy-tensorflow project, once I've typed a full TF module
Shadaj Laddad
@shadaj
@MaximeKjaer this is very exciting! Took a quick glance over the website just now, looks very nice! Also was cool to see the thought process that went into the implementation
Mark Melton
@melton1968
I'm having trouble getting Intellij to work correctly for the cross build. It is very grumpy about CPythonAPI and Platform when I am visiting any files in the shared sources. Anyone else have / solved this issue?
Shadaj Laddad
@shadaj
@melton1968 one strategy could be to just exclude either the JVM or Native sources so that IntelliJ doesn't get confused about the duplicate definitions. I've been using Metals with VS Code these days, which works pretty well for cross builds, though it doesn't have as many of the refactoring abilities as IntelliJ yet.
Piotr Gawryś
@Avasil

Which jep version should be used for 0.3.0+17-2bfe86de ? I tried both 3.8.2 and 3.9.0 and both result in following error:

[error] Native library (darwin/libpython3.7m.dylib) not found in resource path (/var/folders/lr/x_gb25v56fj3_80kyyk17lhc0000gn/T/sbt_dd5aa2c1/job-2/target/90ed79f3/562ee7c8/scala-python-example_2.12-1.0.jar:/var/folders/lr/x_gb25v56fj3_80kyyk17lhc0000gn/T/sbt_dd5aa2c1/target/567bf6c2/9e2a823f/jep-3.9.0.jar:/var/folders/lr/x_gb25v56fj3_80kyyk17lhc0000gn/T/sbt_dd5aa2c1/target/bc8bc145/464fe2e1/scala-library.jar:/var/folders/lr/x_gb25v56fj3_80kyyk17lhc0000gn/T/sbt_dd5aa2c1/target/aa0ecde6/28d0208b/scalapy-macros_2.12-0.3.0%2B17-2bfe86de.jar:/var/folders/lr/x_gb25v56fj3_80kyyk17lhc0000gn/T/sbt_dd5aa2c1/target/1a583173/abb78bc4/jna-5.4.0.jar:/var/folders/lr/x_gb25v56fj3_80kyyk17lhc0000gn/T/sbt_dd5aa2c1/target/26ba9c6e/34d55877/scalapy-core_2.12-0.3.0%2B17-2bfe86de.jar:/var/folders/lr/x_gb25v56fj3_80kyyk17lhc0000gn/T/sbt_dd5aa2c1/target/2ce76994/5a36165d/scala-reflect.jar)
[error]  at com.sun.jna.NativeLibrary.loadLibrary(NativeLibrary.java:302)
  [error]  at com.sun.jna.NativeLibrary.getInstance(NativeLibrary.java:455)
  [error]  at com.sun.jna.Native.register(Native.java:1722)
  [error]  at com.sun.jna.Native.register(Native.java:1443)
  [error]  at me.shadaj.scalapy.py.CPythonAPIInterface.<init>(CPythonAPI.scala:6)
  [error]  at me.shadaj.scalapy.py.CPythonAPI$.<init>(CPythonAPI.scala:74)
  [error]  at me.shadaj.scalapy.py.CPythonAPI$.<clinit>(CPythonAPI.scala)
  [error]  at me.shadaj.scalapy.py.CPythonInterpreter$.<init>(CPythonInterpreter.scala:4)
  [error]  at me.shadaj.scalapy.py.CPythonInterpreter$.<clinit>(CPythonInterpreter.scala)
  [error]  at me.shadaj.scalapy.py.package$.<init>(package.scala:12)
[error]  at me.shadaj.scalapy.py.package$.<clinit>(package.scala)
[error]  at com.sushant.example.PiotrekExample$.delayedEndpoint$com$sushant$example$PiotrekExample$1(PiotrekExample.scala:13)
  [error]  at com.sushant.example.PiotrekExample$delayedInit$body.apply(PiotrekExample.scala:8)
  [error]  at scala.Function0.apply$mcV$sp(Function0.scala:39)
  [error]  at scala.Function0.apply$mcV$sp$(Function0.scala:39)
  [error]  at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:17)
  [error]  at scala.App.$anonfun$main$1$adapted(App.scala:80)
  [error]  at scala.collection.immutable.List.foreach(List.scala:392)
  [error]  at scala.App.main(App.scala:80)
  [error]  at scala.App.main$(App.scala:78)
  [error]  at com.sushant.example.PiotrekExample$.main(PiotrekExample.scala:8)
  [error]  at com.sushant.example.PiotrekExample.main(PiotrekExample.scala)
  [error]  Suppressed: java.lang.UnsatisfiedLinkError: dlopen(libpython3.7m.dylib, 9): image not found
  [error]   at com.sun.jna.Native.open(Native Method)
  [error]   at com.sun.jna.NativeLibrary.loadLibrary(NativeLibrary.java:191)
  [error]   ... 21 more

The same code works with scalapy 0.3.0. Any idea what could be missing from my setup?
It fails on val tf = py.module("tensorflow")

Shadaj Laddad
@shadaj
@Avasil the new ScalaPy release doesn't need Jep anymore, it uses your existing Python installation directly. You can add your installation to the path with javaOptions in Test += s"-Djna.library.path=${"python3-config --prefix".!!.trim}/lib" in your SBT settings
Piotr Gawryś
@Avasil
Thank you, it's working fine now!
Piotr Gawryś
@Avasil

Is there any example of a class which can be created by the user calling constructor directly? e.g.

@py.native class Conv2D(
                       filters: Int,
                       kernel_size: py.|[Int, (Int, Int)],
                       strides: py.|[Int, (Int, Int)] = (1, 1),
                       padding: String = "valid",
                       data_format: String = "channels_last",
                       dilation_rate: py.|[Int, (Int, Int)] = (1, 1),
                       activation: Option[String] = None,
                       use_bias: Boolean = true,
                       kernel_initializer: String = "glorot_uniform",
                       bias_initializer: String = "zeros",
                       kernel_regularizer: Option[String] = None,
                       bias_regularizer: Option[String] = None,
                       activity_regularizer: Option[String] = None,
                       kernel_constraint: Option[String] = None,
                       bias_constraint: Option[String] = None
                       ) extends py.Object

I would like to be able to call it like Conv2D(32, kernel_size = (3, 3)) but I'm not sure how to provide PyValue

Shadaj Laddad
@shadaj
@Avasil there's no class equivalent yet in ScalaPy (this is on my mind but will likely require fairly complex macro transformations), so you can just use objects where the apply method calls a method facade
Piotr Gawryś
@Avasil

@shadaj Thanks, I will try it. Do you mean something like:

@py.native abstract class MyClass(x: String) extends py.Object

@py.native object MyClass extends py.Object {
  def apply(x: String): MyClass = py.native
}

Although I need to provide Reader[MyClass] and PyValue for the object itself
I have some classes working if there is a single parameter (val value: PyValue) but I have trouble to do it if there are more

Shadaj Laddad
@shadaj

@Avasil MyClass would have to be a trait with x being defined as a method inside. Right now, ScalaPy also doesn't support top-level objects as facades, so you would need to define a separate trait which contains the constructor which gets a bit messy. What I'd suggest is defining the constructor inside the facade for the containing module, similar to how https://github.com/shadaj/scalapy-tensorflow/blob/master/src/main/scala/me/shadaj/scalapy/tensorflow/TensorFlow.scala#L52 defines the constructor for Variable.

This is not too elegant unfortunately, but improving this is my top priority after stabilizing the new interpreter backend so should improve soon!