Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Sep 18 01:09
    scala-steward opened #222
  • Sep 15 15:43
    Ang9876 commented #195
  • Sep 15 15:42
    Ang9876 synchronize #195
  • Sep 15 00:26
    scala-steward closed #185
  • Sep 15 00:26
    scala-steward commented #185
  • Sep 15 00:26
    scala-steward opened #221
  • Sep 04 14:34
    Ang9876 commented #195
  • Sep 03 14:57
    Ang9876 synchronize #195
  • Sep 03 14:28
    Ang9876 synchronize #195
  • Sep 03 14:08
    Ang9876 synchronize #195
  • Sep 03 14:05
    Ang9876 synchronize #195
  • Sep 03 12:04
    Ang9876 synchronize #195
  • Sep 01 11:32
    scala-steward closed #193
  • Sep 01 11:32
    scala-steward commented #193
  • Sep 01 11:32
    scala-steward opened #220
  • Sep 01 00:51

    dependabot[bot] on npm_and_yarn

    (compare)

  • Sep 01 00:51
    dependabot[bot] closed #215
  • Sep 01 00:51
    dependabot[bot] commented #215
  • Sep 01 00:51
    dependabot[bot] labeled #219
  • Sep 01 00:51
    dependabot[bot] opened #219
lasimon
@lasimon
@shadaj build.sbt:
fork := true
import scala.sys.process._
javaOptions += s"-Djava.library.path=${"python3-config --ldflags".!! + "/lib"}"
// javaOptions += "-Djna.library.path=/usr/local/opt/python@3.7/Frameworks/Python.framework/Versions/3.7/lib"
[info] running (fork) wumpusWorld.DQNWorld
[error] Exception in thread "main" java.lang.UnsatisfiedLinkError: Unable to load library 'python3':
[error] libpython3.so: cannot open shared object file: No such file or directory
[error] libpython3.so: cannot open shared object file: No such file or directory
[error] Native library (linux-x86-64/libpython3.so) not found in resource path ([file:/tmp/sbt_82d1e13c/job-1/target/55c69968/832b95f1/wumpusworld_2.12-0.1.jar, file:/tmp/sbt_82d1e13c/target/dc7f77ac/b41586e0/scala-library-2.12.12.jar, file:/tmp/sbt_82d1e13c/target/6a07f7c0/c6806aaf/refined_2.12-0.9.15.jar, file:/tmp/sbt_82d1e13c/target/f631ecf1/e854b270/figaro_2.12-5.0.0.0.jar, file:/tmp/sbt_82d1e13c/target/da834eb9/baaae62c/scalapy-core_2.12-0.4.0.jar, file:/tmp/sbt_82d1e13c/target/95d6cb8d/f93cbdeb/scala-compiler-2.12.12.jar, file:/tmp/sbt_82d1e13c/target/17f40488/b6b93b90/scala-reflect-2.12.12.jar, file:/tmp/sbt_82d1e13c/target/df126de0/b50eae97/shapeless_2.12-2.3.3.jar, file:/tmp/sbt_82d1e13c/target/906e7fb1/01c9646c/scala-xml_2.12-1.3.0.jar, file:/tmp/sbt_82d1e13c/target/b81ef6b7/795641c2/asm-3.3.1.jar, file:/tmp/sbt_82d1e13c/target/a5cd33ac/662f0474/commons-math3-3.3.jar, file:/tmp/sbt_82d1e13c/target/0421973b/0603d5cb/jsci-1.2.jar, file:/tmp/sbt_82d1e13c/target/d88c303f/81fcec1e/akka-actor_2.12-2.4.18.jar, file:/tmp/sbt_82d1e13c/target/ea9de1d2/c7269f1b/breeze_2.12-0.13.1.jar, file:/tmp/sbt_82d1e13c/target/29f545e8/0f3e3243/argonaut_2.12-6.2.jar, file:/tmp/sbt_82d1e13c/target/846ef561/db21d5e7/prefuse-beta-20071021.jar, file:/tmp/sbt_82d1e13c/target/7bc43030/382b5f00/scala-swing_2.12-2.0.0.jar, file:/tmp/sbt_82d1e13c/target/8af72ca7/8c48d6d7/scalapy-macros_2.12-0.4.0.jar, file:/tmp/sbt_82d1e13c/target/2b3b90b1/f37928f3/jna-5.5.0.jar, file:/tmp/sbt_82d1e13c/target/f7ef733c/1d4f9fbf/macro-compat_2.12-1.1.1.jar, file:/tmp/sbt_82d1e13c/target/496371c0/d0443e0c/lpsolve-2003-11-03.jar, file:/tmp/sbt_82d1e13c/target/7546bb71/2faab0fb/mathml-dom-java-2.0.jar, file:/tmp/sbt_82d1e13c/target/4399f739/4f47a0d5/xercesImpl-2.8.1.jar, file:/tmp/sbt_82d1e13c/target/c5d711c7/d61d4b86/config-1.3.0.jar, file:/tmp/sbt_82d1e13c/target/f679673a/b69204cf/scala-java8-compat_2.12-0.8.0.jar, file:/tmp/sbt_82d1e13c/target/84087de9/b5315cdf/breeze-macros_2.12-0.13.1.jar, file:/tmp/sbt_82d1e13c/target/419dc0c7/453aa781/core-1.1.2.jar, file:/tmp/sbt_82d1e13c/target/2ac29a43/764709f2/arpack_combined_all-0.1.jar, file:/tmp/sbt_82d1e13c/target/47c97988/b010f01b/opencsv-2.3.jar, file:/tmp/sbt_82d1e13c/target/cfb0976e/6518a453/jtransforms-2.4.0.jar, file:/tmp/sbt_82d1e13c/target/e10d518a/14229e37/spire_2.12-0.13.0.jar, file:/tmp/sbt_82d1e13c/target/cb619cd6/dd21cb48/slf4j-api-1.7.5.jar, file:/tmp/sbt_82d1e13c/target/4a189398/a8781ea4/xml-apis-1.3.03.jar, file:/tmp/sbt_82d1e13c/target/f8a5c36f/ff1b5461/junit-4.8.2.jar, file:/tmp/sbt_82d1e13c/target/84ed269c/8778977b/spire-macros_2.12-0.13.0.jar, file:/tmp/sbt_82d1e13c/target/b67dc7de/2334303a/machinist_2.12-0.6.1.jar])
[error] at com.sun.jna.NativeLibrary.loadLibrary(NativeLibrary.java:302)
[error] at com.sun.jna.NativeLibrary.getInstance(NativeLibrary.java:455)
[error] at com.sun.jna.Native.register(Native.java:1722)
[error] at com.sun.jna.Native.register(Native.java:1443)
[error] at me.shadaj.scalapy.interpreter.CPythonAPIInterface.$anonfun$new$2(CPythonAPI.scala:13)
[error] at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
[error] at scala.util.Try$.apply(Try.scala:213)
[error] at me.shadaj.scalapy.interpreter.CPythonAPIInterface.$anonfun$new$1(CPythonAPI.scala:13)
[error] at me.shadaj.scalapy.interpreter.CPythonAPIInterface.$anonfun$new$1$adapted(CPythonAPI.scala:13)
[error] at scala.collection.LinearSeqOptimized.find(LinearSeqOptimized.scala:115)
[error] at scala.collection.LinearSeqOptimized.find$(LinearSeqOptimized.scala:112)
[error] at scala.collection.immutable.List.find(List.scala:91)
[error] at me.shadaj.scalapy.interpreter.CPythonAPIInterface.<init>(CPythonAPI.scala:13)
[error] at me.shadaj.scalapy.interpreter.CPythonAPI$.<init>(CPythonAPI.scala:95

python3-config --ldflags
-L/usr/lib/python3.6/config-3.6m-x86_64-linux-gnu -L/usr/lib -lpython3.6m -lpthread -ldl -lutil -lm -Xlinker -export-dynamic -Wl,-O1 -Wl,-Bsymbolic-functions
lasimon
@lasimon
larrysimon@inspiron:/usr/lib/python3.6/config-3.6m-x86_64-linux-gnu$ cd /usr/lib/python3.6/config-3.6m-x86_64-linux-gnu
larrysimon@inspiron:/usr/lib/python3.6/config-3.6m-x86_64-linux-gnu$ ls
config.c install-sh libpython3.6m-pic.a libpython3.6.so makesetup python.o Setup.config
config.c.in libpython3.6m.a libpython3.6m.so Makefile python-config.py Setup Setup.local
Shadaj Laddad
@shadaj
Ah, looks like you have Python 3.6, which ScalaPy doesn't load by default (though perhaps it should). For now, could you try adding the environment variable SCALAPY_PYTHON_LIBRARY=python3.6 before launching SBT?
lasimon
@lasimon
larrysimon@inspiron:~/IdeaProjects/wumpus$ echo $SCALAPY_PYTHON_LIBRARY
python3.6
then sbt run
... same error
BTW what I'm doing is a reference implementation of a solution to the classic Russell & Norvig Wumpus World using Deep RL on pyTorch. A Python version is an assignment I set for my Intelligent Agents class at U Toronto Continuing Education.
lasimon
@lasimon
I also have Python 3.7 installed if that's a help (or a conflict).
lasimon
@lasimon
Tried it on my mac with and without setting the env variable and same result (libpython3.dylib image not found). Mac is also Python 3.6.
lasimon
@lasimon
Same thing on another mac running python 3.8
Shadaj Laddad
@shadaj
@lasimon oh, there's actually a big bug in the docs! You should be setting javaOptions += s"-Djna.library.path=${"python3-config --prefix".!!.trim}/lib" (--prefix, instead of --ldflags). I'll update the docs to fix this shortly
lasimon
@lasimon
@shadaj Hmm. Still not working. config --prefix returns /usr. The .so is in /usr/lib/python3.7/config-3.7m-x86_64-linux-gnu/. The directory has two files in it: libpython3.7m.so libpython3.7.so
Shadaj Laddad
@shadaj
hmm, it seems like the python3-config is pointing to a system installation in /usr/lib instead of your python3.7 installation. Do you have a python3.7-config command available?
lasimon
@lasimon
I was able to install it using "sudo apt install python3.7-dev". If I now do "python3.7-config --prefix" it still replies with /usr.
Brandon Stilson
@bbstilson
Hey folks. I'm having trouble running the hello world, likely because my python environment is basically a superfund site.
scalapy-test:>python3-config --ldflags
-L/Users/brandons/.pyenv/versions/3.8.5/lib/python3.8/config-3.8-darwin -lintl -ldl -framework CoreFoundation
When I run the getting started example, i get the following stack trace:
sbt:scalapy-test> run
[info] running (fork) Main 
[error] Exception in thread "main" java.lang.UnsatisfiedLinkError: Unable to load library 'python3':
[error] dlopen(libpython3.dylib, 9): image not found
[error] dlopen(libpython3.dylib, 9): image not found
[error] Native library (darwin/libpython3.dylib) not found in resource path (TOO LONG)
[error]         at com.sun.jna.NativeLibrary.loadLibrary(NativeLibrary.java:302)
[error]         at com.sun.jna.NativeLibrary.getInstance(NativeLibrary.java:455)
[error]         at com.sun.jna.Native.register(Native.java:1722)
[error]         at com.sun.jna.Native.register(Native.java:1443)
[error]         at me.shadaj.scalapy.interpreter.CPythonAPIInterface.$anonfun$new$2(CPythonAPI.scala:13)
[error]         at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
[error]         at scala.util.Try$.apply(Try.scala:210)
[error]         at me.shadaj.scalapy.interpreter.CPythonAPIInterface.$anonfun$new$1(CPythonAPI.scala:13)
[error]         at me.shadaj.scalapy.interpreter.CPythonAPIInterface.$anonfun$new$1$adapted(CPythonAPI.scala:13)
[error]         at scala.collection.immutable.List.find(List.scala:413)
[error]         at me.shadaj.scalapy.interpreter.CPythonAPIInterface.<init>(CPythonAPI.scala:13)
[error]         at me.shadaj.scalapy.interpreter.CPythonAPI$.<init>(CPythonAPI.scala:95)
[error]         at me.shadaj.scalapy.interpreter.CPythonAPI$.<clinit>(CPythonAPI.scala:95)
[error]         at me.shadaj.scalapy.interpreter.CPythonInterpreter$.<clinit>(CPythonInterpreter.scala:9)
[error]         at me.shadaj.scalapy.py.package$.<clinit>(package.scala:15)
[error]         at Main$.main(Main.scala:13)
[error]         at Main.main(Main.scala)
[error]         Suppressed: java.lang.UnsatisfiedLinkError: dlopen(libpython3.dylib, 9): image not found
[error]                 at com.sun.jna.Native.open(Native Method)
[error]                 at com.sun.jna.NativeLibrary.loadLibrary(NativeLibrary.java:191)
[error]                 ... 16 more
[error]         Suppressed: java.lang.UnsatisfiedLinkError: dlopen(libpython3.dylib, 9): image not found
[error]                 at com.sun.jna.Native.open(Native Method)
[error]                 at com.sun.jna.NativeLibrary.loadLibrary(NativeLibrary.java:204)
[error]                 ... 16 more
[error]         Suppressed: java.io.IOException: Native library (darwin/libpython3.dylib) not found in resource path (TOO LONG)
[error]                 at com.sun.jna.Native.extractFromResourcePath(Native.java:1095)
[error]                 at com.sun.jna.NativeLibrary.loadLibrary(NativeLibrary.java:276)
[error]                 ... 16 more
[error] Nonzero exit code returned from runner: 1
[error] (Compile / run) Nonzero exit code returned from runner: 1
[error] Total time: 1 s, completed Nov 24, 2020 2:04:42 PM
sbt:scalapy-test>
this is my build.sbt
ThisBuild / scalaVersion := "2.13.3"

libraryDependencies += "me.shadaj" %% "scalapy-core" % "0.4.0"

fork := true

import scala.sys.process._
javaOptions += s"-Djava.library.path=${"python3-config --ldflags".!! + "/lib"}"
Brandon Stilson
@bbstilson
actually I made a docker container and ran into the same problem
Shadaj Laddad
@shadaj
@bbstilson could you try modifying your SBT build to use python3-config --prefix instead of python3-config --ldflags? There was a bug in the JVM setup instructions on the website.
Brandon Stilson
@bbstilson
I got the same error sadly. I'll open an issue.
Travis
@travissarles
I'm getting the same issue as @bbstilson
Mac OS 10.15, Scala 2.13.4, scalapy-core 0.4.0
Jakub Chrobasik
@jachro
We solved that issue by specifying:
  • VM Option -Djna.library.path=/usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib which is effectively -Djna.library.path=${"python3-config --prefix".!!.trim}/lib
  • environment variable: SCALAPY_PYTHON_LIBRARY = python3.8
    I hope that helps someone :)
1 reply
Peter Jansen
@peterjansen_ai_twitter
I have a similar issue -- complicated slightly by using a conda environment. I posted my (current) solution to the github issue: shadaj/scalapy#125
Though it runs on the command line, it's still throwing the issue when running from intellij, which I don't fully appreciate why (they're using the same build.sbt, etc -- perhaps it's something with an environment variable)
Peter Jansen
@peterjansen_ai_twitter
Is there a fast, cannonical way of packing a multidimensional scala array (e.g. of Ints) into a numpy array?
Peter Jansen
@peterjansen_ai_twitter
image.png
I've put together a fairly hacky way of doing things based off reading the tests, but it's not fast (the array is only ~300x300x3 and takes 500msec to convert) -- but, interestingly, the time appears to be non deterministic (it ranges between 400-4000msec):
Even with the same array content
Peter Jansen
@peterjansen_ai_twitter
Interesting... it also segfaults when using matplotlib to plot that small 300x300x3 image, but not right away. After ~20-30 attempts in a loop, it slows down, then eventually segfaults. I seem to be able to force a segfault by calling System.gc() right before the call to matplotlib to plot the image. I wonder if the converted n-d array is being prematurely garbage collected somehow?
Pedro Luis
@PedroCorreiaLuis
Screenshot 2020-12-18 at 12.58.17.png
I am struggling a lot, can someone help me what should I put in the sbt, thanks!
Shadaj Laddad
@shadaj
@peterjansen_ai_twitter unfortunately converting multidimensional arrays requires a bunch of maps right now, though I hope to improve that by making toPythonCopy/toPythonProxy aware of such types; the segfault is surprising, are you running on the JVM?
@PedroCorreiaLuis this is a bug that will be fixed in the next release of ScalaPy, you can add the environment variable SCALAPY_PYTHON_LIBRARY=python3.6m (replacing 3.6 with the appropriate Python version) to fix it for now
Kamil Kloch
@kamilkloch
Thanks for fixing python library fallback issues. Any chance for a bugfix release? :)
Shadaj Laddad
@shadaj
@kamilkloch v0.4.1 should be out very soon!
Kamil Kloch
@kamilkloch
@shadaj Thanks! I take this opportunity to thank you for all your work within the Scala community. Slinky, ScalaPy, who knows what brings 2021 ;) U da man.
Shadaj Laddad
@shadaj
@kamilkloch thanks so much, that means a lot! it's been very exciting to see the projects being helpful to the community!
Kamil Kloch
@kamilkloch

Hello all, the folowing snippet fails to compile:

import me.shadaj.scalapy.numpy.NumPy
import me.shadaj.scalapy.py

val np = py.module("numpy").as[NumPy]

could not find implicit value for evidence parameter of type me.shadaj.scalapy.readwrite.Reader[me.shadaj.scalapy.numpy.NumPy]
  val np = py.module("numpy").as[NumPy]

Any hints?

Also, py.module("numpy") compiles despite missing implicit jep: Jep, whereas implicitly[Jep] fails to compile.
Kamil Kloch
@kamilkloch
Update: it looks that the impiict jep instance is required by the (legacy?) artifact https://mvnrepository.com/artifact/me.shadaj/scalapy_2.12/0.2.0 sucked in with the scalapy-numpy facade.
Shadaj Laddad
@shadaj
@kamilkloch ah, yeah, I need to re-release those bindings for the new ScalaPy release
Anatolii Kmetiuk
@anatoliykmetyuk

Hi! I was following the Getting Started https://scalapy.dev/docs/ instructions, but when I tried running the code example, I got:

[error] Exception in thread "main" java.lang.ExceptionInInitializerError
[error]     at me.shadaj.scalapy.interpreter.CPythonInterpreter$.<clinit>(CPythonInterpreter.scala:9)
[error]     at me.shadaj.scalapy.py.package$.<clinit>(package.scala:15)
[error]     at Main$package$.hello(Main.scala:5)
[error]     at hello.main(Main.scala:4)
[error] Caused by: java.lang.Exception: Unable to locate Python library, tried python3, python3.7, python3.7m
[error]     at me.shadaj.scalapy.interpreter.CPythonAPIInterface.$anonfun$new$2(CPythonAPI.scala:19)
[error]     at scala.Option.getOrElse(Option.scala:201)
[error]     at me.shadaj.scalapy.interpreter.CPythonAPIInterface.<init>(CPythonAPI.scala:19)
[error]     at me.shadaj.scalapy.interpreter.CPythonAPI$.<init>(CPythonAPI.scala:102)
[error]     at me.shadaj.scalapy.interpreter.CPythonAPI$.<clinit>(CPythonAPI.scala:102)
[error]     ... 4 more

Python was installed via homebrew, the OS is MacOS Catalina. The output of python3-config --configdir is /usr/local/opt/python@3.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/config-3.9-darwin. When I run ls -1 $(python3-config --configdir), I get:

Makefile
Setup
Setup.local
config.c
config.c.in
install-sh
libpython3.9.a
libpython3.9.dylib
makesetup
python-config.py
python.o

So probably javaOptions += s"-Djava.library.path=${"python3-config --configdir".!!.trim}/lib" in the SBT settings cannot locate lib. How do I fix this?

Shadaj Laddad
@shadaj
@anatoliykmetyuk try adding SCALAPY_PYTHON_LIBRARY=python3.9 to your build to override the python version. I'll be publishing an update to the docs with guidance on this situation in just a bit!
Anatolii Kmetiuk
@anatoliykmetyuk
I tried doing so, this did not solve the problem but the error message got changed to include that version of python:
Caused by: java.lang.Exception: Unable to locate Python library, tried python3.9, python3, python3.7, python3.7