Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Jan 20 10:39
    jaroslawk commented #146
  • Jan 15 13:28
    dependabot[bot] labeled #255
  • Jan 15 13:28
    dependabot[bot] opened #255
  • Jan 15 13:28

    dependabot[bot] on npm_and_yarn

    Bump follow-redirects from 1.14… (compare)

  • Jan 15 05:38
    dependabot[bot] labeled #254
  • Jan 15 05:38
    dependabot[bot] opened #254
  • Jan 15 05:38

    dependabot[bot] on npm_and_yarn

    Bump shelljs from 0.8.4 to 0.8.… (compare)

  • Jan 14 20:34

    shadaj on seq-conversions-companion-object

    (compare)

  • Jan 14 20:34

    shadaj on main

    Move Seq element implicit deriv… (compare)

  • Jan 14 20:34
    shadaj closed #253
  • Jan 14 20:34
    shadaj closed #252
  • Jan 14 19:38
    shadaj milestoned #253
  • Jan 14 19:38
    shadaj opened #253
  • Jan 14 19:38

    shadaj on seq-conversions-companion-object

    Move Seq element implicit deriv… (compare)

  • Jan 14 19:27
    shadaj milestoned #252
  • Jan 14 19:27
    shadaj labeled #252
  • Jan 14 19:27
    shadaj commented #252
  • Jan 14 17:22
    nikololiahim commented #251
  • Jan 14 17:22
    nikololiahim commented #251
  • Jan 14 13:06
    nikololiahim commented #251
BISHNU SHANKAR
@bishnushankar95_twitter
It will be helpful if the answer has a python function where I can access the params
scalavision
@scalavision
hello everyone, does it exist any scalapy wrapper for numpy, pytorch etc? Maybe there exists something with scala-native?
shadaj
@shadaj:matrix.org
[m]
@scalavision: at the moment, there aren't well-maintained static-type wrappers for those libraries (though this is something that will be improved soon with static type generation!); you can still use these libraries with the dynamically typed API in the meantime
Scala Native is more of a backend for ScalaPy, in terms of compatibility all Python libraries can be used the same way from either the JVM or Scala Native
1 reply
ScalaWilliam
@ScalaWilliam
Hi, how can you call 'del' in Scalapy?
shadaj
@shadaj:matrix.org
[m]
@ScalaWilliam: shadaj/scalapy#175 :)
ScalaWilliam
@ScalaWilliam
<3 :-) does it include 'del x' where x is a plain object not a dict? for what I'm doing I need to 'del' to delete a process (not my api!)
I did:
CPythonInterpreter.set("something", Writer.anyWriter[py.Any].write(something.as[py.Any]))
CPythonInterpreter.eval("delete something")
shadaj
@shadaj:matrix.org
[m]
ah, it doesn't include that; I'll have to think about how to implement that, since there isn't an equivalent Python API to that function, but the del method should be triggered by cleanup on a reference count decrement
it might be possible to offer an unsafe API on py.Any that lets you release a Python value even if it's still held by Scala code
ScalaWilliam
@ScalaWilliam
coool! :-)
shadaj
@shadaj:matrix.org
[m]
that would let you directly drop the reference instead of being at the mercy of the garbage collector
shadaj
@shadaj:matrix.org
[m]
@ScalaWilliam: just put up shadaj/scalapy#177 to resolve the missing APIs for your use-case!
ScalaWilliam
@ScalaWilliam
amazing @shadaj:matrix.org :-)
BISHNU SHANKAR
@bishnushankar95_twitter

I am trying to run scalapy from intellij in my window machine. I am using anaconda python with python version 3.8.5 . I am getting the below error
`java.lang.UnsatisfiedLinkError: Unable to load library 'python3.8m':
The specified module could not be found.

The specified module could not be found.

The specified module could not be found.

Native library (win32-x86-64/python3.8m.dll) not found in resource path `

shadaj
@shadaj:matrix.org
[m]
@bishnushankar95_twitter: did you make sure to activate the conda environment before launching SBT? (there can be some trickery required when using IDEs, which may launch SBT in the wrong environment); generally ScalaPy should work fine with conda, can you run python-config --ldflags to check where the libraries are installed?
BISHNU SHANKAR
@bishnushankar95_twitter
@shadaj:matrix.org I also tried the same by installing python on my machine. It is throwing the same exception. I tried python-config --ldflags but it is giving me python-config is not recognized as an internal or external command,operable program or batch file. error. Please help me with this
shadaj
@shadaj:matrix.org
[m]
@bishnushankar95_twitter: hmm, what platform are you running on and how did you install Python? I wonder if somehow you don't have the shared libraries
BISHNU SHANKAR
@bishnushankar95_twitter
I am running my scalapy program in a machine machine. I have downloaded python from the official website "https://www.python.org/downloads/release/python-385/"
Let me know the shared libraries that should be there before running scalapy program
@shadaj:matrix.org Please share the list of prerequisite libraries for scalapy
shadaj
@shadaj:matrix.org
[m]

@bishnushankar95_twitter: Hmm, can you try running

from distutils import sysconfig;
print(sysconfig.get_config_var("LIBDIR"))

in Python? that should tell you where the shared libraries are

kartikp4892
@kartikp4892
Is it possible to extend a python class/module using scalapy?
Something like class scala_class extends py_class
Second, I want to access python decorators on scala functions, is this possible with scalapy?
kartikp4892
@kartikp4892
So basically, I want to use base class and decorators from python and want to extend it in scala, the use python decorators on extended scala class functions.
BISHNU SHANKAR
@bishnushankar95_twitter
@shadaj:matrix.org I am getting None as the output of above code. Can you tell me what is the issue?
BISHNU SHANKAR
@bishnushankar95_twitter
@shadaj:matrix.org I have set the LIBDIR = '{pythonbase}/Lib/site-packages' but still i am getting the same error
shadaj
@shadaj:matrix.org
[m]
@kartikp4892: unfortunately you can't with high-level APIs right now, you should be able to create a subclass with https://docs.python.org/3/library/types.html
Decorators have a similar story, there's no native support in ScalaPy but you should be able to use Python APIs for them
@bishnushankar95_twitter: hmm, this makes it seem that you have a Python installation without the shared libraries; do you see any .so (linux) or .dylib (mac) files next to your Python installation?
BISHNU SHANKAR
@bishnushankar95_twitter
@shadaj:matrix.org I am using a window machine. There are many .ddl file in the python installation path
Nir Dunetz
@nirdunetz

Hi all, ive created python module and now I want to call it from scala using ScalaPy.
I started with call the module and the specific function:

 val testScalaPy = py.Module("my_module.test_feature")
 val res = testScalaPy.TestFeature.extract(payload = 1)

And its works perfectly.

Now, i would like to be able to pass case class to the extract function:

 case class Payload(value: Int) 
  val res = testScalaPy.TestFeature.extract(payload(100)) 

but im getting:

type mismatch;
found : com.riskified.poc.ScalaPyPocTest.Payload
required: me.shadaj.scalapy.py.Any
error after rewriting to feature.selectDynamic("TestFeature").applyDynamicNamed("extract")(scala.Tuple2("payload", Payload(200)))
possible cause: maybe a wrong Dynamic method signature?

Any ideas on how to implement writer/reader or if this ability exists already?

shadaj
@shadaj:matrix.org
[m]
@nirdunetz: unfortunately ScalaPy doesn't support converting case classes to Python right now; your best bet would be to create a dictionary and convert each field recursively (and maybe later convert this logic to be handled by a custom Writer derivation)
BISHNU SHANKAR
@bishnushankar95_twitter
@shadaj:matrix.org Any solutions on the issue i am facing?
Nir Dunetz
@nirdunetz
@shadaj:matrix.org Thanks for the quick response.
1.There is a plan to support case class conversation?
2.Wdym convert the logic to Writer? Writer that convert to map? Can you show an example how to convert it to map and than use the writer?
let's say I have:
 case class Payload(total: Int,  info: Info)
 case class Info(firstName: Name, lastName: String)    
 case class Name(nameVal: String)
shadaj
@shadaj:matrix.org
[m]

@nirdunetz: not in the near future, since unlike JavaScript it's not immediately clear exactly what a case class should map to in Python (an object? a dictionary?)

you'll need to use something like https://github.com/softwaremill/magnolia/tree/scala2, which can extract out info on which fields are in each case class and what are their types, and then write code using ScalaPy APIs to construct a Python dictionary (or whatever type you want) and add the fields to it by recursively converting them

basically, you can imagine putting together a writer for one of your classes as
def write(v: Payload): py.Object = {
  val out = py.global.dict()
  out.bracketUpdate("total", implicitly[Writer[Int]].write(out.total)
  ...
  out
the idea is that Magnolia lets you abstract things so that you don't have to write this by hand for every type
Nir Dunetz
@nirdunetz
@shadaj:matrix.org Many thanks!
limar63
@limar63
Hi! I am a bit rusty and just starting with getting comfortable with sbt/projects etc. Can someone tell what is the reason for error Native library (win32-x86-64/python3.dll) not found in resource path ? It happened when I tried to println the example given on the starting page of ScalaPy
val listLengthPython = py.Dynamic.global.len(List(1, 2, 3).toPythonProxy)
println(listLengthPython)
and looks like it's implying I have no Python installed, but I just opened Python 3.7 32 bit version. It's on Windows (you could guess it because DLL). Can someone help with realizing what I am doing wrong?
shadaj
@shadaj:matrix.org
[m]
@limar63: you may want to try the (experimental) setup in shadaj/scalapy#210, which is designed to make finding the native libraries a lot easier
3 replies
Eric K Richardson
@ekrich
@shadaj:matrix.org Most of the Scala world has moved to discord - there were some discussion about ScalaPy there today. https://discord.gg/9G7Cch4q Not sure if you want to setup a server for your projects.
scalavision
@scalavision
@shadaj:matrix.org , will there be a new release soon with support for scala 3? Or is it still a lot of more work to do?
Alexey Novakov
@novakov-alexey
Hi there. Perhaps I have weird question, but is it possible to call Scala code from Python? :smile:
shadaj
@shadaj:matrix.org
[m]
@scalavision will probably cut a release in the next week! although the latest commits are already on Maven Central
1 reply
@novakov-alexey yes! you can pass Scala functions into Python
Alexey Novakov
@novakov-alexey
@shadaj:matrix.org is there some example?
and how that would work? Python VM would start JVM and pass some Scala code snippet to execute?
shadaj
@shadaj:matrix.org
[m]
@novakov-alexey: see https://github.com/shadaj/scalapy/blob/main/core/shared/src/test/scala/me/shadaj/scalapy/py/LambdaTest.scala; ScalaPy works by embedding Python inside your Scala app, so the Scala code would have to initiate things