Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    raproth
    @raproth
    Currently python stopped working in polynote, I get Uncaught exception: <class 'SyntaxError'>: future feature annotations is not defined (jep.JepException) any idea whats wrong? Running Python 3.6.10 on Linux. Do I need Python > = 3.7?
    Clay Lambert
    @claylambert33_twitter
    @raproth it has been my experienxe that 3.7 works better overall. I don’t know if that will resolve your jep exception per se.
    jonathanindig
    @jonathanindig
    @raproth This is due to a recent change in the new release: polynote/polynote#951 Didn’t realize it wouldn’t work in 3.6 but we’re mainly targeting 3.7. Any particular reason you’re using 3.6?
    jeremyrsmith
    @jeremyrsmith
    heh, python is so hard to support with one interpreter, because there's no union of "all of python" even within 3.x. Minor versions have removed stuff and added other stuff, you can't capture that in a union
    I imagine we'll run into the same thing when we start worrying about Scala 3... :grimacing:
    Should there be a Python < 3.7 interpreter and a Python >= 3.7 interpreter? I don't know really... I'd prefer not, but this is the kind of thing that may become necessary :frowning:
    João Costa
    @JD557
    FWIW, according to https://endoflife.date/python, python 3.6 support ends in 1 year and 3 months
    I would say that it might not be worth it to add that support just for one year, but OTOH, all surveys that I can find say that 3.6 is the most used version :grimacing:
    raproth
    @raproth
    @jonathanindig no particular reason, currently python 3.6.10 is installed on our server and I do not have the permissions to change that myself. But don't worry, at least I know what to do
    Lanking
    @lanking520

    Hi all, I am new to PolyNote, currently trying to install it on my mac (Mac OS Catalina) and failed with Python 3.7 virtualenv:

        ld: library not found for -lpython3.7m
        clang: error: linker command failed with exit code 1 (use -v to see invocation)
        error: command 'xcrun' failed with exit status 1

    It seemed to be an issue with jep, did anyone saw this before?

    jonathanindig
    @jonathanindig
    @lanking520 from a quick Google search, it looks like you need to install Xcode tools: https://ma.ttias.be/mac-os-xcrun-error-invalid-active-developer-path-missing-xcrun/
    Lanking
    @lanking520
    I already installed that, 'xcrun' failed
    ld: warning: directory not found for option '-L/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/lib'
    João Costa
    @JD557
    @lanking520 have you tried using the docker image instead? I find that easier than having to fix the local python version
    jonathanindig
    @jonathanindig
    +1 for the Docker version being easier. If you still want to set things up locally, unfortunately you will need to debug your python environment. Are you using the stock python that ships with OS X? Unfortunately Python installation environments tend to be horrible: https://xkcd.com/1987/
    Lanking
    @lanking520
    Nope, I use the virtualenv Python instead, just don't want it messed up local environment. I am trying to use polynote along with my demo on Spark and provide instruction for users to start using it. Using Docker although would work but I am looking for some out-of-box solution for users so they don't have to dive into configuration... Also my application will also use GPU, I am not sure if the docker env will have CUDA driver
    btw:
    (env) 3c22fb785594:polynote qingla$ xcode-select --install
    xcode-select: error: command line tools are already installed, use "Software Update" to install updates
    (env) 3c22fb785594:polynote qingla$ clang --version
    Apple clang version 11.0.3 (clang-1103.0.32.62)
    Target: x86_64-apple-darwin19.6.0
    Thread model: posix
    InstalledDir: /Library/Developer/CommandLineTools/usr/bin
    jonathanindig
    @jonathanindig
    What is the output of python --version and python-config --ldflags
    Also, what command are you running to get that error? pip install jep ?
    Lanking
    @lanking520
    Here you go
    (env) 3c22fb785594:polynote qingla$ python --version
    Python 3.7.3
    (env) 3c22fb785594:polynote qingla$ python-config --ldflags
    -L/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/config -lpython2.7 -ldl -framework CoreFoundation
    Yes, pip install jep failed
    jonathanindig
    @jonathanindig
    and pip --version ?
    Lanking
    @lanking520
    (env) 3c22fb785594:polynote qingla$ pip --version
    pip 19.0.3 from /Users/qingla/Documents/GitHub/polynote/env/lib/python3.7/site-packages/pip (python 3.7)
    Lanking
    @lanking520
    It seemed there is a mismatch on my python-config which is using python2.7... And it mayu cause the issues while jep trying to find python3.7m package
    jonathanindig
    @jonathanindig
    That’s possible, but the ld warning you posted above seemed to be looking for a python 3 path. I wonder where it got /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/lib
    Lanking
    @lanking520
    ls /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/lib
    ls: /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/lib: No such file or directory
    jonathanindig
    @jonathanindig
    Hmm, I don’t really know much about the Python build environment unfortunately. Maybe try python3-config --ldflags?
    My guess is there are two different python installations around that are confusing things
    one of them is no longer there (maybe partially deleted, given the missing directory)
    I’d try reinstalling Python, maybe from homebrew
    or fixing your PATH, in case there are multiple installations there
    maybe your python3-config is from a different python installation than your python? which python3-config and which python should point to the same directory
    for example, on my machine they are both in /usr/local/Cellar/python/3.7.6_1/Frameworks/Python.framework/Versions/3.7/bin/
    (I installed python with homebrew)
    João Costa
    @JD557

    Nope, I use the virtualenv Python instead, just don't want it messed up local environment. I am trying to use polynote along with my demo on Spark and provide instruction for users to start using it. Using Docker although would work but I am looking for some out-of-box solution for users so they don't have to dive into configuration... Also my application will also use GPU, I am not sure if the docker env will have CUDA driver

    yeah... the polynote docker image doesn't support CUDA... I think it's not that hard to build an image that works with CUDA, though (I have a WIP, but my nVidia GPU is on my Windows machine, and getting docker to play well with GPU's in windows is not trivial, so I gave up)

    Lanking
    @lanking520
    @jonathanindig thanks for your help! suprisingly, I didn't have python3-config
    python3-config --ldflags
    -bash: python3-config: command not found
    jonathanindig
    @jonathanindig
    maybe it’s not on your PATH? you might find it in your python installation folder (location of which python, if it’s symlinked make sure to follow them to the actual directory)
    Lanking
    @lanking520
    Thanks for you all, I got my issue fixed: installing python3 on my laptop. the system default python3 seemed not working well: after following the brew install python3 instruction I managed to fix it: https://stackoverflow.com/questions/49704364/make-python3-as-my-default-python-on-mac
    jonathanindig
    @jonathanindig
    Awesome, glad to hear it :smile:
    Lanking
    @lanking520
    Something more to follow up. Is there anyway I can use standalone Scala without importing python dependencies for polynote (if I just need scala)
    jeremyrsmith
    @jeremyrsmith
    @lanking520 you can just use Scala and python stuff shouldn't even be initialized at all... if you mean "can I install it with no python" then I think it ought to be possible except that the run script is itself python. If you start it directly with java then I don't see why it wouldn't work (though I haven't tried)
    Lanking
    @lanking520
    @jeremyrsmith Good to know. So another option is simply java -jar on the package and it should work
    So I didn't even have to install the python dependencies
    jeremyrsmith
    @jeremyrsmith
    @lanking520 it's good for us to know that we need an easy way of running Polynote for people who don't plan to use Python... maybe Python itself ought to be a plug-in 🤔 (not sure if that's possible without making jep nonsense way more complicated though)
    Lanking
    @lanking520
    So here is the thing, I am one of the author of https://github.com/aws-samples/d2l-java this project to allow all Java developers to learn DL in Java. We already onboard with Jupyter Notebook and did several hacks to allow Java runs everywhere within Jupyter Notebook. However, the notebook itself lacks of language feature, such as poping up recommendation functions or help on the grammer issues. I found this project truly helpful to help people leverage Scala. So as a start, I planned to create a short tutorial on Spark with Scala for Deep Learning with PolyNote to see how things goes.
    jonathanindig
    @jonathanindig
    That sounds great! It would be great to see the tutorial once it’s ready :D
    As for running directly without python dependencies, like @jeremyrsmith said, the run script https://github.com/polynote/polynote/blob/master/scripts/polynote.py builds a java command that lets you run Polynote. If you don’t need Python support, you can just take a look at the output of the script once and then use that to run it next time.
    Lanking
    @lanking520
    For sure, I will try to build some shell script that can do similar stuff and launch the project
    Lanking
    @lanking520
    QQ: is there any way we can add scalacOptions += "-target:jvm-1.8" since the package I imported require 1.8 features