Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Kevin Bates
    @kevin-bates
    Hi @komkar123 - thanks for the additional information. The issue is that the Toree kernel instance can’t get fully started. As a result, the kernel restarter detects its death and attempts to start the kernel again (4 more times). So the Jupyter side of things is working as expected and you should focus on resolving the issue in the stack trace. Is there any additional information before or after what is pictured? I’m assuming this occurs immediately at startup, before any cell is executed, but please let us know if that’s not the case.
    Omkar Kalange
    @komkar123
    Yes, this happens when I select Apache Toree as kernel in Jupyter, before executing any piece of code.
    Omkar Kalange
    @komkar123

    image.png

    This is what happens when I start jupyter and select Apache Toree as kernel

    Omkar Kalange
    @komkar123

    jupyter toree install --user --spark_home=$HOME/spark --spark_opts="--master=spark://$MASTER_URL:7077"

    Is in stack to install toree

    Omkar Kalange
    @komkar123
    I searched for the error : java.lang.NoSuchMethodError: scala.Product.$init$(Lscala/Product;)V and it looks like there is a version mismatch of scala
    Luciano Resende
    @lresende
    What version of Scala are you using? For latest Toree you should be using Scala 2.12
    https://github.com/apache/incubator-toree/blob/70cc22012250e09b7061211e8716fffe0ff35d76/Makefile#L34
    Omkar Kalange
    @komkar123
    image.png
    Omkar Kalange
    @komkar123
    Scala code runner version 2.12.2 -- Copyright 2002-2017, LAMP/EPFL and Lightbend, Inc.
    This is the version of scala installed on my machine. However, when I start spark 2.4.6 using spark-shell, it uses scala 2.11.12
    Using Scala version 2.11.12 (OpenJDK 64-Bit Server VM, Java 1.8.0_292)
    Luciano Resende
    @lresende
    You should install Toree 0.4 for Spark 2.4.6 and Scala 2.11
    Toree 0.5, by default, is build with Spark 3 + Scala 2.12
    Omkar Kalange
    @komkar123
    Oh, can u please tell me how can I run install toree command with 0.4?
    Luciano Resende
    @lresende
    https://pypi.org/project/toree/0.4.0/
    pip install toree==0.4.0
    Omkar Kalange
    @komkar123
    Thank you so much, I really appreciate your timely help.
    It worked, the stack was created some time back in last year so running it now installed latest toree
    Luciano Resende
    @lresende
    great to hear that
    Andres Rodriguez
    @acrodriguez_twitter
    Hello. I am trying to pass some data from a Scala cell to a JS cell so that I can perform a (graph) visualization via D3. I cannot seem to find the right documentation. The DataBricks website says to use displayHTML, but that does not seem to be available. Is it possible? Thanks much 🙏
    dhia-gharsallaoui
    @dhia-gharsallaoui
    Hi there, I want to add apache toree to my jupyterhub. my problem is that all my jars are compiled using scala 2.13.5 + spark 3.2.1. when I add toree kernel it doesn't start and this is because of the version I think because toree is built with scala version 2.12. Also I tried to rebuild toree with scala 2.13 but i got many errors. Any propostion thank you
    2 replies
    image.png
    jbguerraz
    @jbguerraz

    Hi there, I want to add apache toree to my jupyterhub. my problem is that all my jars are compiled using scala 2.13.5 + spark 3.2.1. when I add toree kernel it doesn't start and this is because of the version I think because toree is built with scala version 2.12. Also I tried to rebuild toree with scala 2.13 but i got many errors. Any propostion thank you

    Is there any open issue related to scala 2.13 adoption ? I didn't find any on Jira but I'm not so used to it so maybe I just missed it

    1 reply
    Nhat Nguyen
    @ntnhaatj
    hi guys, I got trouble when trying to export environment from toree jupyter, could anyone help to lighten me how can do that please? Thanks a lot
    %export env=val
    Bostonian
    @athensofamerica
    When I install toree with "--interpreters=Scala,PySpark,SparkR,SQL", I only see Scala and SQL. Are PySpark and SparkR interpreters no long supported?
    2 replies
    Rahul Singh
    @RaSi96
    Screenshot_20221104_140426.png
    Greetings all, good day. I've just done a fresh installation of Arch Linux 6.0.6-arch1-1, Jupyter Lab, Spark 3.3.1 (spark-3.3.1-bin-hadoop3-scala2.13) and Toree with everything going well. Upon launching Jupyter and opening a new notebook with Toree, my kernel keeps disconnecting with the above error.
    (I wasn't expecting my photo to upload before my text did, apologies if I confused anybody).
    Some very shallow Google-Fu showed me that it might be a Scala version mismatch? I'm unsure how to go about launching Toree now; my $SPARK_HOME and $JAVA_HOME work fine because I'm able to navigate to $SPARK_HOME/bin and launch a spark-shell.
    Rahul Singh
    @RaSi96

    Hi there, I want to add apache toree to my jupyterhub. my problem is that all my jars are compiled using scala 2.13.5 + spark 3.2.1. when I add toree kernel it doesn't start and this is because of the version I think because toree is built with scala version 2.12. Also I tried to rebuild toree with scala 2.13 but i got many errors. Any propostion thank you

    Well that's the end of that then, I completely scrolled past this exact same issue. Apologies for wasting everyone's time :)

    Rahul Singh
    @RaSi96
    Sorry to revive my problem again, but after matching Scala versions and launching, I'm met with a java.lang.UnsupportedOperationException: The Security Manager is deprecated and will be removed in a future release; consequently my kernel never manages to connect and is left in a perpetual "connecting" status
    Screenshot_20221104_164535.png
    This is with Arch Linux 6.0.6, Spark 3.3.1 with Hadoop 3 and Scala 2.12.15 (direct from Apache's Spark download webpage), and Jupyter Lab 3.5.0
    Is there any way I could look into this or work around it?
    Mikhail
    @Armadik
    How to solve a problem?
    Error parsing magics!: Magics [AddDeps] do not exist!
    2 replies
    Creepysta
    @creepysta

    Hi Team,
    I found some docs on the plugin system of Toree which allows us to add custom magics using %AddJar -magic <url>.
    But I'm unable to find any document for registering Events for something like preRunCell. A sample snippet for what I have seen working in ipython is the following - 

    def load_ipython_extension(ipython):
    ipython.events.register("pre_run_cell", some_func)

    I've noticed that the PluginEvent PreRunCell is fired by Toree too. But I'm not sure how to hook / listen to it using the Toree's plugin system. Could you kindly point me to some documentation.
    Thanks in advance!

    14 replies