almond-sh/almond

Lightweight Scala kernel for Jupyter / IPython 3

Sören Brunk

@sbrunk

@trepidacious that's great! Don't hesitate to ask if you get stuck at some point.

trepidacious

@trepidacious

@sbrunk Thanks :) Really enjoying using almond, I wish I'd found it years ago!

trepidacious

@trepidacious

@sbrunk I've tested that approach with a real notebook and it works well - repl completely suppressed, but html, plotly and println all come through fine, makes the notebook much neater.

Sören Brunk

@sbrunk

@trepidacious awesome! Would you mind opening an issue or a PR to discuss your approach? I guess we also have to look for a way to configure this.

trepidacious

@trepidacious

@sbrunk I will do, I had a look through the issues and couldn't find anything covering it so far, I guess almond-sh/almond#287 might be slightly related, in that suppressing all output would cover that issue :)

Sören Brunk

@sbrunk

@valencik the coursier API hast changed. Try

interp.repositories() ++= Seq(
  coursierapi.MavenRepository.of("https://dl.bintray.com/cibotech/public")
)

Note that I've updated the EvilPlot example accordingly.

Andrew Valencik

@valencik

woooow, thank you! I actually could not come across a fix for that. Out of curiosity, how did you figure this out?

Either way, thank you again.

Sören Brunk

@sbrunk

@valencik I think I saw someone mentioning that in the Ammonite gitter channel. Otherwise, I probably would have asked @alexarchambault, as he’s also the author of Coursier and should know. 😃

trepidacious

@trepidacious

@sbrunk I created a pull request for the "silent repl" feature - seems to work but I'm new to the code so it might not be implemented the best way :) almond-sh/almond#450

Sören Brunk

@sbrunk

@trepidacious thanks! I’m going to give it a try tomorrow

Hubert Plociniczak

@hubertp

Hi all, I'm trying to run a custom Scala kernel for 2.13 but encountering some compilation issues. I used to think it's a bug in Ammonite, but I'm not sure anymore. I can even reproduce it with official instructions available at https://almond.sh/docs/quick-start-install.html#create-a-launcher where I set SCALA_VERSION=2.13.0 (it works with 2.12.x). When running the kernel in jupyter I get:

Compiling (synthetic)/ammonite/predef/interpBridge.sc
ERROR AmmInterpreter$ Caught exception while initializing interpreter
almond.amm.AmmInterpreter$PredefException: 
  almond.amm.AmmInterpreter$.$anonfun$apply$3(AmmInterpreter.scala:130)
  almond.amm.AmmInterpreter$.$anonfun$apply$3$adapted(AmmInterpreter.scala:125)
  scala.Option$WithFilter.foreach(Option.scala:437)
  almond.amm.AmmInterpreter$.apply(AmmInterpreter.scala:125)
  almond.ScalaInterpreter.ammInterp$lzycompute(ScalaInterpreter.scala:97)
  almond.ScalaInterpreter.ammInterp(ScalaInterpreter.scala:61)
  almond.ScalaKernel$$anon$5.run(ScalaKernel.scala:136)
java.lang.NoClassDefFoundError: scala/tools/nsc/reporters/AbstractReporter
  ammonite.interp.Compiler$.apply(Compiler.scala:141)
  ammonite.interp.CompilerLifecycleManager.init(CompilerLifecycleManager.scala:98)
  almond.amm.AmmInterpreter$$anon$1$$anon$2.preprocess(AmmInterpreter.scala:113)
  ammonite.interp.Interpreter.compileRunBlock$1(Interpreter.scala:509)
  ammonite.interp.Interpreter.$anonfun$processAllScriptBlocks$15(Interpreter.scala:569)
  ammonite.util.Res$Success.flatMap(Res.scala:62)
  ammonite.interp.Interpreter.$anonfun$processAllScriptBlocks$14(Interpreter.scala:566)
  ammonite.util.Res$Success.flatMap(Res.scala:62)

which is weird because scala/tools/nsc/reporters/AbstractReporter is a 2.12.x compiler class, not 2.13.x.
Any ideas?

Aleksei Alefirov

@alekseiAlefirov

Hi,
Is it possible to access ScalaInterpreter class from a library?
I don't need any server and concurrency logic, just scala execution, hence the interest.

Aleksei Alefirov

@alekseiAlefirov

Ok, managed to solve it.

"sh.almond" % "scala-interpreter" % V.almond cross CrossVersion.full

worked fine for me, the problem was missing jvm-repr, thankfully, not news in this chat.

Aleksei Alefirov

@alekseiAlefirov

Hi again,
I'd like to use ScalaIntepreter, but when I create it, I get

almond.amm.AmmInterpreter$PredefException: 
    at almond.amm.AmmInterpreter$.$anonfun$apply$3(AmmInterpreter.scala:130)
...
Caused by: scala.reflect.internal.FatalError: Error accessing /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.171-3.b10.fc26.x86_64/jre/lib/ext/java-atk-wrapper.jar
    at scala.tools.nsc.classpath.AggregateClassPath.$anonfun$list$3(AggregateClassPath.scala:99)
...

Can I somehow fixed? Maybe with correct ScalaInterpreterParams?
Thanks.

Aleksei Alefirov

@alekseiAlefirov

Hello again,
Playing with Classloaders (both ScalaIntepreter param and current Thread context one), I've managed to initialize ScalaInterpeter.
But trying to execute code there fails for me with Compilation Failure.
For code val x = 1 I get following error:

cmd0.sc:7: exception during macro expansion: 
java.lang.IllegalArgumentException: argument type mismatch

  .printOnChange(x, "x", _root_.scala.None, _root_.scala.None, _root_.scala.None)) }

Looked at almond code a bit, I've experimented and set ScalaInterpreter params autoUpdateLazyVals and autoUpdateVars both to false, and got slightly different error:

cmd0.sc:8: exception during macro expansion: 
java.lang.IllegalArgumentException: argument type mismatch

          .print(x, "x", _root_.scala.None)

Maybe someone can give me a clue, how to solve it? Thanks.

Hao Sun

@zenhao

java.lang.ClassCastException: almond.ReplApiImpl cannot be cast to ammonite.repl.ReplAPI
I am not sure what is happening. I was trying to load a locally published jar and I got this.

Seems like cats works..
(I am on almond 0.9.0 and scala 2.12.8)

Hao Sun

@zenhao

Solved. I had ammonite in my jar. Exclude it fixed my issue.

Adam Davidson

@Adi255

"Local clusters, Mesos, and Kubernetes, aren't supported by ammonite-spark yet". We use Spark on k8s here, just wondering if this means it does not work at all, or whether you can use it but without ammonite features?

Hubert Plociniczak

@hubertp

almond 0.9.1 (compared to 0.9.0) adds metabrowse dependency to the build. The latter, in turn, brings scala-compiler 2.13.1 dependency. This is problematic for custom kernels that are still on 2.13.0 because ammonite cross compiles to a full Scala version (not just binary one).
In the end one gets nomethod/noclass found exceptions during initialization of the kernel because of compiler interface differences. Would that be considered a bug/feature? I'm guessing metabrowse would need to be crosscompiled to a full Scala version to solve this properly.

Michał Gołębiewski

@mjgolebiewski

hello guys, ive encountered problem when i try to run Scala kernel on my Jupyter Notebook:

./almond --log debug --connection-file scala.kernel.json 
DEBUG ScalaKernel$ Auto dependency:
  Trigger: Module(org.apache.spark, *)
  Adds: Dependency(Module(sh.almond, almond-spark_2.12), 0.9.1)
DEBUG ScalaKernel$ Creating interpreter
DEBUG ScalaKernel$ Created interpreter
DEBUG ScalaKernel$ Running kernel
DEBUG ScalaKernel$ Initializing interpreter (background)
INFO AmmInterpreter$ Creating Ammonite interpreter
DEBUG AmmInterpreter$ Initializing interpreter predef
Compiling (synthetic)/ammonite/predef/interpBridge.sc
DEBUG ZeromqConnection Opening channels for ConnectionParameters(127.0.0.1,tcp,33763,35867,59313,42939,44325,****,Some(hmac-sha256),Some(scala))
ERROR AmmInterpreter$ Caught exception while initializing interpreter
almond.amm.AmmInterpreter$PredefException: 
  almond.amm.AmmInterpreter$.$anonfun$apply$3(AmmInterpreter.scala:130)
  almond.amm.AmmInterpreter$.$anonfun$apply$3$adapted(AmmInterpreter.scala:125)
  scala.Option$WithFilter.foreach(Option.scala:407)
  almond.amm.AmmInterpreter$.apply(AmmInterpreter.scala:125)
  almond.ScalaInterpreter.ammInterp$lzycompute(ScalaInterpreter.scala:97)
  almond.ScalaInterpreter.ammInterp(ScalaInterpreter.scala:61)
  almond.ScalaKernel$$anon$5.run(ScalaKernel.scala:136)
java.lang.NoSuchMethodError: scala.tools.nsc.classpath.ZipAndJarClassPathFactory$.create(Lscala/reflect/io/AbstractFile;Lscala/tools/nsc/Settings;)Lscala/tools/nsc/util/ClassPath;
  ammonite.interp.ExtraCompilerCompatibility.createZipJarFactory(ExtraCompilerCompatibility.scala:10)
  ammonite.interp.Compiler$.$anonfun$prepareJarCp$2(Compiler.scala:372)
  scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:238)
  scala.collection.Iterator.foreach(Iterator.scala:941)
  scala.collection.Iterator.foreach$(Iterator.scala:941)
  scala.collection.AbstractIterator.foreach(Iterator.scala:1429)
  scala.collection.IterableLike.foreach(IterableLike.scala:74)
  scala.collection.IterableLike.foreach$(IterableLike.scala:73)
  scala.collection.AbstractIterable.foreach(Iterable.scala:56)
  scala.collection.TraversableLike.map(TraversableLike.scala:238)
  scala.collection.TraversableLike.map$(TraversableLike.scala:231)
  scala.collection.AbstractTraversable.map(Traversable.scala:108)
  ammonite.interp.Compiler$.prepareJarCp(Compiler.scala:369)
  ammonite.interp.Compiler$.initGlobalClasspath(Compiler.scala:400)
  ammonite.interp.Compiler$$anon$1.<init>(Compiler.scala:206)
  ammonite.interp.Compiler$.apply(Compiler.scala:141)
  ammonite.interp.CompilerLifecycleManager.init(CompilerLifecycleManager.scala:98)
  almond.amm.AmmInterpreter$$anon$1$$anon$2.preprocess(AmmInterpreter.scala:113)
  ammonite.interp.Interpreter.compileRunBlock$1(Interpreter.scala:509)
  ammonite.interp.Interpreter.$anonfun$processAllScriptBlocks$15(Interpreter.scala:569)
  ammonite.util.Res$Success.flatMap(Res.scala:62)
  ammonite.interp.Interpreter.$anonfun$processAllScriptBlocks$14(Interpreter.scala:566)
  ammonite.util.Res$Success.flatMap(Res.scala:62)
  ammonite.interp.Interpreter.$anonfun$processAllScriptBlocks$12(Interpreter.scala:563)
  scala.Option.getOrElse(Option.scala:189)
  ammonite.interp.Interpreter.loop$1(Interpreter.scala:563)
  ammonite.interp.Interpreter.processAllScriptBlocks(Interpreter.scala:601)
  ammonite.interp.Interpreter.$anonfun$processModule$6(Interpreter.scala:397)
  ammonite.util.Catching.flatMap(Res.scala:115)
  ammonite.interp.Interpreter.$anonfun$processModule$5(Interpreter.scala:388)
  ammonite.util.Res$Success.flatMap(Res.scala:62)
  ammonite.interp.Interpreter.processModule(Interpreter.scala:378)
  ammonite.interp.Interpreter.$anonfun$initializePredef$1(Interpreter.scala:126)
  ammonite.interp.Interpreter.$anonfun$initializePredef$1$adapted(Interpreter.scala:126)
  ammonite.interp.PredefInitialization$.$anonfun$apply$7(PredefInitialization.scala:75)
  ammonite.util.Res$.$anonfun$fold$1(Res.scala:32)

here is rest of the error:

 scala.collection.LinearSeqOptimized.foldLeft(LinearSeqOptimized.scala:126)
  scala.collection.LinearSeqOptimized.foldLeft$(LinearSeqOptimized.scala:122)
  scala.collection.immutable.List.foldLeft(List.scala:89)
  ammonite.util.Res$.fold(Res.scala:30)
  ammonite.interp.PredefInitialization$.apply(PredefInitialization.scala:63)
  ammonite.interp.Interpreter.initializePredef(Interpreter.scala:128)
  almond.amm.AmmInterpreter$.apply(AmmInterpreter.scala:125)
  almond.ScalaInterpreter.ammInterp$lzycompute(ScalaInterpreter.scala:97)
  almond.ScalaInterpreter.ammInterp(ScalaInterpreter.scala:61)
  almond.ScalaKernel$$anon$5.run(ScalaKernel.scala:136)

and here are version numbers:

$ scala -version
Scala code runner version 2.12.8 -- Copyright 2002-2018, LAMP/EPFL and Lightbend, Inc.

$ java -version
openjdk version "1.8.0_242"
OpenJDK Runtime Environment (build 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08)
OpenJDK 64-Bit Server VM (build 25.242-b08, mixed mode)

$ javac -version
javac 1.8.0_242

Hubert Plociniczak

@hubertp

@mjgolebiewski That's similar to what I was encountering. Except you seem to be on 2.12.x and you are getting scala-reflect binary incompatibilies. I've also been fighting with that once I resolved scala-compilar compatibility issues.
How did you build that particular almond kernel? Is that from a fresh clone of the repo?
I can show you the (not so pretty) workaround I used, but I think this should be solved in general in almond itself, by fetching dependencies with full scala version, not just the binary compatible one.

Ghost

@ghost~558c900115522ed4b3e2ce89

@mjgolebiewski I ran into the same problem on my FreeBSD dev machine. Just for fun, I tried SCALA_VERSION=2.13.1 ALMOND_VERSION=0.9.1 and it works

Michał Gołębiewski

@mjgolebiewski

@hubertp @oddodaoddo changing scala version helped. thank you guys

Michail Chatzis

@mchatzis

Good morning everyone, I have run to a problem during installation. I would appreciate any help, thank you!

Sudheer Doppalapudi

@sudheer

Hi Team, I am trying to use mermaid JS inside Scala shell launched with the help of almond. I want to draw diagrams right form the Jupiter notebooks. Anybody tried this or have any luck with it ? Thanks in advance.

Shriraj Bhardwaj

@sbhardwaj-mt

I tried SCALA_VERSION=2.13.1 and ALMOND_VERSION=0.9.1 this breaks if you try to import spark-sql for scala. so I downgraded ALMOND_VERSION to 0.8.3 which worked well, if anyone looking for working with spark-sql, use SCALA_VERSION=2.12.8 and ALMOND_VERISON=0.8.3

Will Udstrand

@Udstrand_gitlab

Are there any updates on the issue @mjgolebiewski was seeing? I have tried installing almond with SCALA_VERSION=2.12.8 ALMOND_VERISON=0.8.3, SCALA_VERSION=2.12.8 ALMOND_VERSION=0.9.1, and SCALA_VERSION=2.13.1 ALMOND_VERSION=0.9.1 all to no avail.

David Bouyssié

@david-bouyssie

I'm trying to use Almond on Google Colab. I have followed this tutorial https://gist.github.com/shadaj/323ad2393b46c1b71df435728a052c24 but it doesn't seem to work anymore. Any workaround?

Alfonso Roa

@alfonsorr

@sbhardwaj-mt i hope you find already a solution but spark is not compiled for scala 2.13, you must use a scala version of the 2.12, mainly 2.12.10 with almond 0.9.1 will work

Wojtek Pituła

@Krever

Hey, I'm building a library (with sbt) and would like to add it to my almond container (with custom Dockerfile). I assume sbt publish(on host) -> coursier fetch(in docker file) would work but I would like to skip pushing the artifact to external repo. Do you know how (or which) to copy artifacts from sbt target so it will be properly picked up by kernel (so ammonite and coursier)

Sören Brunk

@sbrunk

@Krever you could do an sbt publishLocal and then copy the local ivy dir into the docker image on build.

You can publish to a dir different than .ivy/local with sbt to avoid copying too much stuff into the dockerfile. See https://www.scala-sbt.org/1.x/docs/Publishing.html

Sören Brunk

@sbrunk

We're using a somewhat similar method (a bit more involved i.e. using a multi stage docker build) to create an almond docker image without having published almond artifacts (for snapshot builds).
https://github.com/almond-sh/almond/blob/master/Dockerfile

Alfonso Roa

@alfonsorr

a question related maybe more with coursier, is there a way to download some dependencies (to create a container) and access them to use in almond? the objective is to create the docker with some dependencies and not have to download them with ivy every time

Sören Brunk

@sbrunk

@alfonsorr adding a RUN coursier fetch <lib>in the docker build should do the trick to prepopulate the coursier cache. almond uses the coursier API through Ammonite so it should find the cached artifacts in the running container then.

Wojtek Pituła

@Krever

@sbrunk thanks!

Wojtek Pituła

@Krever

I have one more question, hopefully, a simple yes/no one but only partially related to almond. Is it possible to store notebook state? I have a use case in which I need to wait in the middle of notebook for a potentially very long time, so I would like to stop it and after I get completion event revive it from where it stopped.

ammonite session persistance comes to my mind, but Im curious if anyone actually tried that

Alfonso Roa

@alfonsorr

@sbrunk great, thanks!

Alfonso Roa

@alfonsorr

i added also coursier fetch --sources <lib> to skip all the downloads in the notebook, it works great thanks!

Sören Brunk

@sbrunk

@alfonsorr I just remembed that you can even take it one step further. You can execute a notebook programatically using nbconvertin the docker build which will do a coursier fetch as well. The advantage is that it will stay in sync with your import $ivy ... statements in the notebook.

I did that for my Scala Days talk about almond because I used a notebook as live slides. https://github.com/sbrunk/scaladays-2019/blob/master/scripts/jupyter.sh#L87

@Krever I think saved ammonite sessions don't persist through different runs of ammonite but I'm not sure

Wojtek Pituła

@Krever

yeah, seems like so ;/ I just went through docs and can find anything about out of process persistance

Alfonso Roa

@alfonsorr

i was thinking to launch a scala script with the imports but that idea is much better

Henry

@hygt

hello, this is more related to ammonite-spark but I'm trying to run the REPL and then hopefully notebooks on my company cluster, it's running Hortonworks/Cloudera HDP 2.6.5

creating the Spark sessions fails

Exception in thread "main" java.lang.NoClassDefFoundError: scala/MatchError
    at org.apache.spark.deploy.yarn.ExecutorLauncher$.main(ApplicationMaster.scala:833)
    at org.apache.spark.deploy.yarn.ExecutorLauncher.main(ApplicationMaster.scala)
Caused by: java.lang.ClassNotFoundException: scala.MatchError
    at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    ... 2 more

Where communities thrive

People

Repo info

Activity