Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • Mar 31 22:21
    dependabot[bot] labeled #68
  • Mar 31 22:21
    dependabot[bot] opened #68
  • Mar 31 22:21

    dependabot[bot] on maven

    Bump guava from 28.2-jre to 29.… (compare)

  • Oct 12 2020 17:03
    markyjackson-taulia commented #35
  • Aug 29 2020 13:49

    Loghijiaha on master

    code cleanup Added HTML view (compare)

  • Aug 29 2020 13:49
    Loghijiaha closed #67
  • Aug 28 2020 18:21
    Loghijiaha commented #67
  • Aug 28 2020 18:20
    Loghijiaha synchronize #67
  • Aug 28 2020 17:48
    Loghijiaha opened #67
  • Aug 28 2020 07:48

    Loghijiaha on master

    Fixed unnecessary SEVERE log (compare)

  • Aug 28 2020 07:48
    Loghijiaha closed #66
  • Aug 27 2020 17:45
    Loghijiaha opened #66
  • Aug 24 2020 20:32

    Loghijiaha on master

    Updated release draft Updated changelog (compare)

  • Aug 24 2020 20:32
    Loghijiaha closed #65
  • Aug 24 2020 20:22
    Loghijiaha commented #65
  • Aug 24 2020 15:44
    Loghijiaha synchronize #65
  • Aug 24 2020 15:43
    Loghijiaha opened #65
  • Aug 24 2020 10:48

    Loghijiaha on master

    [maven-release-plugin] prepare … (compare)

  • Aug 24 2020 10:48

    Loghijiaha on machine-learning-1.0.1

    (compare)

  • Aug 24 2020 10:47

    Loghijiaha on master

    [maven-release-plugin] prepare … (compare)

Bruno P. Kinoshita
@kinow
That way after others watch your demo they can grab this latest release, and you can work on this issues with more calm. I believe we have everything from our design doc submitted to GSoC right?
Loghi
@Loghijiaha
Yh, we all set with the GSoC design doc with some minor changes.
Bruno P. Kinoshita
@kinow
Brilliant. I'll keep looking at this logging issue in the spare time. But in the worst case we just need to tell users about it.
Loghi
@Loghijiaha
Great @kinow :heavy_check_mark:
Loghi
@Loghijiaha

Demo 3 schedule:

Part 1, 2PM UTC, Aug 27 (Thu)

  • Rishabh
  • Kezhi
  • Sladyn

Part 2, 2PM UTC, Aug 28 (Fri)

  • Loghi
  • Buddhika
  • Sumit
Loghi
@Loghijiaha
@kinow there is an issue with default directory settings. Notebooks/scripts are not recognizing the files/dataset eg: example_with_images. We should at least have to change the directory to the workspace for python.
Loghi
@Loghijiaha
procEnvir.png
How can we change the env only for this process this.jupyterKernelProcessLauncher = new JupyterKernelInterpreter.JupyterKernelProcessLauncher(cmd, envs);?
Bruno P. Kinoshita
@kinow
@Loghijiaha most command line libraries have methods to define the working directory. With the Kernels, we rely on Zeppelin providing a method to specify the working directory… let me have a look (just drinking the first cup of coffee of the day.. might need a second one for this issue haha)
@Loghijiaha let us know if you need help with the final work submission, please. Related to this guideline: https://developers.google.com/open-source/gsoc/help/work-product
Loghi
@Loghijiaha
Haha i will let you know, if I need help on final submission.
Bruno P. Kinoshita
@kinow

@Loghijiaha I've rebased the code with your last changes, including the one that removes the error with javascript kernel (trying to change working directory).

With the following diff, the GRPC internal log is hidden, without disabling the logger for other plug-ins. WDYT? Feel free to use this snippet in a PR in case that works.

diff --git a/src/main/java/io/jenkins/plugins/ml/IPythonBuilder.java b/src/main/java/io/jenkins/plugins/ml/IPythonBuilder.java
index 3ae8e04..71423cf 100644
--- a/src/main/java/io/jenkins/plugins/ml/IPythonBuilder.java
+++ b/src/main/java/io/jenkins/plugins/ml/IPythonBuilder.java
@@ -60,6 +60,7 @@ import java.io.InputStreamReader;
 import java.io.Serializable;
 import java.nio.charset.Charset;
 import java.util.List;
+import java.util.logging.Level;
 import java.util.stream.Stream;

 /**
@@ -70,6 +71,8 @@ public class IPythonBuilder extends Builder implements SimpleBuildStep, Serializ
     private static final long serialVersionUID = 1L;

     private static final Logger LOGGER = LoggerFactory.getLogger(IPythonBuilder.class);
+    private static final java.util.logging.Logger GRPC_IO_LOGGER = java.util.logging.Logger.getLogger("io.grpc.internal");
+    private static final Level GRPC_IO_LOGGER_ORIGINAL_LEVEL = GRPC_IO_LOGGER.getLevel();

     private final String code;
     private final String filePath;
@@ -105,6 +108,8 @@ public class IPythonBuilder extends Builder implements SimpleBuildStep, Serializ
     @Override
     @SuppressFBWarnings("NP_NULL_ON_SOME_PATH_FROM_RETURN_VALUE")
     public void perform(@Nonnull Run<?, ?> run, @Nonnull FilePath ws, @Nonnull Launcher launcher, @Nonnull TaskListener listener) throws AbortException {
+        // Disable GRPC internal exceptions due to SEVERE log message when closing kernel interpreter
+        GRPC_IO_LOGGER.setLevel(Level.OFF);
         try {
             if (parserType.isEmpty() || task.isEmpty() || kernelName.isEmpty()) {
                 throw new AbortException("IPython builder is mis-configured ");
@@ -131,12 +136,13 @@ public class IPythonBuilder extends Builder implements SimpleBuildStep, Serializ
             } else {
                 run.addAction(new ResultAction(run, ws));
             }
-
-
         } catch (Throwable e) {
             e.printStackTrace(listener.getLogger());
             throw  new AbortException(e.getMessage());
         }
+        finally {
+            GRPC_IO_LOGGER.setLevel(GRPC_IO_LOGGER_ORIGINAL_LEVEL);
+        }
     }

     @Nullable
diff --git a/src/main/java/io/jenkins/plugins/ml/IPythonInterpreterManager.java b/src/main/java/io/jenkins/plugins/ml/IPythonInterpreterManager.java
index e019d2d..61477e6 100644
--- a/src/main/java/io/jenkins/plugins/ml/IPythonInterpreterManager.java
+++ b/src/main/java/io/jenkins/plugins/ml/IPythonInterpreterManager.java
@@ -157,7 +157,7 @@ public class IPythonInterpreterManager extends InterpreterManager {
         try {
             kernelInterpreter.shutdown();
         } catch (InterpreterException e) {
-            e.printStackTrace();
+            LOGGER.trace("Failed to close interpreter", e);
         }
     }
 }
Might need some further tests, check if everything makes sense, etc. But at least with this I don't see anything not expected in the Jenkins logs.
Bruno P. Kinoshita
@kinow
As for the working directory @Loghijiaha , I think there are two options at least. One is to create an issue in JIRA for Zeppelin asking to have a way to set the working directory, and maybe a PR to simply use a new value from that options-hashmap, and just call the method to setWorkingDirectory over here https://github.com/apache/zeppelin/blob/master/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/util/ProcessLauncher.java#L89

The other option is to find a way to trick the JVM into changing the working directory. Since Zeppelin will run new File("."), the only way would be to bypass that.

If we changed the working directory to the workspace, "." would be correct (instead of the IDE directory for example).

Another way... maybe... would be to monkey-patch the DefaultExecutor code, which is a bit trickier/riskier I think.
Bruno P. Kinoshita
@kinow

@Loghijiaha here's what I meant with monkey-patching... there's another way for that... classpath shadowing? Can't recall it sorry. Spent the day looking at JavaScript code, and quickly switched to Java in lunch break and a bit after work, so didn't have much time to do a good job. Sorry.

The pull request also accidentally includes the logging diff I pasted above hehe. Anyhoo, it worked for me. But this is the kind of code I'm never proud of. It works, but that's really not elegant. Feel free to take the decision on how to fix the log & the working directory. I just want to give a few options, but they are definitely not the ultimate (or even unique) solutions.

Have a great weekend!
Cheers

jenkinsci/machine-learning-plugin#57

Loghi
@Loghijiaha
Thanks @kinow That works. will find a way to change the working directory cheers
Loghi
@Loghijiaha
That draft PR is changing the working directory successfully. With little changes, we can start a review/merge on that PR before we release it. WDYT @kinow?
Bruno P. Kinoshita
@kinow
That'd be fine for me @Loghijiaha . I think it might be easier for you to work on that feature, and prepare a new PR. We can close mine, as that was done in a rather quick way, and there might be some possible improvements for the fix.
If you have time/interest, you can also submit a pull request to Zeppelin later, though that would require unit tests, deciding where to add the parameter for the working directory, follow their contributing guidelines, wait a bit, etc. I think the plug-in might be one of the first projects using their kernel that way :-)
(plus it's always nice to give back to the OSS projects/libs we use, and at the same time that would simplify the plug-in code later after their release)
Loghi
@Loghijiaha
Yup I will do the new PR. After this release, i will submit a PR regarding this change to Zeppelin.
Bruno P. Kinoshita
@kinow
All done @Loghijiaha ! Assuming everything works, I think the code should be pretty close to be released!
Great work!
Loghi
@Loghijiaha
Exactly. I will prepare for release after merging the PRs
Loghi
@Loghijiaha
@all Finally, I think everything is set with the first stable release. Great :tada:
Loghi
@Loghijiaha
What will be a first version number ? Any code of conduct applicable to the first version number?
Bruno P. Kinoshita
@kinow
@Loghijiaha well done!! Normally version number either starts at 0.1 if the release doesn't include the main features of the product (nginx was 0.x for a long time, for example, pandas too), or with 1.0. Either way would be fine IMO. More important is to follow semver to give users/sysadmins/etc a good idea of when an update is safe or not
Loghi
@Loghijiaha
Yes Thanks, @kinow jenkinsci/machine-learning-plugin#61 There was a bunch of Javadoc warning which does not allow me to release.
Bruno P. Kinoshita
@kinow
Yup, I noticed you fixed that, and also the SNAPSHOT version. I think one last check that Jenkins does is SpotBugs. In case you find some issues, it should print the command to review the spotsbugs locally. If no issues found, however, then you should be ready to publish it!
Loghi
@Loghijiaha
I ran mvn clean install, there were no spotbugs.
But the release was failed at the first time.
This message was deleted
<scm>
        <connection>scm:git:git://github.com/jenkinsci/${project.artifactId}-plugin.git</connection>
        <developerConnection>scm:git:git@github.com:jenkinsci/${project.artifactId}-plugin.git</developerConnection>
        <url>https://github.com/jenkinsci/${project.artifactId}-plugin</url>
      <tag>1.0</tag>
Can i do a release for 1.0 now or 1.1?
Bruno P. Kinoshita
@kinow
For 1.0. If it hasn't been released already, you should be able to release 1.0
If there's already a tag, you can - always carefully - remove the tag (do note that in some projects editing tags may be prohibited, with hooks to make them read-only)
Loghi
@Loghijiaha
Released :tada:
Will it take time to reflect on io.jenkins ?
Bruno P. Kinoshita
@kinow
I think it takes about 2 hours? I think there's a process somewhere that runs every 2 hours to sync the update center... Though that value could have changed (or my memory could be wrong, more likely haha).
But congrats Loghi!!! I bet it feels good to have the final version of the plug-in working as expected, and released!
Looking forward to our meeting tomorrow (for me at least it's tomorrow). o/
Loghi
@Loghijiaha
Yes it is now available in the plugin reo.
Ioannis K. Moutsatsos
@imoutsatsos
image.png
1 reply
@kinow I have followed your instructions from Aug 11, and run with a fresh master checkout. Although it seems that I can now start a python3 jupyter kernel it fails with io.grpc error later
image.png
@kinow Python wirh miniconda
Ioannis K. Moutsatsos
@imoutsatsos
image.png