Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • Mar 31 22:21
    dependabot[bot] labeled #68
  • Mar 31 22:21
    dependabot[bot] opened #68
  • Mar 31 22:21

    dependabot[bot] on maven

    Bump guava from 28.2-jre to 29.… (compare)

  • Oct 12 2020 17:03
    markyjackson-taulia commented #35
  • Aug 29 2020 13:49

    Loghijiaha on master

    code cleanup Added HTML view (compare)

  • Aug 29 2020 13:49
    Loghijiaha closed #67
  • Aug 28 2020 18:21
    Loghijiaha commented #67
  • Aug 28 2020 18:20
    Loghijiaha synchronize #67
  • Aug 28 2020 17:48
    Loghijiaha opened #67
  • Aug 28 2020 07:48

    Loghijiaha on master

    Fixed unnecessary SEVERE log (compare)

  • Aug 28 2020 07:48
    Loghijiaha closed #66
  • Aug 27 2020 17:45
    Loghijiaha opened #66
  • Aug 24 2020 20:32

    Loghijiaha on master

    Updated release draft Updated changelog (compare)

  • Aug 24 2020 20:32
    Loghijiaha closed #65
  • Aug 24 2020 20:22
    Loghijiaha commented #65
  • Aug 24 2020 15:44
    Loghijiaha synchronize #65
  • Aug 24 2020 15:43
    Loghijiaha opened #65
  • Aug 24 2020 10:48

    Loghijiaha on master

    [maven-release-plugin] prepare … (compare)

  • Aug 24 2020 10:48

    Loghijiaha on machine-learning-1.0.1

    (compare)

  • Aug 24 2020 10:47

    Loghijiaha on master

    [maven-release-plugin] prepare … (compare)

Bruno P. Kinoshita
@kinow
I have a meeting tonight, and work tomorrow only until 1/2 PM my time. Then on leave for the rest of the week. Can try to join the meeting, but cannot promise. But do have the meeting if other mentors are available. I'm sure @shivaylamba and @imoutsatsos will have great feedback!
Loghi
@Loghijiaha
That's awesome @shivay. I will work on that soon @kinow. For jupyter interpreter, there are a lot of kernels available here https://github.com/jupyter/jupyter/wiki/Jupyter-kernels. Should I include all these manually in the select option? or any special way to do this ?
Bruno P. Kinoshita
@kinow
@Loghijiaha ideally, it would be nice if Jenkins could figure out what are the kernels I have installed.
Here's what I have at the moment.
(seaglass) kinow@ranma:/tmp$ jupyter kernelspec list --debug --json
{
  "kernelspecs": {
    "javascript": {
      "resource_dir": "/home/kinow/.local/share/jupyter/kernels/javascript",
      "spec": {
        "argv": [
          "ijskernel",
          "--hide-undefined",
          "{connection_file}",
          "--protocol=5.1"
        ],
        "env": {},
        "display_name": "Javascript (Node.js)",
        "language": "javascript",
        "interrupt_mode": "signal",
        "metadata": {}
      }
    },
    "kinow-notebooks": {
      "resource_dir": "/home/kinow/.local/share/jupyter/kernels/kinow-notebooks",
      "spec": {
        "argv": [
          "/home/kinow/Development/python/workspace/notebooks/venv/bin/python",
          "-m",
          "ipykernel_launcher",
          "-f",
          "{connection_file}"
        ],
        "env": {},
        "display_name": "kinow-notebooks",
        "language": "python",
        "interrupt_mode": "signal",
        "metadata": {}
      }
    },
    "tooling": {
      "resource_dir": "/home/kinow/.local/share/jupyter/kernels/tooling",
      "spec": {
        "argv": [
          "/home/kinow/Development/python/tooling/venv/bin/python",
          "-m",
          "ipykernel_launcher",
          "-f",
          "{connection_file}"
        ],
        "env": {},
        "display_name": "tooling",
        "language": "python",
        "interrupt_mode": "signal",
        "metadata": {}
      }
    },
    "python3": {
      "resource_dir": "/home/kinow/Development/python/miniconda3/envs/seaglass/share/jupyter/kernels/python3",
      "spec": {
        "argv": [
          "/home/kinow/Development/python/miniconda3/envs/seaglass/bin/python",
          "-m",
          "ipykernel_launcher",
          "-f",
          "{connection_file}"
        ],
        "env": {},
        "display_name": "Python 3",
        "language": "python",
        "interrupt_mode": "signal",
        "metadata": {}
      }
    }
  }
}

4 kernels. See the language attribute. If Jenkins showed me Julia or R, that would probably result in an error, as I don't have the Kernels installed. It would be nice if it gave me the choice to choose the kernel, like when you start Jupyter Lab/Notebook.

And also displayed the language of the Kernel.

Also, if I wrote a script or uploaded a script in JavaScript, would be nice if it gave me the option to pick from a list of kernels that support javascript.

Said all that, I think we should start as simple as possible. So will take a look at your pull request, and see if I can get JS and Python running @Loghijiaha

Loghi
@Loghijiaha
@kinow I was thinking about that. But when it comes to a distributive environment it is hard to get the kernel spec before we start building.
Bruno P. Kinoshita
@kinow
Hmmm, good point (and you are already thinking about this kind of issues, well done @Loghijiaha !!!).
Bruno P. Kinoshita
@kinow
@Loghijiaha sorry was preparing a few things today for a long holiday. Might still have time to review things, but with less bandwidth until Monday 10th o/
Loghi
@Loghijiaha
That's fine @kinow, I am fixing that CI issue. Later will work on persistent data testing.
And enjoy the holiday !! :hotsprings:
Loghi
@Loghijiaha
Im sorry. My battery was down. I didnt see the power. I will join soon
Loghi
@Loghijiaha
JIRA for kernel configuration -> https://issues.jenkins-ci.org/browse/JENKINS-63363
Loghi
@Loghijiaha

ML plugin can run different kernel codes(pure scripts). When I tried to run a javascript based notebook(JS.ipynb), that is not successful. Because of markdown elements should be replaced with the comment tag(# for python) of the language we are running.

     Note n = new JupyterUtil().getNote(inputStreamReader, "python", "\n", "#");
            Gson gson = new Gson();
            JsonElement tree = gson.toJsonTree(n);
            JsonObject obj = tree.getAsJsonObject();

This works only for python. not working for other languages.
https://issues.jenkins-ci.org/browse/JENKINS-63379

Loghi
@Loghijiaha
I found a way. Fixing it. will give PR tommorow.
Bruno P. Kinoshita
@kinow
Great progress so far @Loghijiaha
@imoutsatsos just tested the plugin during a meeting, and it worked on Windows
Anaconda Prompt, with Python 3.7.1. Windows 10 Pro. JDK 1.8
image.png
Bruno P. Kinoshita
@kinow
image.png
image.png
All I did was
  1. start prompt
  2. python -m venv venv (I have always used venv/virtualenv, so I feel more comfortable using it, rather than the conda environments...)
  3. pip install -r requirements.txt
  4. (venv) (base) C:\Users\kinow\Development\java\jenkins\machine-learning-plugin>c:\Users\kinow\Development\java\apache-maven-3.6.1\bin\mvn.cmd clean install
  5. (venv) (base) C:\Users\kinow\Development\java\jenkins\machine-learning-plugin>c:\Users\kinow\Development\java\apache-maven-3.6.1\bin\mvn.cmd hpi:run
Used latest code in master. Hope that helps.
Loghi
@Loghijiaha
Nice work @kinow
Bruno P. Kinoshita
@kinow
Thanks @Loghijiaha !
Loghi
@Loghijiaha
A quick question. Should we need to save images and HTML in different directories associated with build number?
Bruno P. Kinoshita
@kinow
I don't think so. I think we should just output the files in the workspace. If users want, they can use plug-ins or organize their builds to archive artefacts or move the files elsewhere, etc. But maybe @Loghijiaha @imoutsatsos have use cases for that?
1 reply
Loghi
@Loghijiaha
jenkinsci/machine-learning-plugin#51 opened PR for the last task to be finished :hand:
Loghi
@Loghijiaha
Today's meeting :
Zoom link : https://zoom.us/j/98822238159
(LogLevel, not Level I think)
Arg, that's from the plug-in. Let me open the IDE to confirm what methods are aavailable
Bruno P. Kinoshita
@kinow

https://github.com/jenkinsci/jira-trigger-plugin/blob/7b96a75ae7aba5ca5f2828825f1470b9093b77c3/src/integrationTest/groovy/com/ceilfors/jenkins/plugins/jiratrigger/integration/JulLogLevelRule.groovy#L35

It looks like calling the java.util.Logger setLevel works for the NullPointerException. But there's another exception that persists even after this change. I wonder if it wouldn't be the case to dig into the code of Zeppelin to understand why both exceptions are happening @Loghijiaha ? Perhaps it's a parameter that is missing, or some method that was not invoked?

4 replies
Loghi
@Loghijiaha
One of Zeppelin maintainer commented on that issue. https://issues.apache.org/jira/browse/ZEPPELIN-4569
@kinow
Bruno P. Kinoshita
@kinow
Great initiative @Loghijiaha . I've replied to his comment too. Let's see if they know how to silence those exceptions.
Loghi
@Loghijiaha
@all still there is no reply to the issue in https://issues.apache.org/jira/browse/ZEPPELIN-4569. If we are set with the pending issue, We may release the stable version in the next few days.
Bruno P. Kinoshita
@kinow
Let's wait and see if they reply, and keep investigating in the meantime @Loghijiaha . Found this one bug debugging the plug-in: https://issues.jenkins-ci.org/browse/JENKINS-63452
Loghi
@Loghijiaha
Ah. Nice catch @kinow will look into that
Bruno P. Kinoshita
@kinow
@Loghijiaha spent some more time debugging it. Still no luck. But maybe this one can give us some clue: grpc/grpc-java#1577
Loghi
@Loghijiaha
And also we can remove the directory changing line and let it as default.
Bruno P. Kinoshita
@kinow
Yup. I don't think this is really a blocker for a release @Loghijiaha . We can set a deadline, something like Friday or Saturday your time, up to you.
If we can't find a solution, you can prepare a release before the demo.
That way after others watch your demo they can grab this latest release, and you can work on this issues with more calm. I believe we have everything from our design doc submitted to GSoC right?
Loghi
@Loghijiaha
Yh, we all set with the GSoC design doc with some minor changes.
Bruno P. Kinoshita
@kinow
Brilliant. I'll keep looking at this logging issue in the spare time. But in the worst case we just need to tell users about it.
Loghi
@Loghijiaha
Great @kinow :heavy_check_mark:
Loghi
@Loghijiaha

Demo 3 schedule:

Part 1, 2PM UTC, Aug 27 (Thu)

  • Rishabh
  • Kezhi
  • Sladyn

Part 2, 2PM UTC, Aug 28 (Fri)

  • Loghi
  • Buddhika
  • Sumit
Loghi
@Loghijiaha
@kinow there is an issue with default directory settings. Notebooks/scripts are not recognizing the files/dataset eg: example_with_images. We should at least have to change the directory to the workspace for python.
Loghi
@Loghijiaha
procEnvir.png