Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • Mar 31 22:21
    dependabot[bot] labeled #68
  • Mar 31 22:21
    dependabot[bot] opened #68
  • Mar 31 22:21

    dependabot[bot] on maven

    Bump guava from 28.2-jre to 29.… (compare)

  • Oct 12 2020 17:03
    markyjackson-taulia commented #35
  • Aug 29 2020 13:49

    Loghijiaha on master

    code cleanup Added HTML view (compare)

  • Aug 29 2020 13:49
    Loghijiaha closed #67
  • Aug 28 2020 18:21
    Loghijiaha commented #67
  • Aug 28 2020 18:20
    Loghijiaha synchronize #67
  • Aug 28 2020 17:48
    Loghijiaha opened #67
  • Aug 28 2020 07:48

    Loghijiaha on master

    Fixed unnecessary SEVERE log (compare)

  • Aug 28 2020 07:48
    Loghijiaha closed #66
  • Aug 27 2020 17:45
    Loghijiaha opened #66
  • Aug 24 2020 20:32

    Loghijiaha on master

    Updated release draft Updated changelog (compare)

  • Aug 24 2020 20:32
    Loghijiaha closed #65
  • Aug 24 2020 20:22
    Loghijiaha commented #65
  • Aug 24 2020 15:44
    Loghijiaha synchronize #65
  • Aug 24 2020 15:43
    Loghijiaha opened #65
  • Aug 24 2020 10:48

    Loghijiaha on master

    [maven-release-plugin] prepare … (compare)

  • Aug 24 2020 10:48

    Loghijiaha on machine-learning-1.0.1

    (compare)

  • Aug 24 2020 10:47

    Loghijiaha on master

    [maven-release-plugin] prepare … (compare)

Junholv OBO
@junholv
Capture d’écran 2020-07-27 à 01.04.35.png
Loghi
@Loghijiaha
@kinow I have just raised the PR to give an insight into what I m going to do in the next week programmatically explained there. I have not finalized them. I will finalize all the changes after we discussed. there are issues some issues with saving htmls in jenkins workspace. will disscuss it.
Bruno P. Kinoshita
@kinow
Roger that @Loghijiaha
Loghi
@Loghijiaha
Loghi
@Loghijiaha
voting for demo timslot : https://doodle.com/poll/bwqax28iaq6xd7ez
Loghi
@Loghijiaha
After using this System.setProperty("hudson.model.DirectoryBrowserSupport.CSP", ""); the javascript/html is running on the plugin.
Oleg Nenashev
@oleg-nenashev
@Loghijiaha I made a few non-blocking comments in jenkins-infra/jenkins.io#3562 . Would you like to address them? Or should I just ship the blog?
Loghi
@Loghijiaha
I have made the changes @oleg-nenashev . You can proceed with the PR
Oleg Nenashev
@oleg-nenashev
:+1:
Loghi
@Loghijiaha
@kinow there is way after Jenkins 2.200+ an explicitly relaxing the CSP using a root resource URL. But it is too complex to understand to me. Most of the Jupyter's HTML outputs need to run js, I found System.setProperty("hudson.model.DirectoryBrowserSupport.CSP", "sandbox allow-scripts;"); this was a simple way relax CSP, but unsafe. Users should know about this problem when they generate complex js/Html output. What would be our next step on this issue?
Bruno P. Kinoshita
@kinow

Hi @Loghijiaha , that System.setProperty is useful in confirming that enabling that fixes the issue. But changing a configuration like that in a build step is definitely bad.

But it is too complex to understand to me.

Unfortunately I think we have to dig deeper on this option even though it is complex. We have a few options here:

  1. do a deeper investigation into CSP with Jenkins, and either find a way to enable that in jobs with the plugin or let the user choose what to do
  2. document that users will have to enable CSP in their Jenkins installation for testing
  3. check if there are other alternatives, such as the HTML Publisher Plug-in, and if so, document too

I think it'd be best to go with 2. or 3.. If necessary, leave an issue for later. This is not a must-have feature from the plug-in, and we know now that if users enable that env-var or setting in Jenkins, HTML+JS will be correctly rendered.

What do you think @Loghijiaha ?

Loghi
@Loghijiaha
@kinow I agree with the option with 2 and we can document to enable CSP themselves.
Oleg Nenashev
@oleg-nenashev

And a LinkedIn message draft: `Machine Learning Plugin project - Coding Phase 2 project update by Loghi Perinpanayagam: agent Docker image for Jupyter Notebooks, migration to Python 3, user documentation, testing in environment and bugfixes.

https://www.jenkins.io/blog/2020/07/27/machine-learning-plugin-coding-phase2/`

Loghi
@Loghijiaha
Hi @oleg. The open graph is fantastic. I'd 100 for that. The linkedIn draft is looking good for me. How can I add those three images in the blog post in a aligned line ? Should I merge three of them or ?
Oleg Nenashev
@oleg-nenashev
machine-learning.png
Loghi
@Loghijiaha
okay thanks, @oleg I will give PR on that
Oleg Nenashev
@oleg-nenashev
Yeah. Just follow the example in other blogs. I will launch the LinkedIn post, it does not need opengraph metadata
Ioannis K. Moutsatsos
@imoutsatsos
Nice presentation today @Loghijiaha ! It was great to also realize that our DevOps colleagues are also interested in ML analysis of Jenkins artifacts and builds. Hadn't thought about that @oleg-nenashev @markearlwaite. Thanks!
Shivay Lamba
@shivaylamba
Yes great presentation Loghi
Loghi
@Loghijiaha
Thank you @imoutsatsos @shivaylamba .Will finish this and release a stable/RC release soon. :cheers:
Bruno P. Kinoshita
@kinow
@Loghijiaha for testing in a distributed environment, I think first you can try writing a simple pipeline job https://www.jenkins.io/doc/book/pipeline/docker/#execution-environment
But instead of the node command, try running some Python. You'll also need to check if the machine-learning-plugin is compatible with pipelines, or if we need to add the dependency + a few changes in the Java API.
If that works, then you should be able to create a test using the Jenkins testing API (that you are already using in other tests): https://www.jenkins.io/doc/developer/testing/#within-a-pipeline
Bruno P. Kinoshita
@kinow
Here's an example from a plugin that contains a test for the head node, and another for running in an agent: https://github.com/jenkinsci/cmakebuilder-plugin/blob/273b72bcd732718bd17de4c98932ebf806fc5ea4/src/test/java/hudson/plugins/cmake/CmakeBuilderBuildStepTest.java
Loghi
@Loghijiaha
@kinow @imoutsatsos @shivaylamba I found an API Zeppelin-jupyter where we could specify the kernel then it will run the code where you chooses. I have tested with python and R. It works :full_moon:
However, users need to input which kernel they want. For that can we have a drop-down or a textbox? Or we can discuss if we will be able to set up another meeting this week?
Loghi
@Loghijiaha
R_build.png
R_output.png
Loghi
@Loghijiaha
This feature has implemented in 0.9.0-preview1 Zeppelin. More details http://zeppelin.apache.org/docs/0.9.0-preview1/interpreter/jupyter.html
Loghi
@Loghijiaha
It is working for every kernel that already installed in my system(Eg: javascript, R, etc).
javascript_build.png
javascript_output.png
Shivay Lamba
@shivaylamba
Sure we can have a meeting.
A drop down menu should be fine
Bruno P. Kinoshita
@kinow

@Loghijiaha I think configuring the kernel is fine. As long as the user can choose the kernel in the job configuration, that should be enough. Probably the kernel needs to be selected per build step, so you could have

  • a build step to pre-process data with Python/Groovy
  • a build step to train the model with Python/C/C++
  • a build step to validate the model with Shell

etc. For each build step, s/he would just need to pick the right kernel.

Really looking forward to seeing this in action @Loghijiaha !
I have a meeting tonight, and work tomorrow only until 1/2 PM my time. Then on leave for the rest of the week. Can try to join the meeting, but cannot promise. But do have the meeting if other mentors are available. I'm sure @shivaylamba and @imoutsatsos will have great feedback!
Loghi
@Loghijiaha
That's awesome @shivay. I will work on that soon @kinow. For jupyter interpreter, there are a lot of kernels available here https://github.com/jupyter/jupyter/wiki/Jupyter-kernels. Should I include all these manually in the select option? or any special way to do this ?
Bruno P. Kinoshita
@kinow
@Loghijiaha ideally, it would be nice if Jenkins could figure out what are the kernels I have installed.
Here's what I have at the moment.
(seaglass) kinow@ranma:/tmp$ jupyter kernelspec list --debug --json
{
  "kernelspecs": {
    "javascript": {
      "resource_dir": "/home/kinow/.local/share/jupyter/kernels/javascript",
      "spec": {
        "argv": [
          "ijskernel",
          "--hide-undefined",
          "{connection_file}",
          "--protocol=5.1"
        ],
        "env": {},
        "display_name": "Javascript (Node.js)",
        "language": "javascript",
        "interrupt_mode": "signal",
        "metadata": {}
      }
    },
    "kinow-notebooks": {
      "resource_dir": "/home/kinow/.local/share/jupyter/kernels/kinow-notebooks",
      "spec": {
        "argv": [
          "/home/kinow/Development/python/workspace/notebooks/venv/bin/python",
          "-m",
          "ipykernel_launcher",
          "-f",
          "{connection_file}"
        ],
        "env": {},
        "display_name": "kinow-notebooks",
        "language": "python",
        "interrupt_mode": "signal",
        "metadata": {}
      }
    },
    "tooling": {
      "resource_dir": "/home/kinow/.local/share/jupyter/kernels/tooling",
      "spec": {
        "argv": [
          "/home/kinow/Development/python/tooling/venv/bin/python",
          "-m",
          "ipykernel_launcher",
          "-f",
          "{connection_file}"
        ],
        "env": {},
        "display_name": "tooling",
        "language": "python",
        "interrupt_mode": "signal",
        "metadata": {}
      }
    },
    "python3": {
      "resource_dir": "/home/kinow/Development/python/miniconda3/envs/seaglass/share/jupyter/kernels/python3",
      "spec": {
        "argv": [
          "/home/kinow/Development/python/miniconda3/envs/seaglass/bin/python",
          "-m",
          "ipykernel_launcher",
          "-f",
          "{connection_file}"
        ],
        "env": {},
        "display_name": "Python 3",
        "language": "python",
        "interrupt_mode": "signal",
        "metadata": {}
      }
    }
  }
}

4 kernels. See the language attribute. If Jenkins showed me Julia or R, that would probably result in an error, as I don't have the Kernels installed. It would be nice if it gave me the choice to choose the kernel, like when you start Jupyter Lab/Notebook.

And also displayed the language of the Kernel.

Also, if I wrote a script or uploaded a script in JavaScript, would be nice if it gave me the option to pick from a list of kernels that support javascript.

Said all that, I think we should start as simple as possible. So will take a look at your pull request, and see if I can get JS and Python running @Loghijiaha

Loghi
@Loghijiaha
@kinow I was thinking about that. But when it comes to a distributive environment it is hard to get the kernel spec before we start building.
Bruno P. Kinoshita
@kinow
Hmmm, good point (and you are already thinking about this kind of issues, well done @Loghijiaha !!!).
Bruno P. Kinoshita
@kinow
@Loghijiaha sorry was preparing a few things today for a long holiday. Might still have time to review things, but with less bandwidth until Monday 10th o/
Loghi
@Loghijiaha
That's fine @kinow, I am fixing that CI issue. Later will work on persistent data testing.
And enjoy the holiday !! :hotsprings: