Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Jan 30 2019 20:42
    timja commented #56
  • Jan 30 2019 20:32
    imaffe commented #56
  • Jan 30 2019 18:34
    imaffe commented #56
  • Jan 29 2019 11:58
    martinda commented #56
  • Jan 29 2019 07:07
    imaffe commented #56
  • Jan 28 2019 18:35
    imaffe synchronize #56
  • Jan 28 2019 14:28
  • Jan 26 2019 14:50
    martinda commented #56
  • Jan 26 2019 08:28
    imaffe commented #56
  • Jan 26 2019 08:20
    imaffe commented #56
  • Jan 22 2019 16:33
  • Jan 19 2019 17:48
    imaffe commented #56
  • Jan 19 2019 10:52
    oleg-nenashev commented #56
  • Jan 19 2019 04:42
    imaffe commented #56
  • Jan 19 2019 04:31
    imaffe synchronize #56
  • Jan 19 2019 03:38
    imaffe commented #56
  • Jan 19 2019 03:32
    imaffe commented #56
  • Jan 17 2019 07:20
    alexsomai closed #58
  • Jan 17 2019 06:32
    imaffe commented #51
  • Jan 17 2019 03:46
    imaffe synchronize #56
Raghunath Krishnan
@raghunathkrish_twitter
got it, thanks @martinda and @alexsomai
Alexandru Somai
@alexsomai
hi @martinda. I've merged jenkinsci/external-workspace-manager-plugin#68. I'm thinking to do a release with this change, to have support for JCasC, but not very sure when. Also, a double check after the release would be helpful, just to make sure we didn't break anything.
What do you think if I do the release tomorrow sometime? Or would it be better to wait for the next week?
Martin d'Anjou
@martinda
@alexsomai you can go ahead when you are ready I have tried the plugin locally, it looks good. @BenjaminBeggs and I work together and he will do some last checks.
Benjamin Beggs
@BenjaminBeggs
Indeed, I will watch for the release of this and perform some tests to ensure nothing has been broken
Alexandru Somai
@alexsomai
@BenjaminBeggs @martinda hi guys, sorry, but I'm quite busy with something else, and I won't have time to perform the release these days. Most probably I'll do it on Thursday, or Friday. I'll let you know when it's done.
Alexandru Somai
@alexsomai
@BenjaminBeggs @martinda I've finally found some time to do the release with JCasC support. It should be available in the Jenkins Plugin Manager in a few hours I think.
Martin d'Anjou
@martinda
Thanks @alexsomai !
Benjamin Beggs
@BenjaminBeggs
@alexsomai @martinda A set of test jobs have been run to test this new update. Everything appears to be functioning as expected.
Martin d'Anjou
@martinda
Nice!
Alexandru Somai
@alexsomai
Awesome, thanks!
Martin d'Anjou
@martinda
Just want to warn users of the external workspace manager plugin of a problem since Jenkins LTS 2.150.x.
If you write code like this: exws(ws) { stage() { ...} } then you have to switch it around to stage() { exws(ws) { ...} } because the stage appears to reset the workspace path.
Same for withEnv and withCredentials.
It appears to be related to a change in the core, since Jenkins LTS 2.138.x does not have the problem.
Martin d'Anjou
@martinda
Hi Alex. What problem are you seeing with the RunSelector plugin jenkinsci/run-selector-plugin#23 ?
@alexsomai
Alexandru Somai
@alexsomai
Hi Martin @martinda. Are you asking:
  1. why I did the upgrade?
  2. or if I have any issues on that particular PR?
    The answers:
    1. I did the upgrade because jenkinsci/run-selector-plugin#22 would not work otherwise. Not very sure if this is really needed or not actually.
    2. I don't have any other issues on this PR (number 23), the upgrade seems to work, but I was waiting for at least 1 thumbs up from someone within the Jenkins community.
and btw, with regards to what you said previous days, about the compatibility issues between exws() and stage(). Do you think we should do something about it in the plugin? Or just something to be aware of?
Martin d'Anjou
@martinda
Yes I think something needs to be done. See https://groups.google.com/forum/#!topic/jenkinsci-dev/TXBLRt5ZxjI
I started working on it but I have not tried the “BOM” yet.
Alexandru Somai
@alexsomai
that BOM thing is a really cool feature, didn't know about it
Martin d'Anjou
@martinda
I have fixed the issue with Jenkins 2.150.x. See jenkinsci/external-workspace-manager-plugin#70
But some of the clean up tests have become flaky. I posted to the mailing list about it.
Alexandru Somai
@alexsomai
@martinda just FYI - I've triggered a new release with your latest PR #70
Martin d'Anjou
@martinda
Thanks @alexsomai
Justin Dietz
@JoA-MoS
@martinda - In regards to using exws with docker what would be the syntax. I am spinning up multiple docker images in parallel and I want them to all share the same workspace. This is what I am trying so far:
node('dockerserver') {
    withCredentials([
      usernamePassword(
        credentialsId: 'svc',
        usernameVariable: 'SVC_AUTHENTICATION_USR',
        passwordVariable: 'SVC_AUTHENTICATION_PSW'
      ),
      string(
        credentialsId: 'bb',
        variable: 'BB_PAT'
      ),
      string(
        credentialsId: 'nlt',
        variable: 'NPM_LOCAL_TOKEN'
      )
    ]) {
      withEnv([
        "HOME=${workspace}"
      ]) {
        exws(extWorkspace) {
          docker.image('node-builder')
            .inside('--tmpfs /.config --ipc=host', cl)
        }
      }
    }
  }
Martin d'Anjou
@martinda
Hi @JoA-MoS I do not use docker. Maybe you have already seen this page it explains how to allocate, then share a workspace. In your code example, where do you define the value of ${workspace}? I guess that might be part of your question as well. You should try to allocate a workspace first with exwsAllocate, then you can use it with exws. Hope this helps...
Justin Dietz
@JoA-MoS
Thank @martinda - I am allocating the workspace in an earlier step but you did identify the problem ${workspace} is the default jenkins job workspace not the exws is there a way to get the computed exws I would like home to be set to the same value as the exws path ? If I hardcode HOME = /mount-point/job/path/12 my pipeline works so if I can just get this from exws. I tried extWorkspace.pathOnDisk but got an error that the method isn't allowed to be called by a script.
Justin Dietz
@JoA-MoS
ahh... I figured it out by approving the method in manage jenkins --> In-process script approvals.
Jerry Wiltse
@solvingj
Hi, i just learned about this plugin
how does it handle locking and resource contention which was a known issue with regular workspaces, and the reason they had to employ the dynamic workspaces with workspace@1 and workspace@2 ?
also, does this work across multiple agents?
in general it seems very hard to orchestrate a shared folder across multiple jobs all trying to read/write at the same time. Supporting multiple hosts would be yet another major layer of complexity.
Alexandru Somai
@alexsomai

Hi @solvingj Yes, the plugin works across multiple agents, as long as all of them mount the same disk (via NFS, for example). Please make sure that you have the prerequisites set-up correctly before running the plugin.

Now, related to potential multithreading issues that many agents may read/write the same resource - I have to say I didn't have any problems with that. The plugin doesn't do anything special to handle these cases, but I guess there is some resource locking and handling at OS level. Moreover, it depends how you use the plugin. The common approach, I think, is to have one master Jenkins job that writes on the shared disk, then it chains other downstream Jenkins job that reuse the same workspace, but don't necessary run in parallel.

Jerry Wiltse
@solvingj
Ok thanks very much. For the use cases I was thinking of, NFS as a backend won't work. The sharing would have to be implemented purely in Java, which would have to be extremely complicated and robust to make that work... so I was pretty sure that wasn't going to be the case.
ainur-fa
@ainur-fa

Hello.
Why the following doesn't work I can't figure out:

def extWorkspace = exwsAllocate "diskpool1"
node('R584_CrossTests(LSB_x86)_Debian.9.8-i686') {
    exws (extWorkspace) {
        sh 'ls -la'
        sh 'touch test1.txt'
        sh 'ls -la'
    }
}

node('R584_CrossTests(LSB_LicenseFree_x86)_Debian.9.8-i686') {
     exws (extWorkspace) {
        sh 'ls -la'
        sh 'touch test2.txt'
        sh 'ls -la'
    }
}

I get:

Disk allocation strategy was not provided as step parameter. Fallback to the strategy defined in the Jenkins global config
Using Disk allocation strategy: 'Select the Disk with the most usable space'
Selected Disk ID 'disk1' from the Disk Pool ID 'diskpool1'
The path on Disk is: folder1/for_test/Fazlyev/upstream/85
[Pipeline] node
Running on R584_CrossTests(LSB_x86)_Debian.9.8-i686 in /home/tester/workspace/for_test/Fazlyev/upstream
[Pipeline] {
[Pipeline] exws
Searching for disk definitions in the External Workspace Templates from Jenkins global config
Searching for disk definitions in the Node config
Running in /mnt/disk1/folder1/for_test/Fazlyev/upstream/85
[Pipeline] {
[Pipeline] sh

  • ls -la
    итого 8
    drwxr-xr-x 2 root root 4096 июл 17 09:32 .
    drwxr-xr-x 14 root root 4096 июл 17 09:32 ..
    [Pipeline] sh
  • touch test1.txt
    [Pipeline] sh
  • ls -la
    итого 8
    drwxr-xr-x 2 root root 4096 июл 17 09:32 .
    drwxr-xr-x 14 root root 4096 июл 17 09:32 ..
    -rw-r--r-- 1 root root 0 июл 17 09:32 test1.txt
    [Pipeline] }
    [Pipeline] // exws
    [Pipeline] }
    [Pipeline] // node
    [Pipeline] node
    Running on R584_CrossTests(LSB_LicenseFree_x86)_Debian.9.8-i686 in /home/tester/workspace/for_test/Fazlyev/upstream
    [Pipeline] {
    [Pipeline] exws
    Searching for disk definitions in the External Workspace Templates from Jenkins global config
    Searching for disk definitions in the Node config
    Running in /mnt/disk1/folder1/for_test/Fazlyev/upstream/85
    [Pipeline] {
    [Pipeline] sh
  • ls -la
    итого 8
    drwxr-xr-x 2 root root 4096 июл 17 09:32 .
    drwxr-xr-x 14 root root 4096 июл 17 09:32 ..
    [Pipeline] sh
  • touch test2.txt
    [Pipeline] sh
  • ls -la
    итого 8
    drwxr-xr-x 2 root root 4096 июл 17 09:32 .
    drwxr-xr-x 14 root root 4096 июл 17 09:32 ..
    -rw-r--r-- 1 root root 0 июл 17 09:32 test2.txt
    [Pipeline] }
    [Pipeline] // exws
    [Pipeline] }
    [Pipeline] // node
    [Pipeline] End of Pipeline
    Finished: SUCCESS

Why doesn't the second agent see the file created by the first agent?

Alexandru Somai
@alexsomai
Hi @ainur-fa, based on the console output, it looks correct. I just want to double check, did you set-up correctly the prerequisites, especially the Node Properties: https://github.com/jenkinsci/external-workspace-manager-plugin/blob/master/doc/PREREQUISITES.md#node-properties? And please make sure that the mounts are correctly working. Can you list the same files when accessing the mnt path on your nodes?
5 replies
seungilahn
@dkstm95
Hello! Can't use this plug-in on Master(Windows server) and Agent(Windows server)? Even if I set it up as it is in README.md, Agent doesn't seem to find the file.
seungilahn
@dkstm95
image.png
I wrote pipeline script this way, but there was an error(" cannot find '.' ") in Test stage.
seungilahn
@dkstm95
@alexsomai
Alexandru Somai
@alexsomai
The pipeline script looks good, I'm not very sure what could be the issue here ...
2 replies