Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Mark Waite
    @MarkEWaite
    @arpoch I have never investigated the use of SELinux or any variants of SELinux. The last time I interacted with it was many years ago when I made the mistake of enabling it on a CentOS or Red Hat Linux installation. I believe I ultimately replaced the operating system rather than continue wrestling with its restrictions.
    I know that Jim Klimov has done work in the git client plugin to attempt to support it and I suspect that I will eventually need to add it to my test environment, but I've not had time to do that yet.
    Harshit Chopra
    @arpoch
    Hi all,
    I just wanted to clarify that our first meeting for git credentials project will be taking place any day b/w 19-05-21 to 21-05-21.
    Currently I am working on the UI implementation and using credential binding plugin as reference to work on the integration of git credential in pipeline job,alongside creating a more refined roadmap for the project.
    1 reply
    Bassam Khouri
    @bkhouri
    Hi.. I have a declarative pipleine job that have multiple parallel stages. Each parallel stage does a checkout scm and it seem that sometimes, the stages do not clone the same commit. I came accros https://groups.google.com/g/jenkinsci-users/c/P-ErEhzNDOA?pli=1 from years ago which state the issue is fixed in git plugin 2.6.1+. We are running git version 4.6.0. however I still encounter the same symptoms as the original poster.
    2 replies
    Harshit Chopra
    @arpoch
    @MarkEWaite @rishabhBudhouliya @justinharringa , from our discussion earlier today, I have been working on the jenkins hello world archetype to have a more clear insight regarding the extension points, structure of a plugin. Also I guess I have a lead on making git plugin compatible with pipeline script using workflow-scm-step-plugin although it needs more research, thanks for the resources provided. With that I had some questions in mind for tomorrows session.
    3 replies
    Harshit Chopra
    @arpoch
    Screenshot_2021-05-20 Pipeline Syntax Snippet Generator [Jenkins].png
    @MarkEWaite @rishabhBudhouliya @justinharringa , I was working on the idea we were discussing yesterday/today, I was able to develop the UI(attaching the image above) also I have made some modifications such as removing the field of Username and Password variables as all the authentication stuff will be handled in the backend so the user need not to worry about the setting the variables.
    Also another area that I am a bit worried about and going to investigate more on it later is how is a withCredential snippet is handled in a pipeline script and also looking for some guidance on that.
    Also I will keep on updating about any further progress made daily.
    Mark Waite
    @MarkEWaite

    Awesome work @arpoch . That UI screen shot looks great. Well done!

    Would like it very much if you would add an annotation for @Symbol("gitUsernamePassword") so that the $class would be replaced with the gitUsernamePassword string

    Harshit Chopra
    @arpoch
    Thanks @MarkEWaite, added the @Symbol annotation.
    Currently I am working on execution of the context inside or the implementation of the withCredential(gitUsernamePassword()){}. Will update on Monday.
    Justin Harringa
    @justinharringa
    Excellent!
    Harshit Chopra
    @arpoch
    Hi all,
    @justinharringa , @MarkEWaite , @rishabhBudhouliya , I am attaching a script below which performs adding ,removing, pushing tags, making a commit and pushing the changes and all this is done on a private repo 'https://github.com/arpoch/Jenkins-Bug-Forge.git'. Good news everything worked as expected though this is performed on Linux dist(Ubuntu) and uses Username and Password for authentication.
    Also git credential binding implementation is independent of git/git-client plugin so its just a choice to put it these plugins, as all is handed gracefully by Credential Binding plugin API's.
    But I have some concerns-
    • As using sh to perform git operation, it exposes the user to raw commandline thus the operation such as cloning a repo would result in a fatal error if the repo already exists and stops the pipeline. As a result I have used gitsnippet to perform the clone whcih switches to fetch if a repo already exists. So is such a "filter" should be provided for git credential binding as well or dependence on git and checkout snippet is fine for that.
    • Another thing that I wanted to discuss, the authentication process in the git-client plugin and git credential binding is somewhat same but since the git-client-plugin only works on the lines of freestyle project even if used in pipeline. So any improvement made in git credential binding could be made in git-client plugin and vice-versa, so is their any scope of reliability, but how, I am still figuring it out.
    3 replies
    pipeline {
        agent any
        // tools {
        //     // Install the Maven version configured as "M3" and add it to the path.
        //     maven "M3"
        // }
        stages {
            stage('Build') {
                steps {
                    deleteDir()
                    git branch: 'mavenTest', changelog: false, credentialsId: '9f491dfd-135b-4c09-a731-42735ff6fa9a', poll: false, url: 'https://github.com/arpoch/Jenkins-Bug-Forge.git'
    
                    // Run Maven on a Unix agent.
                    sh "mvn -Dmaven.test.failure.ignore=true clean package"
                }
                post {
                    success {
                        junit '**/target/surefire-reports/TEST-*.xml'
                        archiveArtifacts 'target/*.jar'
                    }
                }
            }
            stage('Tag'){
                steps {
                   sh 'echo adding tag'
                    withCredentials([GitUsernamePassword('9f491dfd-135b-4c09-a731-42735ff6fa9a')]) {
                         sh 'git push origin :refs/tags/v2.0'                
                    }
                    sh 'git tag -a v2.0 -m "gitUsernamePassword-TestCase-2"'
                    git branch: 'mavenTest', changelog: false, credentialsId: '9f491dfd-135b-4c09-a731-42735ff6fa9a', poll: false, url: 'https://github.com/arpoch/Jenkins-Bug-Forge.git'
                }
                post {
                    success {
                        withCredentials([GitUsernamePassword('9f491dfd-135b-4c09-a731-42735ff6fa9a')]) {
                         sh 'git push origin v2.0'                
                        }
                         sh 'echo Tag added successfully.'
                    }
                }
            }
            stage('Report') {
                steps{
                    sh 'touch report'
                    sh 'echo Completed Run-2 >> report'
                    sh 'git add report'
                    sh 'git commit report -m "New File"'
                }
            }
            stage('Push') {
                steps{
                    withCredentials([GitUsernamePassword('9f491dfd-135b-4c09-a731-42735ff6fa9a')]) {
                        sh 'git push origin mavenTest'
                    }
                }
                post {
                    success {
                        sh 'echo push successfully.'
                    }
                }
            }
        }
    }
    Rishabh Budhouliya
    @rishabhBudhouliya
    @arpoch great progress Harshit! It seems like you have enough material to start building your design document (you have your own insights from this exercise plus the discussion we had in the last meeting)
    Apart from that,
    • According to me, the git credentials binding step should provide the independence to use a sh or pre-defined git step (I think we get the best of both worlds from this combination)
    • I didn't understand your reference to the similarities in the authentication process between the aforementioned plugins, could you help me by providing more context or maybe point me to the code?
    Harshit Chopra
    @arpoch
    @rishabhBudhouliya, Now as I see my above mentioned concern for the similarities between the authentication process of git-client plugin and git credential binding, I think @MarkEWaite is right about the risks of opening all sorts of compatibility issues also I missed a point that even though the 'logic' might be somewhat similar but since git credential binding depends on credential binding plugin and git plugin doesn't makes them a lot different.
    1 reply

    I am attaching the snippets of the code just to make it a bit more clear. It create a temporary script file to echo username and password.
    Git Client Plugin

     File askpass = createTempFile("pass", ".sh");
            try (PrintWriter w = new PrintWriter(askpass, encoding)) {
                w.println("#!/bin/sh");
                w.println("case \"$1\" in");
                w.println("Username*) cat " + unixArgEncodeFileName(usernameFile.getAbsolutePath()) + " ;;");
                w.println("Password*) cat " + unixArgEncodeFileName(passwordFile.getAbsolutePath()) + " ;;");
                w.println("esac");
            }
            askpass.setExecutable(true, true);

    Git Credential Binding

     final FilePath gitEcho = workspace.createTempFile("auth", ".sh");
                gitEcho.write("#!/bin/sh\n" +
                        "case $1 in\n" +
                        "        Username*) echo " + this.userVariable + "\n" +
                        "                ;;\n" +
                        "        Password*) echo " + this.passVariable + "\n" +
                        "                ;;\n" +
                        "        esac\n", null);
                gitEcho.chmod(0500);
    Harshit Chopra
    @arpoch
    Also as I am working through code could I might have missed judged the dependence of the git credential binding on git-client/git plugin as in some cases the git credential binding could use the functionality provided by git-client plugin. So it's to soon to make any assumption.
    Harshit Chopra
    @arpoch
    Hi,
    @rishabhBudhouliya, @justinharringa, @MarkEWaite, from our pervious meeting I have been working on the ssh binding impl, although am I close to the solution but some work is required. Meet you all soon.
    Also the Bouncycastel API provides support for PEM format and the OPENSSH now generates key in its proprietary format so, the user should convert the key to PEM format first only then the auth process could start else it will return invalid format.
    Mark Waite
    @MarkEWaite
    @rishabhBudhouliya I'm in the session with @arpoch
    3 replies
    Harshit Chopra
    @arpoch
    Hi,
    @MarkEWaite, @rishabhBudhouliya, @justinharringa, I have configured my dev-env so now I have Centos 7.9.2009(core), Ubuntu 20.04(lts), Windows 10 Pro(version 1909 and updating) and the git versions are 1.8.3.1, 2.25.1, and 2.26.0 respectively, will update Ubuntu git to test for GIT_SSH_COMMAND variable. I guess this will be all for the dev setup required for the project.
    Now coming to the SSH binding, as @MarkEWaite we discussed in the previous meeting about the new format used by openssh by default to generate private encryption key and using the bouncycastle api to convert the key first into PEM format and then decrypting the key if protected by passphrase. Well till now what I have learnt is that, to create a PEMEncodable object to be used by bouncy-castle api we require an object of type Key or Keypair for our case but the since the ssh-credential plugin providing the SSHUserPrivateKey interface, used for ssh binding only gives key in string format and it seem right as well because we not have any knowledge about which format and encryption algorithm was used when the key was provided. So keeping it short, we need to have bare minimum knowledge about the algorithm and format used, gone through the JCA api as well.
    I was also wondering if their is any mailing list or chat focused on bouncycastle plugin or encryption/cryptography for Jenkins plugin would be really helpful.
    1 reply
    Harshit Chopra
    @arpoch
    @MarkEWaite - Is there a meeting today, I don't have the link for the meet, can you please share.
    Mark Waite
    @MarkEWaite
    Meeting for git credentials binding GSoC project at https://zoom.us/j/94585783361?pwd=UnJZbTNPVlhvUUxCck9FNG5xK29QUT09
    Harshit Chopra
    @arpoch
    Thanks
    Harshit Chopra
    @arpoch
    I just wanted to share this and was hoping this could help @rishabhBudhouliya in his investigation as well , https://en.wikipedia.org/wiki/ASN.1 here if we look under the protocols using ASN.1 we could find both X.509 and PKCS but noting related to SSH, also if I am correct OpenSSH used the standard ASN.1 formats for private keys. Now, however, OpenSSH has its own private key format. So if we were to work on the conversion of the private key into PEM format following ASN.1 standard then I guess we needed to figure out how it's begin done by ssh-keygen but it just my assumption.
    Gareth Evans
    @garethjevans
    Hi, I'm trying to update the "Branches to build" configuration for an existing job programatically via the Jenkins CLI, I can see the config changes on disk, and the job appears to have updated when I click "Configure", but the "Git Polling Log" always shows the old branch name. From the CLI I have also tried "reload-job" & "reload-configuration" and have tried the "Reload Configuration from Disk" in the Jenkins UI without success. When I click "Build Now" it does update the branch correctly, that seems to be my only workaround. Is this expected behaviour?
    3 replies
    Harshit Chopra
    @arpoch
    @MarkEWaite , @rishabhBudhouliya , @justinharringa , Good news the username password binding for windows is done.
    1 reply
    Rishabh Budhouliya
    @rishabhBudhouliya
    @arpoch did you try the org.bouncycastle.jcajce.spec.OpenSSHPrivateKeySpec to generate a private key?
    this bouncycastle version:
    <dependency>
                <groupId>org.bouncycastle</groupId>
                <artifactId>bcprov-jdk15on</artifactId>
                <version>1.68</version>
            </dependency>
    Mark Waite
    @MarkEWaite
    That is great news! I'm in the Zoom meeting now at https://zoom.us/j/94585783361?pwd=UnJZbTNPVlhvUUxCck9FNG5xK29QUT09
    Harshit Chopra
    @arpoch
    Joining the meeting in a moment.
    Harshit Chopra
    @arpoch
    @MarkEWaite , @rishabhBudhouliya , @justinharringa, I am currently making all the necessary changes for the first GitUsernamePasswrod binding PR. I have created two default environment variable binding i.e Git_Username and Git_Password, these won't be visible in pipeline snippet or modifiable by the user, just in case.
    Also I think I have got a workaround for ssh credential binding but not sure about it yet, will update when explored further.
    Also for the coding phase do I have to the push code on daily or weekly basis.
    Rishabh Budhouliya
    @rishabhBudhouliya
    @arpoch @MarkEWaite @justinharringa please take a look at an edit I've made on the OpenSSH keys decryption experiment: https://docs.google.com/document/d/1gZneYIDWrT5S-1ACG641wfvxs7vnDC0RCYqy-EuuhwY/edit?usp=sharing
    Rishabh Budhouliya
    @rishabhBudhouliya
    @arpoch @MarkEWaite @justinharringa I think I have found a way to decode the SSH keys with passphrase and without passphrase using the sshj library
    I have tested three cases:
    • ssh-keygen -f ssh_key with and without passphrase (the user does not specific an encryption algorithm)
    • ssh-keygen -t rsa -f ssh_key
    • ssh-keygen -t ED25519 -f ssh_key
    I have been able to get the java.security.PrivateKey out of them. How do we want to consume them now?
    I was looking at the Git Client Plugin and as per my understanding we provide the file locations of ssh private key and passphrase and then let git cli talk to ssh?
    The downside is that this is obviously not the BouncyCastle API and I am not sure about our reasons to stick with BouncyCastle. sshj is the java implementation of ssh and it provides much more than just reading OpenSSH keys.
    3 replies
    Will Saxon
    @wsaxon_gitlab

    Hello, we're trying to use the Jenkins Git plugin to clone a repo. Our SCM host requires jumping through an SSH proxy, so we've provided a config file to our agent that sets a ProxyCommand for the SCM host; Jenkins doesn't let us configure a ProxyCommand any other way that I can see.

    Our proxy is an SSH host itself, so we need the SSH key for the ProxyCommand to work. We were hoping to use the SSH Agent plugin for this, but it doesn't seem to work. I wrote a wrapper script around git to dump the environment and run ssh-add -l, and I can see that the agent is set up in the environment and has our key, but e.g. the git fetch --tags ... command that Jenkins runs to fetch objects immediately fails as if the key is not present. If I set up the same scenario manually in a shell it works fine.

    Is this a supported configuration? Should I be able to use the SSH agent plugin to provide the SSH key to Jenkins Git plugin?

    3 replies
    Harshit Chopra
    @arpoch
    Create the first pull request of the coding phase 1-
    jenkinsci/git-client-plugin#712
    3 replies
    Harshit Chopra
    @arpoch
    @justinharringa, @rishabhBudhouliya what are your thoughts on https://github.com/jenkinsci/git-client-plugin/pull/712#discussion_r648520322
    2 replies
    Harshit Chopra
    @arpoch
    @rishabhBudhouliya, @justinharringa, I have committed some changes based on @rishabhBudhouliya sugesstions, Now I will be working on the testcases and if everything works as expected, will move to SSH binding by our next meeting.
    Justin Harringa
    @justinharringa:matrix.org
    [m]
    Awesome news @arpoch ! I'm planning on giving these a better look in the next few days.
    Harshit Chopra
    @arpoch

    @justinharringa:matrix.org , @rishabhBudhouliya , regarding the GitTool impl, I have create this snippet

    default String gitToolName(TaskListener listener) {
            String requiredTool = "Default";
            String actualTool = null;
    
            GitTool gitTool = Jenkins.get().getDescriptorByType(GitTool.DescriptorImpl.class).getInstallation(requiredTool);
            if (gitTool == null) {
                listener.getLogger().println("Selected Git installation does not exist. Using Default");
                gitTool = GitTool.getDefaultInstallation();
                actualTool = gitTool.getName();
            }
            if (actualTool != null) {
                if (actualTool.equalsIgnoreCase(requiredTool)) {
                    return actualTool;
                }
            }
            try {
                gitTool = gitTool.forNode(Jenkins.get(), listener);
                actualTool = gitTool.getName();
            } catch (IOException | InterruptedException e) {
                listener.getLogger().println("Failed to get git tool");
            }
    
            return actualTool;
        }

    I have some concerns related to this-

    • The git tool to be used by default will be the one having the name Default but their is no assurance that path to Git executable is of cli git.
    • If their is not git tool with name Default then the first git tool will be used even if git cli exists
    • The name returned by this snippet will only be checked for jgit and jgitapache, as these two implementations will always return same name.
    Rishabh Budhouliya
    @rishabhBudhouliya
    @arpoch if you're putting Default in GitTool gitTool = Jenkins.get().getDescriptorByType(GitTool.DescriptorImpl.class).getInstallation(requiredTool); then isn't this line essentially
    GitTool.getDefaultInstallation() ?
    I am assuming that there is way to get the user decided GitTool from the context where the credentials are working, if that is not the case please feel free to correct me.
    1 reply
    Harshit Chopra
    @arpoch
    I changed the code, so now its
    default String gitToolName(TaskListener listener) {
            String requiredToolByName = "Default";
            String actualToolByPath = null;
    
            GitTool gitTool = Jenkins.get().getDescriptorByType(GitTool.DescriptorImpl.class).getInstallation(requiredToolByName);
            if (gitTool == null) {
                listener.getLogger().println("Selected Git installation does not exist. Using Default");
                gitTool = GitTool.getDefaultInstallation();
            }
            if(gitTool!=null) {
                try {
                    gitTool = gitTool.forNode(Jenkins.get(), listener);
                    actualToolByPath = FilenameUtils.getBaseName(gitTool.getGitExe());
                } catch (IOException | InterruptedException e) {
                    listener.getLogger().println("Failed to get git tool");
                }
            }
    
            return actualToolByPath;
        }
    Harshit Chopra
    @arpoch
    I have created a commit regarding alot of changes today, things got messy in between and as of now some test are not passing I must have missed something, will be more care full next time. And will make the changes.
    @rishabhBudhouliya, @justinharringa:matrix.org , @MarkEWaite , I will be mailing regarding my University exam dates.
    Rishabh Budhouliya
    @rishabhBudhouliya
    @arpoch Do we have the deliverables for this coding phase defined somewhere? Can we update them according to the recent developments + anticipation of your exams so that we all know how we want to proceed?
    1 reply
    Justin Harringa
    @justinharringa
    Howdy @arpoch @rishabhBudhouliya @MarkEWaite ! I've moved the office hours meet to Jun 16th @ 7.30 AM IST - I wasn't able to edit the original invite so I created a new one
    1 reply
    Harshit Chopra
    @arpoch
    @justinharringa , @rishabhBudhouliya , @MarkEWaite, if possible we shift the meeting to Jun 17th @ 7:30 AM IST or could reduce the meeting time to half hour instead of an hour?
    Justin Harringa
    @justinharringa:matrix.org
    [m]
    @arpochare you thinking just for this week or regularly?
    1 reply
    Justin Harringa
    @justinharringa:matrix.org
    [m]
    Do you just have a conflict at the beginning or end? Certainly want to make sure you get the time you need. 😀