Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Mark Waite
    @MarkEWaite

    There are several issues related to change detection in the git plugin. What you're seeing is not expected behavior. I don't know of anything you're missing in your description of the issue.

    I have several explorations of bugs related to change detection either by polling or by notifyCommit in my jenkins-bugs repository. See https://github.com/MarkEWaite/jenkins-bugs/tree/JENKINS-43468 , https://github.com/MarkEWaite/jenkins-bugs/tree/JENKINS-43754 , and https://github.com/MarkEWaite/jenkins-bugs/tree/JENKINS-52746

    15 replies
    Harshit Chopra
    @arpoch
    Hi,
    I am trying to use SSH Username with private key credentials with passphrase in a pipeline job using ssh-agent plugin, but I keep on getting ssh_askpass: exec(/mycredential/tempfolder/pass_123.sh): No such file or directory, I tried steps in https://support.cloudbees.com/hc/en-us/articles/360027646491-Pipeline-Equivalent-to-Git-Publisher?page=22 also search through stack-overflow but I could not find much help regarding passphrase usage with ssh-key in jenkins pipeline job.
    Any suggestions?
    Harshit Chopra
    @arpoch
    Also can you please provide any documentation source/articles that talk about the usage of SSH_ASKPASS in the git plugin.
    Mark Waite
    @MarkEWaite
    I think the project idea for ssh credentials should find a Java way to convert a passphrase protected private key into a private key without a passphrase. Then we avoid the entire mess with SSH_ASKPASS.
    That was the suggestion from Jesse Glick and I think he's correct.
    He suggested the bouncycastle package that is include in Jenkins as one location that may have code to convert a passphrase protected private key into a private key without passphrase
    Harshit Chopra
    @arpoch
    Hi,
    Just wanted to ask that to include Git credentials in the pipeline job, withCredential wrapper will be used to provide the snippets as currently performed by the Credential binding plugin for ssh and username:password, the git plugin will use the provided information to authenticate the user and perform the operation, or could both withCredential and GitPublisher be used together providing the flexibility and web UI benefits respectively.
    1 reply
    Harshit Chopra
    @arpoch
    Hi, I have created a draft proposal for the GSOC 2021, project Git credentials binding for sh, bat, and powershell , looking for your feedback.
    link- GSOC 2021 PROPOSAL
    @MarkEWaite @rishabhBudhouliya
    Mark Waite
    @MarkEWaite
    Thanks @arpoch . I'll provide review comments in the next 1-2 days. If I haven't provided comments by next Monday, could you remind me?
    Harshit Chopra
    @arpoch
    Thanks for the feedback Mark. I will make the changes soon.
    Ayan Goel
    @GAyan17
    Hey everyone,
    I was researching for SSH Keys and about reading/parsing them in JAVA for Converting a passphrase protected Private Key to a Private Key with no passphrase. And I came to many different formats for the keys being used. As suggested by @MarkEWaite I also looked up in the BouncyCastle library, but the most implementation provided is of OpenSSL, and I suppose OpenSSH is the norm for these days. Mostly every Git Server SSH Authentication uses OpenSSH formats to authenticate. All the more is that PEMDecoders are available but I guess PEM format keys are not used for SSH authentications. Correct me if I'm wrong, it's just an assumption. Also, Jenkins maintain ssh2 library extracted from ganymede, now trilead. But I suppose it has older format decoders.
    I was testing to read key pairs in JAVA using bouncy castle. Reading a public key is supported, but reading a private key is not as it is encrypted with passphrase. That's where I'm stuck.
    4 replies
    Rishabh Budhouliya
    @rishabhBudhouliya

    Hi everyone, I have been reviewing the proposals submitted for this project and have some general feedback for all (from a student's perspective) (@MarkEWaite and @jeffpearce have already given excellent feedback)

    Some points I kept in mind while creating the proposal:

    • I assumed the mentor is not the expert, I am. The mentor takes a learner's role during this phase. The proposal has to be written in a way which has to be descriptive, well-planned and researched thoroughly
    • The current year Git Plugin project is a coding task and as a student, the first thing which comes into my mind is the design of that code. No words can describe your plan/code better than a simple flowchart or better yet, a UML diagram
    • The amount of content someone copies from the basic project idea page is a direct reflection of the amount of research and thought one has put into while writing the abstract and project description.
    • Don't shy away from writing the number of challenges you expect to face, it shows a pragmatic approach towards building the proposal/planning the project
    • Test your hypothesis. For an example, if you think sshj is the correct java library to implement SSH then try to create a proof of concept with it (An archetype plugin with some code testing it would be excellent!)
      IMO, testing your own assumptions gives a much better idea into what the project is actually asking from you.

    To summarise,

    • write incisive descriptions
    • try to include visual elements
    • a POC is always nice
    • try to be the expert (if we're wrong there's always the project mentors and the Jenkins community to humble us)
    2 replies
    raghavgarg098
    @raghavgarg098
    Hi all, here is my initial draft proposal for Gsoc project Git Credentials Binding, please take a look at this proposal and help me in improving this. Link to the proposal https://docs.google.com/document/d/1KAqWYZmCztzu4wYX0ofaJBloHhvwV9hfv6lUKse53a8/edit?usp=sharing.
    Thanks!
    2 replies
    Harshit Chopra
    @arpoch

    Hi,
    I guess I have figured out the conversion of SSH passphrase encrypted key to a decrypted SSH key, thanks to @MarkEWaite for guiding in the right direction. The solution uses bouncycastle api plugin to decode the encypted SSH key.

    passphrase = createPassphraseFile(sshUser);
    String encPrivateKey = sshUser.getPrivateKeys().get(0);
    Secret passPhase = sshUser.getPassphrase();
    char[] passPahseChar = passPhase.getPlainText().toCharArray();
    PEMEncodable decodedPvtKey = PEMEncodable.decode(encPrivateKey,passPahseChar);
    byte[] newKey = decodedPvtKey.toPrivateKey().getEncoded();
    decodedTempFile = createTempFile("decodedsshkey",".key");
    FileUtils.writeByteArrayToFile(decodedTempFile,newKey);

    This code creates a n ByteArray file which then can be referenced in the GIT_SSH env varibale using -i. Also using this code their is no need to create a sperate file for private key and passphrase. This code requires alot of improvement such as

    • The format conversion of SSH key conversion to PEM for keys generated by OpenSSH as it default generates the key in proprietary key format.
    • Understaning of SELinux context and its implementation on the key.
    Mark Waite
    @MarkEWaite
    That's a great results @arpoch . Thanks for doing that investigation!
    Harshit Chopra
    @arpoch
    Hi,
    For the implementation of gitUsernamePassword, I guess GIT_ASKPASS env variable is sufficient unless we need to store the credentials in a cache for which credential.helper will be required.
    Mark Waite
    @MarkEWaite
    Either implementation could work, as far as I recall. I believe the credential helper technique chooses to store the sensitive information in a way that has less dependency on the plugin to write the correct permissions on the file. the SSH_ASKPASS / GIT_ASKPASS setting has the benefit that it is the technique that we know works because that is how the git client plugin currently operates. Credential helpers are the more recent technique, but I believe they are also less explored than the SSH_ASKPASS / GIT_ASKPASS technique
    Harshit Chopra
    @arpoch
    I will work on the credential.helper approach and try to figure out if this approach could work as an alternative to SELinux context or Windows ACL.
    Thanks @MarkEWaite.
    Harshit Chopra
    @arpoch
    Hi,
    I just wanted to ask that selinux feature is not specific to Debian packages as alternatives to this feature are also present such as Apparmor or Tomoyo, also it might the case that selinux has been disabled. So could Access control list(ACL) be seen as an alternative in such cases.
    Mark Waite
    @MarkEWaite
    @arpoch I have never investigated the use of SELinux or any variants of SELinux. The last time I interacted with it was many years ago when I made the mistake of enabling it on a CentOS or Red Hat Linux installation. I believe I ultimately replaced the operating system rather than continue wrestling with its restrictions.
    I know that Jim Klimov has done work in the git client plugin to attempt to support it and I suspect that I will eventually need to add it to my test environment, but I've not had time to do that yet.
    Harshit Chopra
    @arpoch
    Hi all,
    I just wanted to clarify that our first meeting for git credentials project will be taking place any day b/w 19-05-21 to 21-05-21.
    Currently I am working on the UI implementation and using credential binding plugin as reference to work on the integration of git credential in pipeline job,alongside creating a more refined roadmap for the project.
    1 reply
    Bassam Khouri
    @bkhouri
    Hi.. I have a declarative pipleine job that have multiple parallel stages. Each parallel stage does a checkout scm and it seem that sometimes, the stages do not clone the same commit. I came accros https://groups.google.com/g/jenkinsci-users/c/P-ErEhzNDOA?pli=1 from years ago which state the issue is fixed in git plugin 2.6.1+. We are running git version 4.6.0. however I still encounter the same symptoms as the original poster.
    2 replies
    Harshit Chopra
    @arpoch
    @MarkEWaite @rishabhBudhouliya @justinharringa , from our discussion earlier today, I have been working on the jenkins hello world archetype to have a more clear insight regarding the extension points, structure of a plugin. Also I guess I have a lead on making git plugin compatible with pipeline script using workflow-scm-step-plugin although it needs more research, thanks for the resources provided. With that I had some questions in mind for tomorrows session.
    3 replies
    Harshit Chopra
    @arpoch
    Screenshot_2021-05-20 Pipeline Syntax Snippet Generator [Jenkins].png
    @MarkEWaite @rishabhBudhouliya @justinharringa , I was working on the idea we were discussing yesterday/today, I was able to develop the UI(attaching the image above) also I have made some modifications such as removing the field of Username and Password variables as all the authentication stuff will be handled in the backend so the user need not to worry about the setting the variables.
    Also another area that I am a bit worried about and going to investigate more on it later is how is a withCredential snippet is handled in a pipeline script and also looking for some guidance on that.
    Also I will keep on updating about any further progress made daily.
    Mark Waite
    @MarkEWaite

    Awesome work @arpoch . That UI screen shot looks great. Well done!

    Would like it very much if you would add an annotation for @Symbol("gitUsernamePassword") so that the $class would be replaced with the gitUsernamePassword string

    Harshit Chopra
    @arpoch
    Thanks @MarkEWaite, added the @Symbol annotation.
    Currently I am working on execution of the context inside or the implementation of the withCredential(gitUsernamePassword()){}. Will update on Monday.
    Justin Harringa
    @justinharringa
    Excellent!
    Harshit Chopra
    @arpoch
    Hi all,
    @justinharringa , @MarkEWaite , @rishabhBudhouliya , I am attaching a script below which performs adding ,removing, pushing tags, making a commit and pushing the changes and all this is done on a private repo 'https://github.com/arpoch/Jenkins-Bug-Forge.git'. Good news everything worked as expected though this is performed on Linux dist(Ubuntu) and uses Username and Password for authentication.
    Also git credential binding implementation is independent of git/git-client plugin so its just a choice to put it these plugins, as all is handed gracefully by Credential Binding plugin API's.
    But I have some concerns-
    • As using sh to perform git operation, it exposes the user to raw commandline thus the operation such as cloning a repo would result in a fatal error if the repo already exists and stops the pipeline. As a result I have used gitsnippet to perform the clone whcih switches to fetch if a repo already exists. So is such a "filter" should be provided for git credential binding as well or dependence on git and checkout snippet is fine for that.
    • Another thing that I wanted to discuss, the authentication process in the git-client plugin and git credential binding is somewhat same but since the git-client-plugin only works on the lines of freestyle project even if used in pipeline. So any improvement made in git credential binding could be made in git-client plugin and vice-versa, so is their any scope of reliability, but how, I am still figuring it out.
    3 replies
    pipeline {
        agent any
        // tools {
        //     // Install the Maven version configured as "M3" and add it to the path.
        //     maven "M3"
        // }
        stages {
            stage('Build') {
                steps {
                    deleteDir()
                    git branch: 'mavenTest', changelog: false, credentialsId: '9f491dfd-135b-4c09-a731-42735ff6fa9a', poll: false, url: 'https://github.com/arpoch/Jenkins-Bug-Forge.git'
    
                    // Run Maven on a Unix agent.
                    sh "mvn -Dmaven.test.failure.ignore=true clean package"
                }
                post {
                    success {
                        junit '**/target/surefire-reports/TEST-*.xml'
                        archiveArtifacts 'target/*.jar'
                    }
                }
            }
            stage('Tag'){
                steps {
                   sh 'echo adding tag'
                    withCredentials([GitUsernamePassword('9f491dfd-135b-4c09-a731-42735ff6fa9a')]) {
                         sh 'git push origin :refs/tags/v2.0'                
                    }
                    sh 'git tag -a v2.0 -m "gitUsernamePassword-TestCase-2"'
                    git branch: 'mavenTest', changelog: false, credentialsId: '9f491dfd-135b-4c09-a731-42735ff6fa9a', poll: false, url: 'https://github.com/arpoch/Jenkins-Bug-Forge.git'
                }
                post {
                    success {
                        withCredentials([GitUsernamePassword('9f491dfd-135b-4c09-a731-42735ff6fa9a')]) {
                         sh 'git push origin v2.0'                
                        }
                         sh 'echo Tag added successfully.'
                    }
                }
            }
            stage('Report') {
                steps{
                    sh 'touch report'
                    sh 'echo Completed Run-2 >> report'
                    sh 'git add report'
                    sh 'git commit report -m "New File"'
                }
            }
            stage('Push') {
                steps{
                    withCredentials([GitUsernamePassword('9f491dfd-135b-4c09-a731-42735ff6fa9a')]) {
                        sh 'git push origin mavenTest'
                    }
                }
                post {
                    success {
                        sh 'echo push successfully.'
                    }
                }
            }
        }
    }
    Rishabh Budhouliya
    @rishabhBudhouliya
    @arpoch great progress Harshit! It seems like you have enough material to start building your design document (you have your own insights from this exercise plus the discussion we had in the last meeting)
    Apart from that,
    • According to me, the git credentials binding step should provide the independence to use a sh or pre-defined git step (I think we get the best of both worlds from this combination)
    • I didn't understand your reference to the similarities in the authentication process between the aforementioned plugins, could you help me by providing more context or maybe point me to the code?
    Harshit Chopra
    @arpoch
    @rishabhBudhouliya, Now as I see my above mentioned concern for the similarities between the authentication process of git-client plugin and git credential binding, I think @MarkEWaite is right about the risks of opening all sorts of compatibility issues also I missed a point that even though the 'logic' might be somewhat similar but since git credential binding depends on credential binding plugin and git plugin doesn't makes them a lot different.
    1 reply

    I am attaching the snippets of the code just to make it a bit more clear. It create a temporary script file to echo username and password.
    Git Client Plugin

     File askpass = createTempFile("pass", ".sh");
            try (PrintWriter w = new PrintWriter(askpass, encoding)) {
                w.println("#!/bin/sh");
                w.println("case \"$1\" in");
                w.println("Username*) cat " + unixArgEncodeFileName(usernameFile.getAbsolutePath()) + " ;;");
                w.println("Password*) cat " + unixArgEncodeFileName(passwordFile.getAbsolutePath()) + " ;;");
                w.println("esac");
            }
            askpass.setExecutable(true, true);

    Git Credential Binding

     final FilePath gitEcho = workspace.createTempFile("auth", ".sh");
                gitEcho.write("#!/bin/sh\n" +
                        "case $1 in\n" +
                        "        Username*) echo " + this.userVariable + "\n" +
                        "                ;;\n" +
                        "        Password*) echo " + this.passVariable + "\n" +
                        "                ;;\n" +
                        "        esac\n", null);
                gitEcho.chmod(0500);
    Harshit Chopra
    @arpoch
    Also as I am working through code could I might have missed judged the dependence of the git credential binding on git-client/git plugin as in some cases the git credential binding could use the functionality provided by git-client plugin. So it's to soon to make any assumption.
    Harshit Chopra
    @arpoch
    Hi,
    @rishabhBudhouliya, @justinharringa, @MarkEWaite, from our pervious meeting I have been working on the ssh binding impl, although am I close to the solution but some work is required. Meet you all soon.
    Also the Bouncycastel API provides support for PEM format and the OPENSSH now generates key in its proprietary format so, the user should convert the key to PEM format first only then the auth process could start else it will return invalid format.
    Mark Waite
    @MarkEWaite
    @rishabhBudhouliya I'm in the session with @arpoch
    3 replies
    Harshit Chopra
    @arpoch
    Hi,
    @MarkEWaite, @rishabhBudhouliya, @justinharringa, I have configured my dev-env so now I have Centos 7.9.2009(core), Ubuntu 20.04(lts), Windows 10 Pro(version 1909 and updating) and the git versions are 1.8.3.1, 2.25.1, and 2.26.0 respectively, will update Ubuntu git to test for GIT_SSH_COMMAND variable. I guess this will be all for the dev setup required for the project.
    Now coming to the SSH binding, as @MarkEWaite we discussed in the previous meeting about the new format used by openssh by default to generate private encryption key and using the bouncycastle api to convert the key first into PEM format and then decrypting the key if protected by passphrase. Well till now what I have learnt is that, to create a PEMEncodable object to be used by bouncy-castle api we require an object of type Key or Keypair for our case but the since the ssh-credential plugin providing the SSHUserPrivateKey interface, used for ssh binding only gives key in string format and it seem right as well because we not have any knowledge about which format and encryption algorithm was used when the key was provided. So keeping it short, we need to have bare minimum knowledge about the algorithm and format used, gone through the JCA api as well.
    I was also wondering if their is any mailing list or chat focused on bouncycastle plugin or encryption/cryptography for Jenkins plugin would be really helpful.
    1 reply
    Harshit Chopra
    @arpoch
    @MarkEWaite - Is there a meeting today, I don't have the link for the meet, can you please share.
    Mark Waite
    @MarkEWaite
    Meeting for git credentials binding GSoC project at https://zoom.us/j/94585783361?pwd=UnJZbTNPVlhvUUxCck9FNG5xK29QUT09
    Harshit Chopra
    @arpoch
    Thanks
    Harshit Chopra
    @arpoch
    I just wanted to share this and was hoping this could help @rishabhBudhouliya in his investigation as well , https://en.wikipedia.org/wiki/ASN.1 here if we look under the protocols using ASN.1 we could find both X.509 and PKCS but noting related to SSH, also if I am correct OpenSSH used the standard ASN.1 formats for private keys. Now, however, OpenSSH has its own private key format. So if we were to work on the conversion of the private key into PEM format following ASN.1 standard then I guess we needed to figure out how it's begin done by ssh-keygen but it just my assumption.
    Gareth Evans
    @garethjevans
    Hi, I'm trying to update the "Branches to build" configuration for an existing job programatically via the Jenkins CLI, I can see the config changes on disk, and the job appears to have updated when I click "Configure", but the "Git Polling Log" always shows the old branch name. From the CLI I have also tried "reload-job" & "reload-configuration" and have tried the "Reload Configuration from Disk" in the Jenkins UI without success. When I click "Build Now" it does update the branch correctly, that seems to be my only workaround. Is this expected behaviour?
    3 replies
    Harshit Chopra
    @arpoch
    @MarkEWaite , @rishabhBudhouliya , @justinharringa , Good news the username password binding for windows is done.
    1 reply
    Rishabh Budhouliya
    @rishabhBudhouliya
    @arpoch did you try the org.bouncycastle.jcajce.spec.OpenSSHPrivateKeySpec to generate a private key?
    this bouncycastle version:
    <dependency>
                <groupId>org.bouncycastle</groupId>
                <artifactId>bcprov-jdk15on</artifactId>
                <version>1.68</version>
            </dependency>
    Mark Waite
    @MarkEWaite
    That is great news! I'm in the Zoom meeting now at https://zoom.us/j/94585783361?pwd=UnJZbTNPVlhvUUxCck9FNG5xK29QUT09
    Harshit Chopra
    @arpoch
    Joining the meeting in a moment.
    Harshit Chopra
    @arpoch
    @MarkEWaite , @rishabhBudhouliya , @justinharringa, I am currently making all the necessary changes for the first GitUsernamePasswrod binding PR. I have created two default environment variable binding i.e Git_Username and Git_Password, these won't be visible in pipeline snippet or modifiable by the user, just in case.
    Also I think I have got a workaround for ssh credential binding but not sure about it yet, will update when explored further.
    Also for the coding phase do I have to the push code on daily or weekly basis.