Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Diego Henrique Pagani
    @dhpagani
    image.png
    1 reply
    Hi!
    I'm learning how to use JTE 2.0 and I'm a little bit concern about the time to load the libraries in the pipeline. In the console show It took almost 7 seconds two load two libraries. In the future, if I had more it'll take some time to load all of them.
    1 reply
    I'm loading it as SCM from a private repo at github
    Syed Schoaib
    @Schoaib
    Hi @steven-terrana, "stderr: error: could not lock config file .git/config: File exists". Noticed this happening sometimes now while git checkout libraries ran at the same time for multiple jobs and in some cases, the process is not finished gracefully leaving behind the .lock file which requires manual intervention then. I understand this is not an issue with JTE itself but Jenkins git plugin. I tried to use other SCM plugins like bitbucket instead of git but it seems it's not supported. Any leads on how to overcome this and remove any manual interventions to the process will be really helpful. Thanks
    13 replies
    Syed Schoaib
    @Schoaib
    @steven-terrana, Great work on JTE 2.0, got much-awaited updates (y)
    1 reply
    akibabu
    @akibabu
    Started to look at this plugin today, looks great. But all the example I've seen uses zero implementation in the actual projects.
    What we are looking for is for each project to implement their own build() methods and keep that in a Jenkinsfile in the project, not a shared library, is it supported?
    3 replies
    akibabu
    @akibabu
    Or rather the option to do so I guess. So you can either reference a library like maven the JTE way, but if the provided default implementation doesn't work for you, you can choose to implement the build() method instead
    René Zubcevic
    @zubcevic
    Today I noticed that if I export a multi branch pipeline job with a pipeline configuration from console, all configuration is on a single line when imported. This fails the job. However I could fix this by adding ; in between: libraries{ maven ; sonarqube { sast_project_key='WebGoat' } ; docker}
    2 replies
    Ramon Pereira
    @ramondea
    Does anyone have an example of how to run some of the internships in jenkins agents and not in the Master? The libraries are groovy code, and the groovy runs on the master, is that right?
    2 replies
    sebastianchacko89
    @sebastianchacko89
    Hi Team! Good morning!
    Trying to upgrade the JTE Plugin from 1.3 to 2.0. Worked through most of the issues so far, but running into this weird one: "hudson.AbortException: No such saved stash ‘workspace’"
    When i checkout our sdp library, i saw the first think we do is "unstash "workspace"".
    It seemed like the JTE engine used to checkout the workspace and stash it and we needed to unstash it to proceed with our pipeline. Was this behavior changed at some point?
    Wondering how to get across this issue. Appreciate any help! Thanks in advance!
    1 reply
    steven-terrana
    @steven-terrana

    loving all the questions! i’m taking some much needed R&R for the holidays. I’ll be around next week to give responding the time it deserves :)

    appreciate your patience and happy holidays :)

    René Zubcevic
    @zubcevic
    yes do that, you deserve it. happy holidays to all
    René Zubcevic
    @zubcevic
    @sebastianchacko89 sounds strange that you needed a unstash before ever stashing. maybe related to the type of job? i noticed that i need a scm checkout for a multi branch pipeline job
    sebastianchacko89
    @sebastianchacko89
    @zubcevic yeah weird ryt. not sure why we did that way back when. I just tried replacing the "unstash "workspace"" with a "checkout scm" and it worked just fine. :D
    Thanks so much for that suggestion!
    sebastianchacko89
    @sebastianchacko89

    Also, a follow up question, we've been running into some weird seemingly git/JTE related error every so often when there are multiple JTE jobs running on our Jenkins instance, stuff like:
    "org.boozallen.plugins.jte.config.TemplateConfigException: There were library configuration errors."
    "org.eclipse.jgit.errors.MissingObjectException: Missing unknown "

    The most common one is:
    "[JTE] hudson.plugins.git.GitException: Command "git config remote.origin.url ssh://git@blah.com:1234/blah/jte-libraries.git" returned status code 255:
    [JTE] stdout:
    [JTE] stderr: error: could not lock config file .git/config: File exists"

    Like half of the time, if we clear the git cache on jenkins like "rm -rf /var/jenkins_home/caches/*" and kill all the worker nodes(We are using a helm deployment of Jenkins) and restart it works, but of late its just been happening a whole lot which is frustrating.

    Wondering if anyone else has had similar experiences with JTE and had any pointers?
    Again, deeply appreciate any help!

    2 replies
    René Zubcevic
    @zubcevic

    is there a place we can share some repositories with working examples? I am experimenting with JTE in combination with JCASC. Here are two repos: https://github.com/zubcevic/jenkins-controller

    https://github.com/zubcevic/pipeline-config
    the second one is for the jte stuff, the first for the overall simple jenkins setup including a mb pipeline job for building some software

    Carlos OKieffe
    @cokieffebah
    @sebastianchacko89 I believe we removed the auto checkout around 1.4. Most of our teams include the 'sdp' library which has a 'hook' create_workspace_stash, that creates an initial 'workspace' stash
    Like Steven, I will be out until next week.
    Travis Ly
    @TravisLy7_twitter
    @steven-terrana great job on JTE 2.0, we are refactoring our Jenkins Share library and this would give us the governance that we need. I have a quick question regarding the ability to allow the dev team to supply their own libraries in their own feature branch. Currently, if we have the multi-branch or folder configuration as the governance tier, how would the dev team supply their implementation of library in their libraries folder in their code repo? The only way I can see in JTE is to define Default Step in the pipeline_config which they would also need specify the image (or container) that they would use for that step.
    4 replies
    yoav-klein
    @yoav-klein

    Hi @steven-terrana, I encounter the following error:

    // pipeline_config.groovy
    ...
      my_map = ["Major": 1, "Minor": 2, "Revision": 3, "No Change": 4]
    ...
    
    // when running
    hudson.remoting.ProxyException: org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
    Script1.groovy: 4: unable to resolve class No 
     @ line 4, column 16.
                   No Change = 4
                      ^
    
    1 error

    Why isn't it possible to have a string with a space as a key in a map in the config file?

    steven-terrana
    @steven-terrana
    @yoav-klein - probably a bug in the serializing of the pipeline configuration object. We’ll look into it. cc: @cokieffebah
    3 replies
    yoav-klein
    @yoav-klein

    Hi @steven-terrana

    I'm facing the following issue, and would appreciate an advice:

    I have a ant library, which has a build.groovy step. I want it to be as generic as possible,
    so my thought was to set a list of properties in the pipeline_config.groovy, so the call to ant
    would look something like:

    // build.groovy
    def call() {
        String props_str
        config.properties.each { property ->
            props_str += "-D${property.key} ${property.value}"
        }
        bat script: "ant -f ${config.script_path} ${props_str}"
    }

    The problem arises is: One of the properties values I want to pass should be received from the user
    as build parameter. So the above is fine if I hard-code all the properties in the pipeline_config.groovy,
    but how do I do it with arguments?
    I'm looking for something like:

    // pipeline_config.groovy
    libraries {
        ant {
            properties = [[prop1: 'hard_coded1'], [prop2: 'hard_coded2'], [prop3: params.dynamic]]
        }
    }

    Is there a way to tell a step "evaluate the following expression in runtime"?

    5 replies
    SachinMaharana
    @SachinMaharana
    Is there any example repo where libraries are created for nodejs/npm project?
    thetechoddbug (José María Gutiérrez)
    @TheTechOddBug
    What are the minimum requirements (jenkins version, other plugins...) for JTE 2.0?
    akibabu
    @akibabu

    Hi @steven-terrana
    Evaluating the library but having troubles getting started.

    Getting either java.lang.Exception: Could not determine pipeline template. if I don't use any libraries in the project (only default pipeline libraries)

    Or Library gradle not found. if I try to use a library

    These are my configs https://github.com/akibabu/jenkins-jte
    And test project https://github.com/akibabu/jenkins-jte-project

    Of course my github organization templating engine config + libraries both point to the jenkins-jte repo with basepaths pipelines/ci_default and libraries

    1 reply
    Serpa
    @serpa-dystematic

    Hello @steven-terrana . Thanks for the plugin!
    I'm finding a bit hard to understand exactly what is the keywords primitive use case and how it should be used.

    For example, I'm trying to build a "library" pipeline where it can implement a python or a nodejs library:

    node {
      inside_sdp_image docker_image, {
        validate()
        init()
        security_analysis()
        lint()
        unit_test()
        integration_test()
        version_bump()
        publish()
      }
    }

    But I need to set that docker_image.
    I've been able to do it with this configuration on the python lib:

    jte {
      pipeline_template = 'library'
    }
    
    libraries {
      python {
        lib_path = "xxx/yyy"
      }
    }
    
    keywords {
      docker_image = "python-3.8"
    }

    Is this a good practice? If not, how could I set the docker image dynamically?
    Thanks!

    15 replies
    Matteo Cartuccia
    @cartucciam
    Hello all, I am upgrading to 2.0 and started seeing a different behaviour. It picks up the Jenkinsfile even if my config contains allow_scm_jenkinsfile = false, from the documentation it looks like nothing has changed on that front. @steven-terrana maybe you can help?
    5 replies
    Carlos OKieffe
    @cokieffebah
    @/all we are trying to get feedback on the 'Default Step' usage:
    that is when you have a pipeline_config.groovy with a steps section. If you are using the feature, let us know. Because of its dependency on Docker, we feel we need to move this feature out of core JTE. Thanks
    7 replies
    Jamin
    @benjcleveland

    We have been working on upgrading to JTE 2.0 and have run into a problem with the stages and passing in an application_environment. jenkinsci/templating-engine-plugin#151. Basically, we are doing something like the following:

    jenkinsfile

    continuousIntegration dev

    pipeline_config.groovy

    stages {
      continuousIntegration {
          ... 
      }
    }
    application_environment {
      dev {
        ....
      }
    }

    Is this something that should be supported?

    12 replies
    Ghost
    @ghost~5ffbee5ed73408ce4ff87e9f
    This message was deleted
    1 reply
    thetechoddbug (José María Gutiérrez)
    @TheTechOddBug
    Hi @steven-terrana and @cokieffebah. I have a doubt regarding of configuring a Global Governance Tier getting the Source Configuration Pipeline Location from other rather than GitHub. For example, from GitLab but if you use a GitLab with MFA, it seems not allowed to use a Personal Access Token to configure this side of the Global Governance. Am I missing anything? If not, do you have this feature in your roadmap? Thanks in advance!
    8 replies
    Nayan Bhana
    @nayanbhana
    Hi @steven-terrana and @cokieffebah
    I am trying to use pipeline templates and running into an issue where it is trying to find a Jenkinsfile in the root of my governance tier instead of the file defined in my pipeline_templates folder.
    So in my application repository, I have pipeline_template="deploy" as the first line in my pipeline_config.groovy
    In my configuration repository, I have a folder at the root called pipeline_templates and inside is a file called deploy
    In my logs on JTE I get this
    [JTE] -- scm: git https://github.com/contino/bjb-hello-world.git
    [JTE] -- file path: deploy/pipeline_config.groovy
    [JTE] Pipeline Configuration Modifications
    [JTE] Configurations Added:
    [JTE] - pipeline_template set to deploy
    So it looks like it is picking up the pipeline template setting BUT when it comes time to find the Jenkinsfile
    [JTE] deploy/Jenkinsfile does not exist. (show)
    [JTE] Jenkinsfile does not exist.
    [JTE] -- scm: git https://github.com/contino/bjb-jte-config.git
    java.lang.Exception: Could not determine pipeline template.
    Do you know what I am doing wrong? Thanks in advance
    3 replies
    thetechoddbug (José María Gutiérrez)
    @TheTechOddBug

    Hi. I'm running a PoC using a Jenkins instance running in a docker container and configuring JTE to use SCM config and templating and trying to get all the projects inside a group/subgroups structure from GitLab on-premise (using the GitLab group job and its corresponding plugin).But my issue is, quite sure, out of all of this. The "step" i'm trying to run is something like:

    void call(){
        node {
            deleteDir()
            docker.image('node:12.16.1-alpine3.11').inside {
                stage("npm-ng: Build"){
                    println "*Build from the npm-ng library."
                    ... //Some other sh commands here
                }
            }
        }
    }

    And I'm getting always an error due it is not able to find the file of the Angular project to follow the build. The strange thing is that console output shows that it is working at:

    Running on Jenkins in /var/jenkins_home/workspace/pring-petclinic-angular_master_2

    But the project is spring-petclinic-angular not pring-petclinic-angular... Any idea what is happend and how to solve it? I'm getting crazy with this silly thing...

    6 replies
    Ganesh
    @n1603

    Hi @cokieffebah @steven-terrana ,
    I saw some example for template execution based on trigger like on_merge, on_pull. Is there any way to support custom event example like if in PR comments section of github, we type some keyword, the keyword can be caught by JTE and based on it template execution can be done for some specific step. The flow can be used to do somethings like on demand tests etc.

    PS : I am not much experienced with Jenkins as well to JTE, hope I did not miss something obvious to look at for above.

    3 replies
    Nayan Bhana
    @nayanbhana
    I have multiple pipelines that run on the same repo. For each of these pipelines, they follow a different pipeline template.
    For example, I have a pipeline to package an app and publish the artifact, then I have a pipeline to deploy the artifact. I run these using a multi-branch pipeline on Jenkins. At the moment, I have not found a way to specify a custom name for the pipeline_config.groovy, my repo will have a folder for deploy which contains a pipeline_config.groovy and a folder for packaging that will contain the pipeline_config.groovy for that because I can set a "base configuration" setting in Jenkins.
    Is there any way I can have these pipeline_config.groovy files in the same folder with different names?
    4 replies
    Ganesh
    @n1603

    Hi @cokieffebah @steven-terrana, We are trying to implement JTE in our project, want to pass some arguments to the step groovy script (which is build.groovy)
    Below is how we are doing.

    1. libraries/gradle/steps/build.groovy

    void call(args){
    gradle(args)
    }

    def gradle(Map args){
    sh 'printenv'
    println "========"
    }

    1. project Jenkinsfile

    node{
    continous_integration()
    }

    1. pipeline_config.groovy
      stages{
      continous_integration{

        build

      }
      }

    The above works fine. What we want to do is pass some arguments to build.groovy. like one example (build.gradle(branch:"${BRANCH_NAME}", buildNumber:"${BUILD_NUMBER}", localImageName:"test-img") , but this is with calling method which does not work in this case)

    is it possible, How can we achieve this usecase ?

    2 replies
    Carlos OKieffe
    @cokieffebah
    We released 2.0.1 earlier this week. it resolves an issue where maps did not work in the pipeline_config.groovy and some documentation updates, we added the JTE block to the upgrade info
    vincent r kennedy
    @vincent-r-kennedy

    Hi - In the jenkins GUI, configure-job, I am trying to get the Pipeline Configuration, 'Branches to build' section to use the current branch.
    I have a multibranch pipeline and I want to to pull the pipeline config from whatever the current branch is. So if this build was triggered by a change on branch feature/xyz, I want the JTE to also to use feature/xyz as the branch to pull.
    This looks like it should be straightforward as I can go to the 'branch specifier' section and use an environment var as described like:

    ${ENV_VARIABLE}
    It is also possible to use environment variables. In this case the variables are evaluated and the result is used as described above.
    E.g. ${TREEISH}, refs/tags/${TAGNAME}, ...

    I can see in the jenkins env, the current branch is available via a env var called BRANCH_NAME, which if I could get it picked up would solve my problem. But JTE doesn't seem to actually read the env var, it just uses the name of the var directly.
    I've tried ${env.BRANCH_NAME} and ${BRANCH_NAME} - but the value doesn't get read.
    Am I missing something?

    Serpa
    @serpa-dystematic

    Hello, sorry if this is a duplicated question, but could not find an answer. (I think this should be referenced here in docs)

    How can one give a default value to an optional field in library_config.groovy?

    3 replies
    Vincent Letarouilly
    @Opa-

    @serpa-dystematic I think having the availability to set default value directly inside library_config.groovy would be a great feature. I did a workaround by having a special step for that with @Init hook like so:

    @Init
    void call() {
        def DEFAULT_JDK = 'openjdk8'
    
        if (!config.jdk) {
            echo "JDK not specified, using default JDK: $DEFAULT_JDK"
            config.jdk = DEFAULT_JDK
        }
    }

    and this is the related library_config.groovy file

    fields {
        optional {
            jdk = ['openjdk8', 'openjdk11']
        }
    }
    1 reply
    Carlos OKieffe
    @cokieffebah
    @serpa-dystematic @Opa- , feel free to add a feature request ... but ... this change would require changing the structure of the `library_config.groovy' and therefore break existing library_config.
    Acentera
    @ACentera_twitter

    Question, can we create like multistage builds in an JTE Build object?

    ie: Create parralel stage dynamically based on some kind of business logic.

    ie: for the build
    App A might create 5 parralel stage
    App B will only create 1 stage

    3 replies
    Acentera
    @ACentera_twitter
    Just watched https://www.youtube.com/watch?v=pz_kPpb9C1w&feature=youtu.be
    Verry great presentation. JTE Solve a lot of problems. Great library.
    1 reply
    Vincent Letarouilly
    @Opa-

    Does anyone got the new feature "Pipeline Resumability" of JTE 2.0 working ? When I'm restarting Jenkins while a pipeline is running, on resume I get this error:

    an exception which occurred:
        in field groovy.lang.Closure.delegate
        in object org.jenkinsci.plugins.workflow.cps.CpsClosure2@606daccd
        in object of type org.jenkinsci.plugins.workflow.cps.CpsClosure2
        in field groovy.lang.Closure.delegate
        in object org.jenkinsci.plugins.workflow.cps.CpsClosure2@be97210
        in object of type org.jenkinsci.plugins.workflow.cps.CpsClosure2
        in field groovy.lang.Closure.delegate
        in object org.jenkinsci.plugins.workflow.cps.CpsClosure2@7473b201
        in object of type org.jenkinsci.plugins.workflow.cps.CpsClosure2
        in field groovy.lang.Closure.delegate
        in object org.jenkinsci.plugins.workflow.cps.CpsClosure2@3dbcf963
        in object of type org.jenkinsci.plugins.workflow.cps.CpsClosure2
        in object of type java.util.HashMap
        in field org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.closures
        in object org.jenkinsci.plugins.workflow.cps.CpsThreadGroup@303a31dd
        in object of type org.jenkinsci.plugins.workflow.cps.CpsThreadGroup
    Caused: java.io.InvalidClassException: WorkflowScript; Class does not extend stream superclass
        at org.jboss.marshalling.river.RiverUnmarshaller.doReadClassDescriptor(RiverUnmarshaller.java:1046)
    ...

    full stacktrace here https://pastebin.com/TigWEmWf

    3 replies
    Vincent Letarouilly
    @Opa-
    another one with full log of the build https://pastebin.com/sJTVvRJz
    Acentera
    @ACentera_twitter

    https://jenkinsci.github.io/templating-engine-plugin/pages/Job_Configurations/template_step.html

    Is the

    template {
    }

    supposed to be available in an Jenkinsfile ?

    1 reply
    addihorn
    @addihorn
    hi @steven-terrana @cokieffebah
    is there a way to define in my Jenkinsfile to use JTE wand where to find the data?
    Background is: We are using gitea as SCM-Server and we are currently defining the gitea-organizations/projects in Jenkins. At that stage I can define how to identify buildable Projects (by Jenkinsfile or JTE logic)
    But this will apply to all repositories in the project, some of them I do not have in scope. They still want to build the "normal" way by just defining the Jenkinsfile. I want to have my builds Templated by JTE.
    Is there a way to have both solutions in one project/organization without having to import it twice (oone with JTE-Builds, ne as normal builds)?
    4 replies
    yoav-klein
    @yoav-klein

    hi @steven-terrana @cokieffebah

    I found that it could sometimes be useful to be able to configure some library configurations in a higher-level tier, but not necessarily load the library for each of the lower-level tiers' projects. For example, imagine I have a Folder representing a project with several components in it, i.e. several jobs. Some use CMake, some use Maven, etc. I would like to share some CMake configurations among the CMake components, so I'd like to be able to configure those in the high-level folder tier, and same for Ant and Maven etc. But I certainly don't want the CMake library to be loaded in a Ant project. I know I can split my governance so that all CMake projects will be under one tier and all Maven projects will be under another, but that causes the Folder hierarchy to be structured by a technical point of view and not business-logic point of view. Could that be a future feature to be able to configure a library with only explicit loading? Will appreciate your thoughts

    2 replies
    Sean Hoar
    @SHoar

    @cokieffebah @steven-terrana I have a JTE library step which has a call that includes several def methods being exec’d. Within a jte library step, is there a way to turn a passed parameter to the calling step into a groovy global variable in that file? If I were to specify the passed parameter as a new parameter to all the following steps, it is not very DRY.

    i.e.

    jte-repo
    |
    |__libraries
       |__steps
          |__mainStep.groovy

    The mainStep.groovy has one call and several methods

    // mainStep.groovy
    void call(String templateSpecificParameter){
      node{
        method1( templateSpecificParameter)
        method2() // needs access to templateSpecificParameter
        method3() // needs access to same parameter for a compound field
      }
    }
    
    def method1( String templateSpecificParameter ) { // may be called more than once on a jte scan based on the pipeline_template
     ...
    }
    
    def method2(){ ...} // will run if method1 is called & needs passed param
    
    def method3(){ ... } // will run if method1 & method2 are called & needs passed param

    If I just repeated that arg for each method, it solves my problem but isn’t very DRY. However, maybe it’s an acceptable arg to be repeated so that I don’t have to have the mainStep.call create a global var, or is that something that now exists via context? Still new to JTE and Jenkins groovy DSL

    6 replies
    thetechoddbug (José María Gutiérrez)
    @TheTechOddBug

    Hi! I have a doubt. When I run my job that scan the sources repositories to find projects, I see the comments I highlight next (in bold and appear out of the code formatting) regarding to file exist but is not a file:

    Branch indexing
    [JTE] Obtained Pipeline Configuration File (show)
    [JTE] Pipeline Configuration Modifications (show)
    Querying the current revision of branch development...
    Current revision of branch development is dda1dd6181ce6340d736a5b3e2fd44b3c089eb8f

    [JTE] pipeline_config.groovy exists but is not a file. (show)

    [JTE] Loading Library npm_ng (show)
    [JTE] Loading Library dockerBuild (show)
    [JTE] created Library Steps: (show)
    Querying the current revision of branch development...
    Current revision of branch development is dda1dd6181ce6340d736a5b3e2fd44b3c089eb8f

    [JTE] Jenkinsfile exists but is not a file. (show)

    But the code that is in the SCM repo with the template, libraries and config is executed properly. What that messages mean and how can I not get that "strage" errors?

    Thank you very much in advance.