by

Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Gregory Paciga
    @gpaciga
    actually ignore the second part - obvious issue was forgetting override=true on the global pipeline config. The first part holds though, no way to set Jenkinsfile/pipeline config from SCM?
    steven-terrana
    @steven-terrana

    right now for regular pipeline jobs you can only define a template and pipeline configuration in the Jenkins UI.

    it’d be possible to allow it to be defined in an SCM - no one’s asked for it before! i think most folks end up just using a MultiBranch Pipeline Job.

    but feel free to open an issue on the repo (https://github.com/jenkinsci/templating-engine-plugin) and i’d be happy to explain how to do it or get to it when i can

    not a great solution - but right now you could create a multi-branch job and filter it to just the branch you care about.

    i do think it’s worthwhile to add SCM support for regular pipeline jobs

    Gregory Paciga
    @gpaciga
    ok thanks for that confirmation! so far we've done everything with regular pipeline jobs since different branches had to be built different ways - but next thing on my list is to look into using multibranch pipelines instead
    steven-terrana
    @steven-terrana

    might be worth taking a look at the Pipeline Libraries i’ve helped to build for our organization.

    https://boozallen.github.io/sdp-docs/sdp-libraries/2.2/index.html

    specifically - the github library.

    For this use case we typically define Keywords that are regular expressions that map to branch names:

    keywords{
        master  =  /^[Mm]aster$/
        develop =  /^[Dd]evelop(ment|er|)$/ 
        hotfix  =  /^[Hh]ot[Ff]ix-/ 
        release =  /^[Rr]elease-(\d+.)*\d$/
        feature = /^JIRA-(\d+)*$/
    }

    and then those work well with our library so you can write pipelines using our library like:

    on_commit to: feature, {
      // do something on a commit to a feature branch
      // named after some JIRA ticket 
    }
    
    on_pull_request to: develop, from: feature, {
      // do something on PRs to develop
    }
    
    on_merge to: develop , {
      // do something specifically on merge of a PR to develop
    }
    
    on_merge to: master, {
      // do something on a merge to master
    }

    etc

    Gregory Paciga
    @gpaciga
    yeah, I'd like that approach, but would need a bitbucket server library. I did some poking on how to identify the source branch, and it's definitely doable with their REST API and probably their java sdk too, but am not sure how to go about taking it any further than that.
    although i probably don't even need the from: functionality
    Scott Crosby
    @iscooter
    This is curious. I just saw JTE for the first time. I created something similar a few months ago to enable managing hundreds of pipelines easily. https://github.com/iscooter/nopipeline-containers
    I used callable objects within the DSL to control pipeline level logic for integrating apps or stages in containers. Has anyone used JTE to extend app-in-docker to the developer?
    steven-terrana
    @steven-terrana

    @iscooter hello! not exactly sure what you mean by “app-in-docker”.

    i can tell you that i typically recommend libraries use container-images for pipeline runtime environments using the Docker Pipeline Plugin but don’t typically go so far as to recommend the 3 musketeers pattern and rely exclusively on Make/Compose/Docker.

    that being said - there’s no reason you couldn’t build your libraries that way.. where each “step” is really just a Make target.

    JTE might be a nice way to orchestrate your nopipeline-containers framework by being able to centralize the a pipeline template that loads the shared library instead of needing to put that in every repo

    Daniel Fullarton
    @linead
    Hey @steven-terrana - ever tackled mono repos with JTE?
    4 replies
    Scott Crosby
    @iscooter
    Thanks @steven-terrana, that was a dumb question half baked on my part. The route I’ve taken is to define what the pipeline stage order and action in JSON. There’s no Jenkinsfile, JSL or script to expose to the user. They just commit the actionable code and the “type” of pipeline (make, python, container, etc) just does that work. Integrating by simply, from groovy, parsing the JSON and processing it in a runner with python. We have one Jenkinsfile in hundreds of repos where the user is directed to only change the JSON as needed or we define a new param for them. The 3musketeers pattern is only for the....how do we say...Jenkins nuts like us :)
    I am however still interested to dive into these libs you’ve baked up. I see lots of opportunity. Plus the JTE framework sucks it in with less work for me. Just need to see how to add in seemlessly with our set of vars/call() functions.
    2 replies
    Gregory Paciga
    @gpaciga

    so I (perhaps naively) thought that @AfterStep and @BeforeStep would trigger on calls to stage(). I take it they only work on steps defined within JTE? Is there a way to put hooks on stage() calls instead? My use case right now is that I'm trying to put together a way of reporting on the results each stage individually. I'm putting together a Jenkinsfile with a bunch of on_commit and on_pull_request calls (from my own library), each one wrapping a different set of stage() calls that aren't abstracted into JTE yet. Using @AfterStep and @BeforeStep to act as listeners, the list of steps the pipeline goes through looks like

    on_pull_request
    on_commit
    on_commit
    on_commit

    when in fact only one of these will ever actually do anything. What I'd like to see is the list of stages that actually ran. Maybe this is more of a Jenkins-in-general question than a JTE one, just that the lifecycle hooks had me optimistic that I'd be able to do this easily within JTE.

    13 replies
    Gregory Paciga
    @gpaciga
    do stages have to be JTE-defined steps, or can I do arbitrary code? e.g.
    stages {
        some_stage_name {
            stage("my stage") {
                // normal stage as I'd define it in a scripted pipeline
            }
        }
    }
    steven-terrana
    @steven-terrana
    they have to be JTE steps
    Gregory Paciga
    @gpaciga
    hmm I'm this close to defining a JTE step that just wraps a vanilla stage with my before and after logic built in, but that feels pretty hacky
    steven-terrana
    @steven-terrana
    lol it’s one of those… if you really understand what you’re doing then i’d call it more clever than hacky. but with great power…..
    Gregory Paciga
    @gpaciga
    I think I clearly don't :) Something for me to sleep on, though.
    steven-terrana
    @steven-terrana
    if you decide to do that.. feel free to post here and i’d be happy to tell you if there’s any glaring issues you’ll run into.
    freestyler164
    @freestyler164
    Hi There, since the context.status is not available any more , I was trying to use currentBuild.result. Is this available at @Cleanup ? I seem to get a null value in Cleanup
    freestyler164
    @freestyler164

    Hi There, since the context.status is not available any more , I was trying to use currentBuild.result. Is this available at @Cleanup ? I seem to get a null value in Cleanup

    currentBuild.currentResult seems to be the one I was looking for.

    Gregory Paciga
    @gpaciga
    regardling library configs, if I have an optional config, does it make sense to set it to a default value if it's not provided within the @Init or maybe an @validate?
    @Init
    void call(Map args = [:], Closure body) {
        config.myparam = config.myparam ?: 'defaultValue'
    }
    steven-terrana
    @steven-terrana
    i don’t think updates to the config variable will persist so you’d have to do it in each step. It would be an interesting idea to be able to declare the default value for optional steps inside your library configuration file
    Gregory Paciga
    @gpaciga
    they do seem to persist, but sounds like that might be a happy accident rather than purposeful
    i just the usual pattern is to just check if it's defined in the steps that need it?
    steven-terrana
    @steven-terrana

    hahah i feel like that’s a happy accident that i should probably break tbh..

    bc imagine library A contributes a step A and library B contributes a step B..

    in A i could do: B.config = [:] and break another library’s step

    it assumes malicious intent bc you’d have to go pretty out of your way to hit that situation
    but i don’t think it should be possible

    i do think it would make sense for library configs to declare default values.

    or for you to only be able to manipulate that config variable within the same library

    steven-terrana
    @steven-terrana
    Thoughts? i’m open to ideas here on how you all think it should work
    Vincent Letarouilly
    @Opa-

    I was setting the default values with @Init like @gpaciga mentioned and I was thinking it was intended to work this way :D

    I think it'd be better to define the defaults into the library config, that way you define everything in the same file

    Gregory Paciga
    @gpaciga
    defaults in the config make sense to me too. breaking/fixing ability to set config values is fine as long as there's an alternative. though I wonder too about calculated/derived values, like if on init you take a config that's a simple user-friendly value and use that to generate a config value to be used internally in multiple steps. I don't really know the right terminology for it, but more generally, do libraries have a namespace where I can set values, like class members/fields?
    makashu
    @makashu
    Hi, is it possible to load libraries from specific branch in addition to master which is supported by pipeline libraries?
    My use case is something like library "jte-libraries@feature/test-branch" in jenkinsfile or pipeline_config.groovy, is that supported?
    steven-terrana
    @steven-terrana
    @makashu that is not currently possible
    3 replies
    Vearak Thach
    @vthach24
    Hello all, I'm attempting to use JTE and write a pipeline for SonarQube. Can anyone assist in getting me started? Some good documents or examples?
    Daniel Fullarton
    @linead
    @vthach24 I started here - https://boozallen.github.io/sdp-docs/overview/1/index.html once you get used to the model it's pretty straight forward
    steven-terrana
    @steven-terrana

    thanks @linead!

    @vthach24 I usually recommend these Learning Labs when first getting started with JTE

    Once you get through that, you can take a look at Booz Allen’s existing SonarQube library for a starting point if you’re interested.
    Docs: https://boozallen.github.io/sdp-docs/sdp-libraries/2.2/libraries/sonarqube.html
    Library: https://github.com/boozallen/sdp-libraries/tree/master/libraries/sonarqube

    mfuxi
    @mfuxi
    I would really love to get a full example of how to run a simple maven pipeline with custom node since I still can't make it work
    The only method I had managed to run a simple maven pipeline was only on the Jenkins master
    If i'm trying to run on a node i'm getting this error:
    step node from the library node does not have the method call(CpsClosure2)
    my node.groovylooks like that:
    void call(String label= null){
       steps.node( label ?: config.label ?: ""){
           println "test 123"
       }
    }
    my pipeline_config.groovy:
    libraries{
        node {
             label = "jte"
        }
        maven
    }
    mfuxi
    @mfuxi

    Weird, I updated my node.groovy to look like that:

    void call(String label = null, Closure body){
       steps.node( label ?: config.label ?: “” ){
        body()
      }
    }

    And now it works, and runs the build on the node I supported within the pipeline_config.groovy but I don't understand why, can you explain @steven-terrana ?

    Also, my Jenkinsfile looks like that:

    build()

    Why don't I need to call to node() as well?

    steven-terrana
    @steven-terrana

    when you say node(“something”){ // some code }

    that’s the same thing as saying node(“something”, { // some code })

    so the { } is called a Closure and it’s the last input parameter to the node step.

    so when your step was just call(String label = null) it was never accepting the closure parameter.

    you are calling the node step. In your maven library, i’m assuming you have a node{ // some code } block in there somewhere?

    that node block is actually invoking your node library’s node step.

    Gregory Paciga
    @gpaciga
    are there examples of configuring a JTE job with JobDSL? I saw an issue in the old repo discussing it, but can't find anything in the docs
    Gregory Paciga
    @gpaciga
    ah nevermind, found it under factory { templateBranchProjectFactory { ... } }
    Miguel Quintero
    @miqui
    hi, newbie question: how do i configure a specific build agent? (similar to what is done in the traditional Jenkinsfile)
    steven-terrana
    @steven-terrana

    hey @miqui - JTE uses the scripted pipeline syntax. so configuring a build agent would be done the same was as Scripted pipelines.

    if you’re looking to configure a single build agent across the entire pipeline run then you could wrap the whole template in a node block:

    node(“someNodeLabel”){
      step1()
      step2()
    }