Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • May 06 16:03
    tomzo commented #152
  • May 06 16:03

    tomzo on master

    update file pattern documentati… Merge pull request #152 from dA… (compare)

  • May 06 16:03
    tomzo closed #152
  • May 06 15:46
    dAnjou opened #152
  • Jan 24 17:11
    dnewhook commented #136
  • Dec 09 2020 08:36

    tomzo on master

    use newer openjdk Dojo image ku… Merge pull request #151 from xm… (compare)

  • Dec 09 2020 08:36
    tomzo closed #151
  • Dec 09 2020 08:35
    xmik opened #151
  • Dec 03 2020 00:26
    sahlone edited #150
  • Dec 03 2020 00:25
    sahlone edited #150
  • Dec 03 2020 00:25
    sahlone opened #150
  • Dec 01 2020 08:26
    tomzo commented #66
  • Dec 01 2020 01:12
    manojtr commented #66
  • Nov 30 2020 23:54
    manojtr commented #66
  • Oct 27 2020 09:37
    thatsk commented #149
  • Oct 27 2020 09:37
    thatsk commented #149
  • Oct 27 2020 09:35
    thatsk commented #149
  • Oct 27 2020 09:35
    thatsk commented #149
  • Oct 27 2020 09:34
    thatsk edited #149
  • Oct 27 2020 09:33
    thatsk edited #149
Christian Dahlhausen
@cdahlhausen
Is there a priority when using multiple config repositories in yaml?
adamMo
@adamMo
Hello guys,
I would like to develop the possibility to define approval value in environment variable - currently I have one alias referenced in 40+ pipelines and I would like to gradually change them from 'manual' to 'auto'. Do you forsee any problems with that approach?
Matthew Kennedy
@mattkenn4545

Trying to figure out best way to access a private docker registry to do a docker pull from within a dind container.

Ie I have pipeline that does a docker build. I have the artifact working sufficiently to upload to the private repository. What I need is to be able to pull down that artifact/image on other pipelines.

Ideally there would be a simple way to pass the imagePullSecrets to the docker daemon running DinD however I doubt this will work.

The path I'm currently on is passing ENV vars (username/password) via the elastic agent profile. I see the env var in the pod but I can't seem to reference them with a pipeline command/args.

Thatsk
@thatsk
Hello Team, I have question is their way we can restrict when new pipeline.yaml code is pushed to gocd-pipeline.git to build only new pipeline.yaml code
right now when i push new pipeline to repository it starts build all the pipeline
And elastic agents get fully occupied and get exhausted. Why config repository is not building only latest pipeline pused and why its building all the pipeline. is any config is missing
Thatsk
@thatsk
like we have 100 pipelines in gocd-pipeline.git and 101 is added it starts building all 101 pipeline at the same time
2 replies
tsibiski
@tsibiski

Hi, I am having trouble finding out how to trigger multiple GoCD pipelines during checkin.

I currently have a GoCD configuration json file that defines the pipeline I want to run. When I check my code in, the configured pipeline is automatically launched and works just fine.

However, I would like to either:

A) Define multiple pipelines in the same gocd json file, so that when I check my code in, I launch multiple GoCD pipelines at the same time.

B) Define stages in a single pipeline that can all run at the exact same time as one another.

I need to launch several different QA test automation pipelines. One for each browser. I also need them to run simultaneously because they are also testing that the feature works with multiple "people" accessing the APIs at approximately the same time.

I realize that I could define the jobs inside GoCD and then have them triggered by checkins to the same repo, but the lead developer on my project wants all pipelines defined by the json files and parsed by"json.config.plugin". Any advice is greatly appreciated!

marques-work
@marques-work

@tsibiski pipelines are triggered by “materials”; admittedly the name is confusing, but for 95% of pipelines, it refers to a source code repository.

That means that if you want all of your pipelines to be automatically triggered by a single commit to exactly one repository. Thus, for each pipeline you want to trigger, said pipeline must have that repository configured as one of its materials. It is conceptually separate from using pipelines as code to define the pipeline, even though it’s a common case to use the same repository to store pipeline definitions.

tsibiski
@tsibiski
Ah ok, so I can accomplish this if I create the pipeline in the GoCD GUI, and making multiple materials, but I cannot do it using the json.config.plugin and multiple defined pipelines? I wan't to make sure I understand you correctly.
marques-work
@marques-work
You can do it with the JSON config plugin
All I’m saying is that when you define your pipelines in the JSON config plugin you must add that repository as one of the materials of that pipeline
Only then will they be triggered by a commit
tsibiski
@tsibiski
(My apologies if I come off dense here at all, or miss your points) So I have that repo/material defined in my pipeline json. What I am having trouble figuring out is how to define multiple pipelines in the same json file. I was expected that I would have two json objects like this:
{
"format_version": 1,
"label_template": "${COUNT}",
"enable_pipeline_locking": false,
"name": "my-pipeline-01",
"group": "MyPipelines",
And then a second object with all of those stages and tasks (and the same material) would be below this one in the same file, but it might be called "my-pipeline-02".
marques-work
@marques-work

@tsibiski the second part of your question might involve familiarizing yourself with the conceptual structures in GoCD: pipelines, stages, jobs, and tasks.

Pipelines are a container for stages.
Stages are groups of high level work that run sequentially. Stages will always run serially.
A stage is container for jobs. Jobs are meant to run in parallel. This is the answer you are looking for, most likely. Definite multiple jobs to run in parallel
Finally jobs execute a group of 1 or more tasks in serial. Think of tasks as individual steps in a build script

@tsibiski you can define any number of pipelines in a single JSON file. Let me point you to the docs/examples on github to show you how
tsibiski
@tsibiski
ahhhhh. Ok that was a hugely helpful description. Thank you. I have tons of experience with Jenkins, so this was a little different for me. I appreciate the link to examples.
marques-work
@marques-work
Sorry give me a moment. I’m on my phone so this is hard to locate.
tsibiski
@tsibiski
Take your time. Thanks
marques-work
@marques-work
Ok, so I’m mistaken 😅. I got mixed up with the YAML plugin. For JSON, it is one pipeline per file, so define as many files as you need.
tsibiski
@tsibiski
Oh ok. How would I name them, though, so that all of them are recognized and parsed?
right now, its gocd.gopipeline.json. Any other name in place of "gocd"?
Aravind SV
@arvindsv
Yes. *.gopipeline.json is checked by default.
marques-work
@marques-work
The default pattern is *.gopipeline.json so any name will do that fits that pattern
Yes what @arvindsv said
tsibiski
@tsibiski
great. Thanks!
marques-work
@marques-work
That pattern is configurable if need be, but it works for most people so I wouldn’t customize it unless you need to
If I’m not mistaken I think it’s technically */.gopipeline.json so you can put them in a subdirectory if you have a lot of pipelines to define. That way it doesn’t clutter your top level workspace
@tsibiski ☝🏼
Sorry markdown clobbered my glob pattern
**/*.gopipeline.json is what it should read
tsibiski
@tsibiski
interesting. Thank you!
marques-work
@marques-work
👍🏼
Cristiano Fontes
@cfontes
Hello everybody
We are facing a pretty nasty prod issue with our GoCD server
gocd/gocd#8970
please take a look if you can, I have no idea what to do anymore.
I am afraid to restart the server and make it unusable, right now at least the old pipelines still work
Cristian Iaroi
@wirdman
hi everyone, while using the go-config-repos (yaml) we've noticed these types of errors popping up all over the logs: 2021-02-15 15:24:14,977 ERROR [qtp379009144-8053] VariableReplacer:385 - function ${escape:} type 'escape:' not a valid type - how can we fix this? We are currently in the testing phase for the pipeline as code approach and are wondering if there's a limit to how many config repos can be defined on a particular instance of gocd server, we're seeing delays for over 100 config-repos ~ 1k pipelines. Do you have some perf testing data we can look at (or just what's the limit)?
Cristian Iaroi
@wirdman
is there a way to configure a gocd config-repo to not poll on the changes at the server's startup? I did set the "auto_update": False property however this does not prevent fetching all repos at startup time.
Cristian Iaroi
@wirdman
@tomzo apologies for the mention :) are you by any chance able to help me with any of the above?
Aravind SV
@arvindsv

@wirdman Is the VariableReplacer error causing an issue? If so, how does it show up? What happens, and when?

I'm not sure about the ignored auto_update. This code seems to suggest it should be ignored: https://github.com/gocd/gocd/blob/79efd71723694a66b08a715732a719db32556a84/config/config-api/src/main/java/com/thoughtworks/go/config/BasicCruiseConfig.java#L1103

Cristian Iaroi
@wirdman
@arvindsv it is not causing an issue, it is just shown in the log file as an error and I assumed it could be an issue with our pipeline config. Regarding the ignore auto_update - upon server start up the config repos are always pulled from our gitlab instance even though they are set with auto_update=false, which is not what I was expecting. I was expecting the config repos to be ignored untill I trigger a fetch (reload/pull from the ui/api) or if I trigger the pipeline.
Aravind SV
@arvindsv
@wirdman GoCD might not even be aware of a pipeline to trigger, if the repo is not fetched. But yes, I think the expectation of ignore the repo (and so the pipelines) till they are triggered from the config repos UI or API seems to make sense. As I mentioned, I don't know the reason it' s not honored.
Cristian Iaroi
@wirdman
@arvindsv according to this comment: https://github.com/gocd/gocd/issues/3985#issuecomment-343449436 there seems to be a "normal" polling at server start up. I'm not sure however what materials are updated to last known revision before the restart means - does it do a git pull? which seems to be the case - can this behaviour be avoided?
Alex Milstead
@amilstead

Howdy! Does anyone have any experience using the yaml config repo plugin alongside the Github pull request builder plugin? I am trying to use it myself but with limited success. I would like to be able to trigger pipeline builds from a PR, where the version of pipeline definition (i.e. the .yaml file) in the PR is used, rather than the default branch used during initial repo setup.

Is this possible? It feels like I'm just missing something obvious.

3 replies
Cristian Iaroi
@wirdman
hey folks, is there a way to specify the order of the pipelines in the config repository (currently I believe they're created in alfabetical order)? maybe a flag or something to say reverse alfabetical/dev envs first and prd later? anything like that? Thanks.
2 replies
devops-hacks
@devops-hacks
Fellas, I'm experiencing "Check connection failed. Reason(s): ls-remote failed" error with the Git Path Material Plugin. I have tried both ssh & https and the error message persists with both connection protocols. The problem I see so far is the error in the plugin log that says: "ERROR [Thread-78] JsonUtils:127 - go.plugin-settings.get-configuration failed" but not sure what do t apart from re-installing the plugin... Any Ideas?
Eric Low 🇺🇸
@popcorneaterman_twitter
Hi all.
we are using the JSON config plugin also to configure our pipelines. We noticed that while a pipeline is running, we can change the parameters or environment variables, commit to git, and the system will pick up those changes while the pipeline is running and change the parameters and environment variables. Is this expected behavior? Is there an option to only apply parameter changes to future runs?
1 reply
marques-work
@marques-work
@devops-hacks I think I may have replied to you on the Google group — if git ls-remote fails be sure to try that on the command line with the exact url you supplied to GoCD. In the end, that’s what GoCD does — shell out to git. It’d be best to try that from the same server host/node using the same git version and user. My guess is that the URL or the credentials are wrong, and testing it manually using the command line will help you figure out which.
Fwiw the plugin settings error might be unrelated.