by

Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • 16:59
    peterebden opened #1122
  • 16:59
    peterebden review_requested #1122
  • 16:58
    Tatskaari labeled #1108
  • 16:56
    Tatskaari review_requested #1121
  • 16:56
    Tatskaari opened #1121
  • 16:28
    Tatskaari labeled #1074
  • 16:13
    Tatskaari labeled #1075
  • 16:12

    Tatskaari on master

    Make plz update respect plain o… (compare)

  • 16:12
    Tatskaari closed #1120
  • 16:12
    Tatskaari closed #873
  • 15:48
    Tatskaari unlabeled #876
  • 15:48
    Tatskaari labeled #876
  • 15:48
    Tatskaari labeled #876
  • 15:45
    Tatskaari closed #875
  • 15:45
    Tatskaari commented #875
  • 15:42
    Tatskaari synchronize #1120
  • 15:40
    Tatskaari closed #680
  • 15:40
    Tatskaari commented #680
  • 15:36
    Tatskaari closed #646
  • 15:36
    Tatskaari commented #646
Jonathan Poole
@Tatskaari
I think the only requirement is that you have a bunch of object files and it will link them together into a main.go that does basically what go test would do
Márk Sági-Kazár
@sagikazarmark
The problem is the go modules stuff which I still haven't sorted out to be perfect
So I'm guessing the go_test rule won't be able to compile
I want to use it eventually, but without the proper module support, go_library and go_test won't work either. Or will they?
Jonathan Poole
@Tatskaari
It should be able to. Your module rules still spit out .a files no?
Márk Sági-Kazár
@sagikazarmark
I mean, I don't use those rules yet, because they don't really work yet
Jonathan Poole
@Tatskaari
I guess they do but you might have to get them into the right places
Márk Sági-Kazár
@sagikazarmark
so, I want to use those module rules eventually, but we are not there yet
but I do want to utilize the incremental nature of please if possible in some way in the meantime
Márk Sági-Kazár
@sagikazarmark

I'm trying to build a helm lint test rule in my pleasings repo. It looks like the following so far, but I can't really get it work:

CONFIG.setdefault("HELM_TOOL", "//tools/k8s:helm")

def helm_lint(name:str, visibility:list=None):
    """Lints a Helm chart.

    Args:
      name (str): Name of the rule.
      visibility (list): Visibility of the rule.
    """
    return gentest(
        name = name,
        srcs = glob(["**"], hidden = True),
        test_cmd = f'"$(location {CONFIG.HELM_TOOL})" lint "$PKG_DIR" 2>&1 > $RESULTS_FILE',
        data = glob(["**"], hidden = True) + [CONFIG.HELM_TOOL],
        #tools = {"helm": [CONFIG.HELM_TOOL]},
        visibility = visibility,
        no_test_output = True,
    )

I'm currently trying to test it inside the pleasings repo, but obviously I want to reuse it.

So the first question that comes into mind: will the helm tool config work this way? It's actually a target in the same repo, installing helm. Will it work, if I use the pleasings repo as a subrepo somewhere else?

Secondly, I guess I don't really get how tests work yet. So I want to use the downloaded helm tool for linting the helm chart, which consists of every file in the package. Looks like tools doesn't work that way, so I need to add CONFIG.HELM_TOOL as a dependency? Also as a data, so that it's in the test directory?

And the third question: I needed to add the files as both src and data. Is this how it supposed to work?

(Basically what I do here is run the helm lint tool for the chart directory).

Apologies if I'm asking dumb questions, but the current examples seem to work a bit differently than what I'm trying to do here.

Márk Sági-Kazár
@sagikazarmark
Is it possible to exclude certain labels by default and only build/test them when included explicitly?
Integration tests come to mind where this could be handy
Florin STAN
@namtzigla
almost every command has an --exclude option and in https://please.build/config.html look for blacklistdirs
Márk Sági-Kazár
@sagikazarmark
Yeah, I know, but I don't want to manually exclude.
Márk Sági-Kazár
@sagikazarmark
Is it a good practice to leave the rule name empty and fall back to package_name()? In case of a helm chart the only sane thing to build is a helm package.
Márk Sági-Kazár
@sagikazarmark
Like this: name = name or basename(package_name())
Márk Sági-Kazár
@sagikazarmark
Here is the full example:
def helm_package(name:str=None, visibility:list=None):
    """Packages a Helm chart into a versioned chart archive.

    Args:
      name (str): Name of the rule.
      visibility (list): Visibility of the rule.
    """

    return genrule(
        name = name or basename(package_name()),
        srcs = glob(["**"], hidden = True),
        cmd = [
            "mkdir helm-out",
            '"$TOOLS_HELM" package --destination helm-out "$PKG_DIR"',
        ],
        output_dirs = ["helm-out"],
        tools = {"helm": [CONFIG.HELM_TOOL]},
        visibility = visibility,
    )

def helm_lint(name:str=None, visibility:list=None):
    """Lints a Helm chart.

    Args:
      name (str): Name of the rule.
      visibility (list): Visibility of the rule.
    """

    return gentest(
        name = name or "_%s#lint" % (basename(package_name())),
        test_cmd = f'"$(location {CONFIG.HELM_TOOL})" lint "$PKG_DIR" 2>&1',
        data = glob(["**"], hidden = True) + [CONFIG.HELM_TOOL],
        visibility = visibility,
        no_test_output = True,
    )
Peter Ebden
@peterebden
I'd consider it a little unusual to default the name, but might make sense in this case where there's nothing else to do.
We've talked about doing a similar thing for Go to tie up go_library and go_test, and defaulting the name there would be a good idea since it confuses things if they don't match
Márk Sági-Kazár
@sagikazarmark
Hm, how would that look like for go_test? I like the fact that I can have multiple go_test targets in a single package, separating unit, integration and e2e tests
1 reply
Would be nice to have for go_library though
Danetta
@Orlha
image.png
For some reason when I run please build I get this warning from Perl with "en_GB" locale, which I do not use, so it is not genereated anywhere (I use en_US)
Quick test with perl itself
image.png
image.png
If it possible that please somehow defaults to en_GB somewhere even when I don't use it? Or is it a problem on my side?
Jonathan Poole
@Tatskaari
@Orlha Do we have perl rules? I'm not sure how they work but there's a config option for locals that can be set in the .plzconfig: https://please.build/config.html
Maybe they're picking up that
Márk Sági-Kazár
@sagikazarmark

I'm trying to integrate please into a GitHub CI flow with caching.

I have two separate jobs that run sequentially:

  • Build runs plz build and caches the compiled test binaries
  • Test runs the tests

Build saves .plz-cache to the cache, Test restores it and uses it to run the tests. I would like to utilize caching for test executions themselves though. What should I cache (from .plz-cache or plz-out I presume) so that please only runs the tests that changed?

Jonathan Poole
@Tatskaari
What CI platform are you using? I think circleci has a http cache that please is compatible with.
Why do you have 2 stages for build and test? If you just did plz test/cover, the test results would just be cached as you expect.
Márk Sági-Kazár
@sagikazarmark
I use GitHub actions. I actually have more than two stages (docker build, integration tests, etc). The build runs first, then the rest runs in parallel.
Márk Sági-Kazár
@sagikazarmark
Ideally, I would load the cache from the build stage, then I would load the test results from another cache bucket. Is this possible? I can see a test_results.xml in the log directory. Is that it?
6 replies
Márk Sági-Kazár
@sagikazarmark
Also, I thought PLZ_ARGS was recognized by either the pleasew script or plz itself. It's really convenient to define args globally (like PLZ_ARGS="-p --profile ci") and then you can just use the pleasew script without having to duplicate those arguments in every command. I actually copied that env var from the circle config of please, but I realized it was from the bootstrap script.
Márk Sági-Kazár
@sagikazarmark

Here is the PR BTW: banzaicloud/pipeline#3080

I moved onto the next step of building test binaries upfront and then executing them in the following steps.

Márk Sági-Kazár
@sagikazarmark
Is it a bad practice to add a "global" dependency to a rule based on configuration? I'd like to add go.mod files as dependencies to all go_* rules, if it's configured in CONFIG.GOMOD. This way I can avoid generating rules loading go.mod files statically.
Jonathan Poole
@Tatskaari
It's not ideal. I've done something similar in the javascript world where all the rules are dependant on the yarn.lock file.
Márk Sági-Kazár
@sagikazarmark
Well, I've played with please a bit more over the weekend and my efforts actually made the build times worse. :( Right now I just change the directory to the project root and run go build there. I tried moving the build process to the temporary directories, but that made things infinitely worse. I replaced go_library with filegroups and it just exploded the build time. Not sure if it's because I'm doing the caching wrong or something else.
But that was kind of sad anyway. Looks like my efforts to introduce Please incrementally to the project reached the end.
I'm digging into the go compiler now to see what we need for better modules support.
Márk Sági-Kazár
@sagikazarmark
I know I asked this before, but I still don't understand I guess: why isn't there a go_lint test rule for linting code in packages? The current approach requires the traditional go mod cache to be present.
Jonathan Poole
@Tatskaari
Umm I guess we've always invoked the linter from the CI side and have a differentiation between a lint failure and a build failure.
And by the way: thought-machine/please#1106
Márk Sági-Kazár
@sagikazarmark
Oh nice, thanks
cphang99
@cphang99
Hi! I'm looking to test please against some of the remote execution servers that use the REAPI. I'm trying to run the example projects in https://github.com/thought-machine/please-examples but these are currently failing with the go example ./pleasew test -i go -o proto.language:go . It looks like it's trying to construct the python grpc bindings. Are there any projects that use Please that I could have a look at. The smaller the better!
Jonathan Poole
@Tatskaari
@cphang99 I'm not sure about small projects but the core please project has fewer toolchain requirements as it doesn't generate any proto files.
I think in principle please works with REAPI however some of the build rules won't work exactly as they do locally e.g. the binary flag that sets outputs of a rule as executable doesn't get respected.