earthly/community

We are in the process of migrating to Slack: https://earthly.dev/slack

Marshall Culpepper

@marshall

I noticed that sending the build context is a bit slow for my build already. I tried making sure I'm not sending over a lot of files from CWD by adding things to .dockerignore, but it hasn't helped. is there an equivalent for Earthly?

8 replies

also I think the intention is for transfer progress to update inline with terminal escape sequences, but I'm just getting tons of "context | transferring .: 0%" over and over until it's done :)

Marshall Culpepper

@marshall

is there a way to tag an image built by earthly to push to an image registry that isn't docker hub?

22 replies

Vlad A. Ionescu

@vladaionescu

Hey everyone - could use some help with feedback to help guide the direction of the project! Have a questionnaire right here https://docs.google.com/forms/d/e/1FAIpQLScm89g2iHpHZxfi8JqpEKtDFB404QrBY4mPehH-F50f2za5ZA/viewform

GavinRay97

@GavinRay97

I had a question, which basically was just a mirror of an issue previously asked:

earthly/earthly#57

I find the syntax of Earthfiles really appealing, and the re-usability/composability aspect of being able to reference other files. I'm not super familiar with Docker internals, but when it comes to copying artifacts, like binaries or static assets, isn't that the same as the COPY command or using the multi-stage build in Docker?

Often times I find myself needing multiple languages in a Docker project to compile binaries, which then just get copied over as an asset, and I was wondering if there were some unique features that Earthly had compared to Docker with this.

4 replies

GavinRay97

@GavinRay97

Any clues as to what this could mean:

❯ earth build.earth+final-target
buildkitd | Found buildkit daemon as docker container (earthly-buildkitd)
Error: resolve build context for target /build.earth+final-target: Registry  not supported

1 reply

Ahh, oops. Can't pass file references like that, it takes a directory reference :flushed:

Vlad A. Ionescu

@vladaionescu

Hey everyone, Earthly v0.2.0 is out! Includes some fixes related to "out of disk space" error, renames build.earth to Earthfile, introduces a config file and many other goodies. Full details on the releases page: https://github.com/earthly/earthly/releases/tag/v0.2.0

Rey Riel

@rjriel

🥳🥳🥳

Vlad A. Ionescu

@vladaionescu

Hey everyone, looking to get some feedback on a proposal regarding support for running commands on the host: earthly/earthly#167 . Let me know what you think!

Vlad A. Ionescu

@vladaionescu

Earthly v0.3 is out! It includes a new interactive mode (-i) where it drops you in a shell if the build fails. It also adds support for Dockerfiles (FROM DOCKERFILE). Mac builds are now almost 3x faster (!!). And many other goodies! https://github.com/earthly/earthly/releases

Zen Yui

@zenyui

Hi Earthly team!

Question - what's the earth/earthly philosophy around conditional logic for specific commit refs? Do I just have conditional logic in my CI?

For example:

I'd like to have logic for custom artifact versions/tags (that combines tags, commit hashes, dates, etc)
for PR builds of docker assets I'd just like to know that it does build, but not push to my registry.
for PR builds of scala/java assets, I want to compile but not publish

I do realize that I could just have a separate target that publishes assets, like for docker assets I could invoke SAVE IMAGE and have a separate publish target where I invoke SAVE IMAGE --push ....

I'm curious how the earthly team recommends I accomplish this with earth. If possible, I'd like as little logic in my CI as possible.

23 replies

Ivan Sim

@ihcsim

How does the SAVE ARTIFACTS .. AS LOCAL work? The only bind mount I think that is relevant is the one at $HOME/.earthly/run. Is that where it happens?

4 replies

Zen Yui

@zenyui

ive got a target artifact now with a dockerfile and all the required files. is the FROM DOCKERFILE the only way to build a docker asset from that dockerfile?

18 replies

Zen Yui

@zenyui

@vladaionescu got it working :)

one interesting thing I noticed / is expected: I packaged my application as a .deb and installed it into the openjdk-8 debian image, and now I essentially have an image twice the size, as the docker layer has my .deb and the results of installation. Is there a trick in earthly to either copy and install my .deb in one line (to avoid the duplicate layers) or to squash layers?

12 replies

Zen Yui

@zenyui

I believe I found a bug with the EARTHLY_SECRETS env var: earthly/earthly#229

lmk if i'm just doing it wrong, thanks.

3 replies

Ivan Sim

@ihcsim

Multi-platform build isn't supported yet afaict, right?

2 replies

Vlad A. Ionescu

@vladaionescu

Hey everyone, Earthly v0.3.2 is out! Bash autocomplete, new syntax for docker operations (WITH DOCKER), support for generated dockerfiles and a number of bug fixes. More info here https://github.com/earthly/earthly/releases/tag/v0.3.2

Adam Bell

@agbell

Hey everyone, Earthly v0.3.5 is out! It features target aware completion for bash and zsh and a couple bug fixes. More details here https://github.com/earthly/earthly/releases/tag/v0.3.5

Zen Yui

@zenyui

any ideas why this might happen?

ERROR: (helper image for fake dep operations) failed to authorize: failed to fetch anonymous token: context canceled
Error: solve side effects: solve: failed to solve: rpc error: code = Canceled desc = failed to load cache key: failed to authorize: failed to fetch anonymous token: context canceled

14 replies

Zen Yui

@zenyui

do you guys have a doc on best practice for go mod vendor for private repos? i know i could just copy my ssh key into the image

62 replies

Adam Bell

@agbell

I've added python examples to our basic tutorial page https://docs.earthly.dev/guides/basics Let me know if there are other languages you would like to see there.

Adam Bell

@agbell

Hi All, Earthly 0.3.6 is out. We now have an --ssh flag available on the RUN command https://github.com/earthly/earthly/releases/tag/v0.3.6

1 reply

Adam Bell

@agbell

We are exploring a simplified syntax for working with docker compose. Feedback welcome! earthly/earthly#324

Esardes

@Esardes

Hi all
I use earthly to save my docker image directly to a private repo with SAVE IMAGE --push $myrepo:$mytag
What's the best way to push multiple tags?

3 replies

Vlad A. Ionescu

@vladaionescu

Hey everyone - we're showcasing Earthly on Product Hunt today. We're trying to get it to trend! Come help us out with an upvote! https://www.producthunt.com/posts/earthly-2

Vlad A. Ionescu

@vladaionescu

Earthly is on the front page of Product Hunt :tada:

1 reply

Sebastian Radloff

@sebradloff

Hi there. Is there a way to mount docker volumes from the machine running earthly, home directory? for example in the earthfile below, I want to run the drain target. The $(HOME) variable I would love to interpolate to my machine's home /Users/sebastianradloff. Is there a recommended way to do this?

FROM planetlabs/draino:450a853

build:
    USER root
    RUN apk add \
      --update \
      --no-cache \
      --repository http://dl-cdn.alpinelinux.org/alpine/edge/community \
      aws-cli
    SAVE IMAGE local-draino

drain:
    FROM docker:19.03.13-dind
    WITH DOCKER
        DOCKER LOAD +build local-draino
        RUN docker run --rm -v $(HOME)/kube:/root/.kube --entrypoint cat --rm local-draino /root/.kube/config
    END

4 replies

Sanket Dasgupta

@SanketDG

Hi I just saw that the earthly docs are hosted on gitbook, is there an interest in moving away from gitbook, considering it's future roadmap?

2 replies

Zen Yui

@zenyui

hey team, i'm finally playing with the RUN --ssh feature to download private repo go dependencies, and I'm getting an error, but confirmed that from my host machine i can run go mod vendor. I'm on earth version v0.3.8.

what I tried:

RUN --ssh git config --global url."git@github.com:".insteadOf "https://github.com/" && go mod vendor

What I get back:

+build-base | --> RUN git config --global url."git@github.com:".insteadOf "https://github.com/" && go mod vendor
+build-base | go: github.com/namely/go-common@v1.5.4: reading github.com/namely/go-common/go.mod at revision v1.5.4: unknown revision v1.5.4
+build-base | Command /bin/sh -c 'git config --global url."git@github.com:".insteadOf "https://github.com/" && go mod vendor' failed with exit code 1
+build-base | +build-base | ERROR: Command exited with non-zero code: RUN git config --global url."git@github.com:".insteadOf "https://github.com/" && go mod vendor
Repeating the output of the command that caused the failure
=========================== FAILURE ===========================

14 replies

Dieter Lunn

@coder2000

How do I run a shell script included in my repo? I keep getting a "file not found" error. I have tried copying the file too but nothing seems to work.

25 replies

Adam Bell

@agbell

Hey Everyone. We have a new release today https://github.com/earthly/earthly/releases/tag/v0.3.11 and it has a lot of great improvements

4 replies

Artie Pesh-Imam

@apeshimam

HI all, pretty new to Earthly and am working through an issue that I could use some help with. Im trying to create a github action to push our image to AWS ECR.

RUN aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin xxxxxx.dkr.ecr.us-east-1.amazonaws.com t

That's the code that Im using to try to login to ECR and then to the docker repo.

The error that I'm getting is: +build-package failed | /bin/sh: 1: docker: not found
+build-package failed | /bin/sh: 1: aws: not found

So I guess the meat of my question is how do I ensure that those are available to earthly?

19 replies

Adam Bell

@agbell

New Earthly Release! We are seeing a 4.5X improvement in build times Homebrew/homebrew-core#63996.

1 reply

Anthony BOUQUET

@abouquet

Hi guys, do you know if earthly is using .dockerignore to prevent the whole directory to be transmitted to the earth build context ?
I would like to ignore a local node_modules directory when launching my earth command from my project

I can't find any documentation about it

Damn'it, just found the information above on this gitter !
indeed .earthignore could help me :)

1 reply

Adam Bell

@agbell

Hi All, For those of you who prefer slack we have a slack workspace you all are welcome to join as well https://join.slack.com/t/earthlycommunity/shared_invite/zt-ix9rtuv8-DUFl8uxe5bFULxyCGGbqJQ

Zen Yui

@zenyui

Hi all, noticed that one of our pipelines breaks now in v0.3.14 (was working on v0.3.11). We have a target called +docker-build that accepts a SERVICE argument, and we use that to build several downstream docker images like so:

docker-build:
   ...
    ARG VERSION=dev
    ARG SERVICE
    COPY --chown namely:root +docker-stage/${SERVICE} .
    SAVE IMAGE --push registry.namely.land/namely/image-$SERVICE:$VERSION

docker-build-all:
    BUILD --build-arg SERVICE=A +docker-build
    BUILD --build-arg SERVICE=B +docker-build

and now we get the following error:

 +docker-build | ERROR: (COPY +docker-stage/${SERVICE} .) "/${SERVICE}" not found: not found

thoughts?

23 replies

Arsene Tochemey Gandote

@Tochemey

Hello I am getting this error:

Error: --allow-privileged (-P) flag is required

when using docker. This is my build step:

test-all:
    FROM +code
    COPY docker-compose.test.yml ./docker-compose.yml
    WITH DOCKER --pull postgres
        RUN docker-compose up -d && \
            for i in {1..30}; do nc -z localhost 5432 && break; sleep 1; done; \
            sbt coverage test coverageAggregate && \
            docker-compose down
    END

26 replies

Vlad A. Ionescu

@vladaionescu

Hey everyone :wave:

We've been working on a new feature in Earthly which allows you to share build :key: secrets with your team.

It's fresh out of the oven in v0.3.19 and we're super curious what you think. Come over to #cloud-secrets channel on our Slack to give us feedback. Instructions on how to use the feature are linked in that channel.

Use the same secrets (or equivalent secrets) between the CI and the individual developers
No more long lists of --secret x=y in CI to propagate CI secrets into Earthly secrets. You can now place secrets directly into Earthly's cloud secret store.
If you have many secrets in .env files that need to be passed around between developers, it might be easier to manage centrally in the cloud secret store.

All this ultimately helps make builds that much more repeatable!

1 reply

Ido David

@ido-namely

Hi, I realize that you can use FROM and a WORKDIR at the top of the file for other targets to use.
I wonder, what other options can be inherited from definition at the top of the file by other targets?
More specifically, I was hoping that ARG can be shared, similar to what is described in the dockerfile reference - https://docs.docker.com/engine/reference/builder/#understand-how-arg-and-from-interact.
accordingly, here is an example Earthfile:

ARG TST="hello world"
FROM golang:1.13-alpine3.11

test-target:
    ARG TST
    RUN echo $TST

I was hoping earth +test-target would print the var.
Thanks!

13 replies

Cosmin Ciobanu

@cosminci

Hi! Is there an idiomatic way of handling artifact versions? The result of a build usually has the version as part of the name of the artifact.

Looking at https://github.com/earthly/earthly/blob/main/examples/scala/Earthfile, the version is hardcoded, which means the earthfile would need a manual update each time the version changes in the build system's config.

Outside of extracting the version by whatever means and then passing it as an ARG to earthly, is there any other recommended way of handling this? Extracting the version before calling earth usually means bash, which is what we're trying to avoid. Extra scripting glue between the CI and the image builds.

4 replies

Vlad A. Ionescu

@vladaionescu

Hey everyone, we have a roadmap now! Come give us some feedback! https://blog.earthly.dev/the-next-iteration/

Zen Yui

@zenyui

We just discovered you can go mod download -x before copying in the golang code files to prevent duplicate golang dependency downloads during go mod vendor. In hindsight it's obvious, but worth putting somewhere if it's not already in the docs.

Cosmin Ciobanu

@cosminci

Something I'm not getting to the bottom of: When using a private repo in FROM, earth seems to hang indefinitely. i.e.

               +base | --> FROM private-repo.corp.com/team/component-base:latest
               +base | resolve private-repo.corp.com/team/component-base:latest@sha256:aeb6a8acda114643929f631b1b8dc754823d4a7dfaf3b85c12172f096d12f83a 100%

The layer is resolved (the hash is there), but it just hangs. Currently it's going on 30 mins+. If I run a docker build using a FROM private-repo.corp.com/team/component-base:latest it runs w/o a problem, taking around 90 seconds to download the base from the remote. Any idea what's happening or how I can debug it?

4 replies

Cosmin Ciobanu

@cosminci

Noticed some unexpected behavior with build args not being passed through, and I tried running earth +all for https://github.com/earthly/earthly/blob/main/examples/tests/build-arg.earth - and it does fail with +test-global1 | Command /bin/sh -c 'test "$global1" == "abc"' failed with exit code 1 - is this a known issue?

10 replies

Vlad A. Ionescu

@vladaionescu

New release https://github.com/earthly/earthly/releases/tag/v0.4.0 :tada: Note that earth has been renamed to earthly

Ido David

@ido-namely

Hi all,
I'm wondering if there is a way to use a command output string as a sort of an artifact or argument.

For example, what I'm interested in doing is finding the go version used in my module (go list -m -f '{{.GoVersion}}') and accordingly use the right go image in another target to test, build etc:

FROM golang:${GO_VERSION}-alpine
where GO_VERSION would be the output/artifact from the previous command.

Thank you!

7 replies

Vlad A. Ionescu

@vladaionescu

We're moving to Slack. Come join us over there: https://earthly.dev/slack

Where communities thrive

People

Repo info

Activity