Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Siddharth Mundada
    @student-06
    @chanseokoh yeah <container><entrypoint> is way to solve it. can we have shell entrypoint in Jib as feature addition ? It would be great if you can point to me the src code where it can be added.
    Chanseok Oh
    @chanseokoh
    @student-06 you can set whatever you want with <entrypoing> including running sh, for example
               <entrypoint>
                  <arg>/bin/sh</arg>
                  <arg>-c</arg>
                  <arg>java $SOMETHING</arg> <!-- note this contains whitespace -->
                </entrypoint>
    Of course, for this to work, the binary /bin/sh should exist in the base image. Distroless that Jib uses by default doesn't have it, so you will need to set a different image in <from><image>.
    Siddharth Mundada
    @student-06
    @chanseokoh I understand entrypoint can be used, but this way entire java command will be have to be hardcoded. Assuming the base image has /bin/sh can we have functionality in Jib itself to create a image whose entrypoint will be in shell form instead of exec form ?
    Chanseok Oh
    @chanseokoh
    @student-06 hmm... sorry, I don't get it. I don't see how the shell form is different from the exec form in the context of your use case where you just want to do env variable substitution like -Dmyarg=$myarg_env. Maybe I do not fully understand your use case.
    Brian de Alwis
    @briandealwis
    I think @student-06 is asking to be able to reference jib values like in #894
    Thorbjørn Ravn Andersen
    @m86194
    Hi. New to Jib. I managed to package our Spring Boot application quite easily which was really nice, but it appears that the registry is contacted on every build to see if a newer version is available even if it is present in the cache. I had a quick look at the source, but could not easily locate the place where the from image is made available (to see if there was a flag to set or similar). Is this possible out of the box (to always consider the cached copy usable)? If not, what Java source contains the code doing this?
    Brian de Alwis
    @briandealwis
    Hi @m86194. Basically image tags are mutable and may change between invocations (e.g., :latest), so Jib always checks the tag. You have two options: Jib respects Maven and Gradle's --offline flag. You can alternatively use the image digest (e.g., gcr.io/distroless/java@sha256:xxxx.
    Chanseok Oh
    @chanseokoh

    @m86194 note that although Jib will always fetch the image manifest to see if the tag has mutated, but it will use the cached layers if there were no changes, so most of the time, it won't have any noticeable performance hit. But as @briandealwis, you highly suggest you use a digest instead of a tag to maximize reproducibility for production usage.

    BTW, --offline works only with jib:dockerBuild and jib:buildTar but not with jib:build (that builds and pushes to a registry).

    Thorbjørn Ravn Andersen
    @m86194
    @chanseokoh Thank you for coming back to me quickly :) I had a closer look at the "--offline" code handling, and if I use that I cannot push to the remote registry at the end (which is hardcoded, but might be made configurable?). I appreciate the reproducability approach, but locking a digest inside a pom.xml file (or the command invoking it) is not immediately desirable. We still want the "latest". We just don't want to check on each and every build. I think actually that for our purposes we might prepopulate the cache at pod startup so JIB should never consider updating at all.
    Brian de Alwis
    @briandealwis
    @m86194 We've discussed caching the resolved tag result, but it's an unreliable mechanism: it's entirely possible for the that cached result to expire mid-build. The only reliable solution to avoid repeated tag resolves is to resolve the tag at the start and feed those SHA into your build.
    Chanseok Oh
    @chanseokoh

    actually that for our purposes we might prepopulate the cache at pod startup so JIB should never consider updating at all.

    @m86194 if your main motivation is this rather than to avoid making extra network calls to the source repository, I may come up with some workarounds. So, I think I still did not understand your goal clearly. Is your concern the extra network calls or the way control the up-to-date-check the way you desire?

    Thorbjørn Ravn Andersen
    @m86194
    I do not understand why you do not consider "Internet connection to remote repository required for every build" a rather strict requirement? (Unless I've misunderstood something). We are fine with communicating with the target repository which is our internal openshift registry for now, but I'd really, really like to be independent of the rest of the world when building. I do not care in particular if an artifact hosted on dockerhub is used instantly after publishing (unless a severe zero-day exploit needs patching but that is hopefully a rare scenario) but we really care if a broken internet connection has an influence on our builds on our internal build servers. I am aware that this mean we need to proxy docker images for production usage (which also lets us control when "latest" is actuallly updated from upstream) but I would like to keep this to require as little supporting infrastructure as we can.
    I also like that Jib provides a platform independent implementation of both docker image building and docker repository communication which is relevant to us as a multi-platform Java shop, which would like to avoid actually using docker if possible.
    Chanseok Oh
    @chanseokoh
    I do think the Internet connection requirement is a strict requirement. But in your case, if the isolation and controlled image update is important, I'd put the base image into your internal openshift registry. I think that'd give you more control over which base image to pin down and when to update.
    Siddharth Mundada
    @student-06
    @chanseokoh It can simplify the deployment at later stage. For eg, -Darg=$myarg at run time, only populating myarg environment variable will make the image run. One use case is inject bootstrap props using environment variables, this way same image can be used in prod/test. I am afraid this is going toward over-engineering.
    Chanseok Oh
    @chanseokoh
    @student-06 yeah, then you can use either the exec form or the shell form to do -Darg=$myarg that will populate the environment variable at runtime.
    Siddharth Mundada
    @student-06
    @chanseokoh In the exec form it won't happen right ? Because a environment variable is a shell variable and only in shell context those variables will be substituted before running java command . Please correct me if I am wrong here
    Chanseok Oh
    @chanseokoh
    That is why you will execute a shell with -c (check the example I showed you a while a ago). Basically, you do /bin/sh -c <any commands you want with -Darg=...>, and this is basically equivalent to running in the shell form. Theoretically you could even put a whole commands of a long shell script.
    tunaozkasap
    @tunaozkasap

    That is why you will execute a shell with -c (check the example I showed you a while a ago). Basically, you do /bin/sh -c <any commands you want with -Darg=...>, and this is basically equivalent to running in the shell form. Theoretically you could even put a whole commands of a long shell script.

    @chanseokoh Are you saying the environment variables that are used in maven arg tags will be picked up as expected when you DO NOT override the JIB's entrypoint?

    Chanseok Oh
    @chanseokoh
    @tunaozkasap no, Jib's "default" entrypoint is fixed. I am talking about when you go for customizing it using the <entrypoint> config (and running a shell binary in this particular case).
    tunaozkasap
    @tunaozkasap

    @chanseokoh I see, thanks a lot for the info.

    I created the following example to make the problem clear for everyone.

    This is the java class that prints out system properties and environment variables that start with DOCKER string

    package com.exact.jib.jibtest;

    public class PrintEnvironmentVariables {
        public static void main(String[] args) {
            System.out.println("======== SYSTEM PROPERTIES ==========");
            System.getProperties().forEach((key, value) -> {
                if(((String)key).startsWith("DOCKER")) {                
                    System.out.printf("%1$s = %2$s\n", key, value);
                }
            });
            System.out.println("======== ENVIRONMENT VARIABLES ==========");
            System.getenv().forEach((key, value) -> {
                if(key.startsWith("DOCKER")) {                
                    System.out.printf("%1$s = %2$s\n", key, value);
                }
            });
        }
    }

    This is the maven file I have

    <project xmlns="http://maven.apache.org/POM/4.0.0"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
        <modelVersion>4.0.0</modelVersion>
        <groupId>com.exact.jib</groupId>
        <artifactId>jibtest</artifactId>
        <version>0.0.1-SNAPSHOT</version>
    
        <properties>
            <maven.compiler.target>1.8</maven.compiler.target>
            <maven.compiler.source>1.8</maven.compiler.source>
        </properties>
    
        <build>
            <plugins>
                <plugin>
                    <groupId>com.google.cloud.tools</groupId>
                    <artifactId>jib-maven-plugin</artifactId>
                    <version>1.4.0</version>
                    <configuration>
                        <to>
                            <image>tunaozkasap/tunajibtest</image>
                            <auth>
                                <username>{tunaozkasap_dockerio_username}</username>
                                <password>{tunaozkasap_dockerio_password}</password>
                            </auth>
                        </to>
                        <container>
                            <jvmFlags>
                                <jvmFlag>-DDOCKER_FLAG1=$DOCKER_FLAG1</jvmFlag>
                                <jvmFlag>-DDOCKER_FLAG2=$DOCKER_FLAG2</jvmFlag>
                            </jvmFlags>
                            <mainClass>com.exact.jib.jibtest.PrintEnvironmentVariables</mainClass>
                        </container>
                    </configuration>
                    <executions>
                        <execution>
                          <phase>package</phase>
                          <goals>
                            <goal>build</goal>
                          </goals>
                        </execution>
                    </executions>
                </plugin>
            </plugins>
        </build>
    
    </project>

    And finally this is the docker run command I am using and the output is also there

    docker run -e DOCKER_FLAG1=flag1 -e DOCKER_FLAG2=flag2 -e DOCKER_VAR1=var1 -e DOCKER_VAR2=var2 tunaozkasap/tunajibtest
    
    ======== SYSTEM PROPERTIES ==========
    DOCKER_FLAG1 = $DOCKER_FLAG1
    DOCKER_FLAG2 = $DOCKER_FLAG2
    ======== ENVIRONMENT VARIABLES ==========
    DOCKER_VAR1 = var1
    DOCKER_VAR2 = var2
    DOCKER_FLAG1 = flag1
    DOCKER_FLAG2 = flag2

    As you can see the environment variables are not substituted because the entry point is exec command instead of a shell.

    Chanseok Oh
    @chanseokoh
    In the cases like yours, one possible option is to use JAVA_TOOL_OPTIONS: https://github.com/GoogleContainerTools/jib/blob/master/docs/faq.md#how-do-i-set-parameters-for-my-image-at-runtime I believe it works for most JVMs.
    Siddharth Mundada
    @student-06

    That is why you will execute a shell with -c (check the example I showed you a while a ago). Basically, you do /bin/sh -c <any commands you want with -Darg=...>, and this is basically equivalent to running in the shell form. Theoretically you could even put a whole commands of a long shell script.

    @chanseokoh Agree compeletely, just that it would be nice if jib can provide this feature out of box using a flag. But anyways we can solve it without this feature too. Thanks.

    Fabio Lorenzi
    @florenzi002
    ehi guys... any clue why is this happening ? subprojects { apply plugin: 'com.google.cloud.tools.jib'}with result Plugin with id 'com.google.cloud.tools.jib' not found.
    Chanseok Oh
    @chanseokoh
    Do you have the plugins block at the top?
    plugins {
      ...
      id 'com.google.cloud.tools.jib' version '1.4.0' apply false
    }
    Fabio Lorenzi
    @florenzi002
    I do... but it seems this is actually an issue with my gradle script and not a jib one... sorry I posted on the wrong channel
    mik-sher
    @mik-sher
    Hi there,
    Does anybody know how to use jib to create the container with tests? I can’t find compiled test sources in app/classes/
    Chanseok Oh
    @chanseokoh
    @mik-sher I cannot understand why you want test classes at all. Generally speaking, it's not possible. But since the test classes (and test resource files if present) are compiled into target/test-classes, it is not impossible to copy them into the image if you use <extraDirectories> with some post-processing. For example, if you copy or move target/test-classes/* to target/jib-extra-dir/app/classes/* and set <extraDirectories> to target/jib-extra-dir, those test class files will be copied into /app/classes in the image. But I don't really get why you ever want to do that. Perhaps you shouldn't copy test classes after all.
    Brian de Alwis
    @briandealwis
    @mik-sher you might want to look at container-structure-test for a container testing framework. It can check file layouts within the container as well as run and checking outputs.
    pavelbelkevich
    @pavelbelkevich
    Hello! May you help me about adding self-signed certificate following doc. I’v generated jks,and made docker volume with it. But I did not manage to set trust store and password with maven.
    mvn -Djavax.net.ssl.trustStore=path/to/keystore.jks \ -Djavax.net.ssl.trustStorePassword=password \ jib:build
    Properties are empty.Only by setting path to trust store and password by
    System.setProperty( "javax.net.ssl.trustStore" , “path/to/keystore.jks" ); System.setProperty( "javax.net.ssl.trustStorePassword" , “password" );
    worked. That am I doing wrong?
    Thank you
    Brian de Alwis
    @briandealwis
    @pavelbelkevich is this for your packaged application? Setting those values on the mvn command line will affect Maven plugins, not your application.
    You'd need to add to your <jvmFlags>
    pavelbelkevich
    @pavelbelkevich
    yes
    Brian de Alwis
    @briandealwis
    pavelbelkevich
    @pavelbelkevich
    Thank you!I’ll try so
    mik-sher
    @mik-sher
    @chanseokoh @briandealwis Thanks a lot for your answers! I want to create the container with test for integration testing my platform)
    This is necessary for the purpose of running tests in the kubernetes, and parallelizing the launch at an even higher level
    Thorbjørn Ravn Andersen
    @m86194

    @chanseokoh

    I do think the Internet connection requirement is a strict requirement. But in your case, if the isolation and controlled image update is important, I'd put the base image into your internal openshift registry. I think that'd give you more control over which base image to pin down and when to update.

    I agree, we will probably have to do this. During my experiments with an external image, however, I found that our firewall is rather flaky and occasionally denies access to the external registry. This causes the build to fail rather hard. This might not be the desired behavior.

    Chanseok Oh
    @chanseokoh
    @m86194 we will make Jib skip checking up-to-date for external images designated with SHA (if cached, of course).
    kavenr
    @kavenr
    Hi! I'm trying to get jib to work on a gitlab server without access to registry-1.docker.io (corporate firewall...) Is there a way to know which image jib is trying to download so I can provide it?
    Chanseok Oh
    @chanseokoh
    By default Jib uses gcr.io/distroless/java:latest (equivalent of gcr.io/distroless/java:8) for Java 8 apps and gcr.io/distroless/java:11 for Java 11. You can configure your base image too.
    Chanseok Oh
    @chanseokoh

    Hi @/all

    Jib 1.5.0 is released and live with the following major changes:

    • Can now set file timestamps (last modified time) in the image with container.filesModificationTime. The value should either be EPOCH_PLUS_SECOND to set the timestamps to Epoch + 1 second (default behavior), or an ISO 8601 date time parsable with DateTimeFormatter.ISO_DATE_TIME such as 2019-07-15T10:15:30+09:00 or 2011-12-03T22:42:05Z
    • Can now set container creation timestamp with container.creationTime. The value should be EPOCH, USE_CURRENT_TIMESTAMP, or an ISO 8601 date time
    • For Google Container Registry (gcr.io), Jib now tries Google Application Default Credentials (ADC) last when no credentials can be retrieved. ADC are available on many Google Cloud Platform (GCP) environments (such as Google Cloud Build, Google Compute Engine, Google Kubernetes Engine, and Google App Engine). Application Default Credentials can also be configured with gcloud auth application-default login locally or through the GOOGLE_APPLICATION_CREDENTIALS environment variable.
    • When building to a registry, Jib now skips downloading and caching base image layers that already exist in the target registry. This feature will be particularly useful in CI/CD environments. However, if you want to force caching base image layers locally, set the system property -Djib.alwaysCacheBaseImage=true
    • container.useCurrentTimestamp has been deprecated in favor of container.creationTime with USE_CURRENT_TIMESTAMP
    kavenr
    @kavenr
    @chanseokoh I there a way to tell jib to go through the gitlab registry to look for these images? I'm not sure what jib is trying to do when I get this error because there are no details: com.google.cloud.tools.jib.plugins.common.BuildStepsExecutionException: registry-1.docker.io: Name or service not known
    Chanseok Oh
    @chanseokoh
    @kavenr you can configure from.image to point it to your gitlab registry/repository (your image to use as a base image). And for the "Name or service not known", I think it is that your internal network cannot resolve the host name (most likely intentionally). Doing curl registry-1.docker.iowill fail, as much as like curl non-existing.host.name.
    It could also be that your internal network actually allows connecting to registry-1.docker.io but requires you to configure and go through a proxy. I don't know.
    kavenr
    @kavenr
    Correct, it's blocked. I would like to point jib to the gitlab registry which acts as a proxy to the public repos. I'll try with from.image. thanks!
    Chanseok Oh
    @chanseokoh
    No problem!
    kavenr
    @kavenr
    I added --debug to the command line and got Caused by: com.google.cloud.tools.jib.api.InsecureRegistryException: Failed to verify the server at https://registry.gitlab.[...]/v2/integration/jhipster-gradle-docker/manifests/latest because only secure connections are allowed. I added allowInsecureRegistries in addition to specifying the image and the build succeeded. Thank you!
    Attoumane
    @akuma8
    Hi jibbers,
    I would like to know how does jib get project dependencies and put them into /app/libs. My project has a common.jar dependency that has a file src/main/resources/application.properties, I would like to remove that file before jib packaging the docker image.
    The image is built during the package maven's phase, so if I know where jib looks for dependencies I can modify that particular dependency in a previous phase. I would like to modify only the dependency embedded in the docker image. Thanks for your answers
    Chanseok Oh
    @chanseokoh
    @akuma8 if the application.properties is embedded in common.jar(double-check if this is true by unzipping it with unzip common.jar), then there's already not much Jib can do about it, if common.jar is a dependency of the Jib project hence pulled in as-is. That is, I think the packaging of common.jar to include the property file must be happening even before Jib is kicked in, and Jib just takes the JAR as-is. (Maven tells Jib where the JAR comes from.) If you really need to take the file out of common.jar, and I'd say the right way to go is to modify the very project/module that packages the JAR, instead of trying to somehow post-process it to remove the file in a later stage. But I feel it's a bit odd that you need to remove some file packaged in a dependency JAR, because such a JAR should be self-contained and if an embedded file in the JAR is affecting the outside world, I think usually there should be a way to control/override things outside without manipulating the JAR itself (unless the JAR code does things in a weird way, in which case I think it is not a good design)?