Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Attoumane
    @akuma8
    Using @Value( "gs://mybucket/file" ) is a bit restrictive when we have to deal with many files. That's why I set it to the bucket but I think I should review that.
    Artem Bilan
    @artembilan
    Well, then you really need to go to com.google.cloud.storage API and do all your logic with those low-level objects
    Doesn't look like static @Value is going to help you somehow...
    This is already not a Spring responsibility to override GCP API almost for nothing
    Although there is indeed a Spring Integration solution with the GcsMessageHandler which could help you to orchestrate your logic as flows and perform requested operations
    See more about Spring Integration: https://spring.io/projects/spring-integration
    Attoumane
    @akuma8
    Thank you, I'll look over there.
    Mike Eltsufin
    @meltsufin
    @akuma8 Once you get the Bucket from getGoogleStorageResource().getBucket(), you can do whatever you want using the Storage APIs. See: https://github.com/googleapis/google-cloud-java/tree/master/google-cloud-clients/google-cloud-storage
    Attoumane
    @akuma8
    Thanks @meltsufin I should have started by reading the doc before I started ^^
    Gautam Dey
    @gautamdey
    thanks everybody the issue was got resolved adding the following env:
    JBP_CONFIG_SPRING_AUTO_RECONFIGURATION: '{enabled: false}' to manifest.yml
    Mike Eltsufin
    @meltsufin
    @gautamdey Thanks for the update!
    Daniel Zou
    @dzou
    Hey @ryanjbaxter, a while back, we wanted to update our spring.io homepage (https://spring.io/projects/spring-cloud-gcp) with our updated sagan-index.adoc in our repo. Are our updates going to get picked up with the M3 release? Or did we miss something we were supposed to do?
    Spencer Gibb
    @spencergibb
    Ryan is on PTO. I don't think Milestones update the main page, IIRC.
    Daniel Zou
    @dzou
    Ah ok, thanks for clarifying !
    Nuno Marujo
    @nhmarujo
    Hey guys. I'm using spring-cloud-gcp-starter-storage at version 1.1.3.RELEASE and I can see that it has a lot of dependency convergence issues.
    I was able to overcome them by specifying a lot of the dependencies on the dependencyManagement section of Maven, but I was wondering if you are aware of the issue and have a cleaner solution
    (I mean there are a lot of dependencies with convergence issues)
    Elena Felder
    @elefeint
    spring-cloud-gcp 1.1.3 used google cloud BOM version 0.96.0-alpha. So most likely what you are experiencing is related to using newer versions of other dependencies in your project.
    Nuno Marujo
    @nhmarujo
    Thank you so much for your answer @elefeint , but I don’t think that is the issue. All the convergency conflicts happen between the dependencies included by spring-cloud-gcp-starter-storage , as you can see from the dependency report below:
    This message was deleted
    Dependency convergence error for com.google.protobuf:protobuf-java:3.7.1 paths to dependency are:
    +-com.nhmarujo:asset-manager:0.0.1-SNAPSHOT
      +-org.springframework.cloud:spring-cloud-gcp-starter-storage:1.1.3.RELEASE
        +-org.springframework.cloud:spring-cloud-gcp-starter:1.1.3.RELEASE
          +-org.springframework.cloud:spring-cloud-gcp-core:1.1.3.RELEASE
            +-com.google.cloud:google-cloud-core:1.78.0
              +-com.google.protobuf:protobuf-java-util:3.7.1
                +-com.google.protobuf:protobuf-java:3.7.1
    and
    +-com.nhmarujo:asset-manager:0.0.1-SNAPSHOT
      +-org.springframework.cloud:spring-cloud-gcp-starter-storage:1.1.3.RELEASE
        +-org.springframework.cloud:spring-cloud-gcp-starter:1.1.3.RELEASE
          +-org.springframework.cloud:spring-cloud-gcp-core:1.1.3.RELEASE
            +-com.google.cloud:google-cloud-core:1.78.0
              +-com.google.api.grpc:proto-google-common-protos:1.16.0
                +-com.google.protobuf:protobuf-java:3.7.1
    and
    +-com.nhmarujo:asset-manager:0.0.1-SNAPSHOT
      +-org.springframework.cloud:spring-cloud-gcp-starter-storage:1.1.3.RELEASE
        +-org.springframework.cloud:spring-cloud-gcp-starter:1.1.3.RELEASE
          +-org.springframework.cloud:spring-cloud-gcp-core:1.1.3.RELEASE
            +-com.google.cloud:google-cloud-core:1.78.0
              +-com.google.api.grpc:proto-google-iam-v1:0.12.0
                +-com.google.protobuf:protobuf-java:3.5.1
    
    [WARNING] 
    Dependency convergence error for com.google.guava:guava:27.1-android paths to dependency are:
    +-com.nhmarujo:asset-manager:0.0.1-SNAPSHOT
      +-org.springframework.cloud:spring-cloud-gcp-starter-storage:1.1.3.RELEASE
        +-org.springframework.cloud:spring-cloud-gcp-starter:1.1.3.RELEASE
          +-org.springframework.cloud:spring-cloud-gcp-core:1.1.3.RELEASE
            +-com.google.cloud:google-cloud-core:1.78.0
              +-com.google.guava:guava:27.1-android
    and
    +-com.nhmarujo:asset-manager:0.0.1-SNAPSHOT
      +-org.springframework.cloud:spring-cloud-gcp-starter-storage:1.1.3.RELEASE
        +-org.springframework.cloud:spring-cloud-gcp-starter:1.1.3.RELEASE
          +-org.springframework.cloud:spring-cloud-gcp-core:1.1.3.RELEASE
            +-com.google.cloud:google-cloud-core:1.78.0
              +-com.google.http-client:google-http-client:1.30.1
                +-com.google.guava:guava:26.0-android
    and
    +-com.nhmarujo:asset-manager:0.0.1-SNAPSHOT
      +-org.springframework.cloud:spring-cloud-gcp-starter-storage:1.1.3.RELEASE
        +-org.springframework.cloud:spring-cloud-gcp-starter:1.1.3.RELEASE
          +-org.springframework.cloud:spring-cloud-gcp-core:1.1.3.RELEASE
            +-com.google.cloud:google-cloud-core:1.78.0
              +-com.google.api:api-common:1.8.1
                +-com.google.guava:guava:26.0-android
    and
    +-com.nhmarujo:asset-manager:0.0.1-SNAPSHOT
      +-org.springframework.cloud:spring-cloud-gcp-starter-storage:1.1.3.RELEASE
        +-org.springframework.cloud:spring-cloud-gcp-starter:1.1.3.RELEASE
          +-org.springframework.cloud:spring-cloud-gcp-core:1.1.3.RELEASE
            +-com.google.cloud:google-cloud-core:1.78.0
              +-com.google.api:gax:1.45.0
                +-com.google.guava:guava:26.0-android
    (this is just a part, it is quite big)
    Nuno Marujo
    @nhmarujo
    picking up the very first example, all the conflicts for protobuf-java are happening inside spring-cloud-gcp-starter-storage
    see what I mean @elefeint ?
    Elena Felder
    @elefeint
    I do! We are at Spring One right now, but I will look into the dependencies in the evening. Thanks for letting us know!
    Nuno Marujo
    @nhmarujo
    You’re welcome! Thank you for taking a look at it @elefeint . That was just an example, but I saw conflicts in the following dependencies:
    • com.google.api:api-common
    • com.google.api.grpc:proto-google-common-protos
    • com.google.api-client:google-api-client
    • com.google.auth:google-auth-library-oauth2-http
    • com.google.auth:google-auth-library-credentials
    • com.google.http-client:google-http-client
    • com.google.protobuf:protobuf-java
    • io.opencensus:opencensus-api
    • io.opencensus:opencensus-contrib-http-util
    Enjoy the conference! :)
    Mike Eltsufin
    @meltsufin
    It looks like the conflicts are happening under the google-cloud-core:1.78.0 dependency, which we don't really control. You see that we're using the same version of the Google Client libraries, but they have their own dependency conflicts.
    It also looks like we don't have the latest version of Google Cloud Java in 1.1.3.
    Nuno Marujo
    @nhmarujo
    I see what you mean @meltsufin . You are right, those conflicts are on googleapis side
    Do you think that bumping to a newer version of google-cloud-java might fix it?
    Elena Felder
    @elefeint
    @meltsufin But should we file a diamond dependency issue in google cloud java?
    Mike Eltsufin
    @meltsufin
    @elefeint I think we should first see if upgrading to latest google cloud java libs umbrella version resolves the issue.
    Nuno Marujo
    @nhmarujo
    thank you both @meltsufin and @elefeint
    The issue you mentioned doesn’t have the full issue report (gitter code snippet limitations), but I’ll update spring-cloud/spring-cloud-gcp#1975 with it
    Mike Eltsufin
    @meltsufin
    Thanks!
    Nuno Marujo
    @nhmarujo
    You’re welcome! :)
    Nuno Marujo
    @nhmarujo
    Hey everyone. I’m using Spring Webflux and I’m receiving a Flux<FilePart> on a REST endpoint. I would like to stream that to a bucket on Google Storage. Unfortunately I’m failing to find out how. Anyone can point me in the right direction, please?
    Mike Eltsufin
    @meltsufin
    How would you write it to a plain old file?
    Nuno Marujo
    @nhmarujo
    well, for a file on the local filesystem, it has a transferTo(Path path) method
    but none that takes an Outputstream
    Mike Eltsufin
    @meltsufin
    I guess FilePart is designed just to write to local filesystem. You'd have to send the file to GCS separately after it has been written locally. Otherwise, this is really more of a question for Spring Framework Core. https://github.com/spring-projects/spring-framework/issues
    Nuno Marujo
    @nhmarujo
    I would like to just pipe each chunk directly to gcp and avoid a lot of memory consumption
    Will check. Thanks
    stevenweili
    @stevenweili
    I just reported an issue of 1.2.0.M3, project id not found while run in PCF, it works after I downgrade to 1.2.0.M2.
    spring-cloud/spring-cloud-gcp#1994
    @meltsufin can you help check?
    Mike Eltsufin
    @meltsufin
    yeah, I'll respond on the issue
    Looks like SettableApiFuture comes without a message in the exception we have just sent
    Mike Eltsufin
    @meltsufin
    It all comes down to what we get from the Pub/Sub client library.