by

Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Nick
    @gurinderu
    solved
    Dylan Martin
    @dmarticus
    glad you solved it! What was the resolution (just for posterity's sake if anyone ever has that issue again and looks here)?
    Nick
    @gurinderu
    muSrcGenJarNames ++= Seq("protobuf-java")
    Dylan Martin
    @dmarticus
    @gurinderu thanks for sharing!
    Nick
    @gurinderu
    it is necessary if you want types from any jars, in my case every "google/protobuf/*" files are located in protobuf-java
    Nik Gushchin
    @Dr-Nikson
    @gurinderu Hello mate! I’m having some troubles with compiling google related stuff — muSrcGen simply can’t find it
    Nik Gushchin
    @Dr-Nikson
    NWM, just fixed it
    Nik Gushchin
    @Dr-Nikson
    So, it fails when trying to compile https://github.com/googleapis/api-common-protos/blob/master/google/logging/type/http_request.proto
    http_request.proto:20:1: warning: Import google/api/annotations.proto is unused.
    Or even:
    http_request.proto:20:1: warning: Import google/api/annotations.proto is unused.
    [error] stack trace is suppressed; run last protocol / muSrcGen for the full output
    [error] (protocol / muSrcGen) higherkindness.mu.rpc.srcgen.proto.ProtoSrcGenerator$ProtobufSrcGenException
    [error] Total time: 5 s, completed Jun 5, 2020 8:48:21 PM
    [IJ]sbt:sbt-mu-grpc> last protocol / muSrcGen
    [error] higherkindness.mu.rpc.srcgen.proto.ProtoSrcGenerator$ProtobufSrcGenException
    [error] (protocol / muSrcGen) higherkindness.mu.rpc.srcgen.proto.ProtoSrcGenerator$ProtobufSrcGenException
    Nik Gushchin
    @Dr-Nikson
    And there is no descripion of the error at all
    target/streams/_global/muSrcGen/_global/streams
    [0m[error] higherkindness.mu.rpc.srcgen.proto.ProtoSrcGenerator$ProtobufSrcGenException
    [error] (protocol / muSrcGen) higherkindness.mu.rpc.srcgen.proto.ProtoSrcGenerator$ProtobufSrcGenException
    Nik Gushchin
    @Dr-Nikson
    Ooooohhh
    I’ve got the error
    https://github.com/googleapis/api-common-protos/blob/master/google/logging/type/http_request.proto#L17
    Failed to parse package name: <input>:1: error: identifier expected but type found
    google.logging.type
                   ^
    Nik Gushchin
    @Dr-Nikson
    Nik Gushchin
    @Dr-Nikson

    Hm, I’m trying to build my protos, but there a lots of

    object fs2 is not a member of package higherkindness.mu.rpc.internal.server
    [error]   @service(Protobuf, Identity, namespace = Some("uadc.services.v1"), methodNameStyle = Capitalize) trait SystemInstrumentsController[F[_]] {

    What am I doing wrong?)

    Nik Gushchin
    @Dr-Nikson
    Ah, nvm. I missed mu-rpc-fs2. I think it would be great to have mention of the additional deps in the tutorial
    Dermot Haughey
    @hderms
    Are there any plans to make handling forced optionality in proto 3 more idiomatic?
    Alec Zorab
    @aleczorab_twitter
    is there any support for GRPC reflection protocol? I can't find anything in the repo, but maybe someone's written a plugin for it?
    Alec Zorab
    @aleczorab_twitter
    also, if, skeuomorph doesn't seem to know about the standard protobuf types, google/protobuf/empty.proto: File not found. Do I need to go and source those myself?
    Leif Battermann
    @battermann

    Hi, we implemented a gRPC client with mu-scala which worked really well. However when running the client we always get the following error message:

    0: Data Service Response (100): --> io.grpc.StatusRuntimeException: UNAVAILABLE: Channel shutdown invoked
    io.grpc.StatusRuntimeException: UNAVAILABLE: Channel shutdown invoked
            at io.grpc.Status.asRuntimeException(Status.java:533)
            at io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onClose(ClientCalls.java:460)
            at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:426)
            at io.grpc.internal.ClientCallImpl.access$500(ClientCallImpl.java:66)
            at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.close(ClientCallImpl.java:689)
            at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.access$900(ClientCallImpl.java:577)
            at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:751)
            at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:740)
            at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
            at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
            at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
            at java.lang.Thread.run(Unknown Source)

    Network diagnostics show that host and port is accessible.
    Appearently SSL/TLS is used.
    We added the certificate to the keystore as descibed here: https://www.47deg.com/blog/mu-rpc-securing-communications-with-mu/#client-6
    And we still get the same error. It is quite hard to debug this.
    Do you have any pointers what to look into? Is it possible to enable more verbose logging?

    Leif Battermann
    @battermann
    When running this keytool -import -alias sample-ssl -file certificate.pem -keystore sample-ssl-keystore I am asked for a password, an empty password is not allowed.
    How can I tell mu the password for the keystore?
    Leif Battermann
    @battermann
    I found another java option: -Djavax.net.ssl.trustStorePassword… I will try this
    Leif Battermann
    @battermann
    no luck, so far
    Leif Battermann
    @battermann
      def run(args: Args): IO[ExitCode] = {
        val channelFor: ChannelFor                       = ChannelForAddress(args.host, args.port)
        val serviceClient: Resource[IO, DataService[IO]] = DataService.client[IO](channelFor, Nil, null)
    
        for {
          response <- serviceClient.use(client => client.ListOrders(empty.Empty()))
          _        <- IO(response.take(100).dump("Data Service Response (100):").subscribe())
        } yield ExitCode.Success
      }
    I always get the same output (see above) no matter what I pass in or if the keystore exists or anything. So e.g. if I pass a wrong host or port… same output.
    How can I troubleshoot this?
    Leif Battermann
    @battermann
    using rpccurl revealed that the certificate we are using might not be correct…
    Leif Battermann
    @battermann
    I found out that the service uses basic authorization and with this header it works: -H 'Authorization: Basic Zm9vYmFyOmZvb2Jhci1wYXNz’ when using grpcurl
    So now I will try to find out how to set a custom header using the mu-scala client
    Leif Battermann
    @battermann
    looks like it can be done like this:
    case class AuthCredentials(user: String, pass: String) extends CallCredentials {
      override def applyRequestMetadata(requestInfo: RequestInfo, executor: Executor, metadataApplier: MetadataApplier): Unit =
        executor.execute(new Runnable() {
          override def run() =
            try {
              val key            = Metadata.Key.of("Authorization", io.grpc.Metadata.ASCII_STRING_MARSHALLER)
              val userPassBase64 = getMimeEncoder.encodeToString(s"$user:$pass".getBytes(StandardCharsets.UTF_8))
              val value          = s"Basic $userPassBase64"
              val headers        = new Metadata()
              headers.put(key, value)
              metadataApplier.apply(headers)
            } catch {
              case err: Throwable => metadataApplier.fail(Status.UNAUTHENTICATED.withCause(err))
            }
        })
    
      override def thisUsesUnstableApi(): Unit = ()
    }
    haven’t tested it yet
    Leif Battermann
    @battermann
    So, using grpcurl it works with this command:
    grpcurl -v \
      -H 'Authorization: Basic Zm9vYmFyOmZvb2Jhci1wYXNz' \
      -import-path protocol/src/main/resources \
      -proto protocol/src/main/resources/data_service.proto \
      -cacert cert.pem \
      host:12345 \
      my.Service/Method
    This means the issue is not on the server side. How can I implement the same query with mu-scala? I would really appreciate some pointers :pray:
    Chris Birchall
    @cb372
    @battermann wow that's quite a mouthful just to add basic auth! It's probably worth adding that helper to Mu.
    Leif Battermann
    @battermann
    Yes :) I don’t even know if it works
    it compiles and when I run it the output is always the same...
    chain-alex
    @chain-alex
    Do the generated protocol case classes have a fromBytes constructor I can use?
    specifically for proto
    Leif Battermann
    @battermann
    @cb372 we solved the problem
    part of the problem was here:
        for {
          response <- serviceClient.use(client => client.ListOrders(empty.Empty()))
          _        <- IO(response.take(100).dump("Data Service Response (100):").subscribe())
        } yield ExitCode.Success
    the client resource gets destroyed before we run the IO response...
    :see_no_evil:
    chain-alex
    @chain-alex
    I am trying to use PBDirect but it is not clear to me how to do the above
    Dylan Martin
    @dmarticus
    hi @chain-alex I’m sorry for the slow response to your question! We use our fork of PBDirect under the hood of mu-scala for Protobuf serialization, and it’s possible to define custom codecs for both Avro and protobuf serialization by using instances of PBDirect’s PBScalarValueReader and PBScalarValueWriter type classes (see here: https://higherkindness.io/mu-scala/guides/custom-grpc-serialization#custom-codecs). What specifically are you trying to do in your case? FYI, for deserialization, PBDirect supports a pbTo[A] method if that’s what you’re looking for (example: https://github.com/47degrees/pbdirect#deserialization)
    Anton Semenov
    @a7emenov

    Hi, everyone!

    Am I correct that mu only respects the package option (and requires it) when generating server/client definitions from .proto files? Some of the proto files I'm dealing with have package names that I don't want to use when generating Scala code and some have no package names at all.

    I could've manually provided the target directory for generation that points to the package I want (which would only work assuming that all generated classes should reside in the same package), but that will still fail if there is no package name in the .proto file - I always have to provide it.

    I really want to keep the original package names in the .proto definitions, because that's important when communicating with other teams, so is there a way I could "customize" the target package without having to modify the .proto files?

    Dylan Martin
    @dmarticus
    Hi @a7emenov! You're correct that mu requires the package option when generating server/client definitions from .proto files. We do have an option for customizing the target directory (to an extent) that can be configured in your build.sbt (or similar) called muSrcGenIdlTargetDir (more details can be found here: https://higherkindness.io/mu-scala/reference/source-generation). This plugin setting determines the name of the target directory to write your generated protobuf code into (the default being target/scala-2.13/src_managed/main/), so I think it would be possible to use that plugin to specify different target directories for your generated protobuf code if you don't want to modify the package names. That said, I don't know if this is a complete answer to your question, and I'm happy to help you further if you can provide more specific details to what you're trying to do :)
    Anton Semenov
    @a7emenov

    @dmarticus thank you for the response, really appreciate it! Basically confirms all my assumptions. Though I really love Mu, this may be the reason why we'd have to switch to ScalaPB as it allows to be very flexible when generating packages.

    Is it likely that in the future skeuomorph and, by extension, mu will support options like java_package?

    Dylan Martin
    @dmarticus
    @a7emenov great feedback; and I'll circle back with the mu folks at 47 to see what our roadmap for skeuomorph looks like going forward (we're planning on continuing to make mu and skeuomorph more tightly coupled, e.g. we're likely going to move from avrohugger to skeuomorph for the avro schema parsing), so no promises but I'll definitely bring it up. I've added the issue (higherkindness/skeuomorph#318) to the skeuomorph backlog if you'd like to follow along on it.
    Anton Semenov
    @a7emenov
    @dmarticus thanks again - looking forward to this being implemented.