Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Aug 14 21:37

    mergify[bot] on master

    Update scalafmt-core to 3.5.9 (… (compare)

  • Aug 14 21:37
    mergify[bot] closed #1400
  • Aug 14 21:19

    mergify[bot] on master

    Update scalafmt-core to 3.5.9 (… (compare)

  • Aug 14 21:19
    mergify[bot] closed #277
  • Aug 14 21:19
    mergify[bot] synchronize #1400
  • Aug 14 21:13

    mergify[bot] on master

    Update mockito-core to 4.7.0 (#… (compare)

  • Aug 14 21:13
    mergify[bot] closed #1399
  • Aug 14 21:05

    mergify[bot] on master

    Update scalafmt-core to 3.5.9 (… (compare)

  • Aug 14 21:05
    mergify[bot] closed #179
  • Aug 14 21:03
    mergify[bot] closed #198
  • Aug 14 21:03

    mergify[bot] on master

    Update scalafmt-core to 3.5.9 (… (compare)

  • Aug 14 21:03

    mergify[bot] on master

    Update scalafmt-core to 3.5.9 (… (compare)

  • Aug 14 21:03
    mergify[bot] closed #270
  • Aug 14 21:03
    mergify[bot] labeled #277
  • Aug 14 21:03
    mergify[bot] assigned #277
  • Aug 14 21:02
    scala-steward opened #277
  • Aug 14 21:01
    mergify[bot] labeled #179
  • Aug 14 21:01
    mergify[bot] assigned #179
  • Aug 14 21:00
    scala-steward opened #179
  • Aug 14 21:00
    mergify[bot] labeled #1400
Michel Rouly
@jrouly
Hey, just wondering, has anyone had success integrating scalapb (or even specifically akka-grpc) with a tool like buf? I saw there were some messages back in March about buf, but I haven't been able to see any of the examples associated with those conversations or ultimately what the outcome was.
Nadav Samet
@thesamet
@jrouly as far as I recall, the approach was to download the native binary of the plugin (protoc-gen-scala), have it on the PATH, and using buf to invoke it.
1 reply
That's been reported to work.
It would be interesting to see if ScalaPB can be deployed on BSR. It's not something I looked into yet.
Mike Dias
@mikedias
Hi @thesamet did you have a chance to look at thesamet/sparksql-scalapb-test#6 I'm unsure what else I can try from my side and I'd really appreciate your guidance.
Nadav Samet
@thesamet
Hi @mikedias sorry for missing this. The example seems to run with spark-submit (outside databricks), also the exception you are getting is within databricks, although your code doesn't reference anything about databricks. Could you investigate that? Do you have access to databricks support?
Nadav Samet
@thesamet
In the exception it has com.databricks.sql.serialization.marshalling.LiteralMarshaller$.buildProto - I wonder why databricks proto-related code gets kicked in. Maybe this is something that can be disabled somehow.
Mike Dias
@mikedias
Yes, I have access to their support and I'll raise a ticket asking that. Thanks!
Darren Bishop
@DarrenBishop

Hello community. My client has a central repo containing all protobufs for messages going out over the Kafka backbone.
Given we are talking 'backbone' infrastructure, the protobufs are used to generate code for several languages, including Go and Python.
We have specified options for ScalaPB, e.g.

syntax="proto3";

package x.y.z.service.some_service;

import "scalapb/scalapb.proto";
import "google/protobuf/timestamp.proto";
import "messages/common/common.proto";

message SomeServiceEvent {
    option (scalapb.message).extends = "x.y.z.HasMessageMetadata[SomeServiceEvent]";
    option (scalapb.message).companion_extends = "x.y.z.HasKafkaMetadata[SomeServiceEvent]";

    ...
}

but other language (i.e. Python) processors fail with

Import "scalapb/scalapb.proto" was not found or had errors.

Is there a way to use language (ScalaPB) specific options that are ignored by other language processors?

Nadav Samet
@thesamet
Hi @DarrenBishop , this is exactly the scenario auxiliary scoped options are for. You put all these options in a separate proto file and ensure that this file is only available in the ScalaPB build.
Denis Savitsky
@desavitsky
Hi!
Is there ETA on ZIO2-based zio-grpc release?
Or there is anything left that should be done before the release?
Nadav Samet
@thesamet
@desavitsky , I think that the non-trivial parts are behind us (thanks to your help). What remains would be to merge a few version bump PRs and cut the release. I'll see if I can get that this evening or early next week.
Denis Savitsky
@desavitsky
Great! Thanks
Nadav Samet
@thesamet
@desavitsky releasing zio-grpc 0.6.0-test4. Still want to have that flaky test fixed before I'm ready to call it 0.6.0.
Nadav Samet
@thesamet
Looks like the release process failed for a gpg related reason. I'll have more time to look at it early next week.
huangh
@hsl4125
how can i create message by name?? i know c++ can do it with DescriptorPool, but in scalapb it is a little trouble.
Sumiya92
@Sumiya92
Hi Team, I am Trying to implement scalaPB for my project where i have the result in List[Map[String,Any]] and i need to convert that to option[struct],
for ex: val rawdata : List[Map[String,Any]] = List(Map("deviceid" -> 860267 , "altitude " -> 0.062),Map("deviceid" -> 860268 , "altitude " -> 0.062))
i need to store it into decoded struct message IOTDecodeResult {
google.protobuf.Struct decoded = 1;
}
Darren Bishop
@DarrenBishop
@thesamet fantastic! The auxiliary scoped options worked a treat, thanks.
Can this facility be used to switch a field from being a oneof xyz to a oneof sealed_value ?
Darren Bishop
@DarrenBishop

@thesamet apologies, having read over the docs a number of times now, I realise my last question a) does not make sense and b) won't give me mileage
What I was trying to do was create a sealed oneof (idiomatic-Scala) that can take a case object or a case class i.e. something structurally equivalent to Option[Boolean] with values of None and Some(true).
Unfortunately, sealed oneof does not allow enums:

java.lang.UnsupportedOperationException: This field is not of message type.

Is there a trick to generating objects or case objects but extending scalapb.GeneratedMessage?

Kenneth Monteiro
@ManiacViper
@thesamet was thinking of contributing a PR to solve the problem of keywords in package path for a generated scala class but maybe for now it can be specific to
package keyword
we could make it more generic and configurable with follow up Prs
5 replies
Darren Bishop
@DarrenBishop

@thesamet I think the ideal solution would be the ability to specify the name of the Empty case in a sealed oneof to something more meaningful

message Some {
  <your-type> = 1;
}

message Option {
  oneof sealed_value {
    option (scalapb.oneof).empty_name = "None";
    Some some = 1;
  }
}

wdyt?

Nadav Samet
@thesamet
@DarrenBishop Sounds like a feature request to support controlling the name of the empty case. I think it makes sense, can you file this on github so I can think more of it? Can you add more context there on what you're trying to achieve?
jurevreca12
@jurevreca12
Hello everybody. I am new to scala and SBT, so please excuse me if this is a newbie question. I am trying to optimize my build time (I have a sbt project) and I noticed that every time i use "compile" new scala classes are generated. Is this expected behavior? Shouldn't the protocGenerate command only be invoked if there are changes in the proto definition file?
Nadav Samet
@thesamet
Hi @jurevreca12 , this isn't the expected behavior. If the protos don't change, the Scala files wouldn't be generated. If one proto files changed, all the Scala files in the project will be regenerated, but SBT will only compile the minimum necessary.
jurevreca12
@jurevreca12
I see. Is there something wrong with my configuration perhaps?
ThisBuild / scalaVersion     := "2.13.8"
ThisBuild / version          := "0.1.0"
ThisBuild / organization     := "Jure"

Compile / scalaSource := baseDirectory.value / "chisel4ml" / "scala" / "main"
Compile / unmanagedSourceDirectories += baseDirectory.value / "chisel4ml" / "scala" / "lbir"
Compile / unmanagedSourceDirectories += baseDirectory.value / "chisel4ml" / "scala" / "services"


crossTarget := baseDirectory.value / "bin"
assembly / assemblyJarName  := "chisel4ml.jar"
ThisBuild / assemblyMergeStrategy := {
    case PathList("META-INF", "MANIFEST.MF") => MergeStrategy.discard
    case _ => MergeStrategy.first
}

PB.deleteTargetDirectory := false
PB.additionalDependencies := Nil
Compile / PB.includePaths += file(root.base.getAbsolutePath)
Compile / PB.protoSources := Seq(baseDirectory.value / "chisel4ml" / "lbir")
Compile / PB.targets := Seq(
  scalapb.gen(flatPackage = true) ->  baseDirectory.value / "chisel4ml" / "scala"
)

val chiselVersion = "3.5.1"
lazy val root = (project in file("."))
  .settings(
    name := "chisel4ml",
    libraryDependencies ++= Seq(
      "edu.berkeley.cs"            %% "chisel3"              % chiselVersion,
      //"edu.berkeley.cs"            %% "chiseltest"           % "0.5.2"                                 % "test",
      "edu.berkeley.cs"            %% "treadle"              % "1.5.3",
      "com.thesamet.scalapb"       %% "scalapb-runtime"      % scalapb.compiler.Version.scalapbVersion % "protobuf",
      "io.grpc"                    %  "grpc-netty"           % scalapb.compiler.Version.grpcJavaVersion,
      "com.thesamet.scalapb"       %% "scalapb-runtime-grpc" % scalapb.compiler.Version.scalapbVersion,
      "org.slf4j"                  %  "slf4j-api"             % "1.7.5",
      "org.slf4j"                  %  "slf4j-simple"          % "1.7.5"
    ),
    scalacOptions ++= Seq(
      "-language:reflectiveCalls",
      "-deprecation",
      "-feature",
      "-Xcheckinit",
      "-P:chiselplugin:genBundleElements",
    ),
    addCompilerPlugin("edu.berkeley.cs" % "chisel3-plugin" % chiselVersion cross CrossVersion.full),
  )
Nadav Samet
@thesamet
@jurevreca12 Unrelated you probably don't need seting PB.deleteTargetDirectory and PB.additionalDependencies. The latter can cause the generated classes directory get out of sync with the protos in case you remove messages or proto files, removing the latter will allow you to manually remove the dependency on scalapb-runtime since it will get added automatically.
5 replies
Nadav Samet
@thesamet
Maybe the issue is related to generating into an unmanaged source directory (the scalapb.gen line). Have you tried changing that to managed location as in the install instructions? https://scalapb.github.io/docs/installation/#installing-in-sbt-recommended
Stanislav Suslov
@StasSuslov
Guys, hi! I have a question about supporting scalapb repo in buf BSR. Is it actual right now? Can I use it or should I find another way to link scalapb.proto? I'm talking about https://buf.build/scalapb
Nadav Samet
@thesamet
Hey @StasSuslov , looks like scalapb repo on buf was set up by someone else, so not officially supported here. I am not familiar with BSR and how it works. It looks like buf is getting some traction recently so I'd like to see how an integration can work. Please share your findings!
greatmengqi
@greatmengqi
hello everyone!
image.png
I download the demo of scalapb plugin, but the test file can not find the TestMessageFieldNums class in idea. but the sbt shell can work correctlly
Nadav Samet
@thesamet
@greatmengqi Try to re-import the project into IntelliJ after the first sbt compile.
If the project gets imported prior to the first run, it can miss the generated code.
1 reply
greatmengqi
@greatmengqi
the demo use sbt-projectmatrix plugin
i creat this demo by "sbt new scalapb/protoc-gen-template.g8"
image.png
if i set this fold as source, it work correctlly .Unfortunately, when i re import ,this setting will invalid。
Nadav Samet
@thesamet
In sbt you can type sourceDirectories to see whether the said directory is listed. If it is - then it's an intellij bug (most likely the case)
greatmengqi
@greatmengqi
image.png
i have set this manually ,it does not work.
Nadav Samet
@thesamet
doesn't it miss the the jvm-2.12 part?
However, even without this, sourceDirectories should have an entry for the src_managed directory.
greatmengqi
@greatmengqi
image.png
Nadav Samet
@thesamet
Isn't the two overlapping entries (the one marked and the one above it) a potential problem?
So it looks like parent main directory is added automatically which should be sufficient.
So this looks like an intellij bug.
It's probably not the advice you were hoping for, but maybe try vscode with metals on this project. I anticipate this will work flawlessly.
1 reply