Protocol buffer compiler for Scala. Consider sponsoring ScalaPB: https://github.com/sponsors/thesamet
I have recently discovered
prototool and buf and would like to enable one of these linters in my schemas repository. I have initiated a naive CI by running buf on proto resources. Unfortunately import "scalapb/scalapb.proto"; is unresolved and leads to compilation errors. How do you circumvent these types of errors (same problem in the IJ plugin) ? Thanks in advance :)
RootMessage.parseFrom(...) to deserialize which only gives us the RootMessage. So i guess my question is, how can we deserialize the extension fields? Thx for you help :)
{ payload => val msg = RootMessage.parseFrom(payload); (msg, msg.extension(ext))} to return both the message and the extension you expect as a tuple (or define a new case class), and transform it as a dataframe from there.
Hello this might be a simple question but I don't see a straightforward way around it . ScalaPBC generates case classes from proto definitions. The regular protobuf compiler can generate Java classes (that inherit from com.google.protobuf.GeneratedMessageV3 and implements com.google.protobuf.Message).
If Im writing a function that can naively operate on both generated proto classes, how would I do so? There is no common ancestor or interface I can use.
Reason I want this: Writing a small internal library in java that will be used within scala services as well as java services. The input to one of my functions is gonna be a protobuf type. I can't use Message as the type of the argument as it wouldn't work from a scala service
Hey, so is there a generic way to reference the Java class associated with a generated Scala proto?
I wanted to provide an interface that takes A and B that are GeneratedMessages and automatically serialize to/from Java protos and provides a Scala only interface to the user. Currently I have to do:abstract class MyInterface[A, AJava, B, BJava](implicit aJavaSupport: JavaProtoSupport[A, AJava], bJavaSupport: JavaProtoSupport[B, BJava])
Which works, but it's annoying cause the user has to fill out a giant type signature. Ideally they shouldn't have to care about or reference the Java classes. Any ideas?
JavaProtoSupport. Can you provide additional context on what you are trying to enable for the users? What methods will you have in this abstract class? If the intent is to make it easier to convert Java to Scala protos why it needs two types (A and B)?
JavaProtoSupport to create RecordSerializers and RecordDeserializers for the scala protos
It can, but schema registry uses the Java protos (I assume through the file descriptor) to track changes to schema, ensure backwards compatibility, and allow for dynamically deserializing protobuf messages.
But there are lots of tools in the Kafka ecosystem that need to dynamically deserialize messages on topics, and to do that with protobuf, you need to have the schema of the message that was published since protobuf doesn't encode type information into the binary format. So Kafka consumer/producers essentially encode the schema id in schema registry with each message that is published. Then consumers can essentially pull down a message, get the schema id out, reach out to schema registry to get the proto definition, and use that definition to deserialize the message.
1 reply
DynamicMessage and put it in S3 as JSON or whatever
3 replies
@tdbgamer , I played with it a little bit today. It seems like the change would require introducing duplication (two classes for JavaProtoSupport, one with two type parameters and one with dependent-type), or if I eliminate the old one it would make a breaking change, so I want to think about it more. However, another approach could be adding implicit parameter to all method signatures in your interface. This makes the Java type automatically found:
class MyGenericClass[A <: scalapb.GeneratedMessage] {
def toJava[JT](s: A)(implicit jps: scalapb.JavaProtoSupport[A, JT]): JT =
jps.toJavaProto(s)
}This makes the Java type (JT), automatically inferred by the compiler. Your user calls myGenericCall.toJava(scalaProto)
JavaProtoSupport implicit had to be on the constructor or I end up having to do a lot of boilerplate overrides on the subclass. I'll test that out again though in a sec and see if it works
RecordSerializer and RecordDeserializerimplicits are provided as long as you have a ProtobufSettings and JavaProtoSupport implicit.
ProtoReflectionService?
error: overloaded method pack with alternatives:
[A <: scalapb.GeneratedMessage with scalapb.Message[A]](generatedMessage: A, urlPrefix: String): com.google.protobuf.any.Any <and>
[A <: scalapb.GeneratedMessage with scalapb.Message[A]](generatedMessage: A): com.google.protobuf.any.Any
cannot be applied to (generatedMessage: scalapb.GeneratedMessage with scalapb.Message[_])
event = Any.pack(generatedMessage = message),
Hi, I am using Mac M1.
While using the scalapb compiler plugin I am running into trouble.
In plugin.sbt,
addSbtPlugin("com.thesamet" % "sbt-protoc" % "1.0.3")
libraryDependencies += "com.thesamet.scalapb" %% "compilerplugin" % "0.11.5"In build.sbt, I am trying to override the protoc version
PB.protocVersion := "3.17.3"so that I can use the protoc osx-aarch_64 architecture. But I am getting the following error
"Could not find artifact com.google.protobuf:protoc:exe:osx-aarch_64:3.15.6". Not sure why it is still referring to 3.15.6
Any suggestions.
1 reply
ClientCall.Listener from a ForwardingZClientCall and run unsafeRun from onMessage?
message foo {
string cause = 1;
}
message bar {
string data = 1;
}
message my_trait {
string id = 1;
oneof sealed_value {
foo fooEl = 2;
bar barEl = 3;
}
}
