@oshoval Stop (https://godoc.org/google.golang.org/grpc#Server.Stop) force closes all RPCs. But GracefulStop (https://godoc.org/google.golang.org/grpc#Server.GracefulStop) waits for existing RPCs to finish.
Thanks Menghan, as long as they all use context, the logic added by the user, can even create go routines and such right ?
(and GracefulStop would wait the whole handling is finished, if the chain is guarded)
on other words, are there any limitations in some document that tell what the user handlers code should obey please?
we had a bug that serveStreams wasnt waited by Stop, i purposed to use GracefulStop because it does wait for the s.conns to become empty,
it doesnt wait by Stop, because serveStreams runs in a go routinue which isnt waited for, and just remove s.conn when done.
just wondering if there is a limitation on the logic that we adding in our RPC, (i.e dont spawn orphan go routines that we want to wait for)
Hi, everybody. I've been struggling with passing a JSON with Message values back and forth between systems.
message Result {
struct/value/any? variables = 1;
}
// Where variables would contain a (1 level deep) JSON with different types of values, e.g.:
{
"key1": 1,
"key2": true,
"key3": proto.MessageA
}The Result should is passed from a web browser (grpc-web) to a Python backend. The Python backend should serialize Reult to JSON and back again.
I've tried Struct for the variables field but there are problems with turning a Message into proper JSON with grpc-web. I would really like to use proto messages in variables. The reason for picking protobuf/grpc was exactly this, being able to use the same type throughout our complete platform, but this seems to be blocking this goal. Did I miss something? What would you do?
6 replies
Update: Got a bit further but it still doesn't work.
// proto
message Request {
google.protobuf.Struct variables = 1;
}
// obj - Where variables would contain a (1 level deep) JSON with different types of values, e.g.:
{
"key1": 1,
"key2": true,
"key3": proto.MessageA
}
// code
struct = new proto.google.protobuf.Struct(obj);
req = new Request;
req.variables = struct;Checking req.variables before sending it shows that it's indeed a Struct with all the correct fields in it. But once the other end (server) receives it req.variables is an empty Struct. For testing purposes I tried an obj that is simply {'key': 'value'}, but the result was the same.
I'm sending the req from a browser using grpc-web and receiving it in a python grpc-server.
I just found out about proto.google.protobuf.Struct.fromJavaScript and tried it:
// code
struct = proto.google.protobuf.Struct.fromJavaScript(vars);
req = new Request;
req.variables = struct;This works for a simple obj (e.g. {"key": "val"}), but for an obj with a proto message field (such as above) it resulted in :
struct_pb.js:875 Uncaught Error: Unexpected struct type.
at Function.proto.google.protobuf.Value.fromJavaScript (struct_pb.js:875)
at Function.proto.google.protobuf.Struct.fromJavaScript (struct_pb.js:941)
at Function.proto.google.protobuf.Value.fromJavaScript (struct_pb.js:871)
at Function.proto.google.protobuf.Struct.fromJavaScript (struct_pb.js:941)
at Function.proto.google.protobuf.Value.fromJavaScript (struct_pb.js:871)
at Function.proto.google.protobuf.Struct.fromJavaScript (struct_pb.js:941)
at Function.proto.google.protobuf.Value.fromJavaScript (struct_pb.js:871)
at Function.proto.google.protobuf.Struct.fromJavaScript (struct_pb.js:941)
hey all, have a few questions about the scenarios under which certain status codes would be generated by libraries.
To start w/, would "cancelled" only ever be generated client-side automatically when a deadline is exceeded? Would a server-side framework ever automatically send a "cancelled" response back to a client?
2 replies
The same proto files works in a simple node project but the same call fails with compiled typescript. Any ideas? I'm trying to understand where the array buffer is set but debugging compiled code is no fun at all lol
5 replies
2 replies
I have a gRPC-java service, written as spring boot app, that serves about 20 different RPCs. Many of them are server-streaming RPCs. Overall, the response time is anywhere between 150ms-1s, different for different RPCs. The primary client of this gRPC server is a fleet of about 5000 client processes, spread across different datacenters and geographic regions. Sometimes, due to a bug on the client, it can so happen that all these 5000 clients end up making about 10 different RPCs all at almost the same time. This has caused the gRPC server to be in a state where it is not able to accept incoming requests beyond a point (clients start seeing io.grpc.StatusRuntimeException: UNKNOWN: channel closed). And since the clients have some back-off logic they end up making these calls again as they saw channel closed. This eventually leads to cascading failure because the gRPC server is inundated with lots of requests (friendly DDoS) and other clients (not the 5000 primary) start seeing issues connecting to the server. I understand this is a classic use case for having circuit-breakers and rate-limiters. What I'd like to know from gRPC viewpoint is, what's causing the connections to be rejected? My guess is somehow the netty queue/buffer is getting filled up and then requests (maybe response too) are getting dropped. Is there a way to validate this by enabling some detailed logs? Is it possible to somehow monitor this buffer and emit metrics so that I can be alerted? Is the buffer size configurable by the application or is it a low-level gRPC thing which cannot be configured?
@ejona86 or anyone who has experience dealing with gRPC-java servers at scale?
43 replies
message_of_any_type.unpack(Class.forName(message_of_any_type..getClass().getTypeName())) will that service need to have all the proto dependencies of every type of request proto that it parses ?
1 reply
@ejona86 Do you know anyone who can help me with this?
Hello I need some help. I am doing some gRPC C++ development and was wondering if anyone using C++ smart pointers with Arena allocation. In particular, what happens on destroy if I use and arena allocated raw point to create a std:unique_ptr object.
I am really waiting for async support with callbacks without manually controlling dispatch as it is now. Any chance that it's on roadmap?
1 reply
Hi all, I'm new to gRPC and microservices in general, and I'm looking for some advice. I'm migrating from a monolithic low-latency system. The aim of creating the microservices is not one of replication, load balancing etc, but of separation of concerns and configuration flexibility (e.g. being able to swap out components). Each 'run' involves launching the microservices, launching the controllers, and running the master controller, then killing everything at the end.
I have a main controller M that connects to a child-controller A, which runs a data pipeline through other microservices a1 -> a2 -> a3 -> a4. All of these are stateful, in that they need to be aware of their own history (for e.g. detecting changes, or sampling, etc). Note that the a1...4 is a simplification - I have multiple of these child-controllers, each controlling a different number of microservices.
My current approach is
Request : M -> A -> a4 -> a3 -> a2 -> a1
Response: M <- A <- a4 <- a3 <- a2 <- a1I'm finding that this isn't great for throughput due to the requests and the responses being passed along the chain. It'd be fine if I could do it in batch, but I need the result from A as soon as possible for the next stage in the pipeline. When I run these in batch (i.e. collecting results into a repeated field and flushing every 10k of them) it runs almost as fast as the monolithic version (3.9s for the monolithic, 4.1s for the batch microservices). When I run them one-by-one, more closely matching the real-life implementation requirements, it takes 10-100x that.
One solution would be to remove the middle-men and do:
Request: M -> A ----------------------> a1
Response: M <- A <- a4 <- a3 <- a2 <- a1Is it at all possible to do that, bearing in mind that the A that receives the response from a4 is the same A that makes the initial request to a1? I.e. by passing direct connection details down the chain, or something along those lines? All of these components are in C++, using the gRPC C++ API.
11 replies
Is here someone who can take a look at this issue in grpc-dart? grpc/grpc-dart#135
It would be really helpful to have access to the response trailers like in grpc-java.
Hi all,
I'm streaming large arrays using gRPC by streaming a message with:
message Chunk{
bytes payload = 1;
}On our server we serialize the array with:
inline void StreamChunks(ServerWriter<Chunk>* writer, int chunk_size,
const void* array, int n_bytes){
char* bytes = (char*) array;
Chunk chunk;
for (int c = 0; c < n_bytes; c += chunk_size) {
if (c + chunk_size > n_bytes) { // last chunk, send up to the end of the array
chunk.set_payload(&bytes[c], n_bytes - c);
writer->Write(chunk);
}
else { // Send max chunk size (chunk_size)
chunk.set_payload(&bytes[c], chunk_size);
writer->Write(chunk);
}
}
}And deserialize it on the client with:
// size the array
reader->WaitForInitialMetadata();
const int array_size = GetInitialMetadataValue<int>(&context, "size", 0);
int *array = new int[array_size];
char *rawarray = (char*)array;
// Read data from the server in chunks
Chunk chunk;
while (reader->Read(&chunk)) {
const std::string payload = chunk.payload();
memcpy(rawarray, payload.data(), payload.size());
rawarray += payload.size();
}We're getting great performance on both Windows and Linux (around 1 GBps) compared to repeated messages (anywhere between 100 MBps - 400 MBps). But I'd like to get even closer to MEMCPY speeds for IPC.
Is there a way to optimize our client/server model to improve performance? I've looked at grpc arena allocation, but it seems to be suited for messages and not streams. I've also looked into flatbuffers, but it's out of sync with gRPC and won't work with 1.25 (which is a hard requirement).