Hi. before netty 1.0.0, we used tcpServer::bootstrap to register a gauge for pending task queue of the eventloop
factory.addServerCustomizers(httpServer -> httpServer.tcpConfiguration(
tcpServer -> tcpServer.bootstrap(serverBootstrap -> {
serverBootstrap.config().childGroup().forEach(eventExecutor -> {
if (SingleThreadEventExecutor.class.isAssignableFrom(eventExecutor.getClass())) {
SingleThreadEventExecutor singleThreadEventExecutor = (SingleThreadEventExecutor) eventExecutor;
Gauge.builder(SERVER_PENDING_TASK_METRIC, singleThreadEventExecutor::pendingTasks)
.description("Pending Tasks")
.tag(SERVER_THREAD_NAME, singleThreadEventExecutor.threadProperties().name())
.tag(SERVER_THREAD_STATE, singleThreadEventExecutor.threadProperties().state().name())
.register(registry);
}
});
return serverBootstrap;
}
)));starting with netty 1.0.0, bootstrap is not available anymore. how can this be done now? thx for help
2 replies
Hi I've a question. When to use ByteBuf.discardReadBytes()? Does this release the memory of the already read part of the underlying byte array?
It depends on implementation, e.g UnpooledHeapByteBuf has capacity 10, readerIndex set to 4 position and writerIndex to 8, so after discardReadBytes() all bytes between readerIndex and writeIndex will be copy to the beginning and readerIndex set to 0 and writerIndex to 4 position.
:wave: Hi! I'm wondering if anyone knows of an http decoder that would allow me to detect HTTP/2 vs. HTTP/1.x so I can support both versions on the same port. I'd like to configure the channel pipeline after the protocol version has been detected. I've seen the netty example Http2OrHttpHandler but my understanding is that only works if you're using SSL, which I'm not.
If such a decoder doesn't exist would it be possible to write one? I'm imagining something like the PrefaceDecoder in Http2ConnectionHandler that triggers an event after reading the preface instead of throwing an error.
3 replies
Hi! I'm wondering how to make the client of proxy server keep alive. (Thus, I don't want the proxy client to make a tcp close handshake everytime.)
I saw the proxy example in netty
Adding the keepAlive option to this example doesn't seem to work properly. Because it makes a client and connect everytime the server get request and close the client when the response is arrived.
Is there anyone who know how to make the proxy client keepAlive? Is there any reference/example for it?
1 reply
Getting the below error couple of times while uploading same multipart file.
Caused by: java.io.IOException: Out of size: 990 > 989 at io.netty.handler.codec.http.multipart.AbstractMemoryHttpData.addContent(AbstractMemoryHttpData.java:104)
io.netty.handler.codec.http.multipart.HttpPostMultipartRequestDecoder.loadDataMultipartOptimized(HttpPostMultipartRequestDecoder.java:1190)
io.netty.handler.codec.http.multipart.HttpPostRequestDecoder$ErrorDataDecoderException: java.io.IOException: Out of size: 990 > 989
Any guidance will be very much appreciated.
1 reply
I want to know if netty is a good option to use for low latency networking. Can it go below say 10 or even 5 microseconds on an optimized server with a fast NIC (Mellanox)?
Are there any recent benchmarks on latency? Or any benchmarking tools I can use to measure it?
And if not, could anyone point me to the right guides or resources to develop a benchmark? From what I have seen netty has quite a steep learning curve. So I am a bit confused about where to start.
it finally call AbstractOioChannel.doBeginRead() method, and the readTask execute again and again, and the question is the AbstractOioChannel member variable readPending,it has been set true,but another execute it resetting false,it cause execute finally block again and again, I will show the screenshot next, give me some advice tks!
Hello, I have some troubles with tc-native:
Failed to load any of the given libraries: [netty_tcnative_linux_x86_64, netty_tcnative_linux_x86_64_fedora, netty_tcnative_x86_64, netty_tcnative]","logger":"com.onespan.billinginvoicereports.config.HttpClientConfiguration","thread":"main","exception":"java.lang.IllegalArgumentException: Failed to load any of the given libraries: [netty_tcnative_linux_x86_64, netty_tcnative_linux_x86_64_fedora, netty_tcnative_x86_64, netty_tcnative]Epoll loads fine but not openssl, tc-native jar is there, I am using eclipse-temurin:17.0.2_8-jdk-focal, netty 4.1.73 and tc-native v2.0.48
1 reply
public static void zeroCopyFile(FileChannel targetChannel, ByteBuf content) {
targetChannel.transferFrom(new ByteBufChannel(content), 0, content.readableBytes());
}
private static class ByteBufChannel implements ReadableByteChannel {
private final ByteBuf byteBuf;
public ByteBufChannel(ByteBuf byteBuf) {
this.byteBuf = byteBuf;
}
@Override
public boolean isOpen() {
return true;
}
@Override
public void close() throws IOException {
}
@Override
public int read(ByteBuffer dst) throws IOException {
int byteBufReadableBytes = byteBuf.readableBytes();
dst.put(byteBuf.nioBuffer());
return byteBufReadableBytes - byteBuf.readableBytes();
}
};
Hello Netty!
We are currently in the process of porting the Apache James server to Netty 4 and encounters some problems down the way. Those goes beyond my current understanding of Netty, thus is it OK if I ask for help here?
Currently I encounter issues with SMTP pipelining: the client sends all the SMTP requests into a single network hop.
Socket client = new Socket(bindedAddress.getAddress().getHostAddress(), bindedAddress.getPort());
buf.append("HELO TEST");
buf.append("\r\n");
buf.append("MAIL FROM: <test@localhost>");
buf.append("\r\n");
buf.append("RCPT TO: <test2@localhost>");
buf.append("\r\n");
buf.append("DATA");
buf.append("\r\n");
buf.append("Subject: test");
buf.append("\r\n");
buf.append("\r\n");
buf.append("content");
buf.append("\r\n");
buf.append(".");
buf.append("\r\n");
buf.append("quit");
buf.append("\r\n");
OutputStream out = client.getOutputStream();
out.write(buf.toString().getBytes());
out.flush();The way DATA is handled is that it adds a handler prior the core SMTP handler: https://github.com/chibenwa/james-project/blob/ffc0d4a8b22508b8f5b58594d14041d1f6bc3acf/protocols/netty/src/main/java/org/apache/james/protocols/netty/NettyProtocolTransport.java#L162
channel.pipeline().addBefore(eventExecutors, HandlerConstants.CORE_HANDLER, "lineHandler" + lineHandlerCount, new LineHandlerUpstreamHandler(session, overrideCommandHandler));Everything works fine if the core handler is running on the event loop, however once we switch it to a distinct executor, the pipeline modification is no longer applied and the subsequent message content is interpreted as SMTP commands as it the line handler was not there.
// Fails
pipeline.addLast(eventExecutorGroup, HandlerConstants.CORE_HANDLER, createHandler());
// Succeed
pipeline.addLast(HandlerConstants.CORE_HANDLER, createHandler());Is there any way to modify the pipeline 'synchronously' from outside the event loop?
Thanks in advance.
Edit: I found a walkaroud this issue by modifying the core handler to allow overrides of its behaviour, thus not requiring any further modifications of the pipeline. At the price of a not that costly refactoring.
I think I will head toward encapsulating all those behaviors modification into a single handler, thus getting read of all that thread ordering madness.
Things might get bloodier for IMAP though as request decoding is done in a separate handler (request-parsing then request execution) thus overrides happens before request decoding...
Hi everyone!
I just started with Netty -> DotNetty -> SpanNetty. Sorry about the C# code :)
I'm working on a decoder for my custom protocol. Currently I'm using ReplayingDecoder<Enum> decoder.
Now I'm running into IndexOutOfRangeException: readerIndex(4) + length(1280) exceeds writerIndex(10): PooledHeapByteBuffer(ridx: 4, widx: 10, cap: 256)
So I tried to fix it as followed:
if (input.ReadableBytes >= _messageFrame.Length)
{
_messageFrame.Payload = input.ReadBytes(_messageFrame.Length);
Checkpoint(ProtocolDecoderState.ReadEpilog);
}which breaks my next switch case. case ProtocolDecoderState.ReadEpilog because I'm expected epilog to be a certain byte.
Here is the whole code:
protected override void Decode(IChannelHandlerContext context, IByteBuffer input, List<object> output)
{
switch (State)
{
case ProtocolDecoderState.ReadProlog:
_messageFrame.Prolog = input.ReadByte();
Checkpoint(ProtocolDecoderState.ReadMessageType);
break;
case ProtocolDecoderState.ReadMessageType:
_messageFrame.SetMessageType(input.ReadByte());
Checkpoint(ProtocolDecoderState.ReadLength);
break;
case ProtocolDecoderState.ReadLength:
_messageFrame.Length = input.ReadShort();
Checkpoint(ProtocolDecoderState.ReadPayload);
break;
case ProtocolDecoderState.ReadPayload:
if (input.ReadableBytes >= _messageFrame.Length)
{
_messageFrame.Payload = input.ReadBytes(_messageFrame.Length);
Checkpoint(ProtocolDecoderState.ReadEpilog);
}
break;
case ProtocolDecoderState.ReadEpilog:
_messageFrame.Epilog = input.ReadByte();
output.Add(ProtocolHelper.CovertByteFrameToMessage(_messageFrame));
Checkpoint(ProtocolDecoderState.ReadProlog);
_messageFrame = new MessageFrame();
break;
default:
throw new InvalidDataException("Shouldn't reach here.");
}
}Do I need to use the ByteToMessageDecoder instead?
Can I not use the ReadBytes(int length) in the ReplayingDecoder?
Thanks!!
1 reply