Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Kevin
    @KevinRoebert
    Hi I've a question. When to use ByteBuf.discardReadBytes()? Does this release the memory of the already read part of the underlying byte array?
    Peter
    @warmuuh

    Hi. before netty 1.0.0, we used tcpServer::bootstrap to register a gauge for pending task queue of the eventloop

        factory.addServerCustomizers(httpServer -> httpServer.tcpConfiguration(
            tcpServer -> tcpServer.bootstrap(serverBootstrap -> {
                  serverBootstrap.config().childGroup().forEach(eventExecutor -> {
                    if (SingleThreadEventExecutor.class.isAssignableFrom(eventExecutor.getClass())) {
                      SingleThreadEventExecutor singleThreadEventExecutor = (SingleThreadEventExecutor) eventExecutor;
    
                      Gauge.builder(SERVER_PENDING_TASK_METRIC, singleThreadEventExecutor::pendingTasks)
                          .description("Pending Tasks")
                          .tag(SERVER_THREAD_NAME, singleThreadEventExecutor.threadProperties().name())
                          .tag(SERVER_THREAD_STATE, singleThreadEventExecutor.threadProperties().state().name())
                          .register(registry);
                    }
                  });
                  return serverBootstrap;
                }
            )));

    starting with netty 1.0.0, bootstrap is not available anymore. how can this be done now? thx for help

    2 replies
    Andrey Mizurov
    @amizurov

    Hi I've a question. When to use ByteBuf.discardReadBytes()? Does this release the memory of the already read part of the underlying byte array?

    It depends on implementation, e.g UnpooledHeapByteBuf has capacity 10, readerIndex set to 4 position and writerIndex to 8, so after discardReadBytes() all bytes between readerIndex and writeIndex will be copy to the beginning and readerIndex set to 0 and writerIndex to 4 position.

    Kang Breder
    @kangbreder
    Hello Everyone. My name is Kang Breder Mbulle. I am a student from the University of Buea studying for a M.Eng in Network Engineering and Security. I have saw this organisation on the GSoC page and found it interesting. I will like to contribute to this project as I look forward to GSoC 2021. I have some knowledge of Java and I willing to learn. Can someone please guide me on how to get started?
    Vaibhav Patil
    @vaibhavpatil123
    Hello Colleagous, how we can share common SimpleChannelInboundHandler for more then one client request , at last how we can map request -to-response? in netty
    SimpleChannelInboundHandler
    CHEVALIER
    @et00448
    Hello Everyone
    How Can we desactivate reissue on http 408 response with Netty reactor 0.9.13 ?
    1 reply
    Thank'you
    CHEVALIER
    @et00448
    Hi all,
    Is it possible to accept non URLENCODED with NETTY (like TOMCAT's relaxed-query-chars parameter) ?
    Thanks
    bruto1
    @bruto1
    Hi all,
    any tips on whether or not SslProvider.OPENSSL_REFCNT is production-ready?
    looking to reduce memory and CPU impact of JDK's impl on our service - memory profiling shows it allocating alot of byte[], among other things
    Ilya
    @squadgazzz
    Hello! Is it possible to enable TCP SACK (Selective Acknowledgments) via Netty?
    Marco Zanghì
    @marcozanghi
    Hi all, I am trying to resolve a specific problem with a reactor.netty.http.client.PrematureCloseException when I am using to call an endpoint from a webclient. This endpoint is behind a reverse proxy. Do you have any idea?
    Dineth Senevirathne
    @hdineth
    Can someone tell the default log format of the reactor.netty.http.server.AccessLog ?
    2 replies
    cwildman
    @cwildman

    :wave: Hi! I'm wondering if anyone knows of an http decoder that would allow me to detect HTTP/2 vs. HTTP/1.x so I can support both versions on the same port. I'd like to configure the channel pipeline after the protocol version has been detected. I've seen the netty example Http2OrHttpHandler but my understanding is that only works if you're using SSL, which I'm not.

    If such a decoder doesn't exist would it be possible to write one? I'm imagining something like the PrefaceDecoder in Http2ConnectionHandler that triggers an event after reading the preface instead of throwing an error.

    3 replies
    Liu Xinyu
    @Nagisin
    Hi all, I get two handler listen two ports separately,can anyone teaching me how to do that?
    Kirill Marchuk
    @62mkv
    Hi all. Is there a netty-based FTP client implementation? Couldn't find one yet... :(
    MappyLiam
    @MappyLiam

    Hi! I'm wondering how to make the client of proxy server keep alive. (Thus, I don't want the proxy client to make a tcp close handshake everytime.)
    I saw the proxy example in netty
    Adding the keepAlive option to this example doesn't seem to work properly. Because it makes a client and connect everytime the server get request and close the client when the response is arrived.

    Is there anyone who know how to make the proxy client keepAlive? Is there any reference/example for it?

    catiga
    @catiga
    @62mkv u could get my code from https://github.com/catiga/jc , a netty server & client base micro-app server model
    catiga
    @catiga
    any problem can contact by mail
    Valery Lavrentiev
    @lavsurgut
    Hello guys, I'm trying to run netflix zuul 2 and getting warnings like - "Unknown channel option 'SO_KEEPALIVE' for channel "when starting up. Trying to figure out why its the case. I see that its coming from AbstractBootstrap (setChannelOption) function. Zuul passes it on server startup, but netty (4.1.63) seems to ignore it. Why netty might not pick this option up?
    1 reply
    Djem Mustafa
    @JamesMustafa
    Hello guys, I want to monitor the direct memory used by Netty in my application. In the documentation here I saw the usedDirectMemory() method which I think can be useful for my purpose. Can you give me more detail about how can I implement this method and use it in an appropriate way? Thanks in advance :)
    Chris Vest
    @chrisvest
    @JamesMustafa the PooledByteBufAllocator implements that method through the metric() method. So just call those.
    Gurunathan23
    @Gurunathan231_twitter

    Getting the below error couple of times while uploading same multipart file.
    Caused by: java.io.IOException: Out of size: 990 > 989 at io.netty.handler.codec.http.multipart.AbstractMemoryHttpData.addContent(AbstractMemoryHttpData.java:104)
    io.netty.handler.codec.http.multipart.HttpPostMultipartRequestDecoder.loadDataMultipartOptimized(HttpPostMultipartRequestDecoder.java:1190)
    io.netty.handler.codec.http.multipart.HttpPostRequestDecoder$ErrorDataDecoderException: java.io.IOException: Out of size: 990 > 989

    Any guidance will be very much appreciated.

    1 reply
    Chris Vest
    @chrisvest
    You might want to use Wireshark or a LoggingHandler to see if the client is accidentally sending an extra byte. If the client is sending correct amounts of data then it might be a Netty bug, and you can use the captured data when opening an issue.
    2 replies
    Ehsan
    @esihaj
    Hi everyone!
    I want to know if netty is a good option to use for low latency networking. Can it go below say 10 or even 5 microseconds on an optimized server with a fast NIC (Mellanox)?
    Are there any recent benchmarks on latency? Or any benchmarking tools I can use to measure it?
    And if not, could anyone point me to the right guides or resources to develop a benchmark? From what I have seen netty has quite a steep learning curve. So I am a bit confused about where to start.
    lasradoVinod
    @lasradoVinod
    Hi everyone, Is it possible to set the HTTP2 HPACK dynamic table size to 0 using netty?
    zleo
    @zleo:matrix.org
    [m]
    Hi all, newbie question here: I've read netty's versioning strategy here, does this apply to the entire project or individual components as well?
    e.g. is netty-handler4.1.68 supposed to be source backward compatible with netty-transport 4.1.50?
    wuhan
    @wuhan321
    everybody i like
    switlen
    @switlen
    hi everyone,I am new in netty,when I use netty-transport-rxtx(Deprecated) 4.1.73.Final, program is same as netty-example rxtx,I find my program can not get any response,dump the stacktrace,I find AbstractOioByteChannel's doRead() method encounter endless loop,so i have some confusion with it,if condition in it is allocHandle.lastBytesRead() <= 0,but then judge close state use allocHandle.lastBytesRead() < 0, when allocHandle.lastBytesRead() return 0,it seems it can not jump out of the circle,has anyone else encounter this situation? tks
    1643253867(1).png
    switlen
    @switlen
    1643254373(1).png
    switlen
    @switlen
    it's my fault,the endless loop is not there,in AbstractOioByteChannel it can jump out of the circle,but in the finally block it will call read() method again,trace the call chain,
    it finally call AbstractOioChannel.doBeginRead() method, and the readTask execute again and again, and the question is the AbstractOioChannel member variable readPending,it has been set true,but another execute it resetting false,it cause execute finally block again and again, I will show the screenshot next, give me some advice tks!
    switlen
    @switlen
    9313acefa5148cfda631d407422eef5.png
    switlen
    @switlen
    1643276973(1).png
    switlen
    @switlen
    I found isInputShutdown() always return false,it's impl is RxtxChannel.isInputShutdown(),memer variable open is still true,because this channel has not been closed,I will try something else
    croudet
    @croudet

    Hello, I have some troubles with tc-native:

    Failed to load any of the given libraries: [netty_tcnative_linux_x86_64, netty_tcnative_linux_x86_64_fedora, netty_tcnative_x86_64, netty_tcnative]","logger":"com.onespan.billinginvoicereports.config.HttpClientConfiguration","thread":"main","exception":"java.lang.IllegalArgumentException: Failed to load any of the given libraries: [netty_tcnative_linux_x86_64, netty_tcnative_linux_x86_64_fedora, netty_tcnative_x86_64, netty_tcnative]

    Epoll loads fine but not openssl, tc-native jar is there, I am using eclipse-temurin:17.0.2_8-jdk-focal, netty 4.1.73 and tc-native v2.0.48

    1 reply
    Richard
    @konsultaner
    Hi, I need some help to zero-copy a ByteBuf into a FileChannel. I kind of thought that will do, but it doesn't:
    public static void zeroCopyFile(FileChannel targetChannel, ByteBuf content) {
          targetChannel.transferFrom(new ByteBufChannel(content), 0, content.readableBytes());
    }
    
    private static class ByteBufChannel implements ReadableByteChannel {
            private final ByteBuf byteBuf;
    
            public ByteBufChannel(ByteBuf byteBuf) {
                this.byteBuf = byteBuf;
            }
    
            @Override
            public boolean isOpen() {
                return true;
            }
    
            @Override
            public void close() throws IOException {
            }
    
            @Override
            public int read(ByteBuffer dst) throws IOException {
                int byteBufReadableBytes = byteBuf.readableBytes();
                dst.put(byteBuf.nioBuffer());
                return byteBufReadableBytes - byteBuf.readableBytes();
            }
        };
    Benoit TELLIER
    @chibenwa

    Hello Netty!

    We are currently in the process of porting the Apache James server to Netty 4 and encounters some problems down the way. Those goes beyond my current understanding of Netty, thus is it OK if I ask for help here?

    Currently I encounter issues with SMTP pipelining: the client sends all the SMTP requests into a single network hop.

            Socket client = new Socket(bindedAddress.getAddress().getHostAddress(), bindedAddress.getPort());
            buf.append("HELO TEST");
            buf.append("\r\n");
            buf.append("MAIL FROM: <test@localhost>");
            buf.append("\r\n");
            buf.append("RCPT TO: <test2@localhost>");
            buf.append("\r\n");
            buf.append("DATA");
            buf.append("\r\n");
            buf.append("Subject: test");
            buf.append("\r\n");
            buf.append("\r\n");
            buf.append("content");
            buf.append("\r\n");
            buf.append(".");
            buf.append("\r\n");
            buf.append("quit");
            buf.append("\r\n");
    
            OutputStream out = client.getOutputStream();
            out.write(buf.toString().getBytes());
            out.flush();

    CF https://github.com/apache/james-project/blob/634be36800422259e40121248fb0d07475024a9c/server/protocols/protocols-smtp/src/test/java/org/apache/james/smtpserver/SMTPServerTest.java#L1887

    The way DATA is handled is that it adds a handler prior the core SMTP handler: https://github.com/chibenwa/james-project/blob/ffc0d4a8b22508b8f5b58594d14041d1f6bc3acf/protocols/netty/src/main/java/org/apache/james/protocols/netty/NettyProtocolTransport.java#L162

            channel.pipeline().addBefore(eventExecutors, HandlerConstants.CORE_HANDLER, "lineHandler" + lineHandlerCount, new LineHandlerUpstreamHandler(session, overrideCommandHandler));

    Everything works fine if the core handler is running on the event loop, however once we switch it to a distinct executor, the pipeline modification is no longer applied and the subsequent message content is interpreted as SMTP commands as it the line handler was not there.

    CF https://github.com/chibenwa/james-project/blob/ffc0d4a8b22508b8f5b58594d14041d1f6bc3acf/protocols/netty/src/main/java/org/apache/james/protocols/netty/AbstractChannelPipelineFactory.java#L78

            // Fails
            pipeline.addLast(eventExecutorGroup, HandlerConstants.CORE_HANDLER, createHandler());
            // Succeed
            pipeline.addLast(HandlerConstants.CORE_HANDLER, createHandler());

    Is there any way to modify the pipeline 'synchronously' from outside the event loop?

    Thanks in advance.

    Benoit TELLIER
    @chibenwa

    Edit: I found a walkaroud this issue by modifying the core handler to allow overrides of its behaviour, thus not requiring any further modifications of the pipeline. At the price of a not that costly refactoring.

    I think I will head toward encapsulating all those behaviors modification into a single handler, thus getting read of all that thread ordering madness.

    Things might get bloodier for IMAP though as request decoding is done in a separate handler (request-parsing then request execution) thus overrides happens before request decoding...

    Leo
    @Leowuqunqun
    I have a question, why did Netty choose NIO instead of AIO
    Stefan Koch
    @stevo-knievo

    Hi everyone!
    I just started with Netty -> DotNetty -> SpanNetty. Sorry about the C# code :)
    I'm working on a decoder for my custom protocol. Currently I'm using ReplayingDecoder<Enum> decoder.

    Now I'm running into IndexOutOfRangeException: readerIndex(4) + length(1280) exceeds writerIndex(10): PooledHeapByteBuffer(ridx: 4, widx: 10, cap: 256)

    So I tried to fix it as followed:

    if (input.ReadableBytes >= _messageFrame.Length)
    {
        _messageFrame.Payload = input.ReadBytes(_messageFrame.Length);
        Checkpoint(ProtocolDecoderState.ReadEpilog);    
    }

    which breaks my next switch case. case ProtocolDecoderState.ReadEpilog because I'm expected epilog to be a certain byte.

    Here is the whole code:

    protected override void Decode(IChannelHandlerContext context, IByteBuffer input, List<object> output)
    {
        switch (State)
        {
            case ProtocolDecoderState.ReadProlog:
                _messageFrame.Prolog = input.ReadByte();
                Checkpoint(ProtocolDecoderState.ReadMessageType);
                break;
            case ProtocolDecoderState.ReadMessageType:
                _messageFrame.SetMessageType(input.ReadByte());
                Checkpoint(ProtocolDecoderState.ReadLength);
                break;
            case ProtocolDecoderState.ReadLength:
                _messageFrame.Length = input.ReadShort();
                Checkpoint(ProtocolDecoderState.ReadPayload);
                break;
            case ProtocolDecoderState.ReadPayload:
                if (input.ReadableBytes >= _messageFrame.Length)
                {
                    _messageFrame.Payload = input.ReadBytes(_messageFrame.Length);
                    Checkpoint(ProtocolDecoderState.ReadEpilog);    
                }
                break;
            case ProtocolDecoderState.ReadEpilog:
                _messageFrame.Epilog = input.ReadByte();
                output.Add(ProtocolHelper.CovertByteFrameToMessage(_messageFrame));
                Checkpoint(ProtocolDecoderState.ReadProlog);
                _messageFrame = new MessageFrame();
                break;
            default:
                throw new InvalidDataException("Shouldn't reach here.");
        }
    }

    Do I need to use the ByteToMessageDecoder instead?
    Can I not use the ReadBytes(int length) in the ReplayingDecoder?

    Thanks!!

    1 reply
    Sushant Parkhi
    @sushantparkhi

    Hi, I am currently trying to make ECDSA related ciphers to work with TLS 1.2 in Spring Cloud Gateway (Spring Boot Parent 2.6.7 and Spring Cloud 2021.0.2). Here's the snippet of WebServerFactoryCustomizer

        @Bean
        public WebServerFactoryCustomizer<NettyReactiveWebServerFactory> customizer() {
            return factory -> factory.addServerCustomizers(httpServer -> httpServer.secure(sslContextSpec -> {
                try {
                    Ssl ssl = factory.getSsl();
                    KeyStore keyStore = KeyStore.getInstance(KeyStore.getDefaultType());
                    char[] keyStorePassword = ssl.getKeyStorePassword().toCharArray();
                    keyStore.load(resourceLoader.getResource(ssl.getKeyStore()).getInputStream(), keyStorePassword);
                    KeyManagerFactory keyManagerFactory = OpenSslCachingX509KeyManagerFactory
                            .getInstance(KeyManagerFactory.getDefaultAlgorithm());
                    keyManagerFactory.init(keyStore, keyStorePassword);
                    Http11SslContextSpec http11SslContextSpec = Http11SslContextSpec.forServer(keyManagerFactory)
                            .configure(sslContextBuilder -> {
                                sslContextBuilder.sslProvider(SslProvider.OPENSSL);
                                sslContextBuilder.ciphers(Arrays.asList(ssl.getCiphers()));
                                sslContextBuilder.protocols(ssl.getEnabledProtocols());
                                sslContextBuilder.trustManager(InsecureTrustManagerFactory.INSTANCE);
                                sslContextBuilder.clientAuth(ClientAuth.REQUIRE);
                            });
                    sslContextSpec.sslContext(http11SslContextSpec)
                            .handlerConfigurator(sslHandler -> {
                                sslHandler.setCloseNotifyReadTimeout(18000, TimeUnit.MILLISECONDS);
                                sslHandler.setHandshakeTimeout(19000, TimeUnit.MILLISECONDS);
                                SSLParameters sslParameters = sslHandler.engine().getSSLParameters();
                                sslParameters.setUseCipherSuitesOrder(false);
                                sslHandler.engine().setSSLParameters(sslParameters);
                            });
                } catch (UnrecoverableKeyException | IOException | CertificateException | KeyStoreException |
                         NoSuchAlgorithmException e) {
                    throw new RuntimeException(e);
                }
            }));
        }

    But when I try to connect using openssl s_client with ECDHE-ECDSA-AES128-GCM-SHA256 cipher the server returns an error with no shared ciphers, but I do have it in the configuration as

    server.ssl.ciphers=TLS_RSA_WITH_AES_128_GCM_SHA256,\
      TLS_RSA_WITH_AES_256_GCM_SHA384, \
      TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,\
      TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,\
      TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
    server.ssl.enabled-protocols=TLSv1.2

    This behavior was observed when I upgraded versions from Spring Boot 2.3.3.RELEASE and Spring Cloud Hoxton.SR7. Also, if I switch to JDK as a SSL Provider it is working as expected, this issue only occurs with OpenSSL as a provider. Any advice/suggestions would be of great help on fixing or correctly configuring it.

    jmcrawford45
    @jmcrawford45
    Does anyone have a working example of CertificateCompression? I've looked at the relevant PRs in netty/netty-tcnative#689 and netty/netty#12012, but there's not much guidance on how to configure existing compression algorithms. Everything makes sense until I get down to the OpenSslCertificateCompressionConfigBuilder.addAlgorithm call where I'm not clear what I should be doing to leverage existing algorithms.
    Chris Howell
    @chrishowell
    Ahoy maintainers - I wonder if somebody could please take a look at a PR I submitted to fix a compatibility issue? netty/netty#12427 - as a first time maintainer I don't have permission to run checks, but I've run them locally.
    1 reply
    Srikanth Jana
    @janasrikanth
    Hii
    io.netty.channel.unix.Errors$NativeIoException: readAddress(..) failed: Connection reset by peer
    Im getting above error for my springboot application, what could be the possible reason?
    Yann Simon
    @yanns
    Hi I'm using an handler with a state (private attribute) that is set on channelRead if the msg is a HttpRequest.
    Later, in write, I read this private attribute.
    This handler is initialized for each pipeline (it's not shared).
    My goal is to read something from the http headers (in the request) and to push it as http header in the response.
    The handler should save this state per HTTP request / response.
    It's working mostly fine, but sometimes, a HTTP response gets the HTTP header value from another request.
    With http pipeline, I understand that one handler instance can be re-used. Do we have any guarantee that one request is fully processed before the next one is handled? If not, how is it possible to save some state for each HTTP request / response?
    Hamado Dene
    @hamadodene

    Hi,
    We have seen errors in the logs, but we are no longer able to reproduce manually.
    Do you have any idea what the cause might be?
    The exception is:

    Jul 06, 2022 7:13:39 AM reactor.util.Loggers$Slf4JLogger error
    SEVERE: [9b7e395e-1, L:/10.219.105.156:4089 - R:/104.28.98.59:30389] Error finishing response. Closing connection
    java.lang.ArrayIndexOutOfBoundsException: Index 67108863 out of bounds for length 8
            at io.netty.buffer.PoolSubpage.allocate(PoolSubpage.java:92)
            at io.netty.buffer.PoolArena.tcacheAllocateSmall(PoolArena.java:165)
            at io.netty.buffer.PoolArena.allocate(PoolArena.java:134)
            at io.netty.buffer.PoolArena.reallocate(PoolArena.java:287)
            at io.netty.buffer.PooledByteBuf.capacity(PooledByteBuf.java:122)
            at io.netty.buffer.AbstractByteBuf.ensureWritable(AbstractByteBuf.java:333)
            at io.netty.handler.ssl.SslHandler.attemptCopyToCumulation(SslHandler.java:2334)
            at io.netty.handler.ssl.SslHandler.access$2800(SslHandler.java:168)
            at io.netty.handler.ssl.SslHandler$SslHandlerCoalescingBufferQueue.compose(SslHandler.java:2295)
            at io.netty.channel.AbstractCoalescingBufferQueue.remove(AbstractCoalescingBufferQueue.java:176)
            at io.netty.handler.ssl.SslHandler.wrap(SslHandler.java:817)
            at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1439)
            at io.netty.handler.ssl.SslHandler.decodeNonJdkCompatible(SslHandler.java:1246)
            at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1286)
            at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:510)
            at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:449)
            at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:279)
            at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
            at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
            at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
            at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
            at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
            at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
            at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
            at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:800)
            at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480)
            at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378)
            at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
            at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
            at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
            at java.base/java.lang.Thread.run(Thread.java:829)

    Our netty version is 4.1.75-Final

    Thanks in advance