Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
Tuncay
@tuncayofficial
markdown check
cool
jyanJing
@jyanJing
freshly baked Finagle summer update 😃 : https://finagle.github.io/blog/2021/08/03/summer-review/
Alessandro Vermeulen
@spockz
Can I directly refer to a Resolver from a path? /$/ only works with namers iirc
Alessandro Vermeulen
@spockz
@jyanJing i see something mentioned about offload control. Where can i read up on that?
2 replies
Dan Di Spaltro
@dispalt
Is AsyncStream more like a LazyList than say like an Akka Stream?
1 reply
Alexey Shcherbakov
@fuCtor
Hi! I have some question about Finagle sources. Why in this https://github.com/twitter/finagle/blame/develop/finagle-http/src/main/scala/com/twitter/finagle/http/filter/Cors.scala#L85 place are used addUnsafe instead of setUnsafe? In case, when we proxy some response with third party headers, we can get an error CORSMultipleAllowOriginNotAllowed. But if replace addUnsafe on setUnsafe we can avoid this.
Alessandro Vermeulen
@spockz
The enclosing method is a set as well. So I can definitely imagine this should be a set
Maybe create an issue and pr? The folks are pretty quick at accepting these kind of fixes
Roger
@rogern

Hi, we're using finagle-mysql on AWS to talk to Aurora/MySQL in RDS. Recently we've start seeing regular warnings in the logs:

WARN c.t.f.m.RollbackFactory - rollback failed when putting service back into pool, closing connection com.twitter.finagle.ChannelWriteException: com.twitter.finagle.ChannelClosedException: java.nio.channels.ClosedChannelException

It looks to me like there are related to availability because:

Caused by: com.twitter.finagle.ChannelClosedException: java.nio.channels.ClosedChannelException. Remote Info: Not Available ... 61 common frames omitted Caused by: java.nio.channels.ClosedChannelException: null

We're using 10 connections in the pool per instance and since this hasn't led to a degraded system I assume the connections are re-created in the pool.
Anyway, I wonder if this is something we should worry about and if so if there are anything we can try to change to mitigate it.
Any feedback is welcome, thanks!

1 reply
Alessandro Vermeulen
@spockz
Is there a way to set the label of a client when using the methodbuilder? (Aside from resorting to inserting a module that rewrites the label param downstream?)
7 replies
jyanJing
@jyanJing
Alessandro Vermeulen
@spockz

finagle-http, finagle-thriftmux: introduce client.withSni() API. Use this api to specify an SNI hostname for TLS clients. a8ec457b

Will finagle (or netty actually) by default still pass the hostname of the instance as the sni parameter?

Moses Nakamura
@mosesn
@spockz yes! we’re preserving existing behavior. you can see that we only change the behavior if you proactively configure that parameter: https://github.com/twitter/finagle/commit/a8ec457be41271dbfb27f99cdbbe6465961c78c0#diff-bacdd7218926d6c808f04e9281978a3002ebed289d17b19ccd4d28dcee2980c1
Islam Faisal
@decltypeme

Hi, I want to create a finagle HTTP client that would return a custom case class Response (like what we do in finatra controllers enabled by jackson); something like

val customTwitter: Service[Request, MyCustomResponse] = Http.client
  .withTransport.tls("twitter.com")
  .newService("twitter.com:443")

the closest I found was: https://finagle.github.io/featherbed/doc/06-building-rest-clients.html
but it seems that this library has not been updated for 4 years. Could someone point me where to look?

Moses Nakamura
@mosesn
@decltypeme I don’t think what you’re describing exists outside of featherbed. however, if you would like a similar experience, you can provide a Filter that converts from HttpResponse to a case class, and do the jackson handling yourself.
Islam Faisal
@decltypeme
Thanks, that is helpful. Do you know of a maintaned branch for featherbed?
Also, would a diff with the filter you suggested be of intrest to the finagle community or should i just add it in our code?
Moses Nakamura
@mosesn
@decltypeme I think doing it in a truly general way would be pretty complicated (it’s tantamount to implementing featherbed :)). I would probably recommend first trying it out as a standalone project, like featherbed, and then we can consider graduating it to Finatra after we’re satisfied with it. on the other hand, doing a one-off filter for your use case should be relatively easy, so it might be good to start with that and see what your appetite is for starting a new open source project
3 replies
I don’t know of a maintained branch of featherbed, sorry!
Alessandro Vermeulen
@spockz
@mosesn thanks. I saw the patch but wanted to make sure another change somewhere else didn’t change default somehow
@decltypeme in our sdk we have a simple filter than just evokes the Jackson objectmapper for trying to convert the response to the appropriate class. If it fails it wraps the exception and the original http response in a new exception and passes that along.
Ushhud Khalid
@Ushhud

Hey all, I’m trying to have a background thread execute some task (task: => Future[A]) periodically. If it’s relevant,task calls some downstream under the hood and does some processing.

Initial approach was to do something like:

Future.sleep(initialDelay).flatMap { _ =>
  run()
}

…

def run() = task.onSuccess {
  Future.sleep(refreshInterval).map(_ => run())
}

This works fine for scheduling the futures properly (e.g, only when task is complete does the next one get scheduled), however it the task itself ends up running on the finagle netty threads rather than a background job. I tried to schedule the task execution on a FuturePool to try to get around that:

val pool: ExecutorServiceFuturePool = FuturePool.interruptible(
  Executors.newFixedThreadPool(5, new NamedPoolThreadFactory("BackgroundScheduler-ThreadPool", true))
)

pool { 
  logger.info(“Entered pool”)
  Future.sleep(initialDelay).flatMap { _ =>
    run()
  }
}

I noticed that altho “Entered pool” would run on the BackgroundScheduler-ThreadPool, we had thread switches once we entered a Future.sleep (to the Netty Timer thread) or any Future (to Netty finagle) in the task. I’m wondering if there’s any way to have these Futures run in the thread pool we ask for, rather than switching back on the Netty threads. I read in a few places (possibly old?) that this kind of thread scheduling is only possible at a global level, so we’d have to swap out the LocalScheduler with something else — but wanted to confirm. Thanks!

6 replies
Ken Kaizu
@krrrr38
Hi, firstly thank you developing finagle. We use it in production.
I have a question abount jackson. Currently twitter/util and finagle depends on jackson 2.11. Do you have any plan to update jackson to 2.12? Maybe as you know, jackson-scala-module 2.11 DefaultScalaModule depends on jackson 2.11, so we cannot update any libraries which depends on jackson 2.12. I guess that util-jackson make it possible easily. Thanks.
3 replies
Philippe Laflamme
@plaflamme
Hello, can anyone here provide context into this particular twitter/finagle@4f1dbf5? We can see the interface in twitter/util twitter/util@d9c900a, but we cannot find any implementation.
1 reply
Alessandro Vermeulen
@spockz
Any ideas @cacoco ?
Alessandro Vermeulen
@spockz
Okay so maybe it works for stats. I was checking the label in thrown failures. Let me recheck
Alessandro Vermeulen
@spockz
Does anyone have any experience with kTLS and netty/finagle?
sachins301
@sachins301
Hi Im just starting out with a project and was wondering what makes finagle different (or better) from akka-http.
Alessandro Vermeulen
@spockz
Imho there are better load balancers and failure accrual settings.
4 replies
Alessandro Vermeulen
@spockz

Besides. The DTAB in finagle is really powerful. And in general I really like the decoupling interfaces. You can hook in any service discovery mechanism you want.

The main thing akka-http has going over finagle is the ability to do a broadcast and just get the first response to come back or get them all

Oh and gRPC support
sachins301
@sachins301
Thanks, What about with respect to Scala 3. Have you released for it? is it compatible and are there issues as of now?
Alessandro Vermeulen
@spockz
The latest 2.13 releases should be compatible I think. @mosesn any plans on going to scala 3 at Twitter?
sachins301
@sachins301
Would you know which dependency Im missing here. Im using bazel.
This symbol is required by 'trait com.twitter.server.TwitterServer'.
Make sure that type Logging is in your classpath and check for conflicting dependencies with `-Ylog-classpath`.
A full rebuild may help if 'TwitterServer.class' was compiled against an incompatible version of <none>.logging.
object Microservice extends TwitterServer {
Alessandro Vermeulen
@spockz
until-logging is my gues
1 reply
sachins301
@sachins301
I do have this version @maven//:com_twitter_util_logging_2_12_21_8_0
Sudheer Aedama
@avsudheer_twitter
hey everyone, first time user here. I have been banging my head over this exception over the past couple of days and nights com.twitter.finagle.ChannelClosedException: null from service.
Any pointers on how to debug this issue
I have a service running on localhost, that I am able to connect directly via Thrift but not via finagle.Thrift.client. I debugged the issue with breakpoints but no luck :(
Just for reference, the full stacktrace looks something like this:
[error]     at com.twitter.finagle.netty4.transport.ChannelTransport$$anon$2.channelInactive(ChannelTransport.scala:176)
[error]     at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:262)
[error]     at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:248)
[error]     at io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:241)
[error]     at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelInactive(CombinedChannelDuplexHandler.java:418)
[error]     at io.netty.channel.ChannelInboundHandlerAdapter.channelInactive(ChannelInboundHandlerAdapter.java:81)
[error]     at io.netty.channel.CombinedChannelDuplexHandler.channelInactive(CombinedChannelDuplexHandler.java:221)
[error]     at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:262)
[error]     at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:248)
[error]     at io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:241)
[error]     at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelInactive(CombinedChannelDuplexHandler.java:418)
[error]     at io.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:389)
[error]     at io.netty.handler.codec.ByteToMessageDecoder.channelInactive(ByteToMessageDecoder.java:354)
[error]     at io.netty.channel.CombinedChannelDuplexHandler.channelInactive(CombinedChannelDuplexHandler.java:221)
[error]     at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:262)
[error]     at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:248)
[error]     at io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:241)
[error]     at io.netty.channel.DefaultChannelPipeline$HeadContext.channelInactive(DefaultChannelPipeline.java:1405)
[error]     at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:262)
[error]     at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:248)
[error]     at io.netty.channel.DefaultChannelPipeline.fireChannelInactive(DefaultChannelPipeline.java:901)
[error]     at io.netty.channel.AbstractChannel$AbstractUnsafe$8.run(AbstractChannel.java:819)
[error]     at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
[error]     at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
[error]     at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:497)
[error]     at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
[error]     at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
[error]     at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
[error]     at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
[error]     at com.twitter.finagle.util.BlockingTimeTrackingThreadFactory$$anon$1.run(BlockingTimeTrackingThreadFactory.scala:23)
[error]     at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
[error]     at java.base/java.lang.Thread.run(Thread.java:834)
I used scrooge to compile thrift idl files, and created a client using:
com.twitter.finagle.Thrift.client
        .servicePerEndpoint[MyThriftService.ServicePerEndpoint](
          dest = com.twitter.finagle.Resolver.eval("localhost:5610"),
          label = "awesome4s_thrift_client"
        )
Sudheer Aedama
@avsudheer_twitter
appreciate your thoughts :bow: :pray:
Sudheer Aedama
@avsudheer_twitter
I tried all versions of finagle available; tried all the settings I can find on com.twitter.finagle.Thrift.client. No luck so far :(
Alessandro Vermeulen
@spockz
@avsudheer_twitter interesting. I also saw that issue when a tls handshake is reset at a weird moment
Sudheer Aedama
@avsudheer_twitter
@spockz :pray: so good to see your message
Alessandro Vermeulen
@spockz
Does finagle have a warmup functionality? I did see something in netty.
@avsudheer_twitter btw that is with http. We don’t use thrift
Sudheer Aedama
@avsudheer_twitter
This issue is with using Thrift, we don't use http unfortunately.
Alessandro Vermeulen
@spockz
What is wrong with thrift?
Sudheer Aedama
@avsudheer_twitter
This code (which doesn't use Scrooge/Finagle) works without an issue:
val socket   = new TSocket("127.0.0.1", 5610)
socket.open()
val protocol = new TBinaryProtocol(socket)
val client   = new MyThriftService.Client(protocol)