Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
John Schmidt
@schmee
perhaps! Just for some background on why I'm interested in this: we have built a couple of TCP clients with Netty, all with very protocols (basically using Netty's LengthFieldBasedFrameDecoder covers us entirely). However, we find ourselves re-inventing reconnection handling, load balancing, retries... and when I saw Finagle's Client readme I though "THIS is what I've been looking for" :D
random thought: would it make sense to ship a generic TCP client in Finagle, where you specify some stuff like the framing (or maybe even access the underlying Netty pipeline directly), instead of making a full-blown custom client?
also, we are not a Scala shop (we use Java/Kotlin) so it is not possible for us to use Scala in our code base unfortunately
Moses Nakamura
@mosesn
it should be pretty easy to call scala code from java or kotlin
Alessandro Vermeulen
@spockz
@mosesn regarding the java friendliness, we need to put the default values of methods in constants so they can be easily used from Java
Moses Nakamura
@mosesn
sure! if there’s a specific case you’re thinking of, please make a PR
Alessandro Vermeulen
@spockz
Of course :)
Taro L. Saito
@xerial
Hi. I'm trying to use streaming mode with Finagle client, but it seems the chunk size is limited to 16KB. Can we increase such a chunk size limit? I've tried withStreaming(... size), withMaxResponseSize(...), etc. but none of them worked
Taro L. Saito
@xerial
I'm using Finagle 19.8.0
Moses Nakamura
@mosesn
maybe try configuring MaxFrameSize?
are you speaking http/2? or http/1.1?
Moses Nakamura
@mosesn
and how do you know it’s the chunk size limit? what behavior are you seeing?
Taro L. Saito
@xerial
http/1.1. I actually checked Chunk.content.length
Moses Nakamura
@mosesn
withStreaming(... size) should do it then
Taro L. Saito
@xerial
I've tried that too, but it didn't change the chunk size. For each iteration, Chunk.content can have various sizes up to 16KB.
If I enable withHttp2, chunk size becomes 8kb
this seems to be the API you’re running into, but I don’t know why it’s 16KB. I would expect it to be 8KB. this is on a request that you’re sending, or a response that you’re receiving? you’re the client, right?
Taro L. Saito
@xerial
Yes. It's a client side issue. response size is somehow restricted. I'm accessing a server that returns chunked streams of json data
Moses Nakamura
@mosesn
response size is not negotiated between the client and the server. I think this is a problem on your server-side, not the client.
Taro L. Saito
@xerial
That can be the case. Thanks!
Moses Nakamura
@mosesn
(it’s negotiated in http/2, which is why it drops to 8KB in http/2)
np!
Taro L. Saito
@xerial
The server is implemented in jersey, and whose packet size is 16KB by default http://solutionhacker.com/rest-use-streamingout-return-big-content-chunk/
Moses Nakamura
@mosesn
woohoo!
Taro L. Saito
@xerial
:+1:
Taro L. Saito
@xerial
By the way, I'm implementing a simple web framework that uses Scala functions as an IDL for defining APIs https://wvlet.org/airframe/docs/airframe-http.html
I hit the issue when supporting streaming with Reader[Buf]
Moses Nakamura
@mosesn
very cool! I saw that airframe is on the list of companion projects for finagle: http://twitter.github.io/finagle/
Taro L. Saito
@xerial
Yeah. By using finagle as a backend HTTP server, it's quite convenient to quickly implement a web server
Moses Nakamura
@mosesn
:thumbsup:
Clement Emmanuel
@clementemmanuel
Hello, I have service running with an exposed endpoint that when invoked will ultimately make an async call to update an s3 instance. In some particular cases when I hit the endpoint I get 500 back, however the logs from my application layer will indicate success of all the operations, and the database is in fact updated. I can see from finagle metrics that the 500 was observed, but there are no associated logs for this internal server error. I have logback-core/classic on the classpath and i can see that this binding is being used by slf4j at my application level (hence my success logs). Any ideas on how I can surface these logs from finagle if they do in fact exist. I'm able to recreate this "error" by setting the response timeout (via withResponseTimeout) to a small enough value
Christopher Coco
@cacoco
@clementemmanuel Finagle uses JUL (java.util.logging) through a thin scala wrapper: util/util-logging. You need to make sure you’re properly bridging JUL to Logback, per: https://www.slf4j.org/legacy.html
Moses Nakamura
@mosesn
for what it’s worth, we export much more information via metrics rather than via logs. if you find a hole in our logging instrumentation, please make a PR! we’d love to improve it.
Hamdi Allam
@hamdiallam
hey folks, we released the new version of finagle & friends (19.9.0), and it has some new 2.13 stuff too!
Clement Emmanuel
@clementemmanuel

@cacoco Thanks! So I have the slf4j bridge set-up and I can see some internal logs.

@mosesn To your point I don't see any exception logs (specifically in my case of request not being within timeout), am I right to assume things like the default com.twitter.finagle.service.TimeoutFilter don't actually log anything, they just chain the failed future through the filter composition? Maybe I misunderstand the paradigm with filters, but at some point does some process actually encounter the exception and act on it? Or is the suggested usage to have your own implementation of instances of Filter if you want behaviours like logging or other sideffects

ole-magnus
@ole-magnus
Thanks for the release! Great stuff :)
Christopher Coco
@cacoco
@clementemmanuel depending on your server configuration most exceptions in Finagle will usually hit the DefaultMonitor and be logged (unless you configured your server with a NullMonitor). Or yes, you can install a logging Filter to capture.
The TimeoutFilter simply throws a TimeoutException which is captured in a “failed” Future result. You’d get this result if you awaited this Future.
Dejan Lokar
@dejanlokar1
Hi, I am working with an endpoint that has an account id as a subdomain (e.g. accountid.samplehost.com). What is the recommended approach here? I was thinking of using a method builder to instantiate a Finagle client for every account id and caching it for future uses.
Christopher Coco
@cacoco
The Finagle MethodBuilder does not instantiate multiple clients. It's a single Finagle client underneath. It's for filtering calls to the server differently per logical (or RPC) method.
Dejan Lokar
@dejanlokar1
@cacoco thanks, so in my case it is required to init a complete Finagle client per account id.
Christopher Coco
@cacoco
Yes, a client per destination. Just be careful of your cache key....don't use a Java URL.
Jorge Creixell
@jcreixell
Is there a chance that this might change in the future? twitter/finagle@bf09dd4
we are trying to build a twirp implementation on top of finagle and it would be great to be able to use h2c for that https://github.com/soundcloud/twinagle
Moses Nakamura
@mosesn
@jcreixell probably not, but you just need to have one request that doesn’t have a body for the h2c upgrade to succeed
we considered building in support for https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/OPTIONS to finagle to use as a dedicated “h2c upgrade” request, but didn’t end up doing it. but you can still use OPTIONS this way, it should be safe.
Jorge Creixell
@jcreixell
@mosesn thanks for letting me know, we will check if that could work for us. I assume we need to send and OPTIONS request every time a connection is lost and re-established
Jorge Creixell
@jcreixell
actually I think that prior-knowledge will work in our case, so all good :)
Alessandro Vermeulen
@spockz
Is there a way to steer finagle to let it connect to local instances first? Eg with the ewma load balancer s?