Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
Christopher Coco
@cacoco
@bollmann either curl cuts it or your server configuration cuts it (what’s the request timeout configured by your server) or something between the curl client and your server cuts it after a minute (is the request going through some type of proxy) but yes the interrupt happens to cancel unnecessary work in your Finagle server since no one is listening anymore to receive your response.
bollmann
@bollmann

@cacoco: The server config is as follows:

Http.server
      .withTransport.tls(sslConfiguration)
      .withMaxRequestSize(StorageUnit.fromBytes(settings.maxRequestSize.toBytes))
      .withAdmissionControl.concurrencyLimit(settings.maxConcurrencyLimit, settings.maxNumberOfWaitingClients)

So I thought that would mean that there is no request timeout imposed by the finagle server by default. Also, my curl client connects via an AWS ELB to the finagle server (maybe this AWS ELB drops the connection after one minute?)

Christopher Coco
@cacoco
@bollmann very likely the ELB
A minute is a really long time
By default neither the Finagle server or curl have a request timeout so what’s left is the ELB.
bollmann
@bollmann
yes, I'll double check the load-balancer now. I doubt that curl drops the connection itself because even when permitting curl to take longer with --max-time it also drops the connection after one minute
@cacoco exactly. Thanks for making me realise this! :-)
bollmann
@bollmann
@cacoco: thanks, I just found that my ELB's idle timeout was set to just 60 seconds. So that exactly explains the behavior :-)
@cacoco thanks for debugging my head!
Christopher Coco
@cacoco
@bollmann no problem, happy to be a duck
sukanta ghosh
@sontu23_twitter
Hey, I am new to Scala and trying out Finagle, I am stuck with the following configuration where the client is always returning 400. Is there anything missing in the configuration? Some help is appreciated.
 val client: Service[http.Request, http.Response] = Http.client.newService("api.ratesapi.io:443")
  val request = http.Request(http.Method.Get, "/api/latest")
  request.host = "https://api.ratesapi.io"
  request.headerMap.add("Content-Type", "application/json")
  val response: Future[http.Response] = client(request)
  Await.result(response.onSuccess { resp: http.Response => println("Get success" + resp.status) })
5 replies
Pegerto
@pegerto
Hi,
I am trying to understand how to configure a proxy in a http client, should be simple.
      val clientBuilder = Http.client
        .withLabel(settings.label)
        .withRequestTimeout(settings.requestTimeout)
        .withTransport.connectTimeout(settings.connectTimeout)
        .withHttpStats
        .withTransport.httpProxyTo("api-aa:443")
        .withStatsReceiver(TalonStatsReceiver())
        .withStack(x => x.remove(ClientNackFilter.module.role))
        .withHttp2

      val secureClient = if (false) clientBuilder.withTlsWithoutValidation else clientBuilder
      secureClient.newService("localhost:9222")
my proxy is http, but the end connection is https, so I been stuck trying to undertand how can send a connect https:// so the tls handshake happens on the proxy
Instead it connects to the proxy and send a clear http request to an https endpoint 443
Alexey Shcherbakov
@fuCtor
Hi, I know that ThriftMux client support partitioning, finagle-partitioning package exists. How can I use partitioning for Http client or this client doesn't support it?
Moses Nakamura
@mosesn
@pegerto sorry for the delayed response. I don’t think that’s possible today in Finagle, unfortunately. we would potentially be open to a PR to add that though!
it’s a bit tricky though–we would need to figure out how to move the TLS handshake to be after the CONNECT
@fuCtor it doesn’t currently support it, but we would be open to a PR to add support for it! I think it should be relatively straightforward
bollmann
@bollmann

Hi, finagle experts. Suppose, I've got two future computations that return values of different types. Is there a way to easily compose these two computations in parallel? Currently, my solution is the following

val computation1: Future[A] = ...
val computation2: Future[B] = ...

val runInParallel: Future[(A, B)] = Future
  .collect(computation1, computation2)
  .map { case Seq(c1, c2) => (c1.asInstanceOf[A], c2.asInstanceOf[B]) }

However, this looks quite clunky in my opinion, so I wonder whether there is a better way to launch computations in parallel and being able to access the results at their concrete types afterwards.

If anybody knows, I'd appreciate a pointer :-) Thanks!

available both as Future.join[A, B](Future[A], Future[B]): Future[(A, B)] as well as Future[A]#join[B](Future[B]): Future[(A, B)]
Vladimir Ivanovskiy
@vi-p4f
Hi! I created simple finatra http service and see "Elevated metric creation requests" lint warning in admin interface. I tried to find why it complains and see that some metric counters are incremented with every HTTP request served. Example: https://github.com/twitter/finagle/blob/develop/finagle-netty4-http/src/main/scala/com/twitter/finagle/netty4/http/handler/BadRequestHandler.scala#L29 Can you give me an advice what should I do about high metric creation requests or it is known feature which should be ignored?
15 replies
Vladimir Ivanovskiy
@vi-p4f
Hi again, one more question. Is the use of heapster lib (https://github.com/mariusae/heapster) in admin UI recommended/supported way to make heap profiles?
1 reply
Alessandro Vermeulen
@spockz
Is there already a bit of code that I can use to transform a Reader[Buf] into an AsyncStream or Event of Buf?
2 replies
Alessandro Vermeulen
@spockz
Or a way to convert a Stream into a AsyncStream
2 replies
There is one for Seq
Tuncay
@tuncayofficial
hey!
markdown check
cool
jyanJing
@jyanJing
freshly baked Finagle summer update 😃 : https://finagle.github.io/blog/2021/08/03/summer-review/
Alessandro Vermeulen
@spockz
Can I directly refer to a Resolver from a path? /$/ only works with namers iirc
Alessandro Vermeulen
@spockz
@jyanJing i see something mentioned about offload control. Where can i read up on that?
2 replies
Dan Di Spaltro
@dispalt
Is AsyncStream more like a LazyList than say like an Akka Stream?
1 reply
Alexey Shcherbakov
@fuCtor
Hi! I have some question about Finagle sources. Why in this https://github.com/twitter/finagle/blame/develop/finagle-http/src/main/scala/com/twitter/finagle/http/filter/Cors.scala#L85 place are used addUnsafe instead of setUnsafe? In case, when we proxy some response with third party headers, we can get an error CORSMultipleAllowOriginNotAllowed. But if replace addUnsafe on setUnsafe we can avoid this.
Alessandro Vermeulen
@spockz
The enclosing method is a set as well. So I can definitely imagine this should be a set
Maybe create an issue and pr? The folks are pretty quick at accepting these kind of fixes
Roger
@rogern

Hi, we're using finagle-mysql on AWS to talk to Aurora/MySQL in RDS. Recently we've start seeing regular warnings in the logs:

WARN c.t.f.m.RollbackFactory - rollback failed when putting service back into pool, closing connection com.twitter.finagle.ChannelWriteException: com.twitter.finagle.ChannelClosedException: java.nio.channels.ClosedChannelException

It looks to me like there are related to availability because:

Caused by: com.twitter.finagle.ChannelClosedException: java.nio.channels.ClosedChannelException. Remote Info: Not Available ... 61 common frames omitted Caused by: java.nio.channels.ClosedChannelException: null

We're using 10 connections in the pool per instance and since this hasn't led to a degraded system I assume the connections are re-created in the pool.
Anyway, I wonder if this is something we should worry about and if so if there are anything we can try to change to mitigate it.
Any feedback is welcome, thanks!

1 reply
Alessandro Vermeulen
@spockz
Is there a way to set the label of a client when using the methodbuilder? (Aside from resorting to inserting a module that rewrites the label param downstream?)
7 replies
jyanJing
@jyanJing
Alessandro Vermeulen
@spockz

finagle-http, finagle-thriftmux: introduce client.withSni() API. Use this api to specify an SNI hostname for TLS clients. a8ec457b

Will finagle (or netty actually) by default still pass the hostname of the instance as the sni parameter?

Moses Nakamura
@mosesn
@spockz yes! we’re preserving existing behavior. you can see that we only change the behavior if you proactively configure that parameter: https://github.com/twitter/finagle/commit/a8ec457be41271dbfb27f99cdbbe6465961c78c0#diff-bacdd7218926d6c808f04e9281978a3002ebed289d17b19ccd4d28dcee2980c1
Islam Faisal
@decltypeme

Hi, I want to create a finagle HTTP client that would return a custom case class Response (like what we do in finatra controllers enabled by jackson); something like

val customTwitter: Service[Request, MyCustomResponse] = Http.client
  .withTransport.tls("twitter.com")
  .newService("twitter.com:443")

the closest I found was: https://finagle.github.io/featherbed/doc/06-building-rest-clients.html
but it seems that this library has not been updated for 4 years. Could someone point me where to look?

Moses Nakamura
@mosesn
@decltypeme I don’t think what you’re describing exists outside of featherbed. however, if you would like a similar experience, you can provide a Filter that converts from HttpResponse to a case class, and do the jackson handling yourself.
Islam Faisal
@decltypeme
Thanks, that is helpful. Do you know of a maintaned branch for featherbed?
Also, would a diff with the filter you suggested be of intrest to the finagle community or should i just add it in our code?
Moses Nakamura
@mosesn
@decltypeme I think doing it in a truly general way would be pretty complicated (it’s tantamount to implementing featherbed :)). I would probably recommend first trying it out as a standalone project, like featherbed, and then we can consider graduating it to Finatra after we’re satisfied with it. on the other hand, doing a one-off filter for your use case should be relatively easy, so it might be good to start with that and see what your appetite is for starting a new open source project
3 replies
I don’t know of a maintained branch of featherbed, sorry!
Alessandro Vermeulen
@spockz
@mosesn thanks. I saw the patch but wanted to make sure another change somewhere else didn’t change default somehow
@decltypeme in our sdk we have a simple filter than just evokes the Jackson objectmapper for trying to convert the response to the appropriate class. If it fails it wraps the exception and the original http response in a new exception and passes that along.