Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
Moses Nakamura
@mosesn
thanks! and if you wouldn’t mind, if you could publish a local version of the updated Finagle and double check that it solves your problem, that would be great!
Alessandro Vermeulen
@spockz
@mosesn I tried building such an adapter I faced some challenges: controlling wgeee the spans were created. I think this is fixed now by the recreation of the span in the requeuefilter. Then there was the mismatch berwwwb the Tracer api letting the trace is be set externally which is not available in OT. And the third thing was cintrollling which attributes are placed on spans
hllrsr
@hllrsr
@mosesn In the process of compiling finagle with zk on version 3.5.5 I've got some type conversion errors. The latest version that didn't show any errors, only warnings was 3.5.4-beta.
How should I proceed in this case? Should I investigate why this errors are being caused or should I go with the 3.5.4-beta version?
Also, going through change logs they didn't mentioned any breaking change being introduced on 3.5.5.
hllrsr
@hllrsr
Those conversion errors were fixed on 3.5.6. I've manged to compile it on this version.
hllrsr
@hllrsr
I've published a local version and it solve my issue! I've also opened the PR on finagle. Thanks for the directions @mosesn :kudos:
Moses Nakamura
@mosesn
@hllrsr nice work! I commented on the PR
Moses Nakamura
@mosesn
hey folks, we did the January release today. enjoy 21.1.0!
TP Diffenbach
@tpdi
I have a question about Finagle Contexts and com.twitter.util.Local . Documentation suggests both are replacements for ThreadLocals across asynchronous Futures, but I'm not quite sure how to get there. Contexts cannot be updated, only let in order to change their values for execution of a function passed in to let. Locals can be updated, and the updated value is visible to child threads. But updates in child threads are not visible to siblings. That's strictly right in asynchronous programming I suppose, but I'm not sure how to emulate something like the synchronous process A updates ThreadLocal X to 1, process B reads 1 from ThreadLocal X.
In my case, I want parallel Futures in one request to share the same database transaction. If I stick the transaction in a LocalContext or a Context, and then create a bunch of async Futures, they all see the same transaction, which is great! But if transactions are started lazily by the first thread to need to, how can I ensure the other threads see that same transaction? And that different requests/contexts see only their own transaction? Thanks.
Christopher Coco
@cacoco

Does this help at all:

Finagle explicitly manages them for you across threads and execution contexts such as Future composition, FuturePools, Timers, and in some cases — across the client/server boundary.

(https://twitter.github.io/finagle/guide/Contexts.html)

let is a way of scoping a value in a closure
TP Diffenbach
@tpdi
I suppose I could add an extra level of indirection, allowing threads to write to mutable state in the context object, but I'm hoping for something cleaner
Christopher Coco
@cacoco
foo.let(something) { //everthing here sees the something value }
if you’re trying to coordinate state across a lot of futures...
TP Diffenbach
@tpdi
Right, I understand let. But I need updates within "//everthing here sees the something value" to be seen by all threads/Futures within that block
Christopher Coco
@cacoco
right we do this in places by mutating a value held in the context
(like how we propagate slf4j MDC values in Finatra)
TP Diffenbach
@tpdi
So I have in synchronous code some database access object that does a if (myThreadLocal.get() == null) myThreadLocal.set(new Transaction); and then subquestnt calls to it in that thread use the transaction
Ok, so you do use an extra level of indirection
the context is fixed, but the value it holds is mutable
Christopher Coco
@cacoco
exactly
TP Diffenbach
@tpdi
com.twitter.util.Local does almost what I want; in order to replace multiple ThreadLocals, I'll need to establish a key or index for each, com.twitter.util.Local does that, using a counter in the companion object that serves as the array index
I was hoping someone else had already had to do this ;)
Christopher Coco
@cacoco
hah, it is possible there’s something out there
TP Diffenbach
@tpdi
Out of curiosity, when do you use a Local rather than a LocalContext?
Christopher Coco
@cacoco
I think generally when we say local we mean a local context (at least I do)
TP Diffenbach
@tpdi
Sorry, I meant com.twitter.util.Local vs com.twitter.finagle.context.LocalContext.
But thanks so much for clarifying this for me
Alessandro Vermeulen
@spockz
So if you want to share a db transactionwithin the same calls to the database you set the context with .let(mydbkey, newAtomicRef)(... handler code that calls multiple dB calls). On access of the mydbkey you can make sure the transaction is created only once
Prateek Jain
@prateek13688

Hi .. I am new to Finagle and I am using it in my java service. However, I am facing some problem while implementing retries in case of certain HTTP Error codes. The retry policy shouldRetry function does not gets called in case of request failing with certain error codes
here is the snippet of the code i am using to instantiate the client

private Service<Request, Response> buildClient(String host) throws Exception {
    String hostString = String.format("%s:%s", host, "8080");
    PartialFunction<Tuple2<Request, Response>, ResponseClass> typed =
            new AbstractPartialFunction<Tuple2<Request, Response>, ResponseClass>() {
                @Override
                public boolean isDefinedAt(Tuple2<Request, Response> x) {
                    return x._2().status() == Status.InternalServerError() || x._2().status() == Status.ServiceUnavailable();
                    return true;
                }

                @Override
                public ResponseClass apply(Tuple2<Request, Response> x) {
                    return ResponseClass.RetryableFailure();
                }
            };

    return this.hostToClientMap.computeIfAbsent(host, s -> {
        try {
            return ClientBuilder.safeBuild(
                    ClientBuilder.get()
                            .stack(Http.client())
                            .hosts(hostString)
                            .hostConnectionLimit(serviceConfiguration.getHttp_max_conn_per_route())
                            .tcpConnectTimeout(Duration.fromMilliseconds(serviceConfiguration.getHttp_socket_timeout_in_ms()))
                            .keepAlive(true)
                            .tls(LogDispatcherFactory.buildSslContext(serviceConfiguration.getSsm_store_region(),
                                    serviceConfiguration.getPrism_certificate_path(),
                                    serviceConfiguration.getPrism_certificate_key_path(),
                                    serviceConfiguration.getTruststore_password()))
                            .connectTimeout(Duration.fromMilliseconds(serviceConfiguration.getHttp_connect_timeout_in_ms()))
                            .requestTimeout(Duration.fromMilliseconds(serviceConfiguration.getHttp_request_timeout_in_ms()))
                            .retryPolicy(new CustomRetryPolicy())
                            .retryBudget(RetryBudgets.newRetryBudget(Duration.fromSeconds(10), 10, 1))
                            .failFast(false)
                            .responseClassifier(HttpResponseClassifier.apply(typed))
                            .reportTo(DefaultStatsReceiver.get()));
        } catch (Exception e) {
            logger.error(String.format("Error initializing the finagle client for host %s.", hostString), e);
            return null;
        }
    });
}

Any thoughts on this ???

Alessandro Vermeulen
@spockz
Typically you should not use client builder anymore
If you use the methodbuilder from Http.method(“”) then there is a method to specify a partial function which tells the client to retry requests without marking that request as a failure
Moses Nakamura
@mosesn
I’d be interested in what kinds of error codes it doesn’t get called for. Finagle has a mechanism for automatically retrying (without asking the retry policy) if the remote peer has indicated that they are unable to serve the traffic because they’re overloaded
I agree with @spockz that you might want to use the more modern APIs instead of ClientBuilder. maybe check out MethodBuilder?
Prime
@PRIME_LINUX_twitter
where should i begin with ... i am trying to use twitter server for my own work
Alessandro Vermeulen
@spockz
@PRIME_LINUX_twitter maybe try Finatra or Finch. They are a bit more accessible
Jure Malovrh
@JureMalovrh

Hi!
I have some performance questions regarding Finagle. I understand that you should not do any CPU bound operation on Finagle threads and that you should offload I/O work to FuturePool. But checking the comment in FuturePool it states, that this Pool should mainly be used in I/O operations and that CPU-bound operations need some special treatment. Is there any documentation about this "special treatment"?
We currently have a Finagle server, that gets some data from the hash map, but if this data is not found in hashmap, we do a quite expensive regex parsing. Since regex is CPU bound and we are using FuturePool.unboundedPool, we ofter starve out the whole thread pool and our Finagle server becomes unresponsive. Is there any standard way to approach it? I was thinking that this regex parsing should only get a bounded future pool, for example 6 threads, so if there are a lot of cache misses, we should limit the blast radius, by only hogging 6 threads. Would this approach be viable? Or are there any other ways to check it? Should we downsize Finagle event loop threads to avoid hogging?

Thanks

5 replies
Yufan Gong
@yufangong
Hi all, just a heads up that we are starting the February release, we will be cross-building scala 2.11/2.12/2.13 this time, 2.11 will be dropped in March release. Thanks!
Objektwerks
@objektwerks
How does one unit test a Finagle client, service and server combination? Sorry, I'm coming from an Akka-Http / Http4s background. Thanks in advance! :)
Christopher Coco
@cacoco
Objektwerks
@objektwerks
Thanks, Chris! Among the array of test links, would you recommend any one in particular? :)
Christopher Coco
@cacoco
@objektwerks depends on what you want to do but likely the Feature Tests section.
Objektwerks
@objektwerks
@cacoco Some light reading, I see. Thanks! I suspect I might enjoy another IPA - and dive into this twitter bliss tomorrow. Cheers! :)
Abel Miguez
@amiguez
Hi all, quick question here: what would you say is the minimal finagle version supporting scala 2.13.x? I thought Finagle 20.4.1 but I am not sure as I saw some more recent twitter/finatra@26c1e81 indicating that they would get it closer to 2.13 support. From @yugangong message I would understand is the February release.
3 replies
Brian P. Holt
@bpholt
Is there a way to add extra code to the companion objects generated by scrooge? I have some typeclass instances that could usefully be placed there to take advantage of the way implicit resolution works, but it’s not clear to me how one could do that.
2 replies
Niko Dziemba
@dziemba

Hey!

Is it possible to disable finagle's tracing for a given http request after it has already been selected
for sampling? I'm looking for something like c.t.f.tracing.Trace.cancelSampling() which I can call in my http handler.

Usecase: We have an http server with fairly low RPS and thus a high sampling rate.
We now want to exclude all healthcheck requests from tracing though since they are high in volume
and not needed (they cannot be extracted to an admin http server or similar for various reasons)

Cheers

Hamdi Allam
@hamdiallam

@dziemba You can adjust the sampled flag in of the current trace id.

// in handler
Trace.letIdOption(Trace.idOption.map(id => id.copy(_sampled = Some(false)))) { ... continue the http handler here  ... }

This would only prevent the code within the closure from tracing. Depending on where in the stack this closure is placed, some of finagle's default traces could still be emitted. Can you create an GH issue about this? Are you using barebones finagle or finatra for your http server?

Niko Dziemba
@dziemba
@hamdiallam I had a similar idea but I think that won't help much since in the http handler itself nothing happens that would create more traces. We need to cancel the "outer" traces somehow. I'm using bare finagle. Sure, I'll create a GH issue for this. Thanks!
Niko Dziemba
@dziemba
Prateek Jain
@prateek13688
Is there a way to refresh the SSL context of an existing finagle client without disposing and recreating the client again ?