Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
Christopher Coco
@cacoco
right we do this in places by mutating a value held in the context
(like how we propagate slf4j MDC values in Finatra)
TP Diffenbach
@tpdi
So I have in synchronous code some database access object that does a if (myThreadLocal.get() == null) myThreadLocal.set(new Transaction); and then subquestnt calls to it in that thread use the transaction
Ok, so you do use an extra level of indirection
the context is fixed, but the value it holds is mutable
Christopher Coco
@cacoco
exactly
TP Diffenbach
@tpdi
com.twitter.util.Local does almost what I want; in order to replace multiple ThreadLocals, I'll need to establish a key or index for each, com.twitter.util.Local does that, using a counter in the companion object that serves as the array index
I was hoping someone else had already had to do this ;)
Christopher Coco
@cacoco
hah, it is possible there’s something out there
TP Diffenbach
@tpdi
Out of curiosity, when do you use a Local rather than a LocalContext?
Christopher Coco
@cacoco
I think generally when we say local we mean a local context (at least I do)
TP Diffenbach
@tpdi
Sorry, I meant com.twitter.util.Local vs com.twitter.finagle.context.LocalContext.
But thanks so much for clarifying this for me
Alessandro Vermeulen
@spockz
So if you want to share a db transactionwithin the same calls to the database you set the context with .let(mydbkey, newAtomicRef)(... handler code that calls multiple dB calls). On access of the mydbkey you can make sure the transaction is created only once
Prateek Jain
@prateek13688

Hi .. I am new to Finagle and I am using it in my java service. However, I am facing some problem while implementing retries in case of certain HTTP Error codes. The retry policy shouldRetry function does not gets called in case of request failing with certain error codes
here is the snippet of the code i am using to instantiate the client

private Service<Request, Response> buildClient(String host) throws Exception {
    String hostString = String.format("%s:%s", host, "8080");
    PartialFunction<Tuple2<Request, Response>, ResponseClass> typed =
            new AbstractPartialFunction<Tuple2<Request, Response>, ResponseClass>() {
                @Override
                public boolean isDefinedAt(Tuple2<Request, Response> x) {
                    return x._2().status() == Status.InternalServerError() || x._2().status() == Status.ServiceUnavailable();
                    return true;
                }

                @Override
                public ResponseClass apply(Tuple2<Request, Response> x) {
                    return ResponseClass.RetryableFailure();
                }
            };

    return this.hostToClientMap.computeIfAbsent(host, s -> {
        try {
            return ClientBuilder.safeBuild(
                    ClientBuilder.get()
                            .stack(Http.client())
                            .hosts(hostString)
                            .hostConnectionLimit(serviceConfiguration.getHttp_max_conn_per_route())
                            .tcpConnectTimeout(Duration.fromMilliseconds(serviceConfiguration.getHttp_socket_timeout_in_ms()))
                            .keepAlive(true)
                            .tls(LogDispatcherFactory.buildSslContext(serviceConfiguration.getSsm_store_region(),
                                    serviceConfiguration.getPrism_certificate_path(),
                                    serviceConfiguration.getPrism_certificate_key_path(),
                                    serviceConfiguration.getTruststore_password()))
                            .connectTimeout(Duration.fromMilliseconds(serviceConfiguration.getHttp_connect_timeout_in_ms()))
                            .requestTimeout(Duration.fromMilliseconds(serviceConfiguration.getHttp_request_timeout_in_ms()))
                            .retryPolicy(new CustomRetryPolicy())
                            .retryBudget(RetryBudgets.newRetryBudget(Duration.fromSeconds(10), 10, 1))
                            .failFast(false)
                            .responseClassifier(HttpResponseClassifier.apply(typed))
                            .reportTo(DefaultStatsReceiver.get()));
        } catch (Exception e) {
            logger.error(String.format("Error initializing the finagle client for host %s.", hostString), e);
            return null;
        }
    });
}

Any thoughts on this ???

Alessandro Vermeulen
@spockz
Typically you should not use client builder anymore
If you use the methodbuilder from Http.method(“”) then there is a method to specify a partial function which tells the client to retry requests without marking that request as a failure
Moses Nakamura
@mosesn
I’d be interested in what kinds of error codes it doesn’t get called for. Finagle has a mechanism for automatically retrying (without asking the retry policy) if the remote peer has indicated that they are unable to serve the traffic because they’re overloaded
I agree with @spockz that you might want to use the more modern APIs instead of ClientBuilder. maybe check out MethodBuilder?
Prime
@PRIME_LINUX_twitter
where should i begin with ... i am trying to use twitter server for my own work
Alessandro Vermeulen
@spockz
@PRIME_LINUX_twitter maybe try Finatra or Finch. They are a bit more accessible
Jure Malovrh
@JureMalovrh

Hi!
I have some performance questions regarding Finagle. I understand that you should not do any CPU bound operation on Finagle threads and that you should offload I/O work to FuturePool. But checking the comment in FuturePool it states, that this Pool should mainly be used in I/O operations and that CPU-bound operations need some special treatment. Is there any documentation about this "special treatment"?
We currently have a Finagle server, that gets some data from the hash map, but if this data is not found in hashmap, we do a quite expensive regex parsing. Since regex is CPU bound and we are using FuturePool.unboundedPool, we ofter starve out the whole thread pool and our Finagle server becomes unresponsive. Is there any standard way to approach it? I was thinking that this regex parsing should only get a bounded future pool, for example 6 threads, so if there are a lot of cache misses, we should limit the blast radius, by only hogging 6 threads. Would this approach be viable? Or are there any other ways to check it? Should we downsize Finagle event loop threads to avoid hogging?

Thanks

5 replies
Yufan Gong
@yufangong
Hi all, just a heads up that we are starting the February release, we will be cross-building scala 2.11/2.12/2.13 this time, 2.11 will be dropped in March release. Thanks!
objektwerks
@objektwerks
How does one unit test a Finagle client, service and server combination? Sorry, I'm coming from an Akka-Http / Http4s background. Thanks in advance! :)
Christopher Coco
@cacoco
objektwerks
@objektwerks
Thanks, Chris! Among the array of test links, would you recommend any one in particular? :)
Christopher Coco
@cacoco
@objektwerks depends on what you want to do but likely the Feature Tests section.
objektwerks
@objektwerks
@cacoco Some light reading, I see. Thanks! I suspect I might enjoy another IPA - and dive into this twitter bliss tomorrow. Cheers! :)
Abel Miguez
@amiguez
Hi all, quick question here: what would you say is the minimal finagle version supporting scala 2.13.x? I thought Finagle 20.4.1 but I am not sure as I saw some more recent twitter/finatra@26c1e81 indicating that they would get it closer to 2.13 support. From @yugangong message I would understand is the February release.
3 replies
Brian P. Holt
@bpholt
Is there a way to add extra code to the companion objects generated by scrooge? I have some typeclass instances that could usefully be placed there to take advantage of the way implicit resolution works, but it’s not clear to me how one could do that.
2 replies
Niko Dziemba
@dziemba

Hey!

Is it possible to disable finagle's tracing for a given http request after it has already been selected
for sampling? I'm looking for something like c.t.f.tracing.Trace.cancelSampling() which I can call in my http handler.

Usecase: We have an http server with fairly low RPS and thus a high sampling rate.
We now want to exclude all healthcheck requests from tracing though since they are high in volume
and not needed (they cannot be extracted to an admin http server or similar for various reasons)

Cheers

Hamdi Allam
@hamdiallam

@dziemba You can adjust the sampled flag in of the current trace id.

// in handler
Trace.letIdOption(Trace.idOption.map(id => id.copy(_sampled = Some(false)))) { ... continue the http handler here  ... }

This would only prevent the code within the closure from tracing. Depending on where in the stack this closure is placed, some of finagle's default traces could still be emitted. Can you create an GH issue about this? Are you using barebones finagle or finatra for your http server?

Niko Dziemba
@dziemba
@hamdiallam I had a similar idea but I think that won't help much since in the http handler itself nothing happens that would create more traces. We need to cancel the "outer" traces somehow. I'm using bare finagle. Sure, I'll create a GH issue for this. Thanks!
Niko Dziemba
@dziemba
Prateek Jain
@prateek13688
Is there a way to refresh the SSL context of an existing finagle client without disposing and recreating the client again ?
Roger
@rogern
Perhaps this is a feature request for Scrooge but I'd like to know if it is possible to support getting an immutable binary type such as com.twitter.io.Buf when generating scala code? The fact that ByteBuffer is mutable might cause all sorts of trouble if you're not careful. Just copying it without using asReadOnly changes the underlying position in the array and if you try to protect yourself with read only it can fail in the thrift service with java.nio.ReadOnlyBufferException. To me it feels like Buf would be the safer option here, ofc something that can be configurable but please correct me if I'm wrong :)
joybestourous
@joybestourous
@rogern would you mind creating a github issue to explain your feature request a little more? ty!
1 reply
jeroen
@dr-jerry
Hi (this is a crosspost of a SO question: https://stackoverflow.com/questions/66746668/finagle-ketamashardingservicebuilder-is-unknown) I was wondering if someone knows if shardingService and KetamaShardingServiceBuilder have moved to another package or project? To me it looks removed.
11 replies
Dermot Haughey
@hderms
does Finagle use dependency injection? (manual constructor injection or otherwise)
Christopher Coco
@cacoco
@hderms no, Finagle does not.
At least not via a framework. Some code may be structured to use inversion of control. But for what is classically termed dependency injection, no.
Alessandro Vermeulen
@spockz
When using backup requests we see that the first request(s) always have a backup request send, presumably because the average response time is 0 or something. How can we fix this?
Alessandro Vermeulen
@spockz
@cacoco I always envision the params in the stack client to be a kind of DI..
And the use of Java services
Christopher Coco
@cacoco
@spockz since Finatra uses Guice I generally interpret this question as “does Fingale use Guice (or something like it)” and the answer is no. If you’re talking about the general pattern of DI the answer is of course as any framework uses lots of different design patterns and Finagle is no exception.
Alessandro Vermeulen
@spockz
Right 😊
Alessandro Vermeulen
@spockz
When using DTAB, does the | just failover on lookup or does it also force failover if the endpoints on the LHS are down?
Denis Mikhaylov
@notxcain
Hi! We've upgraded to Finagle 21.2.0 and Scala 2.13 and now Finagle can't find HttpZipkinTracer even though it's definitely on the classpath. Any ideas why is this happening? Thank you!
15 replies
Alessandro Vermeulen
@spockz
What happens if you ask the Java service factory for something of that type manually? Do you see it?
Denis Mikhaylov
@notxcain
@spockz was your question addressed to me?