47erbot on scalafmt-core-2.7.5
47erbot on master
Update scalafmt-core to 2.7.5 (compare)
sloshy on scalafmt-core-2.7.5
Update sbt-mdoc to 2.2.10 Update http4s-blaze-client, htt… Update sbt to 1.4.1 and 38 more (compare)
sloshy on compiler-warnings-2020-01-13
sloshy on master
Fix compiler warnings across pr… Merge branch 'master' into fix/… Merge pull request #422 from 47… (compare)
sloshy on compiler-warnings-2020-01-13
Update sbt-scalajs, scalajs-com… Expand dedupe/cache docs in REA… Fix deprecated use of `filterKe… and 2 more (compare)
sloshy on caching-in-readme
liftIO
but need smth like liftEff[F[_]: Concurrent](eff: F[T]): Fetch[F, T]
as not everyone using IO
, as making new DataSources
for every case is quite annoying and instantiating new class on every call with generic isn't a good performance practice it seems, lmk if I should open an issue
Hi, I have an API that provides a method get(key: String): Future[Thing]
. In my implementation, I would like to transparently batch requests within a certain window. As I understand, fetch is not made to do this.
Is my understanding correct? If so, is there an extension point to make it do this? Or do you know an alternative library that would help with such a scenario?
Hi, I have an API that provides a method
get(key: String): Future[Thing]
. In my implementation, I would like to transparently batch requests within a certain window. As I understand, fetch is not made to do this.Is my understanding correct? If so, is there an extension point to make it do this? Or do you know an alternative library that would help with such a scenario?
you mean time window @rethab ?
@purrgrammer ty for liftF
!
I have another case which might be quite interesting related to tracing within Fetch
Context
My underlying Concurrent type is TraceIO[A]
(type TraceIO[A] = TraceT[IO, A]
) which I'm using for tracing, so in FetchTIO
(type FetchTIO = Fetch[TraceIO, A]
)
I wonder how to properly generate new spans (TraceT
has method def newSpan(...)
which returns TraceT
with updated fiber context) when composing, combining, mapping over fetches
Current attempts
I tried using helper
def withTrace[A](ftio: FetchTIO[A], spanInfo: SpanInfo) : FetchTIO[A] = {
Fetch.liftF(Fetch.run(methodTIO(...)).newSpan(spanInfo))
}
but it breaks Fetch optimizations even though does create all spans as I want
Questions/Suggestions
1) Is there anything that can be done to allow this kind of behavior? Maybe smth like
withUpdateF[F[_]: Concurrent, A](f: Fetch[F, A])(upd: F[A] => F[A]): Fetch[F, A]
method? or even more general transformer
withUpdateF[F[_]: Concurrent, G[_]: Concurrent, A](f: Fetch[F, A])(upd: F[A] => G[A]): Fetch[G, A]
?
2) If above isn't possible and/or there's a better alternative wonder what it could be? Can maybe FetchLog
be used somehow to collect execution path info?
FetchLog
stores a log of a Fetch execution, and is a public data structure that users of the library have access to. You can run a fetch recording its execution using `Fetch#runLog, see http://47deg.github.io/fetch/docs.html#debugging-7 for more details.
@purrgrammer
1) yes, I do that as well, but it only gets my leaves of execution tree, I'd like to record branches as well as it is one of the reasons of having local tracing in the first place.
Like if my Fetch.run(returnUser())
10 different method calls away from fetch db calls it's still baffling how we got to these db calls.
And these 10 methods along the way also return Fetch
like
def branchMethod(...): Fetch[..., ...] = for {
(resA, resB) <- (leafMethodA, leafMethodB).tupled
resC <- leafMethodC
} yield {
doSmthWithABC(resA, resB, resC)
}
^ I'd be very nice to get an aggregated span for this guy (even if we managed to get spans for leaves)
does this make sense?
2) yeah, I'm looking right now of how to use/manipulate Round
s to get data I can use once Fetch
part is done to build spans from the resulted Log
Sangria
to use Fetch
for a couple weeks and keep getting the implementation working. Wondering if anyone else has tried at this? Sangria
already supports a concept of Deferred
s which are very similar to the Fetch
but without all the magic that cats
grants and I'm trying to make the Sangria
resolvers accept a Fetch
natively.
I had a question about these lines in the documentation:
Fetch provides its own instance of Applicative[Fetch]. Whenever we use applicative operations on more than one Fetch, we know that the fetches are independent meaning we can perform optimizations such as batching and concurrent requests.
If we were to use the default Applicative[Fetch] operations, which are implemented in terms of flatMap, we wouldn’t have information about the independency of multiple fetches.
I had a conversion yesterday where someone pointed me to parTraverse
and https://typelevel.org/cats/typeclasses/parallel.html?.
In light of all of that, I'd expect Fetch to have, Monad
and Applicative
instances with a Parallel
instance mapping between them, and one would need parTraverse
in order to get batching from the monad.
But it seems like there's really only a Monad
instance. So at the very least is seems like either the documentation is misleading or I can't find the instance of Applicative[Fetch]
that it mentions.