Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Oct 14 10:12

    BenFradet on master

    Update sbt to 1.3.3 (compare)

  • Oct 14 10:12
    BenFradet closed #226
  • Oct 14 09:50
    scala-steward opened #226
  • Oct 02 15:06

    juanpedromoreno on master

    Update sbt-microsites to 0.9.7 … (compare)

  • Oct 02 15:06
    juanpedromoreno closed #225
  • Oct 02 13:44
    scala-steward opened #225
  • Sep 25 16:37

    juanpedromoreno on master

    Update sbt-microsites to 0.9.6 … (compare)

  • Sep 25 16:37
    juanpedromoreno closed #224
  • Sep 25 15:43
    scala-steward opened #224
  • Sep 21 17:41

    juanpedromoreno on master

    Update sbt-scalajs-crossproject… (compare)

  • Sep 21 17:41
    juanpedromoreno closed #221
  • Sep 21 15:39
    scala-steward synchronize #221
  • Sep 20 20:21

    juanpedromoreno on master

    Update sbt-scalajs, scalajs-com… (compare)

  • Sep 20 20:21
    juanpedromoreno closed #222
  • Sep 20 20:20

    juanpedromoreno on master

    Update sbt-org-policies to 0.12… (compare)

  • Sep 20 20:20
    juanpedromoreno closed #220
  • Sep 20 20:20

    juanpedromoreno on master

    Update sbt to 1.3.2 (#223) (compare)

  • Sep 20 20:20
    juanpedromoreno closed #223
  • Sep 20 16:49
    scala-steward opened #223
  • Sep 20 16:49
    scala-steward opened #222
Mikhail Filipchuk
@LiuVII
Hi guys!
I'm tryin to use Fetch over TraceT cedi-dtrace and running into some constraints like the need of implicit ConcurrentEffect which seems like might not even be needed according to 47deg/fetch#163
currently pursing a path of ihaving an instance that's defined within TraceT with some extra implicits but would be awesome to know if it's something that's going to be resolved in foreseeable future
maybe @purrgrammer might now?
also, should I post this on the github issue as well?
Jakub Kozłowski
@kubukoz
hey @LiuVII, I submitted a PR today that relaxes the constraint to Concurrent
needs more work but hopefully it can be merged soon
Jakub Kozłowski
@kubukoz
looks like just some flaky tests
Alejandro Gómez
@purrgrammer
will be merging the PR by Jakub later today, we can probably do a release with less restrictive effect implicits after that
Mikhail feel free to chime in 47deg/fetch#163 detailing your problems, it should be easier for you to use Fetch after we merge Jakub's PR
Jakub Kozłowski
@kubukoz
cool beans
Mikhail Filipchuk
@LiuVII
@kubukoz @purrgrammer that's awesome! is there an ETA of new release that'll include these changes?
Alejandro Gómez
@purrgrammer
@LiuVII the 1.0.1-SNAPSHOT includes both the Fetch#liftIO method and the implicit restriction loosening contributed by Jakub
Jakub Kozłowski
@kubukoz
:)
Mikhail Filipchuk
@LiuVII
@purrgrammer cool beans! ty :)
Jakub Kozłowski
@kubukoz
have I just introduced cool beans to this channel? I can't wait to show you what warm beans are like
Alejandro Gómez
@purrgrammer
you can do so after turning the Fetch[IO, ?] into IO[?]
Omer Zach
@omerzach
yeah i can do that but i lose a bunch of the fetch magic :(
i guess i can build special DataSources with the race logic i need built in
Alejandro Gómez
@purrgrammer
yep, i was going to propose that
you can combine two datasources into one that races both
i have thought about that but haven't gotten to write the code, it should be possible nonetheless
Omer Zach
@omerzach
my use case specifically is combining Fetch[IO, Boolean]s with short-circuiting
so i can race a few sources any of which may return true without waiting for them all, for example
Alejandro Gómez
@purrgrammer
then I would go with building special DataSources with such logic :+1:
TP Diffenbach
@tpdi
Has anyone successfully used fetch from Java code?
Mikhail Filipchuk
@LiuVII
@purrgrammer is there a reason why liftIO requires F to be ConcurrentEffect, Concurrent should be enough, right? maybe even Async
Mikhail Filipchuk
@LiuVII
@purrgrammer overall I'm having similar issue Omer mentioned about liftIO but need smth like liftEff[F[_]: Concurrent](eff: F[T]): Fetch[F, T] as not everyone using IO, as making new DataSources for every case is quite annoying and instantiating new class on every call with generic isn't a good performance practice it seems, lmk if I should open an issue
Alejandro Gómez
@purrgrammer
definitely open an issue @LiuVII , sorry for responding so late but I've been on vacation :)
Reto Habluetzel
@rethab

Hi, I have an API that provides a method get(key: String): Future[Thing]. In my implementation, I would like to transparently batch requests within a certain window. As I understand, fetch is not made to do this.

Is my understanding correct? If so, is there an extension point to make it do this? Or do you know an alternative library that would help with such a scenario?

João Ferreira
@jtjeferreira

Hi, I have an API that provides a method get(key: String): Future[Thing]. In my implementation, I would like to transparently batch requests within a certain window. As I understand, fetch is not made to do this.

Is my understanding correct? If so, is there an extension point to make it do this? Or do you know an alternative library that would help with such a scenario?

you mean time window @rethab ?

Reto Habluetzel
@rethab
yes, eg. batch all requests within 5s into a single request @jtjeferreira
Alejandro Gómez
@purrgrammer
there are options for controling batch size and execution strategy http://47deg.github.io/fetch/docs.html#batching-4-maximum-batch-size-0
but I'm not sure we could support batching over time windows, since Fetch assumes you want to be getting all the data you can at the same time
Reto Habluetzel
@rethab
Thanks @purrgrammer that's what I suspected :wink:
Alejandro Gómez
@purrgrammer
@rethab happy to help, I think fs2 can help you for your use case https://fs2.io/
Mikhail Filipchuk
@LiuVII
@purrgrammer opened one 47deg/fetch#188, lmk you need any extra assistance with that though might need pointers for correct implementation
Alejandro Gómez
@purrgrammer
@omerzach regarding your questions around not caching specific fetches, I've been experimenting a bit and I've come to the conclusion that it better be done with a custom cache 47deg/fetch#183
Mikhail Filipchuk
@LiuVII

@purrgrammer ty for liftF!
I have another case which might be quite interesting related to tracing within Fetch

Context
My underlying Concurrent type is TraceIO[A] (type TraceIO[A] = TraceT[IO, A]) which I'm using for tracing, so in FetchTIO (type FetchTIO = Fetch[TraceIO, A])
I wonder how to properly generate new spans (TraceT has method def newSpan(...) which returns TraceT with updated fiber context) when composing, combining, mapping over fetches

Current attempts
I tried using helper

def withTrace[A](ftio: FetchTIO[A], spanInfo: SpanInfo) : FetchTIO[A] = {
  Fetch.liftF(Fetch.run(methodTIO(...)).newSpan(spanInfo))
}

but it breaks Fetch optimizations even though does create all spans as I want

Questions/Suggestions
1) Is there anything that can be done to allow this kind of behavior? Maybe smth like

withUpdateF[F[_]: Concurrent, A](f: Fetch[F, A])(upd: F[A] => F[A]): Fetch[F, A]

method? or even more general transformer

withUpdateF[F[_]: Concurrent, G[_]: Concurrent, A](f: Fetch[F, A])(upd: F[A] => G[A]): Fetch[G, A]

?

2) If above isn't possible and/or there's a better alternative wonder what it could be? Can maybe FetchLog be used somehow to collect execution path info?

Alejandro Gómez
@purrgrammer
1) If i understood correctly, you should be able to create spans for your TraceIO type. You want to be creating these in the implementation of the DataSource#fetch and DataSource#batch, not on Fetch[TraceIO, A].
2) FetchLog stores a log of a Fetch execution, and is a public data structure that users of the library have access to. You can run a fetch recording its execution using `Fetch#runLog, see http://47deg.github.io/fetch/docs.html#debugging-7 for more details.
Mikhail Filipchuk
@LiuVII

@purrgrammer
1) yes, I do that as well, but it only gets my leaves of execution tree, I'd like to record branches as well as it is one of the reasons of having local tracing in the first place.
Like if my Fetch.run(returnUser()) 10 different method calls away from fetch db calls it's still baffling how we got to these db calls.
And these 10 methods along the way also return Fetch like

def branchMethod(...): Fetch[..., ...] = for {
  (resA, resB) <- (leafMethodA, leafMethodB).tupled
  resC <- leafMethodC
} yield {
  doSmthWithABC(resA, resB, resC)
}

^ I'd be very nice to get an aggregated span for this guy (even if we managed to get spans for leaves)
does this make sense?

2) yeah, I'm looking right now of how to use/manipulate Rounds to get data I can use once Fetch part is done to build spans from the resulted Log

Mikhail Filipchuk
@LiuVII
^ 2nd approach seems to give the same result as the one using logging inside DataSource#fetch/#batch
No way to modify Log it seems when composing fetches and no way to see execution branches, only leaves.
lmk if I missed smth
Justin Heyes-Jones
@justinhj
I noticed that Fetch.run can now take a cache or you can use runLog to get the fetch log, but I don't see a way to run a fetch that takes and returns both a cache and a log. Am I missing it or did it get removed? I think it used to be runEnv and I could get both back?
Alejandro Gómez
@purrgrammer

but I don't see a way to run a fetch that takes and returns both a cache and a log

now that you mention it I think I overlooked this, just opened an issue to address it. Thanks!

Alejandro Gómez
@purrgrammer
@justinhj not sure about the name but there it is 47deg/fetch#194
Justin Heyes-Jones
@justinhj
@purrgrammer looks great thanks!
Alejandro Gómez
@purrgrammer
Fetch#runAll is available in the latest snapshot, I should do a proper release at this point so I'll try to allocate some time at the end of the week. Thanks everyone for your feedback!
Justin Heyes-Jones
@justinhj
:+1:
Paul Daniels
@paulpdaniels
I've been trying to build a simple integration for Sangria to use Fetch for a couple weeks and keep getting the implementation working. Wondering if anyone else has tried at this? Sangria already supports a concept of Deferreds which are very similar to the Fetch but without all the magic that cats grants and I'm trying to make the Sangria resolvers accept a Fetch natively.
Alejandro Gómez
@purrgrammer
hmm that is something really interesting Paul, GraphQL query resolution can exploit a lot of the optimizations that Fetch does
btw Fetch 1.2.0 (upgraded dependency to cats-effect to 2.0.0) should be published, thanks to @paulpdaniels :sparkles: