Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Jan 12 05:16

    47erbot on doobie-core-1.0.0-RC2

    (compare)

  • Jan 12 05:16

    47erbot on main

    Update doobie-core, doobie-h2 t… (compare)

  • Jan 12 05:16
    47erbot closed #584
  • Jan 12 05:16

    47erbot on scala-library-2.13.8

    (compare)

  • Jan 12 05:16

    47erbot on main

    Update scala-library to 2.13.8 (compare)

  • Jan 12 05:16
    47erbot closed #583
  • Jan 12 05:10
    47erbot opened #584
  • Jan 12 05:10

    47erbot on doobie-core-1.0.0-RC2

    Update doobie-core, doobie-h2 t… (compare)

  • Jan 12 05:10
    47erbot opened #583
  • Jan 12 05:10

    47erbot on scala-library-2.13.8

    Update scala-library to 2.13.8 (compare)

  • Jan 10 05:16

    47erbot on sbt-github-0.11.2

    (compare)

  • Jan 10 05:16

    47erbot on main

    Update sbt-github, sbt-github-h… (compare)

  • Jan 10 05:16
    47erbot closed #581
  • Jan 10 05:13
    47erbot commented #580
  • Jan 10 05:13
    47erbot commented #559
  • Jan 10 05:13
    47erbot opened #582
  • Jan 10 05:13

    47erbot on cats-effect-3.3.4

    Update cats-effect to 3.3.4 (compare)

  • Jan 10 05:10
    47erbot opened #581
  • Jan 10 05:10

    47erbot on sbt-github-0.11.2

    Update sbt-github, sbt-github-h… (compare)

  • Jan 03 07:58
    fedefernandez synchronize #580
Omer Zach
@omerzach
is there any way to lift an IO[T] to a Fetch[IO, T] without defining a Data/DataSource for it? i have certain code that’s un-batchable anyway and it would make a lot of my code more straightforward if i could just lift it right into Fetch.
Omer Zach
@omerzach
something along these lines:
  def liftT[T](io: IO[T]): Fetch[IO, T] = {
    object Lift extends Data[IO[T], T] {
      val name = "Lifts an IO[T] to a Fetch[IO, T]"

      def source: DataSource[IO, IO[T], T] = new DataSource[IO, IO[T], T] {
        override def data = Lift

        override def CF: ConcurrentEffect[IO] = ConcurrentEffect[IO]

        override def fetch(id: IO[T]): IO[Option[T]] = (
          id.map(Some(_))
        )
      }
    }

    Fetch(io, Lift.source)
  }
Alejandro Gómez
@purrgrammer

is there any way to lift an IO[T] to a Fetch[IO, T] without defining a Data/DataSource for it? i have certain code that’s un-batchable anyway and it would make a lot of my code more straightforward if i could just lift it right into Fetch.

that's something that I hadn't thought about, right now Fetch assumes you'll use a Data/DataSource for performing requests. If i understood correctly, you want a way to lift IO actions to Fetch without an associated Data/DataSource, knowing that those requests can't be optimized?

Fetch doesn't support it currently but I'm going to open an issue to address this, thanks for bringing this up Omer
Omer Zach
@omerzach
yep exactly what i meant, thanks!
Bijan Chokoufe Nejad
@bijancn

Fetch doesn't support it currently but I'm going to open an issue to address this, thanks for bringing this up Omer

Yeah I would have liked that as well :+1:

hey @bijancn, regarding the "happy path", one asumption that Fetch does is that your identities will be there, and will short-circuit when one is missing. However, for optional identities, you can construct Fetch instances using Fetch#optional. It will yield a Fetch[F, Option[A]], and won't fail if the identity is missing. Let me know if it helps!

Yeah thanks I found that as well at some point and it addresses my problem fully

One thing I am kind of missing is to have custom caches for certain Fetch queries or mark them as uncachable
Bijan Chokoufe Nejad
@bijancn
In any way, refactoring my service to use Fetch has made it really fast and I love the fact that I can do batching, parallel request and caching all with a similar API and not rewrite according to what has to be done first etc
Alejandro Gómez
@purrgrammer
it makes me really happy to read that Bijan, I've opened an issue to address marking fetches as not to be served from the cache
Omer Zach
@omerzach
@purrgrammer another request after liftIO is some sort of “clearCache” command we could use. not sure implementation-wise if that’s much trickier. we want it for this sort of use case:
for {
  user <- Fetch(userId, UsersById)
  organization <- Fetch(organizationId, OrganizationsById.source)

  _ <- Fetch.liftIO(addUserToOrganization(user, organization) // This may modify an organization object in the DB

  _ <- Fetch.clearCache()

  organizationJson <- Fetch(organizationId, OrganizationJsonById.source) // This uses the OrganizationsById DataSource but needs to get a new one, not the cached one!
} yield {
  organizationJson
}

right now we're doing something like this:

for {
  (user, organization) <- Fetch.run[IO])(Fetch(userId, UsersById), Fetch(organizationId, OrganizationsById.source))

  _ <- addUserToOrganization(user, organization) // This may modify an organization object in the DB

  organizationJson <- Fetch.run[IO](Fetch(organizationId, OrganizationJsonById.source)) // This uses the OrganizationsById DataSource but needs to get a new one, not the cached one!
} yield {
  organizationJson
}

which isn’t too bad, but you can see how this can can get out of hand as we have more and more complex cases

Jakub Kozłowski
@kubukoz
hi, does anyone else have problems loading fetch sources into IDEA?
in fact, I get issues when just opening the sbt shell
okay, probably caused by a global bloop plugin...
Mikhail Filipchuk
@LiuVII
Hi guys!
I'm tryin to use Fetch over TraceT cedi-dtrace and running into some constraints like the need of implicit ConcurrentEffect which seems like might not even be needed according to 47deg/fetch#163
currently pursing a path of ihaving an instance that's defined within TraceT with some extra implicits but would be awesome to know if it's something that's going to be resolved in foreseeable future
maybe @purrgrammer might now?
also, should I post this on the github issue as well?
Jakub Kozłowski
@kubukoz
hey @LiuVII, I submitted a PR today that relaxes the constraint to Concurrent
needs more work but hopefully it can be merged soon
Jakub Kozłowski
@kubukoz
looks like just some flaky tests
Alejandro Gómez
@purrgrammer
will be merging the PR by Jakub later today, we can probably do a release with less restrictive effect implicits after that
Mikhail feel free to chime in 47deg/fetch#163 detailing your problems, it should be easier for you to use Fetch after we merge Jakub's PR
Jakub Kozłowski
@kubukoz
cool beans
Mikhail Filipchuk
@LiuVII
@kubukoz @purrgrammer that's awesome! is there an ETA of new release that'll include these changes?
Alejandro Gómez
@purrgrammer
@LiuVII the 1.0.1-SNAPSHOT includes both the Fetch#liftIO method and the implicit restriction loosening contributed by Jakub
Jakub Kozłowski
@kubukoz
:)
Mikhail Filipchuk
@LiuVII
@purrgrammer cool beans! ty :)
Jakub Kozłowski
@kubukoz
have I just introduced cool beans to this channel? I can't wait to show you what warm beans are like
Alejandro Gómez
@purrgrammer
you can do so after turning the Fetch[IO, ?] into IO[?]
Omer Zach
@omerzach
yeah i can do that but i lose a bunch of the fetch magic :(
i guess i can build special DataSources with the race logic i need built in
Alejandro Gómez
@purrgrammer
yep, i was going to propose that
you can combine two datasources into one that races both
i have thought about that but haven't gotten to write the code, it should be possible nonetheless
Omer Zach
@omerzach
my use case specifically is combining Fetch[IO, Boolean]s with short-circuiting
so i can race a few sources any of which may return true without waiting for them all, for example
Alejandro Gómez
@purrgrammer
then I would go with building special DataSources with such logic :+1:
TP Diffenbach
@tpdi
Has anyone successfully used fetch from Java code?
Mikhail Filipchuk
@LiuVII
@purrgrammer is there a reason why liftIO requires F to be ConcurrentEffect, Concurrent should be enough, right? maybe even Async
Mikhail Filipchuk
@LiuVII
@purrgrammer overall I'm having similar issue Omer mentioned about liftIO but need smth like liftEff[F[_]: Concurrent](eff: F[T]): Fetch[F, T] as not everyone using IO, as making new DataSources for every case is quite annoying and instantiating new class on every call with generic isn't a good performance practice it seems, lmk if I should open an issue
Alejandro Gómez
@purrgrammer
definitely open an issue @LiuVII , sorry for responding so late but I've been on vacation :)
rethab
@rethab

Hi, I have an API that provides a method get(key: String): Future[Thing]. In my implementation, I would like to transparently batch requests within a certain window. As I understand, fetch is not made to do this.

Is my understanding correct? If so, is there an extension point to make it do this? Or do you know an alternative library that would help with such a scenario?

João Ferreira
@jtjeferreira

Hi, I have an API that provides a method get(key: String): Future[Thing]. In my implementation, I would like to transparently batch requests within a certain window. As I understand, fetch is not made to do this.

Is my understanding correct? If so, is there an extension point to make it do this? Or do you know an alternative library that would help with such a scenario?

you mean time window @rethab ?

rethab
@rethab
yes, eg. batch all requests within 5s into a single request @jtjeferreira
Alejandro Gómez
@purrgrammer
there are options for controling batch size and execution strategy http://47deg.github.io/fetch/docs.html#batching-4-maximum-batch-size-0
but I'm not sure we could support batching over time windows, since Fetch assumes you want to be getting all the data you can at the same time
rethab
@rethab
Thanks @purrgrammer that's what I suspected :wink:
Alejandro Gómez
@purrgrammer
@rethab happy to help, I think fs2 can help you for your use case https://fs2.io/
Mikhail Filipchuk
@LiuVII
@purrgrammer opened one 47deg/fetch#188, lmk you need any extra assistance with that though might need pointers for correct implementation