typelevel/cats-effect

Comfy Cats Effect room. Ask questions. Rant about config file formats. Be nice!

Christopher Davenport

@ChristopherDavenport

However in F you can do that and then provide the implementations at the edge.

Almost everything in fs2 is in Concurrent and Timer, with the exceptions being IO stuff.

Christopher Davenport

@ChristopherDavenport

I’d need a more concrete example of what you are looking to do to clarify. Final tagless encoding here is what enables a lot of this behavior where functionality only needs to be proved at the edge, where you can just use what you want where you want it otherwise.

Paul Cleary

@pauljamescleary

For us, everything is tied to IO. So, for example, all functions are like def save(repo: Repository, user: User): IO[Unit] for example

in some places, we use par, like parSequence or parMapN

Seems like I would need to add some kind of (implicit x: X) to those functions that require par

Christopher Davenport

@ChristopherDavenport

def save[F[_]: Sync: Par](repo: Repository, user: User): F[Unit]

Is how I would write most of my functions.

Paul Cleary

@pauljamescleary

Yep. Our codebase is big, and we just aren't there atm

just wrapped up standardizing on cats and cats effect

Christopher Davenport

@ChristopherDavenport

Yup, then unfortunately you need the CS at every parallelism point and need to carry it to achieve that.

Paul Cleary

@pauljamescleary

so, we would need def save(repo: Repository, user: User)(implicit p: Par): IO[Unit] then?

Christopher Davenport

@ChristopherDavenport

Nah, Par is a typeclasss for Parallel, so I can’t give you what you don’t have in scope.

It just has a single type as opposed to two.

Paul Cleary

@pauljamescleary

implicit CS: ContextShift[IO] perhaps

Christopher Davenport

@ChristopherDavenport

Yes.

Paul Cleary

@pauljamescleary

I really want to abstract away the effect type

I have to balance scratching that itch with making progress on other things

thanks @ChristopherDavenport for your help

Christopher Davenport

@ChristopherDavenport

Abstracting effect gives greater clarity about what a particular function can do. But getting to IO in general is a major improvement over future.

Fabio Labella

@SystemFw

@pauljamescleary note that you will be able to remove Ec everywhere (apart from legacy blocking code) when you move to F and IOApp

:+1: to what Chris said though

once you're in IO, you have a safe/sane refactoring path to abstract to F later

Paul Cleary

@pauljamescleary

@SystemFw thanks, yea, I may take a stab at doing that and see how far I get

13h3r2

@13h3r2

Hey! I am not sure I understand how Concurrent.start and Fiber works.

I have multiple concurrent computations which should be started by trigger. Then I need to wait for the completion of all of the computations. I am running multiple fibers and use Deferred for synchronization. I realized that after calling Deferred.complete my fibers are processed sequential, but not parallel (as I expected).

I added explicit ContextShift.shift after Deferred.get in my fiber and now it works in parallel.

Here it is a gist - https://gist.github.com/13h3r2/1923169269db6732170c0058d8a869c1 Pay attention to line 26.

Few questions based on this gist:

is it expected behavior?
if yes what causes sequential processing in this case: Fibers, Deferred or something else?
what is supposed way to avoid this kind of problems?

Thanks

Gavin Bisesi

@Daenyth

@13h3r2 if you "just" want to run computations in parallel and collect the answers, you can do

import cats.implicits._
import cats.effect.implicits._
val fa: IO[A]
val fb: IO[B]
val g: (A, B) => C
implicit val cs: ContextShift[IO] = ??? // could get via EC
val fc: IO[C] = (fa, fb).parMapN(g)

in your gist there's some awkward points

fibers.foldLeft(... _.join) means you will join them in order

13h3r2

@13h3r2

@Daenyth thanks. I found this parallel instance for IO, but I would like to figure out what I was doing wrong in my example. Looks like I am missing something.

Gavin Bisesi

@Daenyth

Using Thread.sleep also means you're blocking a real thread instead of async sleeping, not sure how that would interact

Have you looked at fs2?

It makes this kind of thing way easier

Fabio Labella

@SystemFw

@13h3r2 what version are you on, to begin with?

13h3r2

@13h3r2

I do not think join in order is important here. They are already fibers and should be shifted as far as I understand. Thread sleep should be fine here because I have a lot of background threads. In real example I was using timer.

@SystemFw 1.0.0

Fabio Labella

@SystemFw

ok, let me have a look at the code

13h3r2

@13h3r2

Thanks.

Gavin Bisesi

@Daenyth

for example;

fs2.Stream.range(start 0, stopExclusive = 10)
  .mapAsync(parallelLimit: Int)(_ => computation)
  // or mapAsyncUnordered if the order doesn't matter

Fabio Labella

@SystemFw

as a bit of advice, it's better to post example with the proper code (i.e. Timer and not Thread.sleep, which you should never use)

13h3r2

@13h3r2

The gist is runnable. You can remove shift and see the difference.

@SystemFw agree. will do next time

I can fix this one if you think this is important.

Fabio Labella

@SystemFw

I don't think is important enough to say "I don't want to look at it"

I'll fix it

just some advice in general :)

13h3r2

@13h3r2

Thanks.

Gavin Bisesi

@Daenyth

d.get *> this is weird

Fabio Labella

@SystemFw

(also, my biased opinion is also to use fs2, but let's fix this one first)

Gavin Bisesi

@Daenyth

also maybe fibers <- tasks.traverse(_.start) might be clearer

Where communities thrive

People

Repo info

Activity