Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Sep 05 2019 14:43
    @typelevel-bot banned @jdegoes
  • Jan 31 2019 21:17
    codecov-io commented #484
  • Jan 31 2019 21:08
    scala-steward opened #484
  • Jan 31 2019 18:19
    andywhite37 commented #189
  • Jan 31 2019 02:41
    kamilongus starred typelevel/cats-effect
  • Jan 30 2019 00:01
    codecov-io commented #483
  • Jan 29 2019 23:51
    deniszjukow opened #483
  • Jan 29 2019 23:37
  • Jan 29 2019 23:22
  • Jan 29 2019 20:26
    Rui-L starred typelevel/cats-effect
  • Jan 29 2019 18:01
    jdegoes commented #480
  • Jan 29 2019 17:04
    thomaav starred typelevel/cats-effect
  • Jan 28 2019 17:43
    asachdeva starred typelevel/cats-effect
  • Jan 28 2019 07:12
    alexandru commented #480
  • Jan 28 2019 05:45
    codecov-io commented #482
  • Jan 28 2019 05:35
    daron666 opened #482
  • Jan 27 2019 13:56
    codecov-io commented #481
  • Jan 27 2019 13:46
    lrodero opened #481
  • Jan 27 2019 05:47
    codecov-io commented #460
  • Jan 27 2019 05:37
    codecov-io commented #460
Daniel Spiewak
@djspiewak
🎉
Gavin Bisesi
@Daenyth
done
Adam Rosien
@arosien

Sure, for example I'd like to be able to do this kind of a reduce over a stream where I can do a bunch of work on parallel threads first but then reduce on the initial thread. By default Java streams don't guarantee where the work will be executed.

@schrepfler i ran that code and it doesn't guarantee what thread the reduce happens in, like you say. which i would expect.

why do you want to pin the reduce work to one thread? it seems an arbitrary requirement.

(cats-effect works the same way as the java streams example)
Srepfler Srdan
@schrepfler
I can’t pass some thread unsafe dependencies into the map function so was thinking to do the final bit of the processing sequentially in the reduce stage in the starting thread
Adam Rosien
@arosien
parTraverse collects all the results, so if you follow it with a map you can have the isolation
Adam Rosien
@arosien

oh, maybe i didn't parse what you said correctly. you can't pass something into map?

you could use a Semaphore and/or Resource to limit access to the unsafe dependency

1 reply
James Rockett
@jrockett6
Can someone give me some insight as to what's wrong with this:
  private def checkFileExists[F[_]: Sync](file: String): Either[String, Unit] =
    Sync[F].delay(Files.exists(Paths.get(file)))
      .ifM(Right(Unit), Left("Input " + file + " not found"))
Intellij is giving me a error warning after Right(Unit) saying : Right[Nothing, Unit.type]
James Rockett
@jrockett6
Ohh it's because my F type, I needed Sync[Either]. Though now i'm missing a Sync[Either] implicit instance
James Rockett
@jrockett6

Hmm now I ended up with this and ifM can't be found:

  private def checkFileExists(file: String)(implicit v: Sync[Either]): Either[String, Unit] =
    Sync[Either].delay(Files.exists(Paths.get(file)))
      .ifM(Right(Unit), Left("Input " + file + " not found"))

I feel like there's a better way to approach this

Anton Sviridov
@keynmol

@jrrockett ifM in general is defined on anything that implements FlatMap typeclass, but you need a syntax extension import to get this working:

import cats.syntax.flatMap._

or

import cats.syntax.all._

See: https://github.com/typelevel/cats/blob/7c07f71e73e691469d07e79044674c325c7daaa7/core/src/main/scala/cats/syntax/flatMap.scala#L98

Gabriel Volpe
@gvolpe

Sync[Either] that won't compile either @jrrockett . Anyway, returning Either[String, Unit] doesn't seem much more helpful than returning F[Boolean].

This would just work:

 private def checkFileExists[F[_]: Sync](file: String): F[Boolean] =
    Sync[F].delay(Files.exists(Paths.get(file)))

But I'd probably think a bit more about the design. What do you do if it exists? What if it doesn't? Maybe you don't even need to expose an F[Boolean] which is not ideal (see boolean blindness).

James Rockett
@jrockett6
Ok, I'm still trying to wrap my head around returning generic container types and Sync in general..
My current thought is that I would like do do nothing if it exists, but short circuit and return the error message if it doesn't exist, but maybe there's a better way to go about that.
currently it lives in a Either[String, Unit] for comprehension:
  private def parseArgs(args: List[String]): Either[String, Unit] =
    for {
      _ <- checkArgsEmpty(args)
      pair <- args.sliding(2).toList
      _ <- checkIfPair(pair)
      _ <- checkValidFlag(pair.head)
      _ <- checkValidPair(pair)
and it's inside the last function there, something like:
  private def checkValidPair(argsPair: List[String]): Either[String, Unit] = argsPair.head match {
    case "--f" => checkFileExists(argsPair.tail)
but this is the first function out of all of them that has side effects, so I'm not quite sure the appropriate way to handle that
Adam Rosien
@arosien
@jrrockett https://typelevel.org/cats/typeclasses/parallel.html has an example of using Either and Validated to do validation, which may also help
(the validation design is separate from the abstract effect type stuff)
Adam Rosien
@arosien
is this a terrible idea, or an idea that has a better form?
  def mutexResource[A](a: A): IO[Resource[IO, A]] =
    for {
      mutex <- Semaphore[IO](1)
      res = Resource.make(mutex.acquire *> a.pure[IO])(
        _ => mutex.release
      )
    } yield res
Dmitry Polienko
@nigredo-tori

You probably want

res = Resouce.make(mutex.acquire)(_ => mutex.release).as(a)

AFAIK, *> can cause a cancellation check, so you might leak your mutex.

James Rockett
@jrockett6
@arosien Cool thank you I will take a look at that!
James Rockett
@jrockett6
Oh haha I just realized Scala with Cats has a whole chapter (ch 10) dedicated to this as well :smiley:
Dmitry Polienko
@nigredo-tori

@jrrockett, AFAICT, you want the monad you're working in to have two capabilities.

  1. It should be able to check if a file exists (by running some IO operations). Either[String, *] can't do that.
  2. It should have String errors with short-curcuiting. IO and so on can't do that.

So there is no such monad out of the box. However, this is achievable with monad transformers. Something like this:

private def parseArgs(args: List[String]): EitherT[IO, String, Unit] =
    for {
      _ <- EitherT.fromEither[IO](checkArgsEmpty(args))
      pair <- EitherT.fromEither[IO](args.sliding(2).toList)
      _ <- EitherT.fromEither[IO](checkIfPair(pair))
      _ <- EitherT.fromEither[IO](checkValidFlag(pair.head))
      _ <- checkValidPair(pair)
// ...

private def checkValidPair(argsPair: List[String]): EitherT[IO, String, Unit] = argsPair.head match {
  case "--f" => checkFileExists(argsPair.tail)
// ...

Here, EitherT is a transformer that transforms the IO monad by adding the second mentioned capability.

There are ways to reduce the fromEither noise in parseArgs, but that's not important right now. Also, checkFileExists should probably take a Blocker as a parameter (which should be passed through all the functions here).

James Rockett
@jrockett6
Cool that's exactly what I need! That thought process makes a lot of sense. I'll do a refresher on monad transformers
Adam Rosien
@arosien
@nigredo-tori thanks!
Fabio Labella
@SystemFw

@arosien @nigredo-tori

AFAIK, *> can cause a cancellation check, so you might leak your mutex.

make is uncancelable, which means that your acquire is uncancelable, so on one hand you won't leak it, but on the other you have an uninterruptible lock, which isn't great

withPermit on Semaphore has special logic to avoid this problem
Adam Rosien
@arosien
ah interesting. my idea was to try to avoid passing the actual Semaphore around, i was looking for the right interface to do it. i started with withPermit.
Luis Rodero-Merino
@lrodero

Hi, I created the #1122 PR to update tutorial to cats 2.1.4 and fix links.

Beyond that, I discussed with @marcraminv the contents of the current TCP echo server - concurrent system with Fibers section. And we conclude that most of it is hard to read and understand. As I'm the author it pains me to admit it :) , but it is likely we should replace it with something easier to follow. E.g. we considered implementing a solution for the producer-consumer problem. What do you think?

Luis Rodero-Merino
@lrodero
Also, any other comment or suggestion for the tutorial is welcome ;)
Anton Sviridov
@keynmol
Has anyone seen github actions being stuck on cats-effect build? I've opened a PR which seemingly doesn't touch anything and the JDK 11 build has been stuck for 45 minutes now: https://github.com/typelevel/cats-effect/pull/1123/checks?check_run_id=1018303390
Fabio Labella
@SystemFw
@mpilquist @RaasAhsan @djspiewak Do you reckon it's the same issue?
Raas Ahsan
@RaasAhsan
@keynmol @SystemFw I can't tell anymore, but it says the workflow finished in 7 minutes. was the adopt11 build just queued for 45 minutes or something?
Fabio Labella
@SystemFw
didn't check at the time, might well be :)
Anton Sviridov
@keynmol
I force-pushed for a different reason (commit email), which is a dumb thing to do when you're trying to demonstrate a stuck build :D basically previous build on adopt11 spent 45 minutes on " Run sbt ++2.12.11 ci"
Raas Ahsan
@RaasAhsan
ah :D do you know if it was stuck on a particular test?
thankfully actions keeps track of every workflow run
it seems like like all the tests passed and then got stuck before running mima checks
Daniel Spiewak
@djspiewak
on rare occaisions, Actions instances can die randomly
this used to happen a lot
what causes it is the underlying Azure Pipelines instance going away
when that happens, teh build just... hangs roughly forever
that sounds like what happens

Beyond that, I discussed with @marcraminv the contents of the current TCP echo server - concurrent system with Fibers section. And we conclude that most of it is hard to read and understand. As I'm the author it pains me to admit it :) , but it is likely we should replace it with something easier to follow. E.g. we considered implementing a solution for the producer-consumer problem. What do you think?

Producer-consumer seems a lot easier to follow! Like, I love the TCP echo server concept, but it's definitely difficult to follow and requires some setup.

Something I've struggled with sometimes is whether or not it's worth referencing ecosystem projects

Like, you wouldn't implement a TCP server from scratch with vanilla CE, you would use another library which builds on top of CE and exposes that kind of fucntionality, then you would implement an echo server
Daniel Spiewak
@djspiewak
Basically what I'm saying is it's hard to write tutorials :-)
Anand Krishnan
@iamanandkris
Hi is there a way to use ConcurrentEffect[F].start{...} inside Resource.use() in a safe way?
Gavin Bisesi
@Daenyth
Yes, but start is very low level, and you're better off using another abstraction
depending on what you want to do