Where communities thrive

  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
Repo info
  • Sep 05 2019 14:43
    @typelevel-bot banned @jdegoes
  • Jan 31 2019 21:17
    codecov-io commented #484
  • Jan 31 2019 21:08
    scala-steward opened #484
  • Jan 31 2019 18:19
    andywhite37 commented #189
  • Jan 31 2019 02:41
    kamilongus starred typelevel/cats-effect
  • Jan 30 2019 00:01
    codecov-io commented #483
  • Jan 29 2019 23:51
    deniszjukow opened #483
  • Jan 29 2019 23:37
  • Jan 29 2019 23:22
  • Jan 29 2019 20:26
    Rui-L starred typelevel/cats-effect
  • Jan 29 2019 18:01
    jdegoes commented #480
  • Jan 29 2019 17:04
    thomaav starred typelevel/cats-effect
  • Jan 28 2019 17:43
    asachdeva starred typelevel/cats-effect
  • Jan 28 2019 07:12
    alexandru commented #480
  • Jan 28 2019 05:45
    codecov-io commented #482
  • Jan 28 2019 05:35
    daron666 opened #482
  • Jan 27 2019 13:56
    codecov-io commented #481
  • Jan 27 2019 13:46
    lrodero opened #481
  • Jan 27 2019 05:47
    codecov-io commented #460
  • Jan 27 2019 05:37
    codecov-io commented #460
Fabio Labella
the question is whether you need to explicitly say restore around them, or not
Haskell says no, and it blesses some ops as interruptible even inside mask (which the uncancelable we're discussing)
I'm saying that regardless of the operation, if you want interruption you need to call restore
that's especially true when nesting these sections
Gavin Bisesi
I like the idea of "no special cases"
it also seems like ref/deferred/semaphore usage is already an area where you really need expertise
semaphore less so
Fabio Labella
this is an improvement imho
anyway, I'd highly recommend going to the start of this conversation here https://gitter.im/typelevel/cats-effect?at=5d9d0b55874aeb2d2311389f
Gavin Bisesi

restore is composable. uncancelable(_(fa)) <-> fa and cancelable(_(fa)) <-> fa

I like that

I feel like no matter what the answer there will need to be some blog posts and cheat sheets and doc tests...
my takeaway from skimming through that conversation is a big "it's really complicated" :D
Fabio Labella
it sounds insane but it's actually quite a lot easier than what we have now
Gavin Bisesi
I'm willing to take that on faith
Ryan Zeigler
to briefly switch into this fascinating conversation, is there something I can import from to get something like IOApp but for the repl?
Piotr Gawryś


well like I said, I'm more than happy to share the prototype experiment results, design notes, and open questions if people are interested in taking a whack at it :-) PRs very much welcome. If no one else gets to it in the coming months, I'm sure I or someone else will implement it in the 2.x timeline

I would love to try it. I want to have it in Monix but I feel a little bit powerless, not even knowing how to start. :D I have a plenty of free time though and I'm familiar with internals. I would be happy to attempt it for cats.effect.IO first and then port to Monix. I don't get my hopes up but at the very least I might learn something in the process

Gavin Bisesi
@rzeigler I don't think so, but it wouldn't be hard to write such an object
object IORepl {
  implicit class IOOps[A](io: IO[A]) extends AnyVal {
    def yolo: A = io.unsafeRunSync()
  implicit val ec: ExecutionContext = ExecutionContext.global
  implicit val timer: Timer[IO] = IO.timer(ec)
  implicit val cs: ContextShift[IO] = IO.contextShift(ec)
import IORepl._
Ryan Zeigler
the yolo is a nice touch
Fabio Labella
you can make IORepl extend IOApp
  def run(args: List[String]) = ExitCode.Success.pure[IO]
and save the typing of those implicits
object Playground extends IOApp {
  def run(args: List[String]) = ExitCode.Success.pure[IO]

  implicit class Runner[A](s: Stream[IO, A]) {
    def yolo: Unit = s.compile.drain.unsafeRunSync
    def yoloV: Vector[A] = s.compile.toVector.unsafeRunSync
  // put("hello").to[F]
  def put[A](a: A): IO[Unit] = IO(println(a))
this is my standard playground
Gavin Bisesi
"effort" :P
probably also worth a val blocker = Blocker.etc(global)
put is nice
I never end up doing the repl
I just write unit tests if I'm not sure what something will do :)
because that's always clearly the best method :hurtrealbad:
Paul Snively
I just hammer out unsafeRunSync in Ammonite.
Fabio Labella
you write unit tests for people's questions on gitter ? ;)
Gavin Bisesi
nah I just don't test it
often gets close enough ;)
scastie if I'm stuck
or rather, I ask them to put it in scastie if I'm stuck :laughing:
Delegation :sparkles:
Paul Snively
I fire up Ammonite, import stuff from the web, and try stuff out interactively all the flipping time.
Gavin Bisesi
I have a local.sbt that I gitignore and ThisBuild / libraryDependencies ++= ... in
I needs my IDE
Ryan Zeigler
maximum laziness
i frequently find myself trying to find a project on disk that has a library I want to play with that I can bloop console rather than trying to go to scastie
Gavin Bisesi
probably still faster than scastie :D
Ryan Zeigler
its less clicking at least
Daniel Spiewak

@Avasil Here are my notes! https://gist.github.com/c0e731eefd0cc8c82bbbfd9197d08474 Again, massive credit to @rossabaker for doing all the real legwork on figuring this stuff out and benchmarking it. I basically just distilled and came up with some requirements.

So the raw mechanics of how you get the trace boils down to new Throwable().getStackTrace(), which will give you an Array[StackTraceElement]. That's where the easy part ends. Doing this is very expensive, and you need to do it eagerly inside the definition of flatMap, map, delay, and async (at a minimum). Naively implemented, this would result in a lot of overhead every time you call these functions.

So the idea is to not do it naively. If you call getClass on the Function1 passed to flatMap/map, you're going to get something which is reflective of the definition site of the function passed to the method in question. Note that there are some caveats with this:

val f: Int => Boolean = _ % 2 == 0

If you trace that with this technique, both map calls will have the same "call site". I really think that's fine though, and arguably even more useful than the alternative. Anyway…

You need to figure out which stack frame entry actually represents the tru call site, and this is where things get very tricky with Cats Effect because the f in question may be threaded through some other methods, such as monad transformers, libraries like fs2, etc. My spec suggests applying some heuristics to the name of the class you get from the Function1 to take an educated guess, and then go with first best fit. Note that these heuristics can be somewhat expensive at runtime if you need them to be, because you're only doing it once!

Use the Class as a cache key in a global (static) cache. Note that the size of this cache is bounded by the number of distinct call sites in the program, which is not really that many when you think about it. Cache misses are expensive, but they only happen once. Cache lookups are very, very fast. Note that ConcurrentHashMap is heavily read-optimized. The spec references a "slug" mode to tracing, which would basically disable caching entirely. The reason to disable caching is so that you can capture more than one stack frame, which would allow us to give really robust traces when people really need to dig into things. Like we can say things like, "the last few constructors which generated this IO were this map, which had this stack trace, and this flatMap, which had this other stack trace, etc". I imagine this being printed as like a nested bullet list, but hopefully you see where I'm going with this. Slug mode would have a lot of overhead and obviously would only be used when debugging in a dev environment, but that's still really useful! And giving these kinds of robust traces would eliminate the information loss that we would otherwise suffer from with just a single stack frame per call site.

Trace information should be stored in the IO constructors themselves, which avoids the need to maintain a separate data structure representing a "backtrace". In a sense, it's abusing ArrayStack to represent the trace indirectly. This is cool, but unfortunately Map fusion completely defeats it. I note this in my spec, and I have hypothesized that map fusion in IO is entirely pointless in practice and probably doesn't result in any measurable performance gains. Turning it off in IO and then running a sophisticated benchmark suite (like fs2's or Monix's) on top of IO without map fusion, and then again with, should be sufficient evidence to decide. If we can't remove map fusion, then (annoyingly!) we either need to have a separate nested stack structure inside of the IOMap node, or we need to only trace the top-most fused map. Either is probably okay, but not as nice as the unfused alternative. Needs measuring.

@Avasil The biggest problem with all this is configuration. You can't just thread its configuration through the runloop because some of these calls happen before the IO is actually running. (for example, val ioa = pure(42).flatMap(f), the flatMap call site must run before the IO starts executing) So that kicks out some things, unfortunately. ZIO does this nice thing where they have like a notrace function or something (I can't remember what it's called) which presumably threads through the run loop, but there's no possible way it can disable tracing for val examples like this one.

So my spec proposes a two-pronged solution: runloop-threaded configuration, with global defaults set by a system property. So you can still disable tracing entirely if you need to, but the default way you interact with it is via the runloop based configuration (which has nice lexical properties and a better API).

There are some slightly tricky performance-related questions associated with efficiently disabling tracing in such a global way without creating memory barriers. That's all solvable though, just mechanics, really.
Daniel Spiewak

@Avasil Oh, one final bit of trickiness: you can't call getClass on a thunked value in Scala and expect to get the class of the thunk, which is what you need. For example, to trace delay or >>. So you're probably going to need to implement a tiny helper in Java that has the class metadata to trick scalac into passing the thunk along without evaluation. In other words, what scalac does when it sees the following:

def foo(s: => String) = bar(s)
def bar(s: => String) = ...

bar gets the raw thunk that was passed to foo, without re-wrapping. If you can define bar in Java, then you can take that thunk (which will be of type scala.Function0) and call getClass on it without forcing. You don't have that option in Scala, since calling s.getClass will force the thunk and give you the Class of its contents.