Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Sep 05 2019 14:43
    @typelevel-bot banned @jdegoes
  • Jan 31 2019 21:17
    codecov-io commented #484
  • Jan 31 2019 21:08
    scala-steward opened #484
  • Jan 31 2019 18:19
    andywhite37 commented #189
  • Jan 31 2019 02:41
    kamilongus starred typelevel/cats-effect
  • Jan 30 2019 00:01
    codecov-io commented #483
  • Jan 29 2019 23:51
    deniszjukow opened #483
  • Jan 29 2019 23:37
  • Jan 29 2019 23:22
  • Jan 29 2019 20:26
    Rui-L starred typelevel/cats-effect
  • Jan 29 2019 18:01
    jdegoes commented #480
  • Jan 29 2019 17:04
    thomaav starred typelevel/cats-effect
  • Jan 28 2019 17:43
    asachdeva starred typelevel/cats-effect
  • Jan 28 2019 07:12
    alexandru commented #480
  • Jan 28 2019 05:45
    codecov-io commented #482
  • Jan 28 2019 05:35
    daron666 opened #482
  • Jan 27 2019 13:56
    codecov-io commented #481
  • Jan 27 2019 13:46
    lrodero opened #481
  • Jan 27 2019 05:47
    codecov-io commented #460
  • Jan 27 2019 05:37
    codecov-io commented #460
Gavin Bisesi
@Daenyth
:D
brainpiler
often gets close enough ;)
scastie if I'm stuck
or rather, I ask them to put it in scastie if I'm stuck :laughing:
Delegation :sparkles:
Paul Snively
@paul-snively
I fire up Ammonite, import stuff from the web, and try stuff out interactively all the flipping time.
Gavin Bisesi
@Daenyth
I have a local.sbt that I gitignore and ThisBuild / libraryDependencies ++= ... in
I needs my IDE
Ryan Zeigler
@rzeigler
maximum laziness
i frequently find myself trying to find a project on disk that has a library I want to play with that I can bloop console rather than trying to go to scastie
Gavin Bisesi
@Daenyth
probably still faster than scastie :D
Ryan Zeigler
@rzeigler
its less clicking at least
Daniel Spiewak
@djspiewak

@Avasil Here are my notes! https://gist.github.com/c0e731eefd0cc8c82bbbfd9197d08474 Again, massive credit to @rossabaker for doing all the real legwork on figuring this stuff out and benchmarking it. I basically just distilled and came up with some requirements.

So the raw mechanics of how you get the trace boils down to new Throwable().getStackTrace(), which will give you an Array[StackTraceElement]. That's where the easy part ends. Doing this is very expensive, and you need to do it eagerly inside the definition of flatMap, map, delay, and async (at a minimum). Naively implemented, this would result in a lot of overhead every time you call these functions.

So the idea is to not do it naively. If you call getClass on the Function1 passed to flatMap/map, you're going to get something which is reflective of the definition site of the function passed to the method in question. Note that there are some caveats with this:

val f: Int => Boolean = _ % 2 == 0
ioa.map(f)
  .map(f)

If you trace that with this technique, both map calls will have the same "call site". I really think that's fine though, and arguably even more useful than the alternative. Anyway…

You need to figure out which stack frame entry actually represents the tru call site, and this is where things get very tricky with Cats Effect because the f in question may be threaded through some other methods, such as monad transformers, libraries like fs2, etc. My spec suggests applying some heuristics to the name of the class you get from the Function1 to take an educated guess, and then go with first best fit. Note that these heuristics can be somewhat expensive at runtime if you need them to be, because you're only doing it once!

Use the Class as a cache key in a global (static) cache. Note that the size of this cache is bounded by the number of distinct call sites in the program, which is not really that many when you think about it. Cache misses are expensive, but they only happen once. Cache lookups are very, very fast. Note that ConcurrentHashMap is heavily read-optimized. The spec references a "slug" mode to tracing, which would basically disable caching entirely. The reason to disable caching is so that you can capture more than one stack frame, which would allow us to give really robust traces when people really need to dig into things. Like we can say things like, "the last few constructors which generated this IO were this map, which had this stack trace, and this flatMap, which had this other stack trace, etc". I imagine this being printed as like a nested bullet list, but hopefully you see where I'm going with this. Slug mode would have a lot of overhead and obviously would only be used when debugging in a dev environment, but that's still really useful! And giving these kinds of robust traces would eliminate the information loss that we would otherwise suffer from with just a single stack frame per call site.

Trace information should be stored in the IO constructors themselves, which avoids the need to maintain a separate data structure representing a "backtrace". In a sense, it's abusing ArrayStack to represent the trace indirectly. This is cool, but unfortunately Map fusion completely defeats it. I note this in my spec, and I have hypothesized that map fusion in IO is entirely pointless in practice and probably doesn't result in any measurable performance gains. Turning it off in IO and then running a sophisticated benchmark suite (like fs2's or Monix's) on top of IO without map fusion, and then again with, should be sufficient evidence to decide. If we can't remove map fusion, then (annoyingly!) we either need to have a separate nested stack structure inside of the IOMap node, or we need to only trace the top-most fused map. Either is probably okay, but not as nice as the unfused alternative. Needs measuring.

@Avasil The biggest problem with all this is configuration. You can't just thread its configuration through the runloop because some of these calls happen before the IO is actually running. (for example, val ioa = pure(42).flatMap(f), the flatMap call site must run before the IO starts executing) So that kicks out some things, unfortunately. ZIO does this nice thing where they have like a notrace function or something (I can't remember what it's called) which presumably threads through the run loop, but there's no possible way it can disable tracing for val examples like this one.

So my spec proposes a two-pronged solution: runloop-threaded configuration, with global defaults set by a system property. So you can still disable tracing entirely if you need to, but the default way you interact with it is via the runloop based configuration (which has nice lexical properties and a better API).

There are some slightly tricky performance-related questions associated with efficiently disabling tracing in such a global way without creating memory barriers. That's all solvable though, just mechanics, really.
Daniel Spiewak
@djspiewak

@Avasil Oh, one final bit of trickiness: you can't call getClass on a thunked value in Scala and expect to get the class of the thunk, which is what you need. For example, to trace delay or >>. So you're probably going to need to implement a tiny helper in Java that has the class metadata to trick scalac into passing the thunk along without evaluation. In other words, what scalac does when it sees the following:

def foo(s: => String) = bar(s)
def bar(s: => String) = ...

bar gets the raw thunk that was passed to foo, without re-wrapping. If you can define bar in Java, then you can take that thunk (which will be of type scala.Function0) and call getClass on it without forcing. You don't have that option in Scala, since calling s.getClass will force the thunk and give you the Class of its contents.

You used to be able to trick Scala into not forcing thunks (like, back in Scala 2.7), but they closed that loophole (because it was, in fact, a bug in the compiler)

@SystemFw

is Simon Marlow's note on maskUninterruptable: every use should be viewed with extreme suspicion.

well, we can say: every use of uncancelable that ignores restore should be ....

Fair, though maskUninterruptable still has restore as well, and Simon suggests it be viewed with unconditional suspicion.

The "no special cases" thing is very appealing, and in a sense cancelable would create an unbounded number of special cases. I strongly suspect though that the only people who would use it would be authors of things like Deferred, and most code would just ignore it. That would certainly be the best practice.

Piotr Gawryś
@Avasil
Thank you Daniel
etienne
@crakjie
Hello, I'm looking for a structure that act as Semaphore , Defered and MVar but which is a state machine. Aka blocking while the state is not in the expected state.
I think I could do something like that with Ref but I'm not sure how I could make the wait part. Is there is that look like that which exists or is there good exemple that can help me to make this?
Fabio Labella
@SystemFw
that's what Ref + Deferred allows you to do
etienne
@crakjie
Ok, I will reread them and see. I'm doing a kind of transaction where commit can be done if there is not more task running.
Fabio Labella
@SystemFw
you can also use MVar and Semaphore directly ofc
etienne
@crakjie
@SystemFw there is no Ref.update(f : A=F[A]) for exemple if I try to open a transaction closed? Maybe I should encode the error in the State?
oh modify is enought in fact to know what happen inside the update !
Ryan Peters
@sloshy
@crakjie In addition -- modify letting you sidestep the issue lets you concurrently reason about pure data better. The function you pass in can be ran potentially more than once due to the implementation not synchronizing access.
So if there were a variant that took a function to F[A] that would be potentially problematic
Gavin Bisesi
@Daenyth
This message was deleted
etienne
@crakjie
@rzeigler Could you explain what is a variant ?
etienne
@crakjie
@SystemFw I made that with your exemple https://gist.github.com/crakjie/7398a57124690f09a7e2037fc3112457 I still have to try it next week but I'm quite happy.
I was lucky that when looking for a state machine with Ref+ Deferred you actually already made one !
Ryan Zeigler
@rzeigler
@crakjie I think you mentioned the wrong person
etienne
@crakjie
Indeed sorry :x
Paulius Imbrasas
@CremboC
I just noticed, IO.redeem was never added to the typeclasses for cats 2.0?
Rohde Fischer
@rfftrifork
@SystemFw heya, could I please have the link for your talk where you cover the niceness of referential transparancy? I got some colleagues who I think would appreciate it
(and if they don't too bad for them, I did appreciate it, so I'm gonna share it no matter if the do or don't xD )
Fabio Labella
@SystemFw
I don't have a talk specifically on that, but I do cover it in a few talks (shared state in pure FP, my fs2 talk at klarna), I do have a long reddit post on it though. If you browse through https://systemfw.org/talks and https://systemfw.org/writings you should find all the links you need
the reddit posts are called "the benefits of IO"
Rohde Fischer
@rfftrifork
@SystemFw ah ok, if I recall correctly you linked it when I specifically asked a lot of IO questions, but then it's a question of which one that triggered the linking
I think it's the one from Italy, so Shared State in Pure FP :) thanks, mate
Fabio Labella
@SystemFw
:+1:
Yury Ankudinov
@stremlenye
Hi. Does anybody knows by chance which cats-* library exposes the instance of Bracket for Either and if not what would be the reason?
Luka Jacobowitz
@LukaJCB
@stremlenye I’m curious, why would you need a Bracket instance for Either?
Yury Ankudinov
@stremlenye
@LukaJCB I am using Resource to acquire and cleanup dependencies for my IT tests. And unfortunately some of them synchronous.
Luka Jacobowitz
@LukaJCB
You should use SyncIO :)
Yury Ankudinov
@stremlenye
Think of the result they give as Either[Throwable, Assertion]
@LukaJCB I do, but in more up-to-date ones :)
Luka Jacobowitz
@LukaJCB
I don’t think Either is a great replacement for SyncIO
Yury Ankudinov
@stremlenye
It's vice versa. We are slowly replacing Either. It's just it still a lot of the tests still implemented in an old way.