Where communities thrive

  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
Repo info
  • Sep 05 2019 14:43
    @typelevel-bot banned @jdegoes
  • Jan 31 2019 21:17
    codecov-io commented #484
  • Jan 31 2019 21:08
    scala-steward opened #484
  • Jan 31 2019 18:19
    andywhite37 commented #189
  • Jan 31 2019 02:41
    kamilongus starred typelevel/cats-effect
  • Jan 30 2019 00:01
    codecov-io commented #483
  • Jan 29 2019 23:51
    deniszjukow opened #483
  • Jan 29 2019 23:37
  • Jan 29 2019 23:22
  • Jan 29 2019 20:26
    Rui-L starred typelevel/cats-effect
  • Jan 29 2019 18:01
    jdegoes commented #480
  • Jan 29 2019 17:04
    thomaav starred typelevel/cats-effect
  • Jan 28 2019 17:43
    asachdeva starred typelevel/cats-effect
  • Jan 28 2019 07:12
    alexandru commented #480
  • Jan 28 2019 05:45
    codecov-io commented #482
  • Jan 28 2019 05:35
    daron666 opened #482
  • Jan 27 2019 13:56
    codecov-io commented #481
  • Jan 27 2019 13:46
    lrodero opened #481
  • Jan 27 2019 05:47
    codecov-io commented #460
  • Jan 27 2019 05:37
    codecov-io commented #460


Both solutions suffer from instrumenting call sites which evaluate prior to the run loop

Well, all ZIO tracing happens directly in the run loop – it can’t happen prior, since the data is just not created yet.

Really, the problem with bytecode instrumentation is it doesn’t work well at all with polymorphism. ZIO’s ecosystem is pretty monomorphic
Not saying either design is wrong, really. Just different trade offs and challenges, sparked by very different ecosystems and standard usage patterns.

I’ve made ZIO tracing specifically so that it would work well with tagless final – rejecting e.g. static instrumentation with macros because that would never work with TF. Now, it doesn’t work well with monad transformers, that’s unfortunately correct, but I disagree that default even in CE ecosystem is to use monad transformers, total IME ahead: I’ve seen most people stick to F=IO in business logic or use monix Task directly, and the only person I’ve seen use cats-mtl also used it with Ref instances – that tracing is ok with, not transformers.

I sketched the kind of traces you’d get from an exception in place of constructor…

import cats.data.OptionT
import zio._
import zio.interop.catz._
import zio.syntax._

object TestTraceOfOptionT extends zio.App {

  def x(z: Any => Task[Int]): OptionT[Task, Int] = OptionT.liftF[Task, Int](1.succeed).flatMap(_ => y(z))
  def y(z: Any => Task[Int]): OptionT[Task, Int] = OptionT[Task, Int](ZIO.some(()).flatMap {
    case Some(value) => z(value).map(Some(_))
    case None => ZIO.none
  def z: Any => Task[Int] = _ => throw new RuntimeException

  override def run(args: List[String]): UIO[Int] = {
    at TestTraceOfOptionT$.$anonfun$z$1(TestTraceOfOptionT.scala:13)
    at TestTraceOfOptionT$.$anonfun$y$1(TestTraceOfOptionT.scala:10)
    at zio.internal.FiberContext.evaluateNow(FiberContext.scala:272)
    at zio.internal.FiberContext.$anonfun$fork$1(FiberContext.scala:596)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

For OptionT this does look pretty relevant, but I don’t know what’ll happen with suspending transformers like Free or Stream. I do have a gut feeling that supporting every transformer will require a lot of manual work, though.

Daniel Spiewak

Certainly hard coding transformer or third party library call sites is intractable. Stack frame instrumentation though can ameliorate this issue since you have the thunk class and the full runtime stack, so you have the opportunity to trace up. Also, as I mention in the gist, providing extra frame information is really helpful for when that kind of thing fails and you’re trying to track down an error.

To be clear, I don’t think there’s anything wrong with ZIO’s tracing. It gives information that is very useful in a monomorphic or bijective polymorphic context (like your tagless final example). The problem is it is defeated immediately by something like Stream or even some library usage (tracing through an http4s app is varyingly useful, for example), and it doesn’t work at all with transformers. Again, that’s a fair tradeoff, particularly given the ZIO ecosystem’s focus on a single master effect without induction. I just think it’s possible to improve on that, albeit by accepting a totally different tradeoff (greater complexity in implementing lexical configuration), and given how cats IO is often used, it seems worthwhile to at least explore that direction.

Either way, all of these approaches have some unfortunate caveats. It’s possible to build stellar examples of the strengths of both, and also of the weaknesses of both. I’m not sure there’s ever going to be a silver bullet.

Daniel Spiewak
Oh I should clarify that the OptionT example trace does look quite reasonable. Lazy transformers definitely make things worse, as do functions inherited via abstractions (which happens a lot with cats IO). I guess I should be careful when I say “doesn’t work at all with transformers”, because it’s not really that cut and dried (as your example shows).
Rohde Fischer

I'm trying to understand a bit more about cats-effect, and thus did some experiments by modifying the example here: https://typelevel.org/cats-effect/concurrency/basics.html#thread-scheduling

what I did to make it have only one ExecutionContext and ContextShift and a simple producer/consumer using an MVar. However, here I stumbled upon a (to me) puzzling behaviour. In my first example using an explicit ExecutionContext and ContextShift it works as I expect, it keeps running till I terminate it: https://pastebin.com/EdB2k1Cq

in my second example though, I tried to trim down my application to rely on the default in IOApp, but here my program terminates almost immediately: https://pastebin.com/QuC0Putm

why does this happen? What should I read/see/etc to understand this behavior better?

Fabio Labella
do you know about daemon threads?
(a JVM concept, not a cats-effect one)
Rohde Fischer
@SystemFw not really :/ I'm halfway guessing that's the cause from the way you ask
Fabio Labella
depending on your point of view, it's debatable which one of the two examples is puzzling
think about it: start means that you return control immediately, and one fiber keeps going while another is spawn asynchronously
so you have producer.start >> consumer.start >> IO(println("when does this happen"))
the third statement should execute pretty much immediately
which means that the whole application should shut down immediately
does that view point make sense? (as in, do you see how that's coherent, if not correct)
Rohde Fischer
because Fibers aren't threads?
Fabio Labella
no, you can have the same concept with real threads
Rohde Fischer
where you start two threads and it still terminates?
Fabio Labella
Thread1.start(); Thread2.start(), println("when does this happen")
wait, we're getting to what actually happens
but do you see why that semantics is coherent?
start means "spawn something and keep going without waiting for it"
that's the whole point of it actually
so if you start, start, end
end will happen without waiting for the two started things to finish
Rohde Fischer
ah I see the coherency now, yes
Fabio Labella
right, another viewpoint is
if some things have started, I don't want to shut down until they are done
(this is the behaviour you were expecting)
that also makes sense right?
Rohde Fischer
Fabio Labella
whether a thread behaves in the first manner or the second is called daemonicity
well, as in, daemon threads do not prevent the JVM from shutting down
non daemon threads do
Rohde Fischer
yes, so to rephrase, it's basically a matter if I expected to wait for all child threads or if I expect it not to
Fabio Labella
you can rephrase your question as "does the thread pool in IOApp have daemon or not?"
and the answer is no, which is why you see what you see
if you look up the relevant issues you will find the debate as to why that was chosen
it is indeed debatable
Rohde Fischer
as almost all things in programming :)
Fabio Labella
anyway, you can join your fibers are the end to actually wait for them in the cats-effect realm, without knowing about threads
this concept (of not losing spawned concurrent processes) leads to a safer model of concurrency, and it's supported by e.g. fs2 concurrently
you can build it on top of a "lossy" model like cats-effect, which acts like a primitive
gtg now, hopefully it sheds some clarity
Rohde Fischer
a lot more than 10 minutes ago definitely, thanks a lot
Rohde Fischer
per the talk of daemon threads vs. non-daemon yesterday, I was actually wondering, would it be sound to always to join if I expect them to be non-daemon, so I can keep the code independent on which ExecutionContext is in use?
Paul Snively
I prefer to always join if that's the desired behavior.
Rohde Fischer
@paul-snively (y)
Torsten Schmits
do you guys let timers run on the global EC?
Gavin Bisesi
When we were migrating from App to IOApp we used the same pool that backed our ContextShift for the Timer. With IOApp it has its own ScheduledExecutorService