Tracing is directly tied to runtime: you want a tracing id for each fiber on the system. You can hack it into Kleisli or one of the reader-like MTL type classes, but that's just a hack.
@jdegoes I'm curious, if using Kleisli or a reader-like MTL typeclass is a "hack" how does this gets done in pure functional languages like Haskell? I've only seen implementations based on ReaderT and MonadReader but my knowledge is limited :)
trace-id in use.
so, it could be all wrong, but basically I think the current model (interruptible flatMaps, interrupt on async boundaries) is ok except when you are trying to write concurrent code, because then you'd like to know where interruption could happen.
So my idea is a (possibly unimplementable :P) model based on 3 primitives, guaranteeCase, `mask, and poll.guaranteeCase is the same as now.mask is the same as uninterruptible, except when poll appears.poll makes an F wrapped in an interruptible block interruptible again.
So, the typical case where you want uninterruptible flatMaps locally, but interruption on an async thing, for example when using semaphore, could be:
mask {
poll { sem.acquire }.flatMap(yourThing)
}poll encapsulates the flatMap as well, then you can interrupt that as well
guaranteeCase in the places it makes sense too
zio.Resource? (apart from its definition)
poll an action containing that snippet above, its semantics are preserved, and it only gets interrupted where the author deemed it safe to. Otoh, most code keeps the property of not needing any specific "I want it to be interrupted here" that you get with a cancelBoundary sort of model (which is also weird because it only applies to canceling flatMaps, but not async actions)
uninterruptible is needed, which would override any mask/poll thing where you truly want to say that this can never be interrupted in any case
mask) changes to "I don't want what I know about to be interruptible." But things you don't know about (e.g. acquire) may independently decide they are ok with interruption. I think users could be trained to deal with that interpretation. The main drawback I see is that, if you say something's not interruptible (mask), just so you don't have to deal with the possibility of interruption, well, you may still have to add a guaranteeCase to deal with the possibility, because whether or not something is truly interruptible depends on details you'll never know, they're buried deeper. "Inner most interruptible wins".
uninterruptible. If you've handled errors and you make something uninterruptible, you have the strong guarantees necessary to implement higher-level semantics like bracket or Resource. Now you have to think carefully because masking something doesn't guarantee it won't be interrupted unless whatever actions you run do not use poll. Which you may not know (you definitely won't know in the case of combinators).
@jdegoes
I've reread
https://haskell-lang.org/tutorial/exception-safety
http://www.well-typed.com/blog/97/
in detail. These are my takeaways:
- There is a fundamental tradeoff between safety and deadlock prevention
- Allowing some actions to be interruptible even in a masked scenario is essential for deadlock prevention
- Overriding the above sentence is necessary for absolute safety
Overall, what I proposed above is basically how the Haskell model works, if you dive deep enough, but condensed to 4 primitives (which are higher level than Haskell's, none of the following are primitives there).
The primitives are: mask, poll, uninterruptible, guaranteeCase.bracket can be defined as:
mask {
acquire.flatMap { r =>
poll { use(r) }.guaranteeCase { _ => release.uninterruptible }
}
}Consensus seems to be that it's not necessary to make acquire uninterruptible, it can take care of itself. In any case the model allows both.
There is debate on release, but overall it's preferred to make it uninterruptible to err on the side of caution.
Also note that if poll is defined as taking interruption and raising something akin to InterruptedException, guarantee is not needed, it becomes handleErrorWith.
Now, it's true that you need to know which actions are "pollable" (basically things like Semaphore.acquire), but that's an unavoidable limitation due to the tension mentioned above:
to avoid deadlock you need to trust that the implementation of things you rely on is interruptible (e.g using bracket with semaphore), but to guarantee safety in all cases you need to never trust the implementations and make them all uninterruptible. (e.g. using bracket when opening a file).
Therefore, a sane model needs to allow both, or it will be broken in the other case.
I am getting this error:
value foreach is not a member of cats.effect.IO[String]
Code:
val content = for {
content <- fetchUrl(url)
} contentdef fetchUrl(url: String): IO[String] = {
httpClient.expectString
}
Acquire in general cannot be interruptible. openFile <* IO.unit cannot leak resources just because of the unit at the end. Similarly for release, otherwise it will leak resources. Concurrent data structures are a special exception because you can divide the acquisition into a non-interruptible section, followed by an async interruptible section.
The difficulty I have with this new model (which I otherwise quite like) is that if someone gives me an io that I have to run, and my acquire looks like openFile <* io, then the safety of my code depends not on what I write ((openFile <* io).bracket(...)(...)), but on what io I was passed. In other words, local reasoning about resource safety breaks down. It's now necessary to do whole-program analysis to figure out if your code is resource safe.
The current model can be reasoned about locally. That is, you can know just looking locally whether or not you can leak resources. Whole program analysis is not necessary. Of course, you don't know if the program will deadlock but that's beyond the realm of most static type systems anyway and there are lots of ways to deadlock (MVar, Queue, etc.).
bracket would be implemented with uninterruptible and not mask so you wouldn't have this problem. But then you're back in "deadlock" land with semaphore.acquire.bracket(_ => semaphore.release)(...).