Everything about these libraries that makes everithing infinitely composable https://github.com/transient-haskell/transient/blob/master/README.md
Hi @agocorona, your work on transient looks very interesting. I share your philosophy of making everything composable and I'm in search of a way to do distributed computing in a composable way (because actors clearly don't compose).
I don't have time to take a good look at your work yet, but it would be great if you can share the current limitations of transient? And how does your method of doing distributed computing differ from Cloud Haskell? From a cursory glance, it looks like transient
can do quite a lot with just a small number of combinators.
Hi @rahulmutt Thanks for your interest!
This post has something about the differences between the actor model and transient: https://github.com/transient-haskell/transient/wiki/Transient-and-the-Actor-Model-(Erlang-OTP,-Cloud-Haskell,-Scala-Akka)
The tutorial, in the section of remote execution has more information on how it is implemented in transient: https://github.com/transient-haskell/transient/wiki/Transient-tutorial#remote-execution--the-cloud-monad
Limitations: It needs benchmarks and optimizations. For example serialization should be improved. It needs to test for space leaks and bugs in real exploitation cases...
transient
:wormhole
and teleport
. Another benefit is that the transaction becomes first class and you can see exactly what's going on in one place instead of distributing the logic across actors making the code equivalent to event callbacks, as you've stated.<|>
even for distributed computations?runAny
rather than `runAt node.
1) What happens if a node fails a computation? Is resiliency taken care of with <|> even for distributed computations?
yes <|> can provide resilience. It may execute two distributed computations in parallel. collect
can be used to get the first response and discard the others. (although this works, collect
need more testing )
There are other ways: of dealing with errors: There is onFinish construction, that can handle the error and take some corrective action, for example choose another node. look for finalization in the tutorial.
2) Is there a way to abstract over the nodes/threads itself?
there is clustered
and mclustered
which execute a computation in many nodes. But you can construct your owns similarly, using applicative and alternative. In the tutoria. If this does not give you a hint, tell me. I will try to construct the primitive for your case
transient
and most of my initial doubts have been addressed. You've done awesome work! I'll play around with it some more and start going through the codebase when I get time and keep you posted with questions/suggestions.
Transient.Internals
and I'm seeing a lot of unsafeCoerce
- is it essential? I think you can make it more typesafe by using the type-aligned queues mentioned in the Freer monad paper by Oleg Kiselyov:data Continuations a b where
SingleCont :: (a -> TransIO b) -> Continuations a b
ConsCont :: (a -> TransIO x) -> Continuations x b -> Continuations a b
AppendCont :: Continuations a x -> Continuations x b -> Continuations a b
data EventF = forall a b. EventF {
...
, xcomp :: TransIO a
, fcomp :: Continuations a b
...
}
AppendCont
if you don't really need fast appending operation for continuations