Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    wpoosanguansit
    @wpoosanguansit
    that being said, if we look at it optimistically, you can have a lot of room to keep improving.
    Alberto
    @agocorona
    Yes! I have to repeat it when I am less tired. It is hard to explain and my english is not fluent. I posted it here just to have such feedback
    wpoosanguansit
    @wpoosanguansit
    I think there are some good examples we can follow like - http://www.arow.info/blog/posts/2015-07-10-servant-intro.html
    the format is very clear - it starts out with what the framework tries to solve. then it brings in what are needed before going into detail on how those are fit together. it concludes on the whole again at the end.
    and i think it is very clear in what level is expected of the readers to be in and it then brings in, to most who are new to Haskell, quite advanced concepts showing what it solves before how it is being used to achieve that.
    from this i think it is worth while to break things down in a series of articles focusing on digestible chunks following the format.
    that is my 2cents
    Alberto
    @agocorona
    thanks for the feedback @wpoosanguansit
    I have put online an example application: agocorona/transient#5
    wpoosanguansit
    @wpoosanguansit
    you are welcome @agocorona. i am now picking up servant and snap. i will very like ping you when i can get the api for an upload going.
    and i think i will follow the style of the article in explaining how things would be done.
    Alberto
    @agocorona
    hi @geraldus how are you?. I created an organization https://github.com/transient-haskell and put the libraries in there
    Артур Файзрахманов
    @geraldus
    Hi Alberto! I'm mostly fine but still slow'n'lazy :)
    Rahul Muttineni
    @rahulmutt

    Hi @agocorona, your work on transient looks very interesting. I share your philosophy of making everything composable and I'm in search of a way to do distributed computing in a composable way (because actors clearly don't compose).

    I don't have time to take a good look at your work yet, but it would be great if you can share the current limitations of transient? And how does your method of doing distributed computing differ from Cloud Haskell? From a cursory glance, it looks like transient can do quite a lot with just a small number of combinators.

    Alberto
    @agocorona

    Hi @rahulmutt Thanks for your interest!

    This post has something about the differences between the actor model and transient: https://github.com/transient-haskell/transient/wiki/Transient-and-the-Actor-Model-(Erlang-OTP,-Cloud-Haskell,-Scala-Akka)

    The tutorial, in the section of remote execution has more information on how it is implemented in transient: https://github.com/transient-haskell/transient/wiki/Transient-tutorial#remote-execution--the-cloud-monad

    Limitations: It needs benchmarks and optimizations. For example serialization should be improved. It needs to test for space leaks and bugs in real exploitation cases...

    Alberto
    @agocorona
    Feel free to comment your impressions here!
    Rahul Muttineni
    @rahulmutt
    Thanks for the links! Btw, there's a startup in Bangalore using the concepts from your MFlow framework in Javascript and I think they've deployed this in production used by many banks. They said that MFlow gives them what FRP wasn't able to: a way to sequence flows properly. They tried using Elm and had a hard time expressing what they wanted. Hearing this production use case got me more curious about your work :)
    Alberto
    @agocorona
    They are using MFlow or something called HPlayground?
    basically I ported the second to the GHCJS compiler and integrated it with transient
    That is great!!
    Alberto
    @agocorona
    This message was deleted
    Alberto
    @agocorona
    @rahulmutt Do you have the way to contact with them in order to get feedback from them?
    Rahul Muttineni
    @rahulmutt
    Some feedback on transient:
    • It's a bit mind bending in that it's like using a higher-level list monad, but it's very, very cool. For beginning Haskellers, what would be really useful is a visualisation of what happens when you do various distributed/parallel stuff. It's almost shocking how effortlessly you can run computations across threads/nodes.
    • The cool part is the composability in the distributed setting. You can make higher-order monadic functions that allow you to compose & reuse a long chain of distributed transactions via wormhole and teleport. Another benefit is that the transaction becomes first class and you can see exactly what's going on in one place instead of distributing the logic across actors making the code equivalent to event callbacks, as you've stated.
    Rahul Muttineni
    @rahulmutt
    Some questions:
    1) What happens if a node fails a computation? Is resiliency taken care of with <|> even for distributed computations?
    2) Is there a way to abstract over the nodes/threads itself? Basically, I want to be able to describe a problem and have transient automatically distribute the computation across nodes/threads with load balancing with a focus on getting an entire job finished as soon as possible. It a computation fails, it should be restarted automatically and run on another thread/server. The dependency graph is already embedded inside the way you express a monadic action via bind and sending arguments to variables - can we extract this graph and run the computations across nodes accordingly? I guess this means we need something equivalent to runAny rather than `runAt node.
    Rahul Muttineni
    @rahulmutt
    I'll answer 2) - after looking at the distributed primitives it looks like this functionality is already handled. This is pretty cool.
    Alberto
    @agocorona
    Hi @rahulmutt thanks for the feedback! I will answer you along the day
    Alberto
    @agocorona
    Can I use your testimony in some future documentation?
    Alberto
    @agocorona

    1) What happens if a node fails a computation? Is resiliency taken care of with <|> even for distributed computations?

    yes <|> can provide resilience. It may execute two distributed computations in parallel. collect can be used to get the first response and discard the others. (although this works, collect need more testing )

    There are other ways: of dealing with errors: There is onFinish construction, that can handle the error and take some corrective action, for example choose another node. look for finalization in the tutorial.

    2) Is there a way to abstract over the nodes/threads itself?

    there is clustered and mclustered which execute a computation in many nodes. But you can construct your owns similarly, using applicative and alternative. In the tutoria. If this does not give you a hint, tell me. I will try to construct the primitive for your case

    Rahul Muttineni
    @rahulmutt
    @agocorona The testimonial you'll have to ask the guy, as I haven't personally used it. But hopefully I'll be building a nontrivial app in transient to see how it's able to handle big loads.
    So overall, I'm liking transient and most of my initial doubts have been addressed. You've done awesome work! I'll play around with it some more and start going through the codebase when I get time and keep you posted with questions/suggestions.
    Alberto
    @agocorona

    @rahulmutt with testimonial I mean the first impressions

    It's a bit mind bending in that it's like using a higher-level list monad, but it's very, very cool. For beginning Haskellers, what would be really useful is a visualisation of what happens .......

    and so on.

    Rahul Muttineni
    @rahulmutt
    Yeah of course go ahead.
    Alberto
    @agocorona
    nice ;)
    Rahul Muttineni
    @rahulmutt
    Btw, do you accept PRs for cosmetic changes? Like simplifying types using type synonyms, grammatical fixes, etc.?
    Alberto
    @agocorona
    of course
    Rahul Muttineni
    @rahulmutt
    Ok cool
    Alberto
    @agocorona
    that "is like using a high leve list monad" demonstrates that you really understand it
    these cosmetic changes help a lot
    Rahul Muttineni
    @rahulmutt
    Awesome. I'm glad I'm understanding it! There's a bit of a learning curve, but it's far easier than something like lens :)
    Alberto
    @agocorona
    Yes I think that some libraries in haskell like lens are overengineered
    Rahul Muttineni
    @rahulmutt
    I'm taking a look at Transient.Internals and I'm seeing a lot of unsafeCoerce - is it essential? I think you can make it more typesafe by using the type-aligned queues mentioned in the Freer monad paper by Oleg Kiselyov:
    data Continuations a b where
      SingleCont :: (a -> TransIO b) -> Continuations a b
      ConsCont   :: (a -> TransIO x)  -> Continuations x b -> Continuations a b
      AppendCont :: Continuations a x -> Continuations x b -> Continuations a b
    
    data EventF = forall a b. EventF {
      ...
      , xcomp :: TransIO a
      , fcomp :: Continuations a b
      ...
                                     }
    You can add smart constructors so this GADT doesn't leak to the user.
    Rahul Muttineni
    @rahulmutt
    Alternative definition with existential types:
    data Continuations a b =
      | SingleCont (a -> TransIO b)
      | forall x. ConsCont   (a -> TransIO x) (Continuations x b)
      | forall x. AppendCont (Continuations a x) (Continuations x b)
    We could remove AppendCont if you don't really need fast appending operation for continuations
    Alberto
    @agocorona
    If there is possible a Monad instance , even trivial for TransIO constructed with this data type, then yes .
    I tried, time ago but I think that I needed an Indexed monad, which has an extra type parameter
    But you can try, please.