Where communities thrive

  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
    The test.hs contain some expressions that I'm testing
    Tim Pierson
    Michał J. Gajda
    @o1lo01ol1o Nice to see that the work progresses here :-).
    @o1lo01ol1o I saw that you stopped responding to me on private channel. Is it something I said about money?
    Tim Pierson
    @mgajda No, gitter just gets closed and some of the messages fall through the cracks; keybase is useually better
    Michał J. Gajda
    @o1lo01ol1o I am also available on Keybase. Indeed it seems better. I am a bit scared after I heard the news that Zoom took them over.
    Tim Pierson
    Yes, I know, it's a bit worrying
    I have to include a second set of required effects:
    T (required ::[*])  (produced ::[*]) a

    Great. now I can express the pay-> sendStuff constraint:

    data HasPaid
    data SentStuff
    pay  ::Int->  T '[]  '[HasPaid]()
    pay i=  undefined
    sendStuff ::   T '[HasPaid] '[SentStuff] ()
    sendStuff =undefined
    test5=   liftIO (print "hello") >> (return "hello" >> sendStuff)
    test6=  pay 10 >> test5
    *Main> :t test5
    test5 :: T '[HasPaid] '[IOEff, SentStuff] ()    -- test5 need the HasPaid effect, produces IOEff, SendStuff
    *Main> :t test6
    test6 :: T '[] '[HasPaid, IOEff, SentStuff] ()  -- test6  need no effect (since pay is included)

    Now I can also codify get like the state monad:

    get :: T [State a] '[] a
      :: Typeable a =>
         T '[] effs a -> IO (Maybe a)       -- accept only computations with []  requirements
    *Main> :t keep test6                        -- OK
    keep test6 :: IO (Maybe ())
    *Main> :t keep test5
    <interactive>:1:6: error:                  -- does not type check since HasPaid is not presentCouldn't match type ‘'[HasPaid]’ with ‘'[]’
          Expected type: Tnr '[IOEff, SentStuff] ()
            Actual type: T '[HasPaid] '[IOEff, SentStuff] ()In the first argument of ‘keep’, namely ‘test5’
          In the expression: keep test5

    So sendStuff could not be executed without payment.

    (Tnr eff x is an shorthand for T '[] effs x. Probably I will remove it)

    Is that nice and useful or what?
    So a software component can publish the effect required and the effects that it produces, and it cannot compile until all required effects have been given.
    That is what I was looking for since the beginning. Not a mere description of what each program does, but some way to let the compiler verify the requirements that each programmer can use in his domain problem.
    Uploaded the last version with required effects
    I added more modules. The idea is to have a typelevel module for each transient module
    One of the main problems in transient is to know if block of code continues or not, since the execution can be stopped at any moment. Moreover, since the transient is inherently multithreaded and single threading is just a particular case, some threads can follow a path in which they stop and other don´t.

    For example

    test8= do
        x <- choose [1..10]
        if x `rem` 2 == 0 then empty else liftIO $ print x
    *Main> keep' test8

    This program only print the even number since the odd numbers are filtered by if-empty. The threads created by choose, one for each number, stop at guard when they carry an odd number. The type of this program could be like that:

    test8 :: T '[] '[Async, MThread, Streaming, Maybe EarlyTermination, IOEff] ()

    It has Maybe EarlyTerminationsince some threads terminate and others don't

    for this purpose, in the new type-level transient, the type of empty is

    empty :: T ' [] '[EarlyTermination] a

    But this is also a problem since both branches of ifThenElse should have the same signature. Since print x has type T '[] '[IOEff] (), the program does not type match. So it is necessary to use guard instead.

    guard :: Bool -> T '[] '[Maybe EarlyTermination] ()
    test8= do
        x <- choose [1..10:: Int]
        guard $ rem  x 2 == 0 
        liftIO $ print x

    This programs actually type match and has the type desired.

    In the other side:

    test9 =  do
    Main> :t test9
      :: T '[] '[Async, MThread, Streaming, EarlyTermination, IOEff] b

    That is, since empty guarantees that test9 stop for all the threads, then test9 terminates. So the Maybe of test8 is supressed

    So far so good. I renamed the long name Earlytermination to Terminates

    or perhaps "Stop" instead
    Hugh JF Chen

    One of the main problems in transient is to know if block of code continues or not, since the execution can be stopped at any moment. Moreover, since the transient is inherently multithreaded and single threading is just a particular case, some threads can follow a path in which they stop and other don´t.

    I must admit that this is the most confusing problem I'd been facing since the first time I met transient. So a type level safety is really welcome and necessary

    I suspect that if-then-else expressions with monadic code will no type-match in many cases when using this library.

    For example:

    test10 x= if x == 5 then liftIO $ print x else askfornumber
    askfornumber= do
          n <- input (<10) "give me a number >"  
          liftIO $ print (n :: Int)

    The error produced at the else :

    src/Transient/TypeLevel/test.hs:41:48: error:
        • Couldn't match type ‘Terminal’ with ‘IOEff’
          Expected type: T '[] '[IOEff] ()
            Actual type: T '[] '[Terminal, IOEff] ()
        • In the expression: askfornumber
          In the expression:...

    because the "then" side has type T '[] '[Terminal] () since it uses terminal input, while the "else" side uses IO. Since both branches should have the same type, hence the error.

    To solve the problem, the best way I found until now is to use guard in combination with alternative, which is IMHO, a good style that improves readability:

    test11 x=  is5 x <|> askfornumber
     is5 x=  do guard $ x==5 ; liftIO $ print x

    This type check. What is the signature? A mess of Termination-maybe-non-termination?
    Using GHCI:

    *Main> :t test11
      :: (Eq a, Num a, Show a) => a -> T '[] '[IOEff, Terminal] ()

    The type adds the two effects. The same type that the if-then-else expression would have if it would type-check.

    Termination is wiped out since <|> has been redefined under the "RebindableSyntax" extension to cancel any Termination effect of the first term


    Ooops... No. That latter type is not good, since I can cheat the compiler telling that I paid when I don't:

    (do guard False; pay) <|> sendStuff

    This is the same old problem that I had with "empty" and alternatives, now appears again with "guard".

    After some type-level list processing, the type of test11 is the desired:
    *Main> :t   test11
      :: (Eq a, Num a) => a -> T '[] '[Maybe Terminal, Maybe IOEff] ()

    I told that one of the requirements is to allow the definition of contraints between effects defined by the programmer like the pay-> sendStuff ordering.But these are a zero level kind of requirement.

    Even though I thought it couldn't be possible, it's possible to give the programmer detailed control of invariants of the domain problem using self-defined effects by making use of the rebindable syntax, so that the programmer can redefine the monadic,applicative etc operators using the already defined operators as base:

    {-# LANGUAGE RebindableSyntax, TypeFamilies, TypeOperators,DataKinds,RankNTypes, ScopedTypeVariables #-}
    import  Prelude  hiding ((>>=),(>>),return,concat)
    import qualified Prelude as P 
    import qualified Transient.TypeLevel.Effects as Eff
    import Transient.TypeLevel.Base 
    import Transient.TypeLevel.Move
    import Transient.TypeLevel.Indeterminism
    -- Transient.TypeLevel.Effects has >>= redefined using  rebindable syntax too
    data MyEffect1
    data MyEffect2
    -- my own bind ,  for my problem, with more effects rules defined by me. 
    -- He can use the rules already defined in the Eff module to manage effects to sum, subtract, 
    -- test inclusion etc
    -- In this case, not redefined
    (>>=) = (Eff.>>=)
    I'm using transient in a project with web services and I spent some days dealing with timeouts and reconnections in HTTP/1.1 with TLS. and also an obnoxious bug with the interpretation of the command line parameters. In certain precise circumstances and with some parameters these problems produced so crazy outputs that made me think that I was dealing with some fundamental flaw of the execution model.
    At the end I was just a matter of bad management of two varables and some unnecessary complications in the code.
    I learned one thing or two. The HTTP code is producing more headaches than anything else in transient. I will upload a new version as soon as I can.
    watched the superb talk of alexis king about performance and effects.

    This talk inspires me some considerations about effect systems, continuations and performance:
    I wanted continuations to be given by the runtime instead of in programmer space in a continuation monad. In fact I worked in ETA to redefine it so that the IO monad delivers it. That was my dream since many effects are not possible without continuations. But having continuations as a monad in a monadic stack add extra complexity and extra performance loss.

    I tried to define transient so that the primitives are independent of how the effects are implemented, as long as they provide state and continuations over Applicative/Alternative/Monad instances. In fact I have two different engines. The second one uses a different continuations monad and exceptions to implement the Alternative instance. When the patch of Alexis become available, the translation of transient to wathever monad that uses it is relatively straigforward.

    In the meantime, The current monad does not add a lot of boilerplate. The monad implements continuations maintaining a stack of bind parameters which is lazily constructed and discarded when not used. It is possible to use noTrans, to use the underlying monad, when I write ordinary code without async, empty, react or distributed computing. The bind in this case is as fast as one of the ordinary state monad. What may be slower is getState and setState. It is the price to pay for the flexibility of intruducing and retrieving any variable state at any moment, as many of them as you please, without recompiling/redefining/adding instances runners and/or handlers anywhere in the rest of the modules that may be involved. However that performance of state is similar to the best of the extensible record libraries i guess.
    To handle many states is the main chore of monadic stacks and effect systems. since the rest of the effects have limited number of them, but states can be unlimited. However in Transient the need of state is drastically reduced, since basically data is in scope in the form of ordinary variables:
    let  x=  MyData.....
    y <-  multireaded,distributed,streaming...
    --   x still in scope! (even if now it is in a different machine!!)
    Concerning effects, there are different views of what an effect is:
    • In effect libraries, an effect is associated with an interpreter/handler which is called when the effect is needed
    • In Final Tagless/MTL libraries, they are monad transformers in a stack.
    • In libraries like RIO they are class instances that group methods.
    • In Transient, in the type-level transient that I'm developing, effects are salient features of the computation

    The first three can aggregate effects in the type signature and this is good for documentation purposes and to tell the type system what and how the runners and lifters should be inserted in the code to make it type match.

    I think the main functionality of type-level lists of effects in effect libraries is to make type signatures less ugly than monad transformers and to make sure that the runner of each effect is present. and Oleg Kiselyov designed them originally for that purpose. In the case of MTL, the monadic stack signatures are a consequence of his very structure, and in the case of Has.... strategy, it is intended for documentation purposes, to make the program more readable.

    In Transient since the effect can be created with a simple data declaration without runtime representation, they can be used to check the compliance of domain-specific requirements at compile time using type-level logic, as the examples written here above detail.

    In transient any other effect can be created by the combination of the already existent. I don't provide a reader of writer effect since getRData/setRData is equivalent to the reader/writer effects.
    I have to create a table of how-to-do features based on
    I defined whileException:
    -- re execute the first argument as long as the exception is produced within the argument. 
    -- The second argument is executed before every re-execution
    -- if the second argument executes `empty` the execution is aborted.
    whileException :: Exception e =>  TransIO b -> (e -> TransIO())  -> TransIO b 
    whileException mx fixexc =  mx `catcht` \e -> do fixexc e; whileException mx fixexc
    the IO version would be exactly the same, but using catch instead of catchc.

    The motivation is that sometimes it is necessary to retry some task after some fix that tries to solve the problem:

         `whileException` \theException-> fixTheException

    I had this problem with sockets. when a connection is closed by the remote node, the program is not aware of in until he tries to send again, so I can not preemtively reopen the channel. so:

    sendSecure  connection toSend= do
          send connection toSend
      `whileException` \(SomeException _) -> reopen connection
    Transient programs seems to me like an assembly line where there are many workers doing certain precise task, everyone is busy at the same time, while a program with a traditional execution model seems to me like an assembly line with a single man which build the car from the beginning to the end. He may give work to other workers while he is waiting, but, since he is coordinating, many tasks that may be done in parallel can not be done and only a single car can be assembled in the line each time.
    The next car is waiting for the man to finish the previous car and run to the beginning of the assembly line in a loop. He can not ask for something
    [He can not ask for something] <- unfinished phrase
    This message was deleted
    Alberto @agocorona 01:12
    Dialog between a Web server and myself:
    • Me: Why you don't answer my second request after 20 seconds? We agreed that the connection keeps open: I said HTTP/1.1 and you told me "Connection: Keep-Alive"...The connection is open,I know, you receive my request, but you don't answer me after 20 seconds. Why you don't close the connection to signal that you don't want anything more trough this channel?
    • Serv: If you make the request through a Web Browser you will know.
    • Me: Okay, making the request with chrome, you answer that your keep-alive connections have a timeout of 15 seconds. Why don't you say It to the Transient application?
    • Serv: You don't send headers. I like headers...
    • Me: Okay, so I have to send to you "Connection: Keep-Alive"? HTTP/1.1 in the first line says so implicitly...By the way, why don't you simply close the connection or, else send me the timeout info? Why aren't you more explicit in the response?
    • Serv: I am a server and I like headers
    • Me: Okay take it: "Connection: Keep-Alive"
    • Serv: OK "Keep-Alive: 15, max=500"
    • Me: Ok, now my program know that you keep the connection open indefinitely, but after 15 seconds or 500 requests, you won't answer
    • Serv: Yes. I like headers
    Hugh JF Chen
    @agocorona That's why I use req as the http client within my transient service and ask you if we can embed a http server, like warp to expose the transient service.
    @hughjfchen You're right. I have been thinking about leaving this fight. I recommended you at the beginning to use other packages for the HTTP stuff. Integrating another webserver directly in transient is not possible since the communication node-node uses another protocol. The transient server should manage both protocols because the integration with web services is a practical need at the end of the day and is a priority. On the other side, that allows me to use transient primitives for parsing, process management, and also enjoy solving problems in interesting ways.
    Do you use communications node-to-node in your case? runAt etc?
    @hughjfchen Do you use communications node-to-node like runAt etc?

    About backtracking and exceptions
    Going through the code I found this commentary (3 years ago!) from @harendra-kumar that I haven´t noticed

    -- XXX Should we enforce retry of the same track which is being undone? If the
    -- user specifies a different track would it make sense?

    Transient uses a kind of multi-level backtracking to solve a variety of problems, like closing open resources, undoing transactions, or even navigating recursive structures. Also, it is used for the management of exceptions. This backtracking is different from the sigle level backtracking of the alternative.

    As you can see in the example of the tutorial. ( I translated the undo track to am exception Undo. That is semantically similar, with the advantage that this allows to backtrack haskell exceptions when they happens anywhere in the code.

    #!/usr/bin/env stack
    -- stack --install-ghc --resolver lts-6.23 runghc   --package transient 
    import Control.Monad.IO.Class (liftIO)
    import Transient.Base
    import Transient.Backtrack
    instance Exception Undo where
    main= keep $ do
           liftIO $ print "done!"
    productNavigation = liftIO $ putStrLn "product navigation"
    reserve= liftIO (putStrLn "product reserved,added to cart")
                     `onException'` \Undo -> liftIO (putStrLn "product un-reserved")
    updateDB=  liftIO  (putStrLn "update other database necesary for the reservation")
                     `onException'` \Undo -> liftIO (putStrLn "database update undone")
    payment = do
               liftIO $ putStrLn "Payment failed"
               throw  Undo

    Here undo initiates the process of backtracking due to the failure in the payment. This executes from bottom to top the second argument of the onException' until a restarting of the normal flow is found (or the thread stop, as is in this case)

    It makes sense that each onException' does not manipulate this backtracking BY DEFAULT since this enhances composability and modularity. Imagine the real case, with real stuff, databases etc; These steps may be in different modules, in different libraries, maybe by different teams or even companies. Does it make sense to pass to them the responsibility for how the backtracking should continue? It is better to leave this decision to the application programmer which initiates the undo.

    Ok but how the application programmer can taylor the backtracking to his needs?

    onException $ \Undo -> do
                 resp <-         option "retry" "retry all that was undone" <|> 
                                 option "stop" "no further actions" <|>
                                 option "back" "continue backtracking" <|>
                                 option "other"  "throw a different exception" 
                                 option  "bother" "backtrack with other track"
                 case resp of
                          "retry" -> continue
                          "stop"  -> empty
                          "back" -> backtrack
                          "other"-> throw SomeOtherException
                          "bother" ->  back Other

    More succinctly (at the cost of some readability):

                                 (do option "retry" "retry all the undone"; retry) <|> 
                                 (do option "stop" "no further actions"     ; empty) <|>
                                 (do option "back" "continue backtracking" ; backtrack) <|>
                                 (do option "other"  "throw a different exception" ; throw SomeOtherException) <|> 
                                 (do option  "bother" "backtrack with other track"  ; back Other)
    These are the options. The three first are there since the beginning, and the default is backtrack. The other two were close to work reliably, but in the case of exceptions there was some infinite loop when the exception re-raised was of the same type than the original exception. After adding some more code and blowing my mind with many balls in the air: exceptions, continutions, threads... I found that eliminating some code and moving some lines the problem was solved, now working in my box and I will upload it as son as I finish some tests. adn fix some other bugs.