by

## Where communities thrive

• Join over 1.5M+ people
• Join over 100K+ communities
• Free without limits
• Create your own community
##### Activity
Alberto
@agocorona
I have to include a second set of required effects:
T (required ::[*])  (produced ::[*]) a
Alberto
@agocorona

Great. now I can express the pay-> sendStuff constraint:


data HasPaid
data SentStuff

pay  ::Int->  T '[]  '[HasPaid]()
pay i=  undefined

sendStuff ::   T '[HasPaid] '[SentStuff] ()
sendStuff =undefined

test5=   liftIO (print "hello") >> (return "hello" >> sendStuff)

test6=  pay 10 >> test5

*Main> :t test5
test5 :: T '[HasPaid] '[IOEff, SentStuff] ()    -- test5 need the HasPaid effect, produces IOEff, SendStuff

*Main> :t test6
test6 :: T '[] '[HasPaid, IOEff, SentStuff] ()  -- test6  need no effect (since pay is included)

Now I can also codify get like the state monad:

get :: T [State a] '[] a
since
keep
:: Typeable a =>
T '[] effs a -> IO (Maybe a)       -- accept only computations with []  requirements

*Main> :t keep test6                        -- OK
keep test6 :: IO (Maybe ())

*Main> :t keep test5

<interactive>:1:6: error:                  -- does not type check since HasPaid is not present
• Couldn't match type ‘'[HasPaid]’ with ‘'[]’
Expected type: Tnr '[IOEff, SentStuff] ()
Actual type: T '[HasPaid] '[IOEff, SentStuff] ()
• In the first argument of ‘keep’, namely ‘test5’
In the expression: keep test5
Alberto
@agocorona

So sendStuff could not be executed without payment.

(Tnr eff x is an shorthand for T '[] effs x. Probably I will remove it)

Is that nice and useful or what?
Alberto
@agocorona
So a software component can publish the effect required and the effects that it produces, and it cannot compile until all required effects have been given.
Alberto
@agocorona
That is what I was looking for since the beginning. Not a mere description of what each program does, but some way to let the compiler verify the requirements that each programmer can use in his domain problem.
Alberto
@agocorona
Alberto
@agocorona
Uploaded the last version with required effects
Alberto
@agocorona
I added more modules. The idea is to have a typelevel module for each transient module
Alberto
@agocorona
One of the main problems in transient is to know if block of code continues or not, since the execution can be stopped at any moment. Moreover, since the transient is inherently multithreaded and single threading is just a particular case, some threads can follow a path in which they stop and other don´t.
Alberto
@agocorona

For example

test8= do
x <- choose [1..10]
if x rem 2 == 0 then empty else liftIO $print x *Main> keep' test8 2 4 6 8 10 This program only print the even number since the odd numbers are filtered by if-empty. The threads created by choose, one for each number, stop at guard when they carry an odd number. The type of this program could be like that: test8 :: T '[] '[Async, MThread, Streaming, Maybe EarlyTermination, IOEff] () It has Maybe EarlyTerminationsince some threads terminate and others don't for this purpose, in the new type-level transient, the type of empty is empty :: T ' [] '[EarlyTermination] a But this is also a problem since both branches of ifThenElse should have the same signature. Since print x has type T '[] '[IOEff] (), the program does not type match. So it is necessary to use guard instead. guard :: Bool -> T '[] '[Maybe EarlyTermination] () test8= do x <- choose [1..10:: Int] guard$ rem  x 2 == 0
liftIO $print x This programs actually type match and has the type desired. In the other side: test9 = do test8 empty Main> :t test9 test9 :: T '[] '[Async, MThread, Streaming, EarlyTermination, IOEff] b That is, since empty guarantees that test9 stop for all the threads, then test9 terminates. So the Maybe of test8 is supressed So far so good. I renamed the long name Earlytermination to Terminates Alberto @agocorona or perhaps "Stop" instead Hugh JF Chen @hughjfchen One of the main problems in transient is to know if block of code continues or not, since the execution can be stopped at any moment. Moreover, since the transient is inherently multithreaded and single threading is just a particular case, some threads can follow a path in which they stop and other don´t. I must admit that this is the most confusing problem I'd been facing since the first time I met transient. So a type level safety is really welcome and necessary Alberto @agocorona I suspect that if-then-else expressions with monadic code will no type-match in many cases when using this library. Alberto @agocorona For example: test10 x= if x == 5 then liftIO$ print x else askfornumber

askfornumber= do
n <- input (<10) "give me a number >"
liftIO $print (n :: Int) The error produced at the else : src/Transient/TypeLevel/test.hs:41:48: error: • Couldn't match type ‘Terminal’ with ‘IOEff’ Expected type: T '[] '[IOEff] () Actual type: T '[] '[Terminal, IOEff] () • In the expression: askfornumber In the expression:... because the "then" side has type T '[] '[Terminal] () since it uses terminal input, while the "else" side uses IO. Since both branches should have the same type, hence the error. To solve the problem, the best way I found until now is to use guard in combination with alternative, which is IMHO, a good style that improves readability: test11 x= is5 x <|> askfornumber where is5 x= do guard$ x==5 ; liftIO $print x This type check. What is the signature? A mess of Termination-maybe-non-termination? Using GHCI: *Main> :t test11 test11 :: (Eq a, Num a, Show a) => a -> T '[] '[IOEff, Terminal] () The type adds the two effects. The same type that the if-then-else expression would have if it would type-check. Termination is wiped out since <|> has been redefined under the "RebindableSyntax" extension to cancel any Termination effect of the first term Alberto @agocorona Ooops... No. That latter type is not good, since I can cheat the compiler telling that I paid when I don't: (do guard False; pay) <|> sendStuff This is the same old problem that I had with "empty" and alternatives, now appears again with "guard". Alberto @agocorona After some type-level list processing, the type of test11 is the desired: *Main> :t test11 test11 :: (Eq a, Num a) => a -> T '[] '[Maybe Terminal, Maybe IOEff] () Alberto @agocorona I told that one of the requirements is to allow the definition of contraints between effects defined by the programmer like the pay-> sendStuff ordering.But these are a zero level kind of requirement. Even though I thought it couldn't be possible, it's possible to give the programmer detailed control of invariants of the domain problem using self-defined effects by making use of the rebindable syntax, so that the programmer can redefine the monadic,applicative etc operators using the already defined operators as base: {-# LANGUAGE RebindableSyntax, TypeFamilies, TypeOperators,DataKinds,RankNTypes, ScopedTypeVariables #-} import Prelude hiding ((>>=),(>>),return,concat) import qualified Prelude as P import qualified Transient.TypeLevel.Effects as Eff import Transient.TypeLevel.Base import Transient.TypeLevel.Move import Transient.TypeLevel.Indeterminism ... -- Transient.TypeLevel.Effects has >>= redefined using rebindable syntax too data MyEffect1 data MyEffect2 .... -- my own bind , for my problem, with more effects rules defined by me. -- He can use the rules already defined in the Eff module to manage effects to sum, subtract, -- test inclusion etc -- In this case, not redefined (>>=) = (Eff.>>=) Alberto @agocorona I'm using transient in a project with web services and I spent some days dealing with timeouts and reconnections in HTTP/1.1 with TLS. and also an obnoxious bug with the interpretation of the command line parameters. In certain precise circumstances and with some parameters these problems produced so crazy outputs that made me think that I was dealing with some fundamental flaw of the execution model. At the end I was just a matter of bad management of two varables and some unnecessary complications in the code. I learned one thing or two. The HTTP code is producing more headaches than anything else in transient. I will upload a new version as soon as I can. Alberto @agocorona watched the superb talk of alexis king about performance and effects. Alberto @agocorona This talk inspires me some considerations about effect systems, continuations and performance: I wanted continuations to be given by the runtime instead of in programmer space in a continuation monad. In fact I worked in ETA to redefine it so that the IO monad delivers it. That was my dream since many effects are not possible without continuations. But having continuations as a monad in a monadic stack add extra complexity and extra performance loss. I tried to define transient so that the primitives are independent of how the effects are implemented, as long as they provide state and continuations over Applicative/Alternative/Monad instances. In fact I have two different engines. The second one uses a different continuations monad and exceptions to implement the Alternative instance. When the patch of Alexis become available, the translation of transient to wathever monad that uses it is relatively straigforward. Alberto @agocorona In the meantime, The current monad does not add a lot of boilerplate. The monad implements continuations maintaining a stack of bind parameters which is lazily constructed and discarded when not used. It is possible to use noTrans, to use the underlying monad, when I write ordinary code without async, empty, react or distributed computing. The bind in this case is as fast as one of the ordinary state monad. What may be slower is getState and setState. It is the price to pay for the flexibility of intruducing and retrieving any variable state at any moment, as many of them as you please, without recompiling/redefining/adding instances runners and/or handlers anywhere in the rest of the modules that may be involved. However that performance of state is similar to the best of the extensible record libraries i guess. Alberto @agocorona To handle many states is the main chore of monadic stacks and effect systems. since the rest of the effects have limited number of them, but states can be unlimited. However in Transient the need of state is drastically reduced, since basically data is in scope in the form of ordinary variables: let x= MyData..... y <- multireaded,distributed,streaming... -- x still in scope! (even if now it is in a different machine!!) Alberto @agocorona Concerning effects, there are different views of what an effect is: • In effect libraries, an effect is associated with an interpreter/handler which is called when the effect is needed • In Final Tagless/MTL libraries, they are monad transformers in a stack. • In libraries like RIO they are class instances that group methods. • In Transient, in the type-level transient that I'm developing, effects are salient features of the computation Alberto @agocorona The first three can aggregate effects in the type signature and this is good for documentation purposes and to tell the type system what and how the runners and lifters should be inserted in the code to make it type match. I think the main functionality of type-level lists of effects in effect libraries is to make type signatures less ugly than monad transformers and to make sure that the runner of each effect is present. and Oleg Kiselyov designed them originally for that purpose. In the case of MTL, the monadic stack signatures are a consequence of his very structure, and in the case of Has.... strategy, it is intended for documentation purposes, to make the program more readable. In Transient since the effect can be created with a simple data declaration without runtime representation, they can be used to check the compliance of domain-specific requirements at compile time using type-level logic, as the examples written here above detail. Alberto @agocorona In transient any other effect can be created by the combination of the already existent. I don't provide a reader of writer effect since getRData/setRData is equivalent to the reader/writer effects. Alberto @agocorona I have to create a table of how-to-do features based on https://www.47deg.com/blog/io-haskell/ Alberto @agocorona I defined whileException: -- re execute the first argument as long as the exception is produced within the argument. -- The second argument is executed before every re-execution -- if the second argument executes empty the execution is aborted. whileException :: Exception e => TransIO b -> (e -> TransIO()) -> TransIO b whileException mx fixexc = mx catcht \e -> do fixexc e; whileException mx fixexc the IO version would be exactly the same, but using catch instead of catchc. Alberto @agocorona The motivation is that sometimes it is necessary to retry some task after some fix that tries to solve the problem:  do mayRaiseException whileException \theException-> fixTheException I had this problem with sockets. when a connection is closed by the remote node, the program is not aware of in until he tries to send again, so I can not preemtively reopen the channel. so: sendSecure connection toSend= do send connection toSend whileException \(SomeException _) -> reopen connection Alberto @agocorona Transient programs seems to me like an assembly line where there are many workers doing certain precise task, everyone is busy at the same time, while a program with a traditional execution model seems to me like an assembly line with a single man which build the car from the beginning to the end. He may give work to other workers while he is waiting, but, since he is coordinating, many tasks that may be done in parallel can not be done and only a single car can be assembled in the line each time. The next car is waiting for the man to finish the previous car and run to the beginning of the assembly line in a loop. He can not ask for something Alberto @agocorona [He can not ask for something] <- unfinished phrase Alberto @agocorona This message was deleted Alberto @agocorona Alberto @agocorona 01:12 Dialog between a Web server and myself: • Me: Why you don't answer my second request after 20 seconds? We agreed that the connection keeps open: I said HTTP/1.1 and you told me "Connection: Keep-Alive"...The connection is open,I know, you receive my request, but you don't answer me after 20 seconds. Why you don't close the connection to signal that you don't want anything more trough this channel? • Serv: If you make the request through a Web Browser you will know. • Me: Okay, making the request with chrome, you answer that your keep-alive connections have a timeout of 15 seconds. Why don't you say It to the Transient application? • Serv: You don't send headers. I like headers... • Me: Okay, so I have to send to you "Connection: Keep-Alive"? HTTP/1.1 in the first line says so implicitly...By the way, why don't you simply close the connection or, else send me the timeout info? Why aren't you more explicit in the response? • Serv: I am a server and I like headers • Me: Okay take it: "Connection: Keep-Alive" • Serv: OK "Keep-Alive: 15, max=500" • Me: Ok, now my program know that you keep the connection open indefinitely, but after 15 seconds or 500 requests, you won't answer • Serv: Yes. I like headers Hugh JF Chen @hughjfchen @agocorona That's why I use req as the http client within my transient service and ask you if we can embed a http server, like warp to expose the transient service. Alberto @agocorona @hughjfchen You're right. I have been thinking about leaving this fight. I recommended you at the beginning to use other packages for the HTTP stuff. Integrating another webserver directly in transient is not possible since the communication node-node uses another protocol. The transient server should manage both protocols because the integration with web services is a practical need at the end of the day and is a priority. On the other side, that allows me to use transient primitives for parsing, process management, and also enjoy solving problems in interesting ways. Do you use communications node-to-node in your case? runAt etc? Alberto @agocorona @hughjfchen Do you use communications node-to-node like runAt etc? Alberto @agocorona About backtracking and exceptions Going through the code I found this commentary (3 years ago!) from @harendra-kumar that I haven´t noticed -- XXX Should we enforce retry of the same track which is being undone? If the -- user specifies a different track would it make sense? Transient uses a kind of multi-level backtracking to solve a variety of problems, like closing open resources, undoing transactions, or even navigating recursive structures. Also, it is used for the management of exceptions. This backtracking is different from the sigle level backtracking of the alternative. As you can see in the example of the tutorial. ( I translated the undo track to am exception Undo. That is semantically similar, with the advantage that this allows to backtrack haskell exceptions when they happens anywhere in the code. #!/usr/bin/env stack -- stack --install-ghc --resolver lts-6.23 runghc --package transient import Control.Monad.IO.Class (liftIO) import Transient.Base import Transient.Backtrack instance Exception Undo where ... main= keep$ do
productNavigation
reserve
updateDB
payment
liftIO $print "done!" productNavigation = liftIO$ putStrLn "product navigation"

reserve= liftIO (putStrLn "product reserved,added to cart")
onException' \Undo -> liftIO (putStrLn "product un-reserved")

updateDB=  liftIO  (putStrLn "update other database necesary for the reservation")
onException' \Undo -> liftIO (putStrLn "database update undone")

payment = do
liftIO $putStrLn "Payment failed" throw Undo Here undo initiates the process of backtracking due to the failure in the payment. This executes from bottom to top the second argument of the onException' until a restarting of the normal flow is found (or the thread stop, as is in this case) It makes sense that each onException' does not manipulate this backtracking BY DEFAULT since this enhances composability and modularity. Imagine the real case, with real stuff, databases etc; These steps may be in different modules, in different libraries, maybe by different teams or even companies. Does it make sense to pass to them the responsibility for how the backtracking should continue? It is better to leave this decision to the application programmer which initiates the undo. Ok but how the application programmer can taylor the backtracking to his needs?  onException$ \Undo -> do
resp <-         option "retry" "retry all that was undone" <|>
option "stop" "no further actions" <|>
option "back" "continue backtracking" <|>
option "other"  "throw a different exception"
option  "bother" "backtrack with other track"
case resp of
"retry" -> continue
"stop"  -> empty
"back" -> backtrack
"other"-> throw SomeOtherException
"bother" ->  back Other

More succinctly (at the cost of some readability):

                             (do option "retry" "retry all the undone"; retry) <|>
(do option "stop" "no further actions"     ; empty) <|>
(do option "back" "continue backtracking" ; backtrack) <|>
(do option "other"  "throw a different exception" ; throw SomeOtherException) <|>
(do option  "bother" "backtrack with other track"  ; back Other)
Alberto
@agocorona
These are the options. The three first are there since the beginning, and the default is backtrack. The other two were close to work reliably, but in the case of exceptions there was some infinite loop when the exception re-raised was of the same type than the original exception. After adding some more code and blowing my mind with many balls in the air: exceptions, continutions, threads... I found that eliminating some code and moving some lines the problem was solved, now working in my box and I will upload it as son as I finish some tests. adn fix some other bugs.
Alberto
@agocorona
If you don't want to backtrack and want to use the traditional semantics for exceptions catcht works like catch in the transient monad.
Alberto
@agocorona
this program prints "world2..callstack..."
main= keep' $do onException$ \(SomeException e) -> do liftIO (print e); empty

onException $\(SomeException e) -> error "world2" onException$ \(SomeException e) ->error "world"

error "hello"
return()
onException... is the unary equivalent of onException'...
here each onException generates a new exception that is backtracked.
Alberto
@agocorona
when the alexis patch become available It will be straightforward to use it in Transient to make the transient monad as fast as Haskell can be.
Alberto
@agocorona
continue return Unit (()). Sometimes it is necessary to return values of other type to resume execution. In this example a Int is raised and the handler will return the next number:
import Transient.Base
import Control.Exception hiding (onException)
instance Exception Int

main= runTransient $do x<- return 1 onException' \n -> continue >> return (n+1) liftIO$ print (x:: Int)
throw x
return ()

> program
1
2
3
...
if continue is omitted, the program would backtrack after the handler. Since there is no further handlers it will print "1" and that's all.