Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Hugh JF Chen
    @hughjfchen
    @agocorona , yeah, definitely. I'd put a nginx before the service written in transient to handle SSL, now I may remove nginx.
    Alberto
    @agocorona
    @hughjfchen :thumbsup:
    Alberto
    @agocorona
    It also manages HTTP1.1. Requests to the same node may use the same socket . I will upload some example tomorrow
    Raw is a data type defined in the library as:
    newtype Raw= Raw BS.ByteString deriving (Read,Show)
    
    instance Loggable Raw where
      serialize (Raw str)= lazyByteString str
    
      deserialize= Raw <$> do
            s <- notParsed
            BS.length s `seq` return s  --force the read till the end
    Alberto
    @agocorona
    notParsed is defined in Trasient.Parse . It is the whole string still not parsed.
    It also supports chunked-encoded responses
    Alberto
    @agocorona

    I wrote somewhere about the creation of an interpreted language using the transient command line. One of the good things of that command style it that it comes with his command options, his own help and his own log for free:

    This code

    main= keep $ initNode $ inputNodes <|> do.....

    creates a menu of options but also the options can be put in advance in the command line:

    Enter  options          to: show all options
    Enter  ps               to: show threads
    Enter  errs             to: show exceptions log
    Enter  end              to: exit
    Enter  start            to: re/start node
    Enter  cookie           to: set the cookie
    >  start
    "start"
    
    option: start
    hostname of this node. (Must be reachable)? > localhost
    "localhost"
    port to listen? > 80
    80
    Connected to port: 8000

    But also I can write it in the same line, like a programming sentence. Here below I wrote the first line. the rest is generated by the program, as a kind of log.

    >  start localhost 8000    
    "start"
    option: start
    hostname of this node. (Must be reachable)? "localhost"
    port to listen? 8000
    Connected to port: 8000

    Also, I can write it in the command line: program -p start/localhost/8000

    This philosophy can be extended to create an language, maybe a JIT language that can be turning complete and may execute all the transient effects.

    Alberto
    @agocorona
    Here are some bits about the language
    Alberto
    @agocorona

    I just uploaded an update that allows sucesive HTTP/S requests to different services of the same host:port share the same connection with HTTP1.1. The example is in the file https.hs

    The second request which launches a google query within hackage.haskell.org. Also includes a deserializer made with the parser which filters the HTML to extract the list of packages which includes a haskell module:

    #!/usr/bin/env execthirdlinedocker.sh
    --  info: use "sed -i 's/\r//g' yourfile" if report "/usr/bin/env: ‘execthirdlinedocker.sh\r’: No such file or directory"
    -- LIB="/home/vsonline/workspace/transient-stack" && runghc  -i${LIB}/transient/src -i${LIB}/transient-universe/src  -i${LIB}/transient-universe-tls/src -i${LIB}/axiom/src   $1  ${2} ${3}
    
    
    {-# LANGUAGE ScopedTypeVariables, OverloadedStrings   #-}
    module Main where
    
    import Transient.Base
    import Transient.Parse
    import Transient.Move.Internals
    import Transient.Move.Services
    import Transient.TLS
    import Data.List(nub,sort)
    import Data.Char(isNumber)
    import Data.Monoid
    import Control.Monad.State
    import qualified Data.ByteString.Lazy.Char8 as BS
    import Control.Applicative
    import Data.Typeable
    
    getGoogleService = [("service","google"),("type","HTTPS")
                       ,("nodehost","www.google.com")
                       ,("HTTPstr",getGoogle)]
    
    getGoogle= "GET / HTTP/1.1\r\n"
             <> "Host: $hostnode\r\n" 
             <> "\r\n" :: String
    
    getGoogleSearchService = [("service","google"),("type","HTTPS")
            ,("nodehost","www.google.com")
            ,("HTTPstr","GET /search?q=+$1+site:hackage.haskell.org HTTP/1.1\r\nHost: $hostnode\r\n\r\n" )]
    
    
    main=do
      initTLS
    
      keep'  $ do
        Raw r <- runCloud $ callService getGoogleService ()
        liftIO $ do putStr "100 chars of web page: "; print $ BS.take 100 r
    
        Pack packages <- runCloud $ callService getGoogleSearchService ("Control.Monad.State" :: BS.ByteString) 
        liftIO $ do putStr "Search results: " ; print packages
    
    newtype Pack= Pack [BS.ByteString] deriving (Read,Show,Typeable)
    instance Loggable Pack where
        serialize (Pack p)= undefined
        deserialize= do
            Pack .reverse . sort . nub  <$>(many $ do
            tDropUntilToken "hackage.haskell.org/package/" 
            r <- tTakeWhile (\c -> not (isNumber c) && c /= '&' && c /= '/') 
            let l= BS.length r -1
            return $ if ( BS.index r l == '-') then BS.take l r else r)
    Alberto
    @agocorona
    Both requests, to the home page, and to the search uses the same HTTPS connection.
    I need that feature for a service which synchronizes positions of vehicles between two databases very frequently and uses HTTPS
    Alberto
    @agocorona
    My english is too hasty and clumsy. I mean that the parser scraps the google search page and extract Haskell package names. The query includes the module for which I want to search packages. It also restrict the search to the site: hackage.haskell.org
    Alberto
    @agocorona
    @agocorona
    Added HTTP proxy passtrough for HTTP and HTTPS connections using the $http_proxy and $https_proxy environment variables
    Now tryinh to extend that to node-to-node connections. Going for it now.
    Alberto
    @agocorona
    I think that I need to try one more attempt to implement the type-level layer. I didn't came back to it since @o1lo01ol1o gave me some type level set skeleton that I needed for that. I'm looking for it. Until then I used type level set from Dominic Orchard that was difficult to extend.
    Alberto
    @agocorona
    Now transient calls between nodes bypass HTTP proxies! This is very important for WAN networks. Probably this is one of the first distributed systems that do so. The code is in the transient-stack repo.
    I need to fix the TLS option for secure communications which appears to be broken in that repo.
    Alberto
    @agocorona
    Now that is fixed.
    Now transient nodes can communicate across HTTP proxies and manage proxy authentication as well as secure TLS communications, so NASA, now you can type check your interplanetary communications:
    Alberto
    @agocorona
    To use a proxy, just set the environment variables:
    > export http_proxy=http://user:password@host:port
    > export https_proxy=http://user:password@host:port
    Alberto
    @agocorona
    I'm now doing some tentative implementation of the cloudshell language
    Alberto
    @agocorona
    image.png
    That effect system is awesome. https://www.youtube.com/watch?v=yicXcdLI2YA
    Alberto
    @agocorona
    Seeing that effect system, the closest that Haskell can go is something like my type-level effects since at least there are no lift-run cluttering
    Although the implicit applications and the non distinction between pure and efectful terms in Unison are not possible to achieve in Haskell. Now I understand the efforts of Conor McBride with the idiom brackets to make applicatives implicit.
    The problem is that a and m a do not type match in Haskell
    Alberto
    @agocorona
    image.png
    Alberto
    @agocorona

    It seems that the type level stuff finally works, thanks to the new type-level list from @o1lo01ol1o .

    It does most of what I was looking for:

    Above, the IDE infers correctly the type of test: It has two states (Int, and String) It handles SomeException exceptions, it can terminate early (since it uses empty), perform IO, do streaming (because waitEvents) and it is multithreaded.

    It also has subtle effect cancellation rules. For example, EarlyTermination is cancelled by the operator <|> :

    test2 = getState <|> return (0 :: Int)
    
    -- >>> :t getState
    -- getState
    --   :: Typeable a =>
    --      T '[EarlyTermination] a
    --
    
    -- >>> :t test2
    -- test2 :: T '[] Int

    I can force the compiler to check for the presence of an effect. for example, runAtwould not compile if his argument is not logged and notifies that it is a cloud computation:

    runAt :: (Loggable a, Member Logged effs ~ True) 
           => Tr.Node 
           -> T effs a 
           -> T (effs <: Cloud) a
    All the primitives use the same monad T, in contrast with transient, which use different monads: TransIO, Cloud, Widget, StateIO..
    Tim Pierson
    @o1lo01ol1o
    :100:
    @agocorona Is the type-level effects stuff integrated into transient-universe?
    Alberto
    @agocorona
    @o1lo01ol1o It is a fine layer over the transient libraries, with only type level definitions, no runtime code. My idea is to make it a drop-in replacement so the programmer can use it optionally with the same code (with the exception of the type signatures)
    I don't know I adding modules to the current packages or making a new package
    Tim Pierson
    @o1lo01ol1o
    kk; when you do decide and it's useable, can you point me to it? this covid buisness has everything out of whack but I'd like to look at doing something
    Alberto
    @agocorona
    Sure.
    Alberto
    @agocorona

    I'm still testing for some improvements and completing the definitions

    One really cool thing of this system , in my opinion, is the really low cost for creating custom effects in order to control invariants of the domain problem effectively.

    data HasPaid
    
    pay ::  T '[HasPaid,IOEff,BlaBlaBlah...]()
    pay....
    
    sendStuff :: (Member  HasPaid  effs)=> Address  -> T '[IOEff,.....] ()

    This would type check

    do
         pay
         ...
         ...
         sendStuff

    But if pay is not present before sendStuff that would produce a compilation error

    This has even more value in Transient, since the execution can span across different computers. It can support shutdowns and restarts, so sendStuff can execute in a remote computer some time after. Maintaining the paid flag in a single database and make programmers aware of that requirement are the kind of problems that can be eliminated.

    Alberto
    @agocorona

    @o1lo01ol1o a question: I need to creata a return that include some effect. sort of:

    reteff    ::  a -> (eff :: [*]) -> T eff a

    but it complains with:

    • Expected a type, but ‘(eff :: [])’ has kind ‘[]’
    • In the type signature: reteff :: a -> (eff :: [*]) -> T eff a

    Alberto
    @agocorona
    Ok I found a workaround
    Tim Pierson
    @o1lo01ol1o
    You'll probably need something like
    data AKnownEffect a where 
        FooE :: AKnownEffect Foo
        BarE :: AKnownEffect Bar
    
    appendEff :: T es a -> AKnownEffect e -> T (e':es) a
    Tim Pierson
    @o1lo01ol1o
    or
    type family AppendEffect es e where 
        AppendEffect es e = e:es
    
    reteff :: forall e a. a -> T (AppendEffect [] e) a
    
    >>> reteff @EffectFoo foo
    Alberto
    @agocorona
    Tanks!
    Alberto
    @agocorona

    The problem is: I can handle the effects of the alternative operator carefully since they may or may not be executed. If I simply add the effects of both terms (minus the early termination) then this code:

    (empty >> pay)  <|>  return()

    would have HasPaid as if this effect has been executed, and this is not the case. To make sure that HasPaid is executed, i should demand that both terms have the same effects, so either one or the other term has pay on it.

    But thar is too restrictive, since async... <|> waitEvents have somme common effects and some that are differents and the alternative operator would not type check.

    Tim Pierson
    @o1lo01ol1o
    IIUC, you would need proper dependent types to handle that case. You could potentially do it so long as the Haskell expression is handled in an existentially quantified continuation, but even then you wouldn't know statically (ie, compile time) if something paid or not.
    Alberto
    @agocorona
    I created this hack orElse for a relaxed alternative, besides the strict alternative <|>:
    -- >>> :t async
    -- async :: IO a -> T '[Async, MThread] a
    --
    
    -- >>> :t waitEvents
    -- waitEvents :: IO a -> T '[Streaming, MThread] a
    --
    
    test =   async (P.return "hello")  `orElse` waitEvents (P.return "world")
    
    -- >>> :t test
    -- test :: T '[Maybe Streaming, MThread, Maybe Async] [Char]
    --
    assigns Maybe to the effects that are not present in both sides
    Tim Pierson
    @o1lo01ol1o
    That might get you somewhere, although I think you may need a type-level Maybe to correctly pattern match via a type family
    in general, i'm not sure where this model of effect tracking will fall down since it relies on the ad-hoc assurances and not term-level proof, but certainly worth exploring (and those types help with documentation quite a bit -- you could add a phantom type to MThread (n :: Nat) and restrict the number of threads sub-processes can use)
    It also looks like you may actually want an Either for orElse, no? Since the types will be either the first of the alternatives or the second.
    Alberto
    @agocorona
    About the latter, the problem si that I'm not sure where is empty in the first expression, so it can execute half of the effects of the first therm and all of the others, or execute all of the first term and none of the other etc
    What do you mean by term-level proof?
    Tim Pierson
    @o1lo01ol1o
    Well, using reteff @EffectFoo foo, anyone can add or remove effects from the list as they want and it wouldn't change the runtime semantics of the expression.