Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Hugh JF Chen
    @hughjfchen

    I understand that if HTTP is 1.0, it the server should close the connection as you said. If not (it is 1.1) only it should close when "Connection" in the request is "close". Additionally, the server should close the connection after some timeout, after sending a "Connection: close" back (to do soon).

    Hi, @agocorona , that's correct. And the server can send back a Keep-alive: timeout=xxx;request=xxx header to indicate the timeout(in seconds) and pipeline requests. Then after the timeout reaches, it can close it after sending a response with the Connection: close header, which transient doesn't implement yet

    Because I believe there's need to negotiate with the client to comply to the http spec

    hi @hughjchen That would need a layer to adapt the server to transient that would be equally or more cumbresome. I think it can be programmed that way gradually according with the needs, since the scope of transient is very wide and the resources are limited. The disconnection protocol https://www.w3.org/Protocols/rfc2616/rfc2616-sec8.html is not really a protocol with his own set of messages. It is more like a set of rules.

    That's my concern, too. That's why I'm also thinking integrating transient into a web framework(servant for example) and calling transient services within a web application handler as a transient client. What do you think about this?

    Hugh JF Chen
    @hughjfchen

    I understand that if HTTP is 1.0, it the server should close the connection as you said. If not (it is 1.1) only it should close when "Connection" in the request is "close". Additionally, the server should close the connection after some timeout, after sending a "Connection: close" back (to do soon).

    Hi, @agocorona , that's correct. And the server can send back a Keep-alive: timeout=xxx;request=xxx header to indicate the timeout(in seconds) and pipeline requests. Then after the timeout reaches, it can close it after sending a response with the Connection: close header, which transient doesn't implement yet

    That requires transient keep track of the state and a timer for EACH connection. I guess it may need lots of effort to implement it. That's why I'm thinking incorporating an existing http server to do that.

    Alberto
    @agocorona
    I need it for any connection not only for HTTP connections and I don't think it is so difficult.
    To integrate with a web server you need some communication between the two since transient is also a server and should keep execution state between requests.
    Alberto
    @agocorona
    I could do the timeout quite fast but I want to integrate it in a new release which I'm testing now. It could be ready in two weeks
    Hugh JF Chen
    @hughjfchen

    I could do the timeout quite fast but I want to integrate it in a new release which I'm testing now. It could be ready in two weeks

    Great to hear that. What other features will be included except the connection timeout, by the way?

    And looking at the api source code, I saw that you get the connection from the transient state, and if I remember correctly, the objects got from the transient state by the type. So how shall we distinguish connections and get the right one if there are more than one connection from the client to the transient api?
    I asked this because I noticed a strange problem when I test my transient api with browser.
    Hugh JF Chen
    @hughjfchen
    The browser(chrome in this case) will always keep 2 connections to the transient api. The first time I input the api URL in the address bar and hit return, the browser gets the result from the api quickly and I can see 2 connection open to the transient api.
    However, if I hit the refresh button of the browser to reload the URL, the browser just hang and says it's waiting for server and never get any response. When I check the network connections, I found one of the 2 previous connections had been closed and another one open.
    Then I hit stop button to stop the request, and hit the refresh button again, the browser get response from the api very quickly. Then I hit refresh, browser hang again, then stop, then refresh, it works again...
    Hugh JF Chen
    @hughjfchen
    That pattern make me wonder if the transient api get the wrong connection from the transient state and can't match with the one the browser uses to send request so the browser never gets response from api
    Alberto
    @agocorona

    And looking at the api source code, I saw that you get the connection from the transient state, and if I remember correctly, the objects got from the transient state by the type. So how shall we distinguish connections and get the right one if there are more than one connection from the client to the transient api?

    Each call has his own thread and his own state with his own connection data. You can not access to the state of other requests.

    However, if I hit the refresh button of the browser to reload the URL, the browser just hang and says it's waiting for server and never get any response. When I check the network connections, I found one of the 2 previous connections had been closed and another one open.

    mmm I have to check it. I do not know it this makes another requests.

    Then I hit stop button to stop the request, and hit the refresh button again, the browser get response from the api very quickly. Then I hit refresh, browser hang again, then stop, then refresh, it works again...

    Probably, close will kill the connection on the browser wich send a termination to the server which kill the thread which handles the response , and the next request creates a new connection. I have to look

    Alberto
    @agocorona
    I definitively have to fix these things
    Alberto
    @agocorona

    However, if I hit the refresh button of the browser to reload the URL, the browser just hang and says it's waiting for server and never get any response. When I check the network connections, I found one of the 2 previous connections had been closed and another one open

    I'm trying to reproduce that behaviour using chrome but I can't. I use the simple api example. ...My app respond correctly.

    Hugh JF Chen
    @hughjfchen
    mmm, look like I need to do more tests for my api to see what happens.
    Alberto
    @agocorona
    I will use the trick used in this article to implement the timeout.
    Hugh JF Chen
    @hughjfchen
    When I saw this, I said to myself what a good timing!
    Would you also update the network package dependency to the 3.1.1?
    Alberto
    @agocorona
    I don't know if it will change it in this release but the trick can be used in the current version anyway
    Alberto
    @agocorona
    testing the last bugs of the new serialization mechanism, which uses byteString builders rather than strings.
    After that, I will implement the connection "lyfecycle" : closing connections on timeouts and so on.
    Until now connections among non-websocket nodes were considered as permanent, as is the csde in other distributed computing settings, more appropriate for controlled networks such is intranets. But HTTP and socket connections in the internet need some tigh control of connection resources.
    Alberto
    @agocorona

    the new serialization is very intuitive to debug, since 1) it contains REST paths basically and 2) any closure in the node can receive tests request using HTTP, and will receive responses using a simple Web browser.

    See https://gitter.im/Transient-Transient-Universe-HPlay/Lobby?at=5c14d70db4c74555cccf975f to know what the latter means.

    Hugh JF Chen
    @hughjfchen
    Thanks for the update, @agocorona , looking forward to the new release.
    Hugh JF Chen
    @hughjfchen
    Well, got a confusing problem. I can't get a state which set with setData
    newtype MqttClientWrapper = MqttClientWrapper MQTTClient deriving (Typeable)
    
    myQoSToQoS :: MyQoS -> QoS
    myQoSToQoS MyQoS0 = QoS0
    myQoSToQoS MyQoS1 = QoS1
    myQoSToQoS MyQoS2 = QoS2
    
    sendMsg' :: (SendMqttMsg, Topic, DBL.ByteString, Bool, MyQoS) -> Cloud ()
    sendMsg' (_, topic, payload, retail, qos) = local $ do
             MqttClientWrapper mc <- getSData <|> initMqttSenderStat defaultConfForCloud
             liftIO $ publishq mc topic payload retail (myQoSToQoS qos)
    
    initMqttSender :: Configuration -> IO MQTTClient
    initMqttSender conf = do
      mc <-
        runClient
          mqttConfig
            { _hostname = (mqttHost conf)
            , _port = (mqttPort conf)
            , _connID = (dataSenderConnectionId conf)
            -- _cleanSession=False,
            , _lwt =
                Just $
                mkLWT (dataSenderLWTTopic conf) "MqttSenderDown" False
            }
      -- r <- waitForClient mc
      -- r' <- case r of
      --   Left e -> do
      --              print e
      --              disconnect mc
      --              initMqttSender conf
      --   Right () -> return mc
      -- return r'
      return mc
    
    initMqttSenderStat :: Configuration ->  TransIO MqttClientWrapper
    initMqttSenderStat c = do
         mc <- async $ initMqttSender c
         setData $ MqttClientWrapper mc
         return $ MqttClientWrapper mc
    
    main :: IO (Maybe ())
    main = keep initMqttSender'
      where
        initMqttSender' = do
         -- runCloud $ connect mqttSenderLocalNode localConfigNode
         -- aConfig <- runCloud $ callService' localConfigNode getConfig
         -- let realConf = case aConfig of
         --                  Just c -> c
         --                  Nothing -> defaultConfForCloud
         -- initMqttSender realConf
         (void $ initMqttSenderStat defaultConfForCloud) <|>     (void (runService        mqttSenderServiceDef        mqttSenderServicePort        [ serve sendMsg'        ]        empty))
    the following expression:
             MqttClientWrapper mc <- getSData <|> initMqttSenderStat defaultConfForCloud
    getSData won't get the MqttClientWrapper which I set within the initMqttSenderStat
    So each time I call sendMsg', initMqttSenderStat will get called and reconnect to my mqtt broker.
    That's definitely not what I want
    Hugh JF Chen
    @hughjfchen
    But I can't figure out why
    Hugh JF Chen
    @hughjfchen
    Is it because I didn't call initNode? Hence the service method sendMsg' and initMqttSenderStat not run within the same node?
    Alberto
    @agocorona
    Hi @hughjfchen I see why.
    the reason is because initMqttSenderStat call async before setData. async is asynchronous, and initaties a new thread , which executes setDatain that new thread. It also return empty to the current thread, which executes the second term of <|> which executes mqttSenderServiceDef which invoes getData. But In that thread, setData has not been executed.
    Please call setData before async
    in that case, both threads, the current one and the new one created by async would have your state set.
    Alberto
    @agocorona
    otherwise, setData is executed only in the new thread
    I don't know If I explained it well
    Hugh JF Chen
    @hughjfchen
    Well, I see what you means. I'll try that and see what happens.
    Hugh JF Chen
    @hughjfchen
    Well, I remove async but the problem still persist.
    {-# LANGUAGE FlexibleContexts, TypeOperators #-}
    module Main
    (main
    ) where
    
    import Protolude hiding (async, local, state)
    
    import Transient.Base
    import Transient.Move
    import Transient.Move.Services
    
    import qualified Data.ByteString.Lazy as DBL
    
    import Network.MQTT.Client
    
    import PTM.Core.Conf
    import PTM.Core.Types
    import PTM.Core.ServiceDefinitions
    
    newtype MqttClientWrapper = MqttClientWrapper MQTTClient deriving (Typeable)
    
    -- instance Show MqttClientWrapper
    -- instance Read MqttClientWrapper
    
    myQoSToQoS :: MyQoS -> QoS
    myQoSToQoS MyQoS0 = QoS0
    myQoSToQoS MyQoS1 = QoS1
    myQoSToQoS MyQoS2 = QoS2
    
    sendMsg' :: (SendMqttMsg, Topic, DBL.ByteString, Bool, MyQoS) -> Cloud ()
    sendMsg' (_, topic, payload, retail, qos) = local $ do
             MqttClientWrapper mc <- getSData <|> initMqttSenderStat
             liftIO $ publishq mc topic payload retail (myQoSToQoS qos)
    
    initMqttSender :: Configuration -> IO MQTTClient
    initMqttSender conf =
      -- mc <-
        runClient
          mqttConfig
            { _hostname = mqttHost conf
            , _port = mqttPort conf
            , _connID = dataSenderConnectionId conf
            -- _cleanSession=False,
            , _lwt =
                Just $
                mkLWT (dataSenderLWTTopic conf) "MqttSenderDown" False
            }
      -- r <- waitForClient mc
      -- r' <- case r of
      --   Left e -> do
      --              print e
      --              disconnect mc
      --              initMqttSender conf
      --   Right () -> return mc
      -- return r'
      -- return mc
    
    initMqttSenderStat :: TransIO MqttClientWrapper
    initMqttSenderStat = do
         c <- runCloud getCfg
         mc <- liftIO $ initMqttSender c
         setData $ MqttClientWrapper mc
         return $ MqttClientWrapper mc
    
    getCfg :: Cloud Configuration
    getCfg = do
      aConfig <- callService' localConfigNode getConfig
      let realConf = fromMaybe defaultConfForCloud aConfig
      localIO $ print realConf
      return realConf
    
    main :: IO (Maybe ())
    main = keep initMqttSenderService
      where
        initMqttSenderService = void $ runService mqttSenderServiceDef mqttSenderServicePort [serve sendMsg'] empty
    Each time I call sendMsg', will connect to my mqtt broker
    Alberto
    @agocorona

    where is setData there? if it is in initMqttSenderStat

    MqttClientWrapper mc <- getSData <|> initMqttSenderStat

    mind that the later is after getSData so getSData is executed before the data is set. since <|> is executed from left first, right next (if the left fails). Also, for that matter, initMqttSenderStat is executed because getSData fails!!

    Just put setData before async in your first snippet.