Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    kumarsen26
    @kumarsen26
    Hi,
    I am new to SwimOS, If Swim server restart happens, how the data persistence or data integrity is maintained? is there a way where my data is saved.. In tradiditonal applications, we will store in Database.. How in Swim it is maintained?
    brohitbrose
    @brohitbrose

    @kumarsen26 Being a vertically integrated stack, SwimOS comes with a built-in database that tracks the state of all lanes that are not explicitly declared transient, but only if you opt into persistence. The easiest way to do this is via the recon configuration file; see https://github.com/swimos/traffic/blob/master/server/src/main/resources/server.recon#L14-L16 for an example. Note that the directory to which the database files are written is configurable.

    Upon restart, Swim will load the same values into the lanes that they had immediately before the shutdown.

    gkaur2294
    @gkaur2294
    @brohitbrose When I specify the @store {
    path: "/tmp/swim-test/"
    } in recon file
    Only empty tmp folder is getting created. Nothing inside...
    I am just newly exploring swim. Kindly suggest.
    brohitbrose
    @brohitbrose
    @gkaur2294 Have you confirmed that at least one Web Agent was started (easiest to do with print statements on the didStart callback of the agent) and that at least one of its (non-transient) lanes had a non empty value (easiest to do with print statements on didSet or didUpdate)?
    kumarsen26
    @kumarsen26

    @brohitbrose yes WebAgent is started verified it by
    @Override
    public void didStart() {
    logMessage("Web Agent Started");
    }
    Also created one lane
    @SwimLane("hello")
    ValueLane<String> info = this.<String>valueLane()
    .didSet((newValue, oldValue) -> {
    logMessage("info set to " + newValue + " from " + oldValue);
    });

    Both are non empty. Also pushed some data into other lanes.
    But, Still no store generated and on restart other lanes products are empty

    brohitbrose
    @brohitbrose

    @kumarsen26 I think I've got it. You're likely missing a module dependency. Please make sure that you have the following (or something similar to it) in the dependencies { ... } section of build.gradle:
    api group: 'org.swimos', name: 'swim-store-db, version: '3.10.2'

    Furthermore, if you're using Java 9+ with modules, you'll need the following in your module-info.java:
    requires swim.store.db;

    kumarsen26
    @kumarsen26
    Thank you. Now the Database is getting created.
    Now I am trying to deploy Swim server in Heroku. I am able to build and deploy but But able to connect server from Client which is running in my local. Kindly suggest.
    Ajay.Gov
    @ajay-gov
    @kumarsen26 Swim applications bind to a port specified in the server.recon file. Unfortunately in Heroku you don't have control over which port is exposed on the server since the port is assigned dynamically. Heroku does expose this port as an environment variable, so you'll have to read this property and inject that into the server configuration.
    @kumarsen26 Are you using gradle in Heroku or are you deploying the application as a service?
    kumarsen26
    @kumarsen26
    @ajay-gov yes I am using Gradle.. in build goal i given "build" and in Procfile I have unpacked the tar and run the sh file inside bin
    kumarsen26
    @kumarsen26
    How can we enable data persistance in using swim+docker..
    I am referring https://github.com/swimos/greenhouse/tree/feature/docker
    But not able to find persistance example with docker.
    Ajay.Gov
    @ajay-gov
    @kumarsen26 As mentioned earlier, Heroku opens up the port in the firewall dynamically. This means that the swim application needs to know about this port at run-time. So please do the following:
    • Remove the line ServerLoader.loadServer(); in your main class. Instead use the CustomServerLoader provided here: https://gist.github.com/ajay-gov/282ac4c28ed5027b0cecb75eb3f7c918. So your code then would be CustomreServerLoader.loadServer() (Note: You need to change the package name of the CustomServerLoader to match your package name.
    • Not the line here: https://gist.github.com/ajay-gov/282ac4c28ed5027b0cecb75eb3f7c918#file-customserverloader-java-L113, this is the part which reads the system property called port and uses the value in that property and gets the swim application to bind to that port
      • You need to pass the System property to the gradle run task. Unfortunately gradle doesn't do this by default. So add the following line to the gradle run task: (using this as an example gradle file: https://github.com/swimos/cookbook/blob/master/planes/build.gradle#L59)
        systemProperties System.getProperties(). So the run task would look like this:
        run {
        dependsOn jar
        systemProperties System.getProperties()
        doFirst {
          jvmArgs += [
            '--module-path', files(configurations.runtimeClasspath, jar.archivePath).asPath,
            '--module', "${moduleName}/${mainClassName}"
          ]
          classpath = files()
        }
        }
    • Override the gradle run task that heroku invokes by doing the following: heroku config:set GRADLE_TASK="run -Dport=$PORT". If this is not how you run then modify the run task suitably so that the -Dport variable is assigned to the $PORT value
    @kumarsen26 In order to use persistence with docker, get the mount point of the docker persistent volume you are using and then modify the server.recon file to use that mount point. So if your mount point of the docker persistent volume is /mnt/docker/v1 then change the store path in the server.recon file as path: "/mnt/docker/v1/swim-test/"
    Shafqat Ullah
    @shafqatevo
    Hi, I just got introduced to Swim. I am just curious to know whether Akka was considered for the actor model infrastructure. I can see the abstractions are little different in Swim but all that could be implemented on top of Akka too. As an Akka user that would ease the learning curve. Was there any particular reason not to base Swim on Akka?
    9 replies
    theseus yang
    @theseusyang
    How to invoke the external http webservice interface to get json data in swimos? which api should be used, please?
    Ajay.Gov
    @ajay-gov
    @theseusyang Would you like to use http to access the data externally from the lane of a WebAgent? Or would you like to get data from an external http API and then inject it into the lane of a swim WebAgent? Both are possible with swim. Please clarify and we will reply based on your clarification.
    theseus yang
    @theseusyang
    @ajay-gov thanks, I got it . I can get data from an external http API by using WebAgent.
    another thing, how to create a CEP application in swimos? like make Storm realtime streaming analysis application, using rule to filter and search data in swimos streaming.
    Ajay.Gov
    @ajay-gov
    @theseusyang regarding the external http API access, you can look at our sample repo here. The http API access is here and it feeds data into web agents.
    Ajay.Gov
    @ajay-gov
    Regarding CEP applications- swimOS works slightly different from CEP applications since it is a stateful application application platform. So you design the application using WebAgents. WebAgents can be used to model fundamental entities in your application. Higher order WebAgents can also be used to model your aggregations, this could include your search/filter criteria. Please look at the example apps in our github repo, eg: traffic, transit etc.
    theseus yang
    @theseusyang
    Ok, I need add esper into swimos alone.
    Ajay.Gov
    @ajay-gov
    @theseusyang Sorry we are not that familiar with esper. However, you can include any java library into your swimOS application.
    kumarsen26
    @kumarsen26

    Hi,
    Could you please tell me how to get particular values from a Maplane (ie I am expecting only some values based on query params like http get method)

    Currently Plane and Agent is like below
    in Plane
    @SwimRoute("/customer/:name")
    private AgentRoute<CustomerAgent> customer;

    in Agent
    @SwimLane("view")

    Guidance is appreciated.

    brohitbrose
    @brohitbrose

    You may refer to this: https://www.swimos.org/tutorials/map-lanes/ and scroll to the section that says “Reading from Map Lanes”.

    After opening a downlink to the lane, you can simply call get() on the key for a one-time read. To receive streaming updates instead, you can utilize the didUpdate(), callback, and ignore values for which the key doesn’t match your target.

    kumarsen26
    @kumarsen26

    @brohitbrose Thank you !!!

    If i have one client and multiple server applications.. How do we share the data since it is stateful?
    How to load balance?

    is this needed? or it will be only one client and one server?

    I am just thinking in prodution perspective and it will be good some architecture is shared..

    sorry if i missed if it is available.

    brohitbrose
    @brohitbrose

    @kumarsen26
    While we don't have a full-fledged tutorial for what you're looking for, it follows the process here: https://github.com/swimos/transit#run-as-a-fabric
    The main idea is that Web Agents can be distributed over multiple machines, yet each machine can serve data from Web Agents that run on a different machine (e.g. downlinking to a Web Agent that runs on machine2 will work, even if you set your hostUri=machine1). This works because Swim servers automatically abstract over the load balancing logic that you described.

    You'll have to seed each server is seeded with the right recon file. I recommend reading the run scripts and the .recon configuration files, then asking us any questions if anything is unclear.

    kumarsen26
    @kumarsen26

    Hi,

    In client, I am getting below response in client side when trying to read value from mapLane.

    What I am getting.. (@Customer and @ContactInfo is the object name)

    {
      "@Customer": null,
      "firstName": "kumar",
      "lastName": "Raju",
      "dob": "1990-09-09",
      "contactInfo": [
        {
          "@ContactInfo": null,
          "email": "test@gmail.com",
          "phone": "1252425262"
        }
      ]
    }

    Expected is

    {
      "firstName": "kumar",
      "lastName": "Raju",
      "dob": "1990-09-09",
      "contactInfo": [
        {
          "email": "test@gmail.com",
          "phone": "1252425262"
        }
      ]
    }

    My Client code:

    swim.downlinkMap().hostUri(HOST).nodeUri('/customer').laneUri('details').open().didUpdate((key,value) => {
              console.log({...value.toAny()});})

    I am not sure what I am missing.. Could you point the mistake.

    brohitbrose
    @brohitbrose

    Hard to say for sure without seeing the server code. But you can skip to past the --- for my suggested fix.

    My hypothesis is that this is actually the expected behavior. I assume that

    1. You have a Customer Java object with firstName, lastName, dob, and contactInfo
    2. The type of contactInfo is ContactInfo[]

    Then the recon (not JSON) serialization of your object will look like:
    @Customer{firstName:kumar,lastName:Raju,dob:"1990-09-09",contactInfo:{@ContactInfo{email:"test@gmail.com",phone:"1252425262"}}}, assuming that you used annotations instead of manually writing the Form<Customer>.

    When a serialization looks like @foo{bar:a,baz:b}, the swim.structure representation of this is actually a three-length Record (i.e. a collection of Items) whose first item is an Attr with key foo and undefined value; second Item is a Slot with key bar and value a; last Item is a Slot with key baz and value b. It is by design that an Attr appears "raised" even though it is in the same structural level. JSON does not differentiate between "special" key-value pairs like Attr vs Slot, so when you translate an Attr-containing object to JSON, it simply appears as a "regular" key-value pair with @ in it.

    ---

    To get everything except for the first Item in a Record, you should be able to call tail() on it. On the shorter example above, you should see (in JSON) {bar:a,baz:b}. In your example, you would need to do this twice to handle the nested ContactInfo type object.

    @kumarsen26 ^
    kumarsen26
    @kumarsen26

    @brohitbrose
    Please find server code.. I hope this helps..

    Plane:

    @SwimRoute("/customer")
      private AgentRoute<Customer> customer;

    Lane:

    @SwimLane("details")
        MapLane<String, Customer> view = this.<String, Customer>mapLane().didUpdate((key, newValue, oldValue) -> {
            System.out.println("the new value is " + newValue.toString());
        });
    brohitbrose
    @brohitbrose
    @kumarsen26 and what happens when you replace your Client code with:
    didUpdate((key,value) => {
        console.log(value.tail());
    });
    kumarsen26
    @kumarsen26

    @brohitbrose I am getting values as attached in the screenshot.

    image.png

    brohitbrose
    @brohitbrose

    good, the lower and upper bounds to the Record are correct. Let's now try:

    didUpdate((key,value) => {
        let customerDetails = value.tail();
        let contactInfoDetails = value.get("contactInfo");
    });

    Can you log these values using your method of choice and show the results?

    @kumarsen26 if you need the final result to be in a specific JSON format, it would be best to have a separate function to return a new object containing the final result. You should be able to build this object from customerDetails and contactInfoDetails (this might take a little more processing, maybe something like contactInfoDetails.getItem(0).tail() for the first value since it's an Array)as defined above
    NITESH
    @NITESHMEHTA68_twitter
    i want to learn swin from basic
    and want to become contributor
    any one help me
    Dobromir Marinov
    @DobromirM
    @NITESHMEHTA68_twitter You can get familiar with the core Swim concepts here: https://www.swimos.org/concepts/ and check the tutorials: https://www.swimos.org/tutorials/ to get started.
    Rikki
    @Yukaira
    Hey, i was reading the readme on swimos LED matrix driver and i saw that you could both run the server and client on the Pi, how would that work?
    kumarsen26
    @kumarsen26
    Hi Team,
    I am trying to access my swim-server using wss://localhost:9001 (secure websocket). Is it supported? if so do I need to do some configuration at server side? Kindly let me know if any example which i can refer
    brohitbrose
    @brohitbrose
    @Yukaira any device, including a Pi, that can concurrently run multiple processes will be able to run the server and client at once
    @kumarsen26 doing so requires setting up the server in a specific way that includes, among other things, certs on the machine and a Swim configuration that points to those certs. If you have certs ready, then we can walk you through how to do that
    brohitbrose
    @brohitbrose
    @kumarsen26 note that no additional configurations are needed if you configure the server without wss, i.e. use ws://localhost:9001
    kumarsen26
    @kumarsen26
    @brohitbrose Thank you .. Ok I will check and get back once I get the certificate..
    Thee, Meepeek
    @meepeek
    I'm new here. I read the web and have some questions
    1. How to scale out ? Can I run a dynamic cluster ?
    2. how to backup and administrate ? can I store a file as well ?
    3. Can I just run the Java server then use a nodejs server to handle logic ? or I need to program everything in Java for the server ?
    4. How the cache and security works ? should let the client contact swim server directly or through proxy ?
    5. How about authorize and access control ?
    Ajay.Gov
    @ajay-gov

    @meepeek Good questions.

    1. You can run swim in a cluster. Please look at this example of the transit application in a 2 node cluster: https://github.com/swimos/transit#run-as-a-fabric
    2. All swim lanes can be persisted if you turn the persistence option on. This is part of our enterprise offering. If you turn the persistence option on, then the data stored in the lanes for all agents of a specific type will get written into files on disk which you could then back up.
    3. Ideally it's best to program all the server side logic in java using the SwimOS platform
    4. The client can connect to the swim server directly. Web agents expose endpoints and streaming APIs, so data from the lanes will get streamed to your client
    5. You can configure access control and authorization programmatically. This typically done by injecting an Authenticator and a Policy to the Swim Plane. Here is an example from the cellular app: https://github.com/swimos/swim-cellular/blob/master/src/main/java/swim/cellular/CellularPlane.java#L24-L30

    Please do let us know if you have any other questions. Also one question for you, how did you hear about the Swim platform?

    Thee, Meepeek
    @meepeek
    I searched for something like web socket or web server communication. I forgot how i end up there but just yesterday as i lookup for easier way to deal with rpc or graphql. Swim concept is quite hit what i want, if it was nodejs, i'm sure i'll dig the code.
    @ajay-gov
    Thee, Meepeek
    @meepeek
    @ajay-gov as for the 2nd answer, does that mean without enterprise feature, the test server will run in memory only and the data will reset every time it restart ?
    Ajay.Gov
    @ajay-gov
    @meepeek yes that is correct. You could write through the data you want from the Web Agent to a database or a message broker. Here is an example of how to write to a database from a Web Agent: https://github.com/swimos/cookbook/tree/master/egress_bridges