Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Vishnu Prassad
    @vishnuprasad73_twitter
    @ajay-gov
    Ajay.Gov
    @ajay-gov
    @vishnuprasad73_twitter Great, so it works now?
    Vishnu Prassad
    @vishnuprasad73_twitter
    Yes. Thank a lot.
    Vishnu Prassad
    @vishnuprasad73_twitter
    @ajay-gov Do you know any services or any simulator which provides sample IoT data which i can use and play around with swim ?
    Ajay.Gov
    @ajay-gov
    Sure, glad to hear that it works. A bit more detail- when you open a downlink the swim client establishes a connection to the server. The data is streamed from the server to the client using that connection. You may open multiple downlinks i.e to other nodes/lanes on the same host and the swim client will use the same connection to stream data to the different downlinks. However when you call client.close the connection to the server is closed and all downlinks will stop getting the data from the server.
    Ajay.Gov
    @ajay-gov
    @vishnuprasad73_twitter we work with customer related data for our enterprise apps. We have built a couple of sample apps that use public data sources.
    1. Swim Transit (Repo for code here: https://github.com/swimos/transit) is a real-time application that uses bus location information from multiple cities in the US. The bus location is sent from sensors on these buses. The docs for their API is here: https://gist.github.com/grantland/7cf4097dd9cdf0dfed14. You can look at the https://github.com/swimos/transit code to see how you can create web-agents from it and then build a real-time streaming web application using those web agents from a REST API data source.
    2. Swim Flight Tracker is a real-time application that uses flight location information. The flight location information is available here: https://opensky-network.org/apidoc/
    Vishnu Prassad
    @vishnuprasad73_twitter
    @ajay-gov Thanks again.SWIM is really interesting :) Does swim have any plans to have support for the server-side in other languages other than Java?
    Vishnu Prassad
    @vishnuprasad73_twitter
    Im a noob at Java, Learned a bit of basic of it for SWIM. :)
    Ajay.Gov
    @ajay-gov
    @vishnuprasad73_twitter We are working on moving the swim-core into Rust. Since Rust will run natively, we plan on supporting other language bindings into Rust. At that point you could use other languages on the server-side
    @vishnuprasad73_twitter Glad you like swim! Please do spread the word.
    Scott Clarke
    @scottswim
    you can find the source for the Flight Tracker demo here https://github.com/swimos/flight-tracker. It demos using both 3rd party APIs (OpenSky) as well as static CSV files. The project itself is split into 2 parts. The main webapp and a data simulator. The data simulator takes data from the APIs and CSV files and sends data into the main app in a timed way using swim command lanes. This allows for starting/stopping/replaying the data in the main app.
    We also have a simpler demo where we use NodeJS to bridge sensor data from an Arduino into Swim. The code for that demo is here: https://github.com/swimos/plant-monitor
    Pasquale Vazzana
    @Pasvaz
    @scottswim are you guys aware that the flight-tracker doesn't work on Safari?
    Brad Johnson
    @johnson_brad_twitter
    Hey @Pasvaz - just tagged @scottswim and some of our other devs in Slack. I think it's a known issue?
    Scott Clarke
    @scottswim
    I was just made aware of that a few days ago :D I have not had a chance to fix it yet.
    Pasquale Vazzana
    @Pasvaz
    I'll check the code and see if it's a real issue, it doesn't seem to like Classes but afaik Safari supports classes since v9
    Scott Clarke
    @scottswim
    thanks for letting us know.
    Pasquale Vazzana
    @Pasvaz
    no I stand corrected, classes are supported but no public/private fields, which is where it complains indeed
    Scott Clarke
    @scottswim
    yeah, safari does not like how i am doing member vars in the class.
    Pasquale Vazzana
    @Pasvaz
    yeah
    Scott Clarke
    @scottswim
    should be an easy enough fix, just have not had the time just yet.
    Pasquale Vazzana
    @Pasvaz
    I guess only babel can, there is no way to make Safari use public members apart transpiling them
    anyway, it's not a major issue, Chrome works fine
    Scott Clarke
    @scottswim
    firefox works as well.
    Pasquale Vazzana
    @Pasvaz
    not on android apparently
    Scott Clarke
    @scottswim
    oh, yeah, i am not surprised there. I have not done any optimization for mobile for flight-tracker
    that is another good thing to add to the list however.
    mobile support should be fairly easy to add.
    Scott Clarke
    @scottswim
    thanks.
    really this is where things like babel are perfect, however i am not using it in the project because i wanted to keep things as simple as possible.
    Pasquale Vazzana
    @Pasvaz
    I know what you mean
    Ajay.Gov
    @ajay-gov
    @Pasvaz thanks for pointing the issue with the flight tracker. traffic (https://github.com/swimos/traffic), transit (https://github.com/swimos/transit) work on safari and on android.
    https://traffic.swim.ai/ and https://transit.swim.ai/ are the hosted versions of those repos respectively
    Scott Clarke
    @scottswim
    flight tracker should be working better now in safari too.
    just pushed some quick fixes for safari to github and the live demo site.
    Pasquale Vazzana
    @Pasvaz
    Thanks @ajay-gov @scottswim
    Pasquale Vazzana
    @Pasvaz
    transit is quite cool and probably a better place to start
    Ajay.Gov
    @ajay-gov
    @Pasvaz thanks. Please do ping us if you have any questions.
    Peace Michaels
    @oexza

    Hi @ajay-gov

    I am a two day old swimer, so the workings of swim are not yet very clear to me. I'm exploring the possibility of using a webagent as single writer per stream in an eventstore.

    How does one co-ordinate planes in such a manner that only one instance of a webagent can exist in a "cluster" of servers, and ensure that requests are routed/re-routed to the node that has that instance running? this would be kind of like what you can achieve with Akka Cluster Sharding. Is this possible with swim?

    Ajay.Gov
    @ajay-gov
    @oexza great question and this feature is supported In swimos. The way swimos distributes webagents in a cluster is very similar to how data is distributed across a Cassandra cluster(https://docs.datastax.com/en/cassandra-oss/3.0/cassandra/architecture/archDataDistributeDistribute.html). swimos computes a hash token for each agent's uri and the hash token is configured to a certain partition which pins agents to specific servers in the fabric. You can then address that agent using a swim client (or a browser running a swim app) by pointing the client to any plane in the fabric. Swim will transparently demultiplex links opened by external clients, and route them to the appropriate server in the fabric.
    For a concrete example on how to do this please refer to the swimos transit repo- https://github.com/swimos/transit. Specifically look at the https://github.com/swimos/transit#run-as-a-fabric section which guides you through the steps of how to setup a two server fabric. The partitions are configured in the server-a.recon and in the server-b.recon. Here we have a partition with two servers with agents being evenly distributed across both servers. Agent Uris that hash to a token that lies between 0x00000000 and 0x7fffffff are pinned to server-a and agent uris that hash to a token that lies between 0x80000000 and 0xffffffff are pinned to server-b
    Peace Michaels
    @oexza
    thanks @ajay-gov thanks, I looked at it and it seems pretty solid, the next logical question is how would you run this in a cloud environment or in kubernates where Ip Addresses are not static?
    Ajay.Gov
    @ajay-gov
    @oexza, thanks so much for checking it out! The k8s question is the next logical step indeed. Since swimos applications are stateful they need to deployed using k8s statefulsets
    The pods need to communicate internally but they can use the POD DNS names to look each other up (instead of using static IP addresses).
    External access to the k8s cluster can be made using the DNS name provided by the k8s load balancer for this cluster, swimos will then route requests to the appropriate pod internally.

    Using the transit as a concrete example, say you have the following set up in your k8s yaml file

    Service (ns/name): swimos/transit
    StatefulSet (ns/name): default/transit

    then the DNS and Hostnames will be

    StatefulSet Domain: transit.swimos.svc.cluster.local
    Pod DNS: transit-{0..N-1}.transit.swimos.svc.cluster.local
    Pod Hostname: transit-{0..N-1}

    For the 2 pod example with server-a as index 0 and server-b as index 1

    POD DNS for server-a will be: transit-0.transit.swimos.svc.cluster.local
    POD DNS for server-b will be: transit-1.transit.swimos.svc.cluster.local
    Also when running in a k8s cluster it will be best to run the app on the same port in all the pods. In order to do that change the port in @web section of the recon file. If you'd like to
    run it on port 9002 change the @web section of the recon file to be: @web(port: 9002) {

    With the apps running on port 9002, the next thing to fix is the the mesh part for the server recon.
    For server-a.recon this will be:

      @mesh {
        @part {
          key: partA
          predicate: @hash(0x00000000, 0x7fffffff)
          @host {
            primary: true
          }
    
        }
        @part {
          key: partB
          predicate: @hash(0x80000000, 0xffffffff)
          @host {
            uri: "warp://transit-1.transit.swimos.svc.cluster.local:9002"
            primary: true
          }
        }
      }

    For server-b.recon this will be:

      @mesh {
        @part {
          key: partA
          predicate: @hash(0x00000000, 0x7fffffff)
          @host {
            uri: "warp://transit-0.transit.swimos.svc.cluster.local:9002"
            primary: true
          }
        }
        @part {
          key: partB
          predicate: @hash(0x80000000, 0xffffffff)
          @host {
            primary: true
          }
        }
      }
    Ajay.Gov
    @ajay-gov
    @oexza A lot to take in, so please do let us know if you have more questions. As you can see with the current setup you will need a recon configuration file for each pod in your k8s cluster which is a tedious manual process. We are planning to add a k8s setup feature in our enterprise product swim continuum that will simplify this setup.
    Peace Michaels
    @oexza
    thanks so much for the detailed answer @ajay-gov a very lot to take in. this truly is tedious and unsustainable for on demand scaling. One thing that would really ease the pain, is a way to programatically configure the mesh instead of using the server.recon file. this is so that some kind of service discovery mechanism can be used in code instead. this kind of configuration is used alot in Apache Ignite, where discovery is so flexible to the point that even a good ol' database table can be used a registration and discovery point for the cluster. this FLEXIBILITY is HIGLY important. Would there be a way to achieve something similar today in swim?
    Ajay.Gov
    @ajay-gov
    @oexza programmatically configuring the meshes is definitely the way to go about it. The mesh configurations in the recon file is for convenience (for non-prod or single node deployments) but everything in the recon configuration file can be done programmatically. In fact we did exactly what you are asking for one of our customers where we run a multi-node k8s cluster. We can share this code with you if you'd like. Please do let us know.