Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Ajay.Gov
    @ajay-gov

    With the apps running on port 9002, the next thing to fix is the the mesh part for the server recon.
    For server-a.recon this will be:

      @mesh {
        @part {
          key: partA
          predicate: @hash(0x00000000, 0x7fffffff)
          @host {
            primary: true
          }
    
        }
        @part {
          key: partB
          predicate: @hash(0x80000000, 0xffffffff)
          @host {
            uri: "warp://transit-1.transit.swimos.svc.cluster.local:9002"
            primary: true
          }
        }
      }

    For server-b.recon this will be:

      @mesh {
        @part {
          key: partA
          predicate: @hash(0x00000000, 0x7fffffff)
          @host {
            uri: "warp://transit-0.transit.swimos.svc.cluster.local:9002"
            primary: true
          }
        }
        @part {
          key: partB
          predicate: @hash(0x80000000, 0xffffffff)
          @host {
            primary: true
          }
        }
      }
    Ajay.Gov
    @ajay-gov
    @oexza A lot to take in, so please do let us know if you have more questions. As you can see with the current setup you will need a recon configuration file for each pod in your k8s cluster which is a tedious manual process. We are planning to add a k8s setup feature in our enterprise product swim continuum that will simplify this setup.
    Peace Michaels
    @oexza
    thanks so much for the detailed answer @ajay-gov a very lot to take in. this truly is tedious and unsustainable for on demand scaling. One thing that would really ease the pain, is a way to programatically configure the mesh instead of using the server.recon file. this is so that some kind of service discovery mechanism can be used in code instead. this kind of configuration is used alot in Apache Ignite, where discovery is so flexible to the point that even a good ol' database table can be used a registration and discovery point for the cluster. this FLEXIBILITY is HIGLY important. Would there be a way to achieve something similar today in swim?
    Ajay.Gov
    @ajay-gov
    @oexza programmatically configuring the meshes is definitely the way to go about it. The mesh configurations in the recon file is for convenience (for non-prod or single node deployments) but everything in the recon configuration file can be done programmatically. In fact we did exactly what you are asking for one of our customers where we run a multi-node k8s cluster. We can share this code with you if you'd like. Please do let us know.
    @oexza also these programmatic APIs are going to be used by our swim continuum product which will provide a UI console from which you can set up the meshes for your k8s cluster. Hopefully we will have that soon.
    Peace Michaels
    @oexza
    @ajay-gov thanks!!!. yes I'd tremendously appreciate it if you share the code with me. Thanks!
    Ajay.Gov
    @ajay-gov
    @oexza sure no problem, will do. Need to clean that code up a bit. Will have it by end of day (we are in California) if that is okay with you.
    Peace Michaels
    @oexza
    Thanks @ajay-gov I'll be waiting.
    Ajay.Gov
    @ajay-gov
    @oexza Committed the code in a branch of the transit repo. It can be found here: https://github.com/swimos/transit/tree/feature/cluster-config
    The TransitPlane class has the code that programmatically loads the mesh configurations. Please do check out this section of the readme as well
    Ajay.Gov
    @ajay-gov
    @oexza you should run it using k8s stateful sets. k8s stateful set will start up node 0 to node 5 in that order. When the TransitPlane in node 0 starts up it will try to connect to nodes 1-5 and you will see some ConnectionExceptions since those nodes aren't up yet. As the nodes come up these ConnectionExceptions will go away as the swimos kernel will try to reconnect and will eventually connect to those nodes. Usually you will see no exceptions in node 5 since nodes 0-4 would have already started up. Do let us know if you have any questions.
    Peace Michaels
    @oexza
    Thanks!!! @ajay-gov i'll test it out and give feedback.
    Dimitris Kakanis
    @itsjimos
    Hello guys, I'd like to understand more about swimos and how I can benefit from it being a full stack javascript developer
    Ajay.Gov
    @ajay-gov
    @itsjimos Hi Dimitris, swimOS is a software platform for building real-time applications. It's a self-contained vertically integrated stack, so you don't need an app server or a database or a message broker. swimOS also has UI libraries for real-time visualizations. Please refer to the docs at https://www.swimos.org/.
    Scott Clarke
    @scottswim
    Our UI libraries are javascript and integrate nicely with existing JS frameworks as well.
    Dimitris Kakanis
    @itsjimos
    My only experience with real-time applications was through socket.io for a small side project. If I'm correct swimOS is destined to a wider variety of applications like IoT.
    I lean towards web development and I'm curious of the use cases that there might be. I also see it a chance to get into DevOps. Thank you for your replies!
    Scott Clarke
    @scottswim
    on the UI side of things our Swim Client uses websockets for communication but all that is handled by our client for you. We have a number of demos, many of which have web based UIs. You can find those on our github @ https://github.com/swimos
    many of those demos have a live version as well. Some of those are:
    http://traffic.swim.ai/
    http://transit.swim.ai/
    http://swim.flights/
    http://swim.farm/
    I am a web dev as well and often build fun things with swim. My latest uses satellite data and three.js to create whats basically a pretty screensaver :D http://satellites.swim.ai/neon.html
    Scott Clarke
    @scottswim
    Swim also can act as a web server and is very easy to setup. This way you can easy fire up a web server if you dont have one already.
    Dimitris Kakanis
    @itsjimos
    Yea, these demos look great. I remembered that a couple of years ago I had some time with InfluxDB a time series database. To my eyes its philosophy is close to real time application technologies and therefore Swim. I will look at the docs and experiment a bit the following weeks. Keep it up!
    Scott Clarke
    @scottswim
    Awesome. let us know if you have questions or get stuck. We are here to help!
    Abhishek Khuntia
    @abhishaked_twitter
    hi Swim team, any updates on adding auth to server.recon? I am testing out swim now, looks very interesting. I see the Authenticator module but there are no examples , so am not sure how exactly to use it. Also can you comment on RBAC implementation with swim. Can we also control actions and roles with annotation or we write it from scratch ? Thanks!
    Ajay.Gov
    @ajay-gov

    @abhishaked_twitter We support authentication using the server.recon file now. If you are using google authenticator it will look something like this:

    tutorial: @fabric {
      @plane(class: "swim.tutorial.TutorialPlane")
      myauth: @googleId {
        @audience("yourkey.apps.googleusercontent.com")
        @email("youremail@gmail.com")
      } 
    }

    There is a similar way to add OpenId Authenticator parameters as well.

    Ajay.Gov
    @ajay-gov
    @abhishaked_twitter We also support authorization as well. You may intercept the incoming requests at different levels i.e on connect, on link to the server, on link to an agent, on link to a lane etc. and perform RBAC style authorizations at any of these levels.
    You are right though, we haven't documented any of these cleanly yet. Our top priority right now is to revamp our developer site. We will definitely add sample code for authentication and authorization as part of this process.
    Neelam Jain
    @neeluuuuuu_twitter
    Hi, I am new to SwimOs can we make logic configurable instead of hard coded in code ? please suggest.
    Ajay.Gov
    @ajay-gov
    @neeluuuuuu_twitter thanks for your question, please do look at https://www.swimos.org/ for more information on how to use swimOS
    Anoukh Ashley
    @anoukh_ashley_twitter
    Hi, I was going through https://www.swim.ai/architecture/ and it says web agents do aggregate, transform, analyze. Is this for SwimOS or Swim Continuum?
    brohitbrose
    @brohitbrose

    @anoukh_ashley_twitter Those features are available in SwimOS, requiring only that you express them in terms of Web Agents, Lanes, and lifecycle callbacks.

    Please see the tutorials for examples: https://www.swimos.org/tutorials/ . Aggregation is outlined under Downlinks, Join Value Lanes, and Join Map Lanes. Transformation and analysis take various forms through all the tutorials, but notably bridging to and from external processes is explained in Ingress Bridges and Egress Bridges.

    kumarsen26
    @kumarsen26
    Hi,
    I am new to SwimOS, If Swim server restart happens, how the data persistence or data integrity is maintained? is there a way where my data is saved.. In tradiditonal applications, we will store in Database.. How in Swim it is maintained?
    brohitbrose
    @brohitbrose

    @kumarsen26 Being a vertically integrated stack, SwimOS comes with a built-in database that tracks the state of all lanes that are not explicitly declared transient, but only if you opt into persistence. The easiest way to do this is via the recon configuration file; see https://github.com/swimos/traffic/blob/master/server/src/main/resources/server.recon#L14-L16 for an example. Note that the directory to which the database files are written is configurable.

    Upon restart, Swim will load the same values into the lanes that they had immediately before the shutdown.

    gkaur2294
    @gkaur2294
    @brohitbrose When I specify the @store {
    path: "/tmp/swim-test/"
    } in recon file
    Only empty tmp folder is getting created. Nothing inside...
    I am just newly exploring swim. Kindly suggest.
    brohitbrose
    @brohitbrose
    @gkaur2294 Have you confirmed that at least one Web Agent was started (easiest to do with print statements on the didStart callback of the agent) and that at least one of its (non-transient) lanes had a non empty value (easiest to do with print statements on didSet or didUpdate)?
    kumarsen26
    @kumarsen26

    @brohitbrose yes WebAgent is started verified it by
    @Override
    public void didStart() {
    logMessage("Web Agent Started");
    }
    Also created one lane
    @SwimLane("hello")
    ValueLane<String> info = this.<String>valueLane()
    .didSet((newValue, oldValue) -> {
    logMessage("info set to " + newValue + " from " + oldValue);
    });

    Both are non empty. Also pushed some data into other lanes.
    But, Still no store generated and on restart other lanes products are empty

    brohitbrose
    @brohitbrose

    @kumarsen26 I think I've got it. You're likely missing a module dependency. Please make sure that you have the following (or something similar to it) in the dependencies { ... } section of build.gradle:
    api group: 'org.swimos', name: 'swim-store-db, version: '3.10.2'

    Furthermore, if you're using Java 9+ with modules, you'll need the following in your module-info.java:
    requires swim.store.db;

    kumarsen26
    @kumarsen26
    Thank you. Now the Database is getting created.
    Now I am trying to deploy Swim server in Heroku. I am able to build and deploy but But able to connect server from Client which is running in my local. Kindly suggest.
    Ajay.Gov
    @ajay-gov
    @kumarsen26 Swim applications bind to a port specified in the server.recon file. Unfortunately in Heroku you don't have control over which port is exposed on the server since the port is assigned dynamically. Heroku does expose this port as an environment variable, so you'll have to read this property and inject that into the server configuration.
    @kumarsen26 Are you using gradle in Heroku or are you deploying the application as a service?
    kumarsen26
    @kumarsen26
    @ajay-gov yes I am using Gradle.. in build goal i given "build" and in Procfile I have unpacked the tar and run the sh file inside bin
    kumarsen26
    @kumarsen26
    How can we enable data persistance in using swim+docker..
    I am referring https://github.com/swimos/greenhouse/tree/feature/docker
    But not able to find persistance example with docker.
    Ajay.Gov
    @ajay-gov
    @kumarsen26 As mentioned earlier, Heroku opens up the port in the firewall dynamically. This means that the swim application needs to know about this port at run-time. So please do the following:
    • Remove the line ServerLoader.loadServer(); in your main class. Instead use the CustomServerLoader provided here: https://gist.github.com/ajay-gov/282ac4c28ed5027b0cecb75eb3f7c918. So your code then would be CustomreServerLoader.loadServer() (Note: You need to change the package name of the CustomServerLoader to match your package name.
    • Not the line here: https://gist.github.com/ajay-gov/282ac4c28ed5027b0cecb75eb3f7c918#file-customserverloader-java-L113, this is the part which reads the system property called port and uses the value in that property and gets the swim application to bind to that port
      • You need to pass the System property to the gradle run task. Unfortunately gradle doesn't do this by default. So add the following line to the gradle run task: (using this as an example gradle file: https://github.com/swimos/cookbook/blob/master/planes/build.gradle#L59)
        systemProperties System.getProperties(). So the run task would look like this:
        run {
        dependsOn jar
        systemProperties System.getProperties()
        doFirst {
          jvmArgs += [
            '--module-path', files(configurations.runtimeClasspath, jar.archivePath).asPath,
            '--module', "${moduleName}/${mainClassName}"
          ]
          classpath = files()
        }
        }
    • Override the gradle run task that heroku invokes by doing the following: heroku config:set GRADLE_TASK="run -Dport=$PORT". If this is not how you run then modify the run task suitably so that the -Dport variable is assigned to the $PORT value
    @kumarsen26 In order to use persistence with docker, get the mount point of the docker persistent volume you are using and then modify the server.recon file to use that mount point. So if your mount point of the docker persistent volume is /mnt/docker/v1 then change the store path in the server.recon file as path: "/mnt/docker/v1/swim-test/"
    shafqatevo
    @shafqatevo
    Hi, I just got introduced to Swim. I am just curious to know whether Akka was considered for the actor model infrastructure. I can see the abstractions are little different in Swim but all that could be implemented on top of Akka too. As an Akka user that would ease the learning curve. Was there any particular reason not to base Swim on Akka?
    9 replies
    theseus yang
    @theseusyang
    How to invoke the external http webservice interface to get json data in swimos? which api should be used, please?
    Ajay.Gov
    @ajay-gov
    @theseusyang Would you like to use http to access the data externally from the lane of a WebAgent? Or would you like to get data from an external http API and then inject it into the lane of a swim WebAgent? Both are possible with swim. Please clarify and we will reply based on your clarification.
    theseus yang
    @theseusyang
    @ajay-gov thanks, I got it . I can get data from an external http API by using WebAgent.
    another thing, how to create a CEP application in swimos? like make Storm realtime streaming analysis application, using rule to filter and search data in swimos streaming.
    Ajay.Gov
    @ajay-gov
    @theseusyang regarding the external http API access, you can look at our sample repo here. The http API access is here and it feeds data into web agents.
    Ajay.Gov
    @ajay-gov
    Regarding CEP applications- swimOS works slightly different from CEP applications since it is a stateful application application platform. So you design the application using WebAgents. WebAgents can be used to model fundamental entities in your application. Higher order WebAgents can also be used to model your aggregations, this could include your search/filter criteria. Please look at the example apps in our github repo, eg: traffic, transit etc.
    theseus yang
    @theseusyang
    Ok, I need add esper into swimos alone.
    Ajay.Gov
    @ajay-gov
    @theseusyang Sorry we are not that familiar with esper. However, you can include any java library into your swimOS application.