Hi @ajay-gov
I am a two day old swimer, so the workings of swim are not yet very clear to me. I'm exploring the possibility of using a webagent as single writer per stream in an eventstore.
How does one co-ordinate planes in such a manner that only one instance of a webagent can exist in a "cluster" of servers, and ensure that requests are routed/re-routed to the node that has that instance running? this would be kind of like what you can achieve with Akka Cluster Sharding. Is this possible with swim?
The pods need to communicate internally but they can use the POD DNS names to look each other up (instead of using static IP addresses).
External access to the k8s cluster can be made using the DNS name provided by the k8s load balancer for this cluster, swimos will then route requests to the appropriate pod internally.
Using the transit as a concrete example, say you have the following set up in your k8s yaml file
Service (ns/name): swimos/transit
StatefulSet (ns/name): default/transitthen the DNS and Hostnames will be
StatefulSet Domain: transit.swimos.svc.cluster.local
Pod DNS: transit-{0..N-1}.transit.swimos.svc.cluster.local
Pod Hostname: transit-{0..N-1}For the 2 pod example with server-a as index 0 and server-b as index 1
POD DNS for server-a will be: transit-0.transit.swimos.svc.cluster.local
POD DNS for server-b will be: transit-1.transit.swimos.svc.cluster.local@web section of the recon file. If you'd like torun it on port 9002 change the
@web section of the recon file to be: @web(port: 9002) {
With the apps running on port 9002, the next thing to fix is the the mesh part for the server recon.
For server-a.recon this will be:
@mesh {
@part {
key: partA
predicate: @hash(0x00000000, 0x7fffffff)
@host {
primary: true
}
}
@part {
key: partB
predicate: @hash(0x80000000, 0xffffffff)
@host {
uri: "warp://transit-1.transit.swimos.svc.cluster.local:9002"
primary: true
}
}
}For server-b.recon this will be:
@mesh {
@part {
key: partA
predicate: @hash(0x00000000, 0x7fffffff)
@host {
uri: "warp://transit-0.transit.swimos.svc.cluster.local:9002"
primary: true
}
}
@part {
key: partB
predicate: @hash(0x80000000, 0xffffffff)
@host {
primary: true
}
}
}The TransitPlane class has the code that programmatically loads the mesh configurations. Please do check out this section of the readme as well
TransitPlane in node 0 starts up it will try to connect to nodes 1-5 and you will see some ConnectionExceptions since those nodes aren't up yet. As the nodes come up these ConnectionExceptions will go away as the swimos kernel will try to reconnect and will eventually connect to those nodes. Usually you will see no exceptions in node 5 since nodes 0-4 would have already started up. Do let us know if you have any questions.
I lean towards web development and I'm curious of the use cases that there might be. I also see it a chance to get into DevOps. Thank you for your replies!
http://traffic.swim.ai/
http://transit.swim.ai/
http://swim.flights/
http://swim.farm/
@abhishaked_twitter We support authentication using the server.recon file now. If you are using google authenticator it will look something like this:
tutorial: @fabric {
@plane(class: "swim.tutorial.TutorialPlane")
myauth: @googleId {
@audience("yourkey.apps.googleusercontent.com")
@email("youremail@gmail.com")
}
}There is a similar way to add OpenId Authenticator parameters as well.
You are right though, we haven't documented any of these cleanly yet. Our top priority right now is to revamp our developer site. We will definitely add sample code for authentication and authorization as part of this process.
@anoukh_ashley_twitter Those features are available in SwimOS, requiring only that you express them in terms of Web Agents, Lanes, and lifecycle callbacks.
Please see the tutorials for examples: https://www.swimos.org/tutorials/ . Aggregation is outlined under Downlinks, Join Value Lanes, and Join Map Lanes. Transformation and analysis take various forms through all the tutorials, but notably bridging to and from external processes is explained in Ingress Bridges and Egress Bridges.
@kumarsen26 Being a vertically integrated stack, SwimOS comes with a built-in database that tracks the state of all lanes that are not explicitly declared transient, but only if you opt into persistence. The easiest way to do this is via the recon configuration file; see https://github.com/swimos/traffic/blob/master/server/src/main/resources/server.recon#L14-L16 for an example. Note that the directory to which the database files are written is configurable.
Upon restart, Swim will load the same values into the lanes that they had immediately before the shutdown.
@brohitbrose yes WebAgent is started verified it by
@Override
public void didStart() {
logMessage("Web Agent Started");
}
Also created one lane
@SwimLane("hello")
ValueLane<String> info = this.<String>valueLane()
.didSet((newValue, oldValue) -> {
logMessage("info set to " + newValue + " from " + oldValue);
});
Both are non empty. Also pushed some data into other lanes.
But, Still no store generated and on restart other lanes products are empty
@kumarsen26 I think I've got it. You're likely missing a module dependency. Please make sure that you have the following (or something similar to it) in the dependencies { ... } section of build.gradle:api group: 'org.swimos', name: 'swim-store-db, version: '3.10.2'
Furthermore, if you're using Java 9+ with modules, you'll need the following in your module-info.java:requires swim.store.db;
server.recon file. Unfortunately in Heroku you don't have control over which port is exposed on the server since the port is assigned dynamically. Heroku does expose this port as an environment variable, so you'll have to read this property and inject that into the server configuration.
I am referring https://github.com/swimos/greenhouse/tree/feature/docker
But not able to find persistance example with docker.
- Remove the line
ServerLoader.loadServer();in your main class. Instead use the CustomServerLoader provided here: https://gist.github.com/ajay-gov/282ac4c28ed5027b0cecb75eb3f7c918. So your code then would beCustomreServerLoader.loadServer()(Note: You need to change the package name of the CustomServerLoader to match your package name. - Not the line here: https://gist.github.com/ajay-gov/282ac4c28ed5027b0cecb75eb3f7c918#file-customserverloader-java-L113, this is the part which reads the system property called
portand uses the value in that property and gets the swim application to bind to that port- You need to pass the System property to the gradle run task. Unfortunately gradle doesn't do this by default. So add the following line to the gradle run task: (using this as an example gradle file: https://github.com/swimos/cookbook/blob/master/planes/build.gradle#L59)
systemProperties System.getProperties(). So the run task would look like this:run { dependsOn jar systemProperties System.getProperties() doFirst { jvmArgs += [ '--module-path', files(configurations.runtimeClasspath, jar.archivePath).asPath, '--module', "${moduleName}/${mainClassName}" ] classpath = files() } }
- You need to pass the System property to the gradle run task. Unfortunately gradle doesn't do this by default. So add the following line to the gradle run task: (using this as an example gradle file: https://github.com/swimos/cookbook/blob/master/planes/build.gradle#L59)
- Override the gradle run task that heroku invokes by doing the following:
heroku config:set GRADLE_TASK="run -Dport=$PORT". If this is not how you run then modify the run task suitably so that the-Dportvariable is assigned to the$PORTvalue