Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • 15:42
    Aaronontheweb synchronize #4086
  • 15:42
    Aaronontheweb closed #4083
  • 15:42

    Aaronontheweb on dev

    Fix #4083 - Endpoint receive bu… (compare)

  • 15:42
    Aaronontheweb closed #4089
  • 15:42
    Aaronontheweb labeled #4093
  • 15:42
    Aaronontheweb labeled #4093
  • 15:42
    Aaronontheweb labeled #4093
  • 15:42
    Aaronontheweb opened #4093
  • 14:20
    Aaronontheweb commented #4092
  • 14:14
    Aaronontheweb labeled #4089
  • 14:14
    Aaronontheweb labeled #4089
  • 14:11
    Aaronontheweb synchronize #4089
  • 14:10
    Aaronontheweb synchronize #4086
  • 14:09

    Aaronontheweb on dev

    Convert to ImmutableHashSet for… (compare)

  • 14:09
    Aaronontheweb closed #4090
  • 12:04
    nagytech synchronize #4092
  • 11:53
    nagytech synchronize #4092
  • 11:49
    nagytech edited #4092
  • 11:40
    nagytech opened #4092
  • 11:32
    nagytech edited #4091
jweimann
@jweimann
but if they just took a command for input then updated their position every x seconds or whatever it should be fine. i had a similar system with 50k+ actors moving around that worked well
but they didn't do anything if the velocity didn't change
Aaron Stannard
@Aaronontheweb
namespace WebCrawler.CrawlService
{
    public class WorkerRole : RoleEntryPoint
    {
        protected ActorSystem ClusterSystem { get; set; }

        private readonly ManualResetEvent _runCompleteEvent = new ManualResetEvent(false);

        public override void Run()
        {
            Trace.TraceInformation("Crawler is running");
            try
            {
                this.Run(this._runCompleteEvent);
            }
            finally
            {
                this._runCompleteEvent.Set();
            }
        }

        public override bool OnStart()
        {
            // Set the maximum number of concurrent connections
            ServicePointManager.DefaultConnectionLimit = 12;

            // For information on handling configuration changes
            // see the MSDN topic at http://go.microsoft.com/fwlink/?LinkId=166357.



            bool result = base.OnStart();

            Trace.TraceInformation("Crawler has been started");

            return result;
        }

        public override void OnStop()
        {
            Trace.TraceInformation("Crawler is stopping");

            this._runCompleteEvent.Set();

            base.OnStop();

            Trace.TraceInformation("Crawler has stopped");
        }

        private void Run(ManualResetEvent shutdownCalled)
        {
            var config = AzureRuntimeBootLoader.CreateRuntimeConfig("WebCrawler.TrackingService", "Akka.Cluster");
            ClusterSystem = ActorSystem.Create("webcrawler", config.Result);

            while (!shutdownCalled.WaitOne())
            {
                ClusterSystem.Shutdown();
                ClusterSystem.AwaitTermination(TimeSpan.FromSeconds(5));
            }
        }
    }
}
And the magic that makes this whole thing work is the AzureRuntimeBootLoader class I wrote
jweimann
@jweimann
nice :) gonna try that out tonight
Aaron Stannard
@Aaronontheweb
public static class AzureRuntimeBootLoader
    {
        public static async Task<Config> CreateRuntimeConfig(string seedRoleName, string akkaEndpoint)
        {
            // Need to wait until at least 1 instance of our seed role is up and available
            while(!IsSeedRoleAvailable(seedRoleName))
            {
                await Task.Delay(TimeSpan.FromSeconds(1.5));
            }

            return CreateConfig(seedRoleName, akkaEndpoint);
        }

        public static bool IsSeedRoleAvailable(string seedRoleName)
        {
            if (!RoleEnvironment.Roles.ContainsKey(seedRoleName))
                throw new ArgumentException("Undefined role " + seedRoleName + " in Azure service definition");
            return RoleEnvironment.Roles[seedRoleName].Instances.Any();
        }

        private static Config CreateConfig(string seedRoleName, string akkaEndpoint)
        {
            var section = (AkkaConfigurationSection)ConfigurationManager.GetSection("akka");
            var clusterConfig = section.AkkaConfig;

            var selfAzureAddress = RoleEnvironment.CurrentRoleInstance.InstanceEndpoints[akkaEndpoint];
            var selfAkkaAddress = selfAzureAddress.IPEndpoint.Address;
            var selfPort = selfAzureAddress.IPEndpoint.Port;
            var seedNodes = new List<string>();

            var trackerRoles = RoleEnvironment.Roles[seedRoleName].Instances;
            foreach (var trackerEndpoint in trackerRoles.Select(x => x.InstanceEndpoints[akkaEndpoint]))
            {
                seedNodes.Add(string.Format(@"""akka.tcp://{0}@{1}:{2}""", "webcrawler", trackerEndpoint.IPEndpoint.Address, trackerEndpoint.IPEndpoint.Port));
            }

            var remoteConfig = ConfigurationFactory.ParseString(string.Format(@"akka.remote.helios.tcp.public-hostname = ""{0}"" 
            akka.remote.helios.tcp.port = {1}", selfAkkaAddress, selfPort));
            var clusterSeedConfig = ConfigurationFactory.ParseString(string.Format(@"akka.cluster.seed-nodes = [{0}]", string.Join(",", seedNodes)));
            var finalConfig = clusterSeedConfig.WithFallback(remoteConfig).WithFallback(clusterConfig);
            return finalConfig;
        }
    }
jweimann
@jweimann
have an azure meetup starting today locally
Aaron Stannard
@Aaronontheweb
the AzureRuntimeBootloader is able to specify a type of Azure role that can be used as a seed node
and the name of a specific internal endpoint
you want to use internal endpoints for Akka.NET
so you can bypass the load-balancer and talk to each VM instance directly
does that make sense @jweimann ?
jweimann
@jweimann
yep :)
Aaron Stannard
@Aaronontheweb
:+1:
I'm going to OSS that whole sample soon
working on smoothing out some issues with CPU utilization on the worker nodes
trying to introduce throttling there to stop them from maxing out on massive domains
for the exact diminishing returns reasons I was talking about earlier :p
Coinio
@Coinio
@Aaronontheweb I was testing with MaxNumberOfBalls at around 100,000-500,000 just for kicks (comparing it against a single threaded non-akka version). I had a previous design where I was passing in a Ball class from a giant list to a BallUpdateActor pool which only needed a pool size of around 10. That worked okay, but required the Ball class to have mutable properties for the BallUpdateActor to work on which didn't seem to nice. So I thought having individual BallActors which managed their own state would be better. Need a rethink I guess...
@jweimann That's currently all they are doing. Just updating position from the velocity on receiving an update message.
Emil Ingerslev
@emilingerslev
@Aaronontheweb didn't see your response before now. Join seems good, but will gossip spread likewise, using no seed notes? Like if B joins C and A joins B, then all nodes know when A leaves, etc.?
Geert Van Laethem
@GeertVL
The LastSender in the TestKitBase is that also the replier when you do a actor.AskAndWait() ??
Suhas Chatekar
@schatekar
I was going though the docs and notices that there is lot of explanation of concepts which is good but code samples are missing
is this on purpose or a general lack of contribution in this area?
Bartosz Sypytkowski
@Horusiath
guys, your The Inevitable Rise of the Stateful Web Application post reached the latest highscalability.com
Roger Johansson
@rogeralsing
btw. we just reached 1100 stars on GitHub.. exactly the same time as Orleans :P
Aaron Stannard
@Aaronontheweb
guys, your The Inevitable Rise of the Stateful Web Application post reached the latest highscalability.com
:sparkles:
@schatekar I'd chalk it up to a combination of lack of attention / not knowing what should be included as an example there
the docs definitely need some more love
@emilingerslev yep, gossip will still spread - as long as you can connect to a node that's already in the cluster, you're good to go
Aaron Stannard
@Aaronontheweb
seed nodes aren't special - they're just the first nodes that new nodes contact when they attempt to join
@Coinio ah, so under that scenario, since you're using a broadcast router, all 100,000-500,000 balls are going to attempt to update at once
still though, that's pretty cool :p
@rogeralsing hehe - nice.
Emil Ingerslev
@emilingerslev
@Aaronontheweb I hoped that was true. I makes the setup very reliable, which is good. Thanks :)
jweimann
@jweimann
@Aaronontheweb @Coinio I don't think the broadcast router makes sense for those balls, unless they're all supposed to change velocity at the same exact second to the same value, but I doubt that's the expected case
Bartosz Sypytkowski
@Horusiath
with akka-jvm 2.4, there are a lot of changes in cluster sharding:
  1. They moved persistence backed for sharding coordinator from Akka.Persistence to CRDTs
  2. Protobufed messages
  3. Graceful shutdown
Aaron Stannard
@Aaronontheweb
quite the moving target
but I get it - that stuff's not trivial to implement
Birnsen
@Birnsen

Hey everyone. Fantastic project! Following a while now... Watched your cluster contrib meeting on YT and there you mentioned the need of a MultiNode Visulaizer.
So I did a little something :). It is not pretty and it is written in F# (currently learning so code is probably a mess), so do not expect too much.
But maybe I can do something out of it (spit out static HTML) if needed.

https://github.com/Birnsen/MultiNodeVisualizer/blob/master/mnv.png

The numbers are the amount of events at that given moment. If an test fails is also not implemented yet but should be easy to do...
Had only one file to test it so others will probably fail.

derrickcrowne
@derrickcrowne
Are there any docs or examples on how to use UdpConnected in Akka.IO? There are extension methods on ActorSystem for Udp() and Tcp() but I couldn't find one for UdpConnected().
derrickcrowne
@derrickcrowne
I tried sending the UdpConnected.Connect message to ActorSystem.Udp() but it said it only responds to Bind and SimpleSender
Joshua Benjamin
@annymsMthd
@Aaronontheweb @derrickcrowne is starting to use Akka.IO to build a vehicle simulator
Suhas Chatekar
@schatekar
@Birnsen is that something you are outputting on the console?
Birnsen
@Birnsen
@schatekar yes
but the json file as input not real time
Suhas Chatekar
@schatekar
How difficult would it be to make your program spit out an html that would render the same thing?
Birnsen
@Birnsen
not that much... all the data should be given. only have to do a the skeleton to put the data in.