Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • Dec 08 18:55
    Zetanova synchronize #5416
  • Dec 08 18:55
    Zetanova commented #5416
  • Dec 08 18:52
    Zetanova synchronize #5416
  • Dec 08 17:53
    Zetanova commented #5416
  • Dec 08 16:03
    Arkatufus commented #5371
  • Dec 08 15:49
    Aaronontheweb commented #5371
  • Dec 08 15:24
    Aaronontheweb closed #5419
  • Dec 08 15:24
    Aaronontheweb commented #5419
  • Dec 08 15:19

    Aaronontheweb on dev

    Fixes performance issue with Hi… (compare)

  • Dec 08 15:19
    Aaronontheweb closed #5420
  • Dec 08 15:13
    Aaronontheweb commented #5419
  • Dec 08 15:09
    Aaronontheweb labeled #5420
  • Dec 08 15:09
    Aaronontheweb labeled #5420
  • Dec 08 15:09
    Aaronontheweb auto_merge_enabled #5420
  • Dec 08 15:07
    Aaronontheweb synchronize #5420
  • Dec 08 14:54
    lucavice opened #5420
  • Dec 08 14:47
    Aaronontheweb commented #5419
  • Dec 08 14:47
    Aaronontheweb milestoned #5419
  • Dec 08 14:47
    Aaronontheweb labeled #5419
  • Dec 08 14:47
    Aaronontheweb labeled #5419
AndreSteenbergen
@AndreSteenbergen
An exception occurred while trying to apply plugin of type Akka.Actor.ActorStashPlugin to the newly created actor (Type = AnalyserService.Actors.Analyser
Cause: System.NotSupportedException: DequeBasedMailbox required, got: Mailbox
I am making the proxy actor like this:
public class ActorProxy : ReceiveActor
{
    public ActorProxy(string typeName, string actorName, params object[] constructorParameters)
    {
        var actorBaseType = typeof(ActorBase);
        var t = Type.GetType(typeName);

        if (!t.IsSubclassOf(actorBaseType))
        {
            Console.WriteLine($"{t} is not a subtype of actorbase");
            throw new ArgumentException("typeName");
        }

        //try and find an actor by type
        IActorRef targetActor = Context.ActorOf(Props.Create(() => (ActorBase)Activator.CreateInstance(t, constructorParameters)), actorName);

        Console.WriteLine(targetActor);
        ReceiveAny((msg) => {
            targetActor.Tell(msg, Sender);
        });
    }
}
All is well until I make the target actor IWithUnboundedStash
Any ideas?
Bartosz Sypytkowski
@Horusiath
@AndreSteenbergen did you configured a custom mailbox?
AndreSteenbergen
@AndreSteenbergen
no
that's the weird thing, or do I need to do that
AndreSteenbergen
@AndreSteenbergen
Thanks, just forced a mailbox using the withMailbox as parameter, works as a charm.

I do have 1 thing I don't know if it's a bug or something else, I occasionally see this error, after that the flow just stops ... I am using the example of the web crawler

[ERROR][11-4-2018 10:47:00][Thread 0018][akka://crawler/remote/akka.tcp/crawler@127.0.0.1:61662/user/api/XXXXXXX/coordinators/c1/StreamSupervisor-1/Flow-0-1-actorRefSource] The LinkedList is empty.
Cause: System.InvalidOperationException: The LinkedList is empty.
at System.Collections.Generic.LinkedList1.RemoveLast() at Akka.Streams.Implementation.ActorRefSourceActor1.ReceiveElement(T message)
at Akka.Actor.ActorBase.AroundReceive(Receive receive, Object message)
at Akka.Actor.ActorCell.ReceiveMessage(Object message)
at Akka.Actor.ActorCell.Invoke(Envelope envelope)

I can't seem to pinpoint the problem. And I don't know is it's a bug in my code or akka.net. I did tweak the numbers a bit, to not let websites crash. The code example really is a crawler of doom ;)
AndreSteenbergen
@AndreSteenbergen
250 per minute with 2 maxconcurrent.
        var htmlFlow = Flow.Create<PageToCrawl>()
            .Throttle(job.Config.ThottleAmount, job.Config.PerTimeUnit, job.Config.ThottleAmount, ThrottleMode.Shaping)
            .Via(DownloadFlow.ProcessHtmlDownloadFor(MaxConcurrentDownloads, job, pageRequester));
AndreSteenbergen
@AndreSteenbergen
When that happens the entire actor won't receive anything anymore
or the stream at least
AndreSteenbergen
@AndreSteenbergen
I have a near infinite buffer: var source = Source.ActorRef<PageToCrawl>(int.MaxValue, OverflowStrategy.DropTail);
AndreSteenbergen
@AndreSteenbergen
I did find RemoveLast is called by DropTail, but I know for sure I have not reached int.MaxValue items in the buffer.
Marc Piechura
@marcpiechura
@AndreSteenbergen could you try to reproduce it in a simpler example?
AndreSteenbergen
@AndreSteenbergen
I tried ....
It isn't actually that hard, it's also doesn't happen evry time
30% of the time I have a crawl of about 600 pages
I have also tried debug messages, but the actors are quite chatty in debug mode. Can't seem to find any useful data
AndreSteenbergen
@AndreSteenbergen
Or can it be as simple of choosing a smaller buffer e.g. 2048 * 2048? 4 million in that queue is quite a lot I guess ...
AndreSteenbergen
@AndreSteenbergen
I have test run my crawl 5 times now, still no issue ... Could Buffer.IsFull be a bit problematic with queues of size int.MaxValue?
Marc Piechura
@marcpiechura
Not sure, I don’t know how the web crawler example looks like, maybe @Aaronontheweb knows more
Joshua Garnett
@joshgarnett
Are there any known performance issues I should watch out for when running a debug vs a release build? We are seeing significant performance hits on debug builds, more then I would have expected.
Aaron Stannard
@Aaronontheweb
@AndreSteenbergen I've had the WebCrawler sample, as it currently stands, process 20GB+ of pages before
didn't have any issues really
@joshgarnett other than the usual stuff in the CLR that we don't control, none that I am aware
the Akka.NET binaries we ship on NuGet don't have any conditional code that checks for the presence of an attached debugger or anything
Joshua Garnett
@joshgarnett
hmm I might have some profiling in my future, now to find a profiler that works well on OSX for .net core
Aaron Stannard
@Aaronontheweb
I assume you're not using a local build of Akka.NET or anything, right?
Joshua Garnett
@joshgarnett
Akka.Persistence.MySQL is being built locally
otherwise using some nuget packages
but that should never affect you if you're using the NuGet packages
only comes into play when running Akka.NET locally in debug
Joshua Garnett
@joshgarnett
interseting
I’m not seeing any ifdefs for the Akka.Persistence.MySQL build
Aaron Stannard
@Aaronontheweb
I assume what you're seeing is a drop in your entire app
performance, right?
Joshua Garnett
@joshgarnett
Yeah
AndreSteenbergen
@AndreSteenbergen
@Aaronontheweb I understand, it can really crawl a lot, it seems changing the buffer size fixed the issue, don't know why or how, but the issue is gone, and a waiting list of 4 million pages is already absurd, so if the buffer would be full, I would have used too much memory already, so going back to a few 1000 is goog enough as well.
Aaron Stannard
@Aaronontheweb
is this the Akka.Remote buffer size or the crawler's stream setup?
because yeah, the Akka.Streams stuff needs to be tuned a bit - I think it's a bit too aggressive in terms of throttling down the crawl rate and level of concurrency at the moment
AndreSteenbergen
@AndreSteenbergen

Crawler stream set-up, with the smaller buffer I don't seem to get any issues.

I still need to tweak the config somewhat, on all node the standard config creates 2 download actors (per domain as I see it), with the maxConcurrency, making it distributed over multiple nodes will look a ddos attack in no time, don't want to bring down sites.

nathvi
@nathvi
I have a question with Akka Remote Deployment. If I deploy actor A from process A to process B, and process B stops or gets terminated, where does the state of actor A reside?
Aaron Stannard
@Aaronontheweb
actor A lives physically in process B
so actor A's state will be destroyed if B is terminated
you'd want to persist that state to Akka.Persistence or replicate it elsewhere
to make it recoverable or otherwise available