These are chat archives for akkadotnet/akka.net

11th
Apr 2018
Bartosz Sypytkowski
@Horusiath
Apr 11 2018 04:58
@AndreSteenbergen did you configured a custom mailbox?
AndreSteenbergen
@AndreSteenbergen
Apr 11 2018 08:06
no
that's the weird thing, or do I need to do that
AndreSteenbergen
@AndreSteenbergen
Apr 11 2018 10:49
Thanks, just forced a mailbox using the withMailbox as parameter, works as a charm.

I do have 1 thing I don't know if it's a bug or something else, I occasionally see this error, after that the flow just stops ... I am using the example of the web crawler

[ERROR][11-4-2018 10:47:00][Thread 0018][akka://crawler/remote/akka.tcp/crawler@127.0.0.1:61662/user/api/XXXXXXX/coordinators/c1/StreamSupervisor-1/Flow-0-1-actorRefSource] The LinkedList is empty.
Cause: System.InvalidOperationException: The LinkedList is empty.
at System.Collections.Generic.LinkedList1.RemoveLast() at Akka.Streams.Implementation.ActorRefSourceActor1.ReceiveElement(T message)
at Akka.Actor.ActorBase.AroundReceive(Receive receive, Object message)
at Akka.Actor.ActorCell.ReceiveMessage(Object message)
at Akka.Actor.ActorCell.Invoke(Envelope envelope)

I can't seem to pinpoint the problem. And I don't know is it's a bug in my code or akka.net. I did tweak the numbers a bit, to not let websites crash. The code example really is a crawler of doom ;)
AndreSteenbergen
@AndreSteenbergen
Apr 11 2018 10:56
250 per minute with 2 maxconcurrent.
        var htmlFlow = Flow.Create<PageToCrawl>()
            .Throttle(job.Config.ThottleAmount, job.Config.PerTimeUnit, job.Config.ThottleAmount, ThrottleMode.Shaping)
            .Via(DownloadFlow.ProcessHtmlDownloadFor(MaxConcurrentDownloads, job, pageRequester));
AndreSteenbergen
@AndreSteenbergen
Apr 11 2018 13:47
When that happens the entire actor won't receive anything anymore
or the stream at least
AndreSteenbergen
@AndreSteenbergen
Apr 11 2018 15:54
I have a near infinite buffer: var source = Source.ActorRef<PageToCrawl>(int.MaxValue, OverflowStrategy.DropTail);
AndreSteenbergen
@AndreSteenbergen
Apr 11 2018 16:00
I did find RemoveLast is called by DropTail, but I know for sure I have not reached int.MaxValue items in the buffer.
Marc Piechura
@marcpiechura
Apr 11 2018 16:23
@AndreSteenbergen could you try to reproduce it in a simpler example?
AndreSteenbergen
@AndreSteenbergen
Apr 11 2018 16:24
I tried ....
It isn't actually that hard, it's also doesn't happen evry time
30% of the time I have a crawl of about 600 pages
I have also tried debug messages, but the actors are quite chatty in debug mode. Can't seem to find any useful data
AndreSteenbergen
@AndreSteenbergen
Apr 11 2018 16:43
Or can it be as simple of choosing a smaller buffer e.g. 2048 * 2048? 4 million in that queue is quite a lot I guess ...
AndreSteenbergen
@AndreSteenbergen
Apr 11 2018 16:48
I have test run my crawl 5 times now, still no issue ... Could Buffer.IsFull be a bit problematic with queues of size int.MaxValue?
Marc Piechura
@marcpiechura
Apr 11 2018 17:03
Not sure, I don’t know how the web crawler example looks like, maybe @Aaronontheweb knows more
Joshua Garnett
@joshgarnett
Apr 11 2018 22:37
Are there any known performance issues I should watch out for when running a debug vs a release build? We are seeing significant performance hits on debug builds, more then I would have expected.
Aaron Stannard
@Aaronontheweb
Apr 11 2018 22:37
@AndreSteenbergen I've had the WebCrawler sample, as it currently stands, process 20GB+ of pages before
didn't have any issues really
@joshgarnett other than the usual stuff in the CLR that we don't control, none that I am aware
the Akka.NET binaries we ship on NuGet don't have any conditional code that checks for the presence of an attached debugger or anything
Joshua Garnett
@joshgarnett
Apr 11 2018 22:39
hmm I might have some profiling in my future, now to find a profiler that works well on OSX for .net core
Aaron Stannard
@Aaronontheweb
Apr 11 2018 22:40
I assume you're not using a local build of Akka.NET or anything, right?
Joshua Garnett
@joshgarnett
Apr 11 2018 22:40
Akka.Persistence.MySQL is being built locally
otherwise using some nuget packages
but that should never affect you if you're using the NuGet packages
only comes into play when running Akka.NET locally in debug
Joshua Garnett
@joshgarnett
Apr 11 2018 22:43
interseting
I’m not seeing any ifdefs for the Akka.Persistence.MySQL build
Aaron Stannard
@Aaronontheweb
Apr 11 2018 22:43
I assume what you're seeing is a drop in your entire app
performance, right?
Joshua Garnett
@joshgarnett
Apr 11 2018 22:43
Yeah