@Aaronontheweb I understand, it can really crawl a lot, it seems changing the buffer size fixed the issue, don't know why or how, but the issue is gone, and a waiting list of 4 million pages is already absurd, so if the buffer would be full, I would have used too much memory already, so going back to a few 1000 is goog enough as well.
is this the Akka.Remote buffer size or the crawler's stream setup?
because yeah, the Akka.Streams stuff needs to be tuned a bit - I think it's a bit too aggressive in terms of throttling down the crawl rate and level of concurrency at the moment
Crawler stream set-up, with the smaller buffer I don't seem to get any issues.
I still need to tweak the config somewhat, on all node the standard config creates 2 download actors (per domain as I see it), with the maxConcurrency, making it distributed over multiple nodes will look a ddos attack in no time, don't want to bring down sites.
I have a question with Akka Remote Deployment. If I deploy actor A from process A to process B, and process B stops or gets terminated, where does the state of actor A reside?
so actor A's state will be destroyed if B is terminated
you'd want to persist that state to Akka.Persistence or replicate it elsewhere
to make it recoverable or otherwise available
when doing cross cluster messaging do my messages have to be mutable?
just need to be serializable
they should always be immutable ideally
I am trying to figure out why a EventStream.Publish object is coming back with null fields where those fields are pocos with private setters
everything else (mutatable fields) are fine
if i make them private set they appear null too
will dig further
the JSON serializer just may not be able to figure out how to populate those values since it relies on conventions / magic IMHO to do it
but imho
oh so between clusters it uses JSON and not Hyperion as default?
yeah Hyperion isn't used as a default anywhere at the moment
if you're serious about using Akka.Remote or Akka.Cluster in production though
you're better off going with hand-rolled serialization using something that has an explicit scheme like Google.Protobuf
better performance, first and foremost
but more importantly, this type of issue doesn't happen
since you can explicitly control serialization
also makes it easier to version messages on the wire too
we will probably end up there, right now were doing a "large" proof of concept
we ship with JSON.NET / Hyperion in the future as a default because it's easy to get up and running
but those polymorphic serializers aren't perfect
might have to give them some hints on how to do their job
I'm having an issue with message delivery in a setup remoting between a windows service and an asp.net site. I'm using an aggreggator to send a large amount of messages to keep the size small, but out of 3500 messages only 1700 or so are making it. In smaller quantities, when there's only a few hundred there's no problems.
My agreggator and the hocon config for the two is here: https://gist.github.com/jesseconnr/256da997b49e38026c32f4bc2987adf5
and shutting down?
Both are on the same machine. I get errors like this
Error while decoding incoming Akka PDU of length 28885 but I'm not sure what to make of it.
that's a serialization problem
do you get that issue without it?
and is this on Linux?
so we've had reports of DotNetty fucking up its length-frame encoding
on v0.4.6
originally the reports were Windows-only
err
Linux only
I send the messages in the same order and tried to figure out if it was a particular one that was having a serialization issue, but the point at which the error occurs seems random.
no no, this isn't something you're doing wrong
it's DotNetty
Yep, as far as I can tell, DotNetty is all 0.4.7
like, their API is outright broken there
I've literally never had this issue happen to me on v0.4.6 but other users have reported it
I would strongly, strongly encourage rolling back to v0.4.6
I was having the issue on 0.4.6 and an older version of akka and upgraded to see if it would fix it and no dice.
can you guys do me a favor and open an issue on the DotNetty github
I don't have a window where I can even look into the issue for another 5 days
you're going to get a faster resolution on it if we can get their org to start looking at the problem
this is not an Akka.NET issue
it's a problem with length-frame encoding not being written or read correctly
Ah yeah I see the bug report and the mention. I get that same error.
DotNetty.Codecs.TooLongFrameException: Adjusted frame length exceeds 128000: 419561476 - discarded
hmm, v5 was next week in february, but I guess it never got there
I have the DotNetty guys on the horn in their gitter now
please go post a copy of your error message et al
that would help
have to demonstrate the pervasiveness of the issue
Would having an old 1.3.0 version of Akka.Serialization.Hyperion possibly have an effect?
I just realized that when everything else was updated, that wasn't because I didn't check the prerelease in visual studio.
but this looks really clearly like a DotNetty issue
Can you use a Lighthouse type of solution with Akka.Cluster.Sharding?
thanks
Ah well, I would try, but using nuget from within visual studio to downgrade was apparently a bad idea, everything is wonderfully jacked now. :)
well, got some answers from the DotNetty crew that make some sense
due to some issues with .NET Core, it's possible that the byte buffer pools DotNetty uses can be released early
by the socket itself
especially true on Linux
we use the default byte buffer allocator in our DotNetty transport, which uses pooling
this normally helps performance big time, because it eliminates allocations and keeps GC pressure down
guess I could expose some sort of setting to change the pooling strategy within Akka.Remote's DotNetty transport ("none" being an option)
but anyway, that issue is fixed in DotNetty v0.5.0 and they have one last PR they're trying to sort out before moving ahead with that
@Aaronontheweb thanks for the help that did it. now onto the next problem.
If I have 2+ nodes (cluster sharding) running, and I start sending messages, none are getting processed, once I drop down to 1, processing starts on that node, then I can later add additional nodes and they will sometimes pick up the load.. Is this by design?
am I missing another config?
beta60
that we're trying to ship in 1.3.6
my bad on the delay on the 1.3.6 release
when do you think that the cluster sharding stuff will be out of beta?
soon, hopefully - we just fixed the biggest bug with it in 1.3.6
Ah well it's good an update is coming. Almost got this downgrade worked out, hopefully it works, otherwise it seems my only option is going to be custom compiling dependencies to get this app live.
Awesome, I don't know why it didn't work before my upgrade, perhaps cleaning everything out and reinstalling the old version fixed something. Every message came through near instantly on 0.4.6.
glad to hear it
after moving to nightlys, I am getting this error:
System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.MissingMethodException: Method not found: 'Void Hyperion.SerializerOptions..ctor(Boolean, Boolean, System.Collections.Generic.IEnumerable
System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.MissingMethodException: Method not found: 'Void Hyperion.SerializerOptions..ctor(Boolean, Boolean, System.Collections.Generic.IEnumerable
1<Hyperion.Surrogate>, System.Collections.Generic.IEnumerable1<Hyperion.SerializerFactories.ValueSerializerFactory>, System.Collections.Generic.IEnumerable`1<System.Type>)'.
gotta upgrade the version of the Hyperion serializer too
I suspect
<PackageReference Include="Hyperion" Version="0.9.8" />
^^^
I didnt have a reference to it at all
@Aaronontheweb ok nightly builds gave me much better results
thanks again for the help
one question, what triggers a rebalance?
never mind, it looks like I just didnt have enough unique shardsIds in the dataset I was using.