@Silv3rcircl3 Sorry, was out there. The relevant code for disposing the stream was this:
var materializer = Context.Materializer();
var source = StreamConverters.FromInputStream(() => stream, STREAM_CHUNK_SIZE)
.Select(bs => new ApiResultV1.ResourceChunkContainer(API_NAME, bs.ToArray()));
var sink = Sink.ActorRef<ApiResultV1.ResourceChunkContainer>(sendingProxy, new ApiResultV1.ResourceStreamCompleted(API_NAME, resourceId, "UTF-8"));
var result = source.To(sink)
.Run(materializer);
result.ContinueWith((task) =>
{
stream.Dispose();
if (task.Result.WasSuccessful)
{
log.Debug($"Successfully streamed {result.Result.Count} packages for '{resourceId}'");
}
else
{
log.Error(task.Result.Error, "Streaming failed");
}
});When I dispose the materializer where the stream is disposed, the data does not get send at all.
Guys, do you use anything for mocking in the unit tests?
@Aaronontheweb Could you please have a look: #2923
@Aaronontheweb double checked the cluster singleton with the clean setup today. It does not take over in case of node death. Here is how I start it:
actorSystem.ActorOf(ClusterSingletonManager.Props(
singletonProps: Props.Create<SingleActor>(), // Props used to create actor singleton
terminationMessage: PoisonPill.Instance, // message used to stop actor gracefully
settings: ClusterSingletonManagerSettings.Create(actorSystem).WithRole("workingNode")),// cluster singleton manager settings
name: "widgetmanager");
@Aaronontheweb I don't know if it's related or not, but remote death watch does not work as well in case of node sudden death. If node leaves gracefully, death watch works fine. Same behavior with the cluster singleton.
@ZoolWay I'm only guessing but I'm not sure if the task you get back is complete once the stream is completed but rather when the end of the stream is reached. So MAYBE what happens is, the STREAM_CHUNK_SIZE is higher than the number of bytes inside the stream, so only a single read operation occurs, the bytes are send to the next stage and the task is completed, but the task completion is executed before the bytes arrive at the sink and therefore doesn't get send to the actor.
maybe you could try to dispose the materialiazer a second after the task has completed to check if that helps
hi @Horusiath, may I ask you something about cluster singleton. It seems that this is expected behavior, when 1 node dies unexpected, cluster singleton is not moved to another node up until cluster is healthy again. Is this correct? I don't see this in the documentation. https://stackoverflow.com/questions/38821243/akka-net-cluster-singleton-handover-not-occurs-when-current-singleton-node-shu
I'm trying to use Akka.Persistence with in-memory snapshot for the AtLeastOnceDeliveryReceiveActor. If everything is left with defaults, it works. The actor "stops" receiving any message, if I use the following serialization in my HOCON:
serializers {
wire = "Akka.Serialization.WireSerializer, Akka.Serialization.Wire"
}
serialization-bindings {
"System.Object" = wire
}I also tried Hyperion, same issue. If I remove this configuration, it works .
Is this a bug or misconfiguration ?
Okay, I just noticed this #2743
@Silv3rcircl3 To be honest I was curious about when that Task completes and when the continue is triggered by myself. There is some kind of
OnCompleted for an Akka.Stream but I did not figure out how to apply it to my scenario. The docs are rather limited about Akka.Stream to my feelings.
@Silv3rcircl3 You are right, when I apply a small delay before disposing the materializer, it works. So the idea to do it when the returned
Task completes was bad. The streaming process is not completed at that point in time yet. Will have to find out how to have a callback are message to myself send the stream completed.
@ZoolWay maybe a better solution would be to use a
Select stage where you send the bytes to your actor and simply return Unit.Default. As Sink use Sink.Ignore which provides a Task which is completed once the stream has actually processed all elements
or
SelectAsnyc + actor.Ask if you want backpressure
but that's obviously quite unsafe over the network ;)
Hello everyone! I'm currently working on a clustered Akka.net application. While trying to catch some exceptions thrown in one microservice, aggregate them into a list, and send to another microservice I've run into the following error:
08/03/2017 17:31:00 [ERROR] [akka://TestHub/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FTestHub%4010.16.4.33%3A57197-2] "No parameterless constructor defined for this object."
System.MissingMethodException: No parameterless constructor defined for this object.
at System.RuntimeTypeHandle.CreateInstance(RuntimeType type, Boolean publicOnly, Boolean noCheck, Boolean& canBeCached, RuntimeMethodHandleInternal& ctor, Boolean& bNeedSecurityCheck)
at System.RuntimeType.CreateInstanceSlow(Boolean publicOnly, Boolean skipCheckThis, Boolean fillCache, StackCrawlMark& stackMark)
at System.RuntimeType.CreateInstanceDefaultCtor(Boolean publicOnly, Boolean skipCheckThis, Boolean fillCache, StackCrawlMark& stackMark)
at System.Activator.CreateInstance(Type type, Boolean nonPublic)
at System.Activator.CreateInstance(Type type)
at Hyperion.SerializerFactories.ExceptionSerializerFactory.<>c__DisplayClass9_0.<BuildSerializer>b__0(Stream stream, DeserializerSession session)
at Hyperion.ValueSerializers.ObjectSerializer.ReadValue(Stream stream, DeserializerSession session)
at Hyperion.SerializerFactories.EnumerableSerializerFactory.<>c__DisplayClass5_0.<BuildSerializer>b__1(Stream stream, DeserializerSession session)
at Hyperion.ValueSerializers.ObjectSerializer.ReadValue(Stream stream, DeserializerSession session)
at lambda_method(Closure , Stream , DeserializerSession )
at Hyperion.ValueSerializers.ObjectSerializer.ReadValue(Stream stream, DeserializerSession session)
at lambda_method(Closure , Stream , DeserializerSession )
at Hyperion.ValueSerializers.ObjectSerializer.ReadValue(Stream stream, DeserializerSession session)
at Hyperion.Serializer.Deserialize[T](Stream stream)
at Akka.Serialization.HyperionSerializer.FromBinary(Byte[] bytes, Type type)
at Akka.Serialization.Serialization.Deserialize(Byte[] bytes, Int32 serializerId, String manifest)
at Akka.Remote.MessageSerializer.Deserialize(ActorSystem system, SerializedMessage messageProtocol)
at Akka.Remote.DefaultMessageDispatcher.Dispatch(IInternalActorRef recipient, Address recipientAddress, SerializedMessage message, IActorRef senderOption)
at Akka.Remote.EndpointReader.<Reading>b__11_1(InboundPayload inbound)
at lambda_method(Closure , Object , Action`1 , Action`1 , Action`1 )
at Akka.Tools.MatchHandler.PartialHandlerArgumentsCapture`4.Handle(T value)
at Akka.Actor.ReceiveActor.ExecutePartialMessageHandler(Object message, PartialAction`1 partialAction)
at Akka.Actor.ReceiveActor.OnReceive(Object message)
at Akka.Actor.UntypedActor.Receive(Object message)
at Akka.Actor.ActorBase.AroundReceive(Receive receive, Object message)
at Akka.Actor.ActorCell.ReceiveMessage(Object message)
at Akka.Actor.ActorCell.Invoke(Envelope envelope)
--- End of stack trace from previous location where exception was thrown ---
at Akka.Actor.ActorCell.HandleFailed(Failed f)
at Akka.Actor.ActorCell.SysMsgInvokeAll(EarliestFirstSystemMessageList messages, Int32 currentState)Both Exception and List<T> have parameterless constructors so I'm at a loss here. Does anybody know how to figure out which type "this object" is?
Question for anyone that might know: In reading the documentation, it sounds as though if you have a cluster with multiple seed nodes, and one seed node goes down, new nodes can't be joined intot eh cluster until that seed node comes back up. Thats all fine and good. However, in my testing, its seems that this limitation also happens when a normal non-seed node goes down. I can't get any new nodes to join the cluster until that node that died, comes back up and rejoins the cluster. Is this the way it is supposed to work?
Aha. I found the '''auto-down-unreachable-after''' configuration
@mithril52 if any node is unreachable, no new node can join the cluster.
It's also considered a bad thing to use auto-downing.
What would I use besides auto-downing?
It's stated you should not use it. I guess some kind of external monitoring tools, probably with semi automatic downing.
Apart from this there is always a way to down the unreachable node manually through console.
Yeah, manually kinda goes against elasticity :) I've tried that route, but I'm having problems getting pbm to work. Its unable to load System.Collections.Immutable, and I havn't been able to figure out why yet. But, for my purposes, I think auto-down will work perfectly
Yeah, manual doesn't fit the idea:-)
@maxcherednik regarding cluster singleton - by default singleton lives on the oldest node in the cluster. I'm not sure, but I think that singleton may not move from the unreachable node, but only from removed one (since the actual alive/dead state of unreachable is unknown, and we don't want to risk having 2 singletons at the same time) - if you're using singleton proxies, they will buffer messages until new singleton location will be known, and then they will forward their buffers to it.
Ok, that means it's by design.
Cause I couldn't find it on the documentation.
Again, it pushes me to turn auto-downing.
Still can't wrap my head around. "Do not do auto downing. It's going to be a split brain". But nothing works without it:-)
@maxcherednik this phrase about autodowning is more valid in case if you have alternatives (i.e. on JVM side you can buy subscription with better solutions for that issue ;) )
What about remote death watch? It seems it's also not working when node is unreachable. It needs to be down.
but unless you clusters grow big, this can work fairly well
I see.
So it's the way to go until we really face a split brain:-)
if I were you I'd just set auto downing (otherwise implement custom split brain resolver, but it's not out of the box)
Ok, thanks. Could you please confirm the remote death watch behavior?
Again my expectation would be: I receive Terminated event in case of network issue.
Is this correct?
@maxcherednik I wouldn't count on it (Unreachable is treated as sort of temporal undefined state, so in most of the cases this state change is undecisive to perform any actions), but you can check it easily if you have some cluster already spinning up. /cc @Aaronontheweb
@Horusiath I did actually... but as far as I remember I had it working before.
and I am also reading this: http://getakka.net/docs/remoting/deathwatch
This section: When Will You Receive a Terminated Message?
As far as I remember, I was receiving Terminated message right away...
I start to remember something. It seems I already asked exactly the same question and the answer was: Remote and Cluster watch works slightly different and there is no documentation for this. @Aaronontheweb