These are chat archives for akkadotnet/akka.net

8th
Aug 2015
Ryan Davis
@rdavisau
Aug 08 2015 02:08

Trying Real Hard to set up remote deployment, nailed it locally using the existing examples but now trying to work between two machines.. finally getting closer and seeing this:

on the "local" system, which is actually an azure box:

let aref =  
    spawnRemote localSystem "akka.tcp://ryans-house@*ryan's house's ip*:54053/" "hello"
       <@ actorOf (fun msg -> printfn "received '%A'" msg) @>
aref <! "sup"

seen in the log of the "remote" system, my house:

[ERROR][8/08/2015 1:56:14 AM][Thread 0020][[akka://ryans-house/system/endpointManager/reliableEndpointWriter-akka.tcp%3a%2f%2fryans-cloud%40*the cloud ip*%3a54053-1/endpointWriter]] AssociationError [akka.tcp://ryans-house@*the house ip*:54053] <- akka.tcp://ryans-cloud@*the cloud ip*:54053: Error [Error deserializing object of type 'System.Object'.] [   at Nessos.FsPickler.RootSerialization.readRootObject[T](IPicklerResolver resolver, ReflectionCache reflectionCache, IPickleFormatReader formatter, FSharpOption`1 streamingContext, FSharpOption`1 sifted, Pickler`1 pickler) in c:\Users\eirik\Development\nessos\FsPickler\src\FsPickler\FsPickler\RootSerialization.fs:line 61
   at Nessos.FsPickler.FsPicklerSerializer.Deserialize[T](Stream stream, FSharpOption`1 pickler, FSharpOption`1 streamingContext, FSharpOption`1 encoding, FSharpOption`1 leaveOpen) in c:\Users\eirik\Development\nessos\FsPickler\src\FsPickler\FsPickler\Serializer.fs:line 65
   at Akka.FSharp.Serialization.deserializeFromBinary[t](BinarySerializer fsp, Byte[] bytes) in C:\Users\rdavis\Source\Repos\akka.net\src\core\Akka.FSharp\FsApi.fs:line 352
   at Akka.FSharp.Serialization.ExprSerializer.FromBinary(Byte[] bytes, Type _arg1) in C:\Users\rdavis\Source\Repos\akka.net\src\core\Akka.FSharp\FsApi.fs:line 361
   at Akka.Remote.Serialization.DaemonMsgCreateSerializer.Deserialize(ByteString bytes, Type type) in C:\Users\rdavis\Source\Repos\akka.net\src\core\Akka.Remote\Serialization\DaemonMsgCreateSerializer.cs:line 51
   at Akka.Remote.Serialization.DaemonMsgCreateSerializer.GetArgs(DaemonMsgCreateData proto) in C:\Users\rdavis\Source\Repos\akka.net\src\core\Akka.Remote\Serialization\DaemonMsgCreateSerializer.cs:line 172
   at Akka.Remote.Serialization.DaemonMsgCreateSerializer.FromBinary(Byte[] bytes, Type type) in C:\Users\rdavis\Source\Repos\akka.net\src\core\Akka.Remote\Serialization\DaemonMsgCreateSerializer.cs:line 117
   at Akka.Serialization.Serialization.Deserialize(Byte[] bytes, Int32 serializerId, Type type) in C:\Users\rdavis\Source\Repos\akka.net\src\core\Akka\Serialization\Serialization.cs:line 106
   at Akka.Remote.MessageSerializer.Deserialize(ActorSystem system, SerializedMessage messageProtocol) in C:\Users\rdavis\Source\Repos\akka.net\src\core\Akka.Remote\MessageSerializer.cs:line 31
   at Akka.Remote.DefaultMessageDispatcher.Dispatch(IInternalActorRef recipient, Address recipientAddress, SerializedMessage message, IActorRef senderOption) in C:\Users\rdavis\Source\Repos\akka.net\src\core\Akka.Remote\Endpoint.cs:line 59
   at Akka.Remote.EndpointReader.OnReceive(Object message) in C:\Users\rdavis\Source\Repos\akka.net\src\core\Akka.Remote\Endpoint.cs:line 1569
   at Akka.Actor.UntypedActor.Receive(Object message) in C:\Users\rdavis\Source\Repos\akka.net\src\core\Akka\Actor\UntypedActor.cs:line 21
   at Akka.Actor.ActorBase.AroundReceive(Receive receive, Object message) in C:\Users\rdavis\Source\Repos\akka.net\src\core\Akka\Actor\ActorBase.cs:line 155
   at Akka.Actor.ActorCell.ReceiveMessage(Object message) in C:\Users\rdavis\Source\Repos\akka.net\src\core\Akka\Actor\ActorCell.DefaultMessages.cs:line 125
   at Akka.Actor.ActorCell.Invoke(Envelope envelope) in C:\Users\rdavis\Source\Repos\akka.net\src\core\Akka\Actor\ActorCell.DefaultMessages.cs:line 55]
Cause: Unknown

I think it means the message made it over, but things went bad trying to unpack it. Given that it works using Akka.Remote within the same process, maybe there is something different about the states of the two environments that could cause it? Has anyone seen this before? The hint being Error [Error deserializing object of type 'System.Object'.]

Ryan Davis
@rdavisau
Aug 08 2015 04:42
Ok I moved everything up to 1.04 and the same applies :cry:
Bartosz Sypytkowski
@Horusiath
Aug 08 2015 05:04
@rdavisau I need more. I've tried this myself and I haven't any of your problems
Ryan Davis
@rdavisau
Aug 08 2015 05:07
@Horusiath ok, good to know. I am going to start a clean self-contained test and if it still doesn't work, I'll share it
Ryan Davis
@rdavisau
Aug 08 2015 05:19
btw all your writeups have been very helpful.. thanks! :+1:
Bartosz Sypytkowski
@Horusiath
Aug 08 2015 05:21
actually I feel bad about them... this is not how things should look like
Ryan Davis
@rdavisau
Aug 08 2015 05:22
anything I should keep in mind?
Bartosz Sypytkowski
@Horusiath
Aug 08 2015 05:23
don't use Discriminated Unions between F# and C# actors
Bartosz Sypytkowski
@Horusiath
Aug 08 2015 08:02
I'm trying to update Akkling to the latest changes, and I've got a serious mindfuck - our remote serialization/deserialization test is failing, while exactly the same piece of code executed as separate script is passing
Ryan Davis
@rdavisau
Aug 08 2015 08:06
what kind of error?
Bartosz Sypytkowski
@Horusiath
Aug 08 2015 08:07
timeout, message is not arriving to actor
Ryan Davis
@rdavisau
Aug 08 2015 08:34
hm, no advice.. unless you are using a let rec loop style actor deployment and messing up the continuation signature like I was this morning :-1:
Bartosz Sypytkowski
@Horusiath
Aug 08 2015 08:35
ok, I think I've got it
Google.ProtocolBuffers dependency version missmatch
I hate all of these runtime exceptions
Ryan Davis
@rdavisau
Aug 08 2015 08:37
Mmm
I just fixed my remote deployment situation; I can't say for sure it was due to upgrading one end of the connection to the matching .NET runtime / F# 4.0 because I also simplified the solution a little, but I think it may have been the former
actually I did have an issue at one point with protocol buffers today, I saw repeated "Binding session to {google protocol buffers dll}" on the receiving end log when a message should have come through, is that what you would expect to see in the case of a version mismatch?
Christian Sparre
@christiansparre
Aug 08 2015 08:53
I'm wondering if there is a way to know if a PersistentView is "up to date"?, especially during a clean startup. I know snapshotting can minimize the replay time, however it would be nice to know if the view is going through a long restore.
Bartosz Sypytkowski
@Horusiath
Aug 08 2015 10:09
@rdavisau no, it's usually a runtime error specifying version number of and fully qualified assembly name
@christiansparre when PersistentView actor is created it won't respond on any custom messages until it fully and successfully recovers.
Roger Johansson
@rogeralsing
Aug 08 2015 10:14
@skotzko great job on the cluster docs!!
Bartosz Sypytkowski
@Horusiath
Aug 08 2015 10:20
@christiansparre you can however apply some custom metric i.e. in the receive function to count the number of messages actor received during recovery. You can also check what messages persistent view would receive - since whole API is message based you may simply recreate messages send to journal by persistent view to mock it and verify messages received.
Christian Sparre
@christiansparre
Aug 08 2015 11:40
@Horusiath thanks for clearing that up, just thought I experienced receiving something else (in my case a query) before the view was recovered, but guess I was mistaken :) just tried again and it behaves as expected...
Suhas Chatekar
@schatekar
Aug 08 2015 11:59
@Aaronontheweb I and @barambani have both submitted a PR for #1088. Could you take a look please?
Roger Johansson
@rogeralsing
Aug 08 2015 17:03
Need some more eyes on this, akkadotnet/akka.net#1222 am I right there or am I misreading something?
Suhas Chatekar
@schatekar
Aug 08 2015 18:21
How do I specify fully qualified name of a multinode test to run a single test?
```Akka.MultiNodeTestRunner.exe Akka.MultiNodeTests.dll -Dmultinode.enable-filesink=off -Dmultinode.test-spec="Akka.MultiNodeTests.ClusterConsistentHashingGroupSpec.AClusterRouterWithConsistentHashingGroupMustSendToSameDestinationsFromDifferentNodes"
E:\Source\akka.net\src\core\Akka.MultiNodeTestRunner\bin\Debug>Akka.MultiNodeTestRunner.exe Akka.MultiNodeTests.dll -Dmultinode.enable-filesink=off -Dmultinode.test-spec="Akka.MultiNodeTests.ClusterConsistentHashingGroupSpec.AClusterRouterWithConsistentHashingGroupMustSendToSameDestinationsFromDifferentNodes"
This does not work
Bartosz Sypytkowski
@Horusiath
Aug 08 2015 19:33
@schatekar isn't your problem related to #1199 ?
Chris Ray
@chris-ray
Aug 08 2015 20:01
I'm trying to run the basic akka.io examples using 1.0.4 from nuget and I cannot even call system.Tcp() without it going infinite loop
Chris Ray
@chris-ray
Aug 08 2015 20:07
namespace Proto1.Client
{
    using Akka.Actor;
    using Akka.IO;
    using System;

    class Program
    {
        static void Main(string[] args)
        {
            var system = ActorSystem.Create("proto1");
            var manager = system.Tcp(); //this causes 100% CPU
            Console.ReadLine();
        }
    }
}
Suhas Chatekar
@schatekar
Aug 08 2015 20:44
@Horusiath That could be the problem
may be I would look at that first
how do I assign github issues to myself?
Bartosz Sypytkowski
@Horusiath
Aug 08 2015 20:55
@schatekar under the labels on the right in issue view you have assignee field, from there you should be able to assign it to yourself. If you can't let me know, I will assign you to it.
@chris-ray have you played with TcpEchoService in the examples section?
Chris Ray
@chris-ray
Aug 08 2015 20:58
@Horusiath no, I'm working off the client/server example here: http://getakka.net/docs/IO
just calling system.tcp shoots the cpu to 100%. If I put a Thread.Sleep(5) in first line of Select() method in SelectionHandler.cs, it fixes it, but that's obviously a terrible solution
Roger Johansson
@rogeralsing
Aug 08 2015 20:59
@chris-ray I tried it from the source code and it works good there
Chris Ray
@chris-ray
Aug 08 2015 21:00
@rogeralsing I'm using dev branch, just cloned about 15 min ago... hmm...
so if you run the 3-line project I posted a bit earlier, your cpu stays low? Mine hits 100
Roger Johansson
@rogeralsing
Aug 08 2015 21:02
just tried it, works fine. im also on dev branch
Chris Ray
@chris-ray
Aug 08 2015 21:02
hmm
Roger Johansson
@rogeralsing
Aug 08 2015 21:02
try stepping into the Tcp extension method
it should pretty much just register the tcp extension and return the manager actor
Bartosz Sypytkowski
@Horusiath
Aug 08 2015 21:04
I've run it just a moment ago. It works fine
Chris Ray
@chris-ray
Aug 08 2015 21:05
that's strange, i'm running it and watching a pegged cpu...
but the thread sleep brings it back to 0% util
Chris Ray
@chris-ray
Aug 08 2015 21:15
would my environment matter in that three-line program? I'm running VS2015 on a server 2012 instance in AWS (waiting on my new computer to come in the mail)
t2.large
just trying to think of why the same program would work fine for everyone but me
Bartosz Sypytkowski
@Horusiath
Aug 08 2015 21:17
@chris-ray we're not runing it in aws :)
my guess is something with VM config or security
Chris Ray
@chris-ray
Aug 08 2015 21:18
could be... I didn't do any config, just straight out of the box, fired it up and installed VS about 3 hours ago
just ran it outside VS, as a straight exe and it's still pegging
unfortunately I don't have a windows computer to copy the DLL to and test to see if it's in fact the VM until my new one comes on Wednesday. If anyone wants to compile their three-liner and shoot it up to an AWS server 2012 instance to see if it presents for them, it would go a long way towards my sanity :smile:
Evert Mulder
@evertmulder
Aug 08 2015 21:29
@chris-ray I also tried you sample using the latest nightly build. Cpu is also rater high around 30% with no debugger attached (50% with debugger). (Core i7). Seems too much...
Chris Ray
@chris-ray
Aug 08 2015 21:31
@evertmulder good news (for my sanity anyway). What's your OS?
Evert Mulder
@evertmulder
Aug 08 2015 21:31
win8.1 x64 ent
Chris Ray
@chris-ray
Aug 08 2015 21:32
alright. there's def a problem then.
Evert Mulder
@evertmulder
Aug 08 2015 21:34
Also the sample server app http://getakka.net/docs/IO, runs with high CPU usage on my machine (no clients connected)
Bartosz Sypytkowski
@Horusiath
Aug 08 2015 21:34
could you guys set an issue on that? some exaples and screens/logs with perf monitors would be very helpfull
Chris Ray
@chris-ray
Aug 08 2015 21:35
I only have about 5 min before I'm dark the rest of the weekend. @evertmulder could you do that?
Evert Mulder
@evertmulder
Aug 08 2015 21:39
Yep. I can create an issue. Any perf mon counters (apart from cpu) that could help?
Bartosz Sypytkowski
@Horusiath
Aug 08 2015 21:44
I think that code sample to reproduce the problem and CPU stats to compare results should be sufficient.
Maybe machine spec (if it's cloud VM, it's type) also :)
Evert Mulder
@evertmulder
Aug 08 2015 21:57
@Horusiath created #1225
Bartosz Sypytkowski
@Horusiath
Aug 08 2015 21:58
thx
:)
Suhas Chatekar
@schatekar
Aug 08 2015 22:44
@Horusiath I do not see anything under the labels on the right to assign the issue to myself
do I need to be given some kind of permission on the repo?
Roger Johansson
@rogeralsing
Aug 08 2015 22:59
ah, yes, one needs to be in the contributor team list.. Ill add
invite sent. that should allow you to assign the task to you
Suhas Chatekar
@schatekar
Aug 08 2015 23:04
thanks
still cannot assign to myself. This is what I see
blob
Roger Johansson
@rogeralsing
Aug 08 2015 23:11
strange, I've assigned you now
Suhas Chatekar
@schatekar
Aug 08 2015 23:16
thanks
one question - after my PR is merged, can I just continue to work on the same fork and local clone for some other issue?