These are chat archives for akkadotnet/akka.net

7th
Jun 2016
qwoz
@qwoz
Jun 07 2016 01:45
when I use the sample config at http://getakka.net/docs/Logging I can receive _log.Info() calls on stdout but _log.Debug() calls don't output. Is there something else needed to enable debug level logging?
the config on start output:
  akka : {
    log-config-on-start : on
    stdout-loglevel : DEBUG
    loglevel : DEBUG
    actor : {
      debug : {
        receive : on
        autoreceive : on
        lifecycle : on
        event-stream : on
        unhandled : on
      }
    }
  }
qwoz
@qwoz
Jun 07 2016 03:51

I'm also running into a strange issue with ScheduleTellOnceCancelable and testing. My test code uses:

        private TestScheduler Scheduler
        {
            get
            {
                return (TestScheduler)Sys.Scheduler;
            }
        }

and inside the actor I want to test I have:

        Context.System.Scheduler.ScheduleTellOnceCancelable(TimeSpan.FromSeconds(60), ...);

When the Tell fires, I send a message back to the calling actor. If my test code uses:

        Scheduler.Advance(TimeSpan.FromSeconds(61));
        ExpectMsg<Message>();

then it fails. If I change this to:

        //Scheduler.Advance(TimeSpan.FromSeconds(61));
        ExpectMsg<Message>(TimeSpan.FromSeconds(61));

then everything works. Is there something I need to do inside the actor I'm testing to have it use the appropriate scheduler, whether production or testing?

Aaron Stannard
@Aaronontheweb
Jun 07 2016 03:52
@alexvaluyskiy I started working on some of the dispatcher and actorcell issues
figured that those are needed to ensure some of the correctness around routers
the dispatcher system is pretty thread-bare at the moment
working on the accounting code needed to cleanly terminate them
which should help with the race conditions in specs that make heavy use of custom dispatchers
Akka.Remote.Tests, Cluster.Tests, Persistence.Tests, Streams.Tests
other good news, if this is implemented correctly
this will massively reduce the memory footprint of all actors across the board
by eliminating a second concurrent queue inside the mailbox for system messages and using a specialized lightweight structure instead
in line with what the JVM does
@qwoz hmmm
I'm not sure what the issue is there...
might have to do with the implementation of the TestScheduler
ExpectMsg might call it differently internally in a way that triggers the schedule to fire for your actor properly
not sure why - not as familiar with that code internally
but it sounds like a bug
qwoz
@qwoz
Jun 07 2016 03:58
yeah, the actor code used to use SetReceiveTimeout(TimeSpan.FromSeconds(60)) which works well with Scheduler.Advance() and passes all tests but I changed the design to use ScheduleTell... and now it fails.
Aaron Stannard
@Aaronontheweb
Jun 07 2016 03:58
we'd appreciate it if you'd file a bug for that
sounds like there's some case where it doesn't fire properly
if you could include that detail you just shared on the Github issue
that would be :+1:
qwoz
@qwoz
Jun 07 2016 03:58
will do
Aaron Stannard
@Aaronontheweb
Jun 07 2016 03:58
thanks!
qwoz
@qwoz
Jun 07 2016 03:59
any thoughts on the logging issue I mentioned above? not urgent, just a bit of a nuisance.
Aaron Stannard
@Aaronontheweb
Jun 07 2016 04:00
hmmm that's weird
don't have a clear idea offhand
@rogeralsing or @danthar might have an idea as to why the Debug messages aren't showing up
qwoz
@qwoz
Jun 07 2016 04:03

The first few output lines show:

[DEBUG][2016-06-07 3:44:21 AM][Thread 0016][EventStream] subscribing [akka://all-systems/] to channel Akka.Event.Info
[DEBUG][2016-06-07 3:44:21 AM][Thread 0016][EventStream] subscribing [akka://all-systems/] to channel Akka.Event.Warning
[DEBUG][2016-06-07 3:44:21 AM][Thread 0016][EventStream] subscribing [akka://all-systems/] to channel Akka.Event.Error
[DEBUG][2016-06-07 3:44:21 AM][Thread 0016][EventStream] StandardOutLogger started

Note there's no Akka.Event.Debugthere. Later it shows:

[DEBUG][2016-06-07 3:44:21 AM][Thread 0016][EventStream] subscribing [akka://MySystem/system/log1-DefaultLogger] to channel Akka.Event.Debug
[DEBUG][2016-06-07 3:44:21 AM][Thread 0016][EventStream] subscribing [akka://MySystem/system/log1-DefaultLogger] to channel Akka.Event.Info
[DEBUG][2016-06-07 3:44:21 AM][Thread 0016][EventStream] subscribing [akka://MySystem/system/log1-DefaultLogger] to channel Akka.Event.Warning
[DEBUG][2016-06-07 3:44:21 AM][Thread 0016][EventStream] subscribing [akka://MySystem/system/log1-DefaultLogger] to channel Akka.Event.Info
[DEBUG][2016-06-07 3:44:21 AM][Thread 0016][EventStream] subscribing [akka://MySystem/system/log1-DefaultLogger] to channel Akka.Event.Warning
[DEBUG][2016-06-07 3:44:21 AM][Thread 0016][EventStream] subscribing [akka://MySystem/system/log1-DefaultLogger] to channel Akka.Event.Error
Aaron Stannard
@Aaronontheweb
Jun 07 2016 04:07
hmmmm
looks like the StandardOutDebugger isn't reading its configuration
let's take a look
private void SubscribeLogLevelAndAbove(LogLevel logLevel, IActorRef logger)
        {
            //subscribe to given log level and above
            foreach (var level in AllLogLevels.Where(l => l >= logLevel))
            {
                Subscribe(logger, level.ClassFor());
            }
        }
Aaron Stannard
@Aaronontheweb
Jun 07 2016 04:12
I'm going to guess that code is whiffing hard on the math somewhere
but after combing through the codebase that's my best guess
either that or the wrong value being read
should be fairly easy to debug
but I can't pull off of the bit I'm on right now to do that
qwoz
@qwoz
Jun 07 2016 04:15
not a problem... I'm using _log.Info()for anything I want to see, so there's a trivial workaround.
Aaron Stannard
@Aaronontheweb
Jun 07 2016 04:17
definitely recommend logging a bug
people do go through those
and fix them
qwoz
@qwoz
Jun 07 2016 04:20
:plus1:
Bart de Boer
@boekabart
Jun 07 2016 08:17
@Aaronontheweb I can't seem to figure out why (which) exactly perf.test failed on PR #2051
Bart de Boer
@boekabart
Jun 07 2016 08:23
Found it, [FAIL] Expected Elapsed Time to must be less than or equal to 2.00 ms; actual value was 3.33 ms.
Marc Piechura
@marcpiechura
Jun 07 2016 08:26
@boekabart I create a PR later today to increase the expected duration, has nothing to do with your PR
There is another streams performance benchmark which failed now and then, I set the expectations a little bit to close ;)
Bart de Boer
@boekabart
Jun 07 2016 08:27
I was hoping that, couldn't really see the relationship directy
Peter Bergman
@peter-bannerflow
Jun 07 2016 09:49
I could need some help interperting log messages in my cluster setup, I have a Lighthouse instance up and running which I use as seed node in my cluster, in the log from Lighthouse I see a lot of messages like this:
Dropping message [Akka.Actor.ActorSelectionMessage] for non-local recipient [[akka.tcp://bf@100.118.4.31:4053/]] arriving at [akka.tcp://bf@100.118.4.31:4053] inbound addresses [akka.tcp://bf@127.0.0.1:4053]
Could this mean that gossip messages aren't delivered to Lighthouse?
MartinNiemandt
@MartinNiemandt
Jun 07 2016 11:41
Is it fine to spam an actor every say 30 seconds, the SetReceiveTimeout is set to one minute?
Roger Johansson
@rogeralsing
Jun 07 2016 12:04
@peter-bannerflow looks like you are mixing 127.0.0.1 with remote systems
Peter Bergman
@peter-bannerflow
Jun 07 2016 12:09
@rogeralsing It looks like that in the log but I'm not sure where that comes from. I have configured my nodes to use a LAN IP as public-hostname and then I've just used 0.0.0.0 as hostname to bind to all interfaces...
Finn Newick
@finnnewick-justgiving
Jun 07 2016 12:13
@Horusiath all working now - turned out that using 'object' as the TEvent parameter was the problem, changing it to a marker interface fixed the issue of it not transitioning states - is that by design or is it a bug? Thanks for all the help!
Roger Johansson
@rogeralsing
Jun 07 2016 12:16
@finnnewick-justgiving what is the context here? using object as argument/property etc have been a notorious problem due to Json.NET as it does not carry type information for primitives, e.g. guids/dates/etc
Deniz ─░rgin
@Blind-Striker
Jun 07 2016 13:04
Hi, We created an Akka Cluster infrastructure for Sms, Email and Push notifications. But we have some stability problems on production. Can i write my problem here or should i send somewhere else?
Bart de Boer
@boekabart
Jun 07 2016 13:19
Since you'll probably write a lot, and the answer might be useful for many, stackoverflow (and then paste the link here for good measure)
Deniz ─░rgin
@Blind-Striker
Jun 07 2016 13:35
Thank you. Here is the link
Bartosz Sypytkowski
@Horusiath
Jun 07 2016 13:43
@Blind-Striker - @Aaronontheweb will be probably the best expert here.
Aaron Stannard
@Aaronontheweb
Jun 07 2016 17:03
@Blind-Striker left an answer for you
TL;DR; known bug, should be fixed in 1.1
we're furiously working on validating Akka.Cluster's behavior with multi-node tests, fixing low-level issues with it - all manner of things
going pretty smoothly thus far
@MartinNiemandt you can use the scheduler to deliver a message every 30 seconds
ScheduleTellRepeatedly
our poor little build server lol
has a backlog of work like 90 minutes long
ah, just an hour actually
still though
Aaron Stannard
@Aaronontheweb
Jun 07 2016 17:09
@rogeralsing I'm going to skip using the https://msdn.microsoft.com/en-us/library/system.nonserializedattribute(v=vs.110).aspx attribute on this message
because I'd need to make some other changes to the way ISystemMessages are implemented
well, hmmm
allows us to turn all ISystemMessages into a linked list
unfortunately, I can't do that on an interface
so I'd need to add a new abstract base class that also derives from the interface
which exposes this stuff
and it might be pretty hacky
so in the future if we were to do a new major version, I'd probably drop the ISystemMessage interface and turn it into a base class
I have an alternative way of doing this which uses a wrapper class
Aaron Stannard
@Aaronontheweb
Jun 07 2016 17:15
eh... you know what... let me try doing this the JVM's way and we'll see what the fallout is on the PR
ok, I totally underestimated resharper. That was easy as hell. I can go ahead and do this the JVM's way
Finn Newick
@finnnewick-justgiving
Jun 07 2016 17:27
@rogeralsing it's the TEvent parameter on the PersistentFSM base class. This was only in a throw-away spike so it's not really an issue. I suspect because the state transition events were sharing the same base type as the application events maybe they were being replayed incorrectly - but haven't looked into it yet.
Roger Johansson
@rogeralsing
Jun 07 2016 17:43
@Aaronontheweb oh, never seen that sys messages can be linked, weird
Aaron Stannard
@Aaronontheweb
Jun 07 2016 17:43
I think it's new
ish
Roger Johansson
@rogeralsing
Jun 07 2016 17:43
why is that? wouldnt it have been better to make some sort of envelope containing multiple sys messages instead? linking them seems super odd imo
Aaron Stannard
@Aaronontheweb
Jun 07 2016 17:43
but yeah, the way the system message queue in all mailboxes is implemented is essentially a linked list
it's a memory optimization to some extent
Roger Johansson
@rogeralsing
Jun 07 2016 17:44
umm there cant really be so many sys messages that they cause a mem footprint to care about?
Aaron Stannard
@Aaronontheweb
Jun 07 2016 17:44
it's to eliminate the need to keep a second collection object around
Roger Johansson
@rogeralsing
Jun 07 2016 17:44
its not like they are flooding the system
Aaron Stannard
@Aaronontheweb
Jun 07 2016 17:44
for that exact reason
the overhead of allocating a second queue
if you're only going to have a handful of system messages
may as well use something simple like this
the other reason relates to how termination is handled with the dispatchers
Roger Johansson
@rogeralsing
Jun 07 2016 17:45
so we need to do the whole CAS/lock dance on them then?
not seeing any CAS stuff yet
looks like the processMailbox command
which is what enforces memory consistency, is where they deque - so it makes sense that the reads would be safe
but the writes on the other hand....
ah yeah
Aaron Stannard
@Aaronontheweb
Jun 07 2016 17:49
there you go
good old CAS
jinx
Roger Johansson
@rogeralsing
Jun 07 2016 17:50
this is going to open up a whole can of worms I tell ya :) we have not respected the "never send the same system message to two actors" anywhere... so.. yak yak..
Aaron Stannard
@Aaronontheweb
Jun 07 2016 17:50
a shit sandwich I am prepared to eat my friend
lol
in order to fix some issues with the dispatcher shutdown and child attach stuff
when I started digging through what the real issues were, this came up
phase two of the SendSystemMessage stuff I did in the previous PR
problem with this one is it introduces more "public" API changes
but it's parts of the public API that no one should be using - all internals
inside SysMsg namespace
Roger Johansson
@rogeralsing
Jun 07 2016 17:52
btw. running multinode with "Dmultinode.spec="ClusterSingletonManagerStartupSpecs"" passes locally. that is the correct spec to run for the issue Im seeing in UID fix?
Aaron Stannard
@Aaronontheweb
Jun 07 2016 17:52
yep
Alex Valuyskiy
@alexvaluyskiy
Jun 07 2016 17:52
@Aaronontheweb could you make a milestone for v1.1 plz Right now is not possible to see all scope of work and what should we done before release.
Aaron Stannard
@Aaronontheweb
Jun 07 2016 17:52
could have been an issue where the server just timed out
it's there
you can use it
Alex Valuyskiy
@alexvaluyskiy
Jun 07 2016 17:55
@Aaronontheweb it has outdated issues, maybe we should clear all issues in that milestone?
Aaron Stannard
@Aaronontheweb
Jun 07 2016 17:55
it has some
others are ones that need to be reviewed
and marked as fixed
which I'm doing now
start adding the new issues to it
hehe
just used the bulk editor to do that
Aaron Stannard
@Aaronontheweb
Jun 07 2016 18:02
k, the milestone should be pretty current now
Alex Valuyskiy
@alexvaluyskiy
Jun 07 2016 18:05
Few notes
  • remove almost all closed issues (they don't belong to this release)
  • add all closed issues since 26.04
Aaron Stannard
@Aaronontheweb
Jun 07 2016 18:08
eh
no point in adding the stuff that's already merged in - it gets included in the release notes
hmm
I'll look into cleaning it up a bit later - we should be using this more
now that I think about it
but yeah, good suggestions
I'll do that
done
updated that
Alex Valuyskiy
@alexvaluyskiy
Jun 07 2016 18:25
These also could be in the scope
akkadotnet/akka.net#1572
akkadotnet/akka.net#1762
akkadotnet/akka.net#1577
But this one is definitely not
akkadotnet/akka.net#2012
Aaron Stannard
@Aaronontheweb
Jun 07 2016 18:27
guess I grabbed #2012 by accident while I was labelling everything
alright, that milestone is looking good
I'm going to knock out the last remaining Akka.Remote MNTR specs
plenty of cluster specs for others to port
simonpetty
@simonpetty
Jun 07 2016 18:55
thanks for the tip @Silv3rcircl3
Marc Piechura
@marcpiechura
Jun 07 2016 19:15
@simonpetty I can't remember which one, but np ;-)