Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Pantazis Deligiannis
    @pdeligia

    Btw, just to add to this, if you run the test from the command line tool (coyote test or coyote replay) you can add the option --break (-b) which will start the debugger before it runs the test (it basically instruments a System.Diagnostics.Debugger.Launch()). This typically opens a Visual Studio selector for the debug session, as @akashlal described. But I have only tried this approach of debugging using the VS IDE on Windows so far.

    Another way, is to run the Coyote test programmatically. We plan to add documentation on this asap, but basically you can programmatically create a Coyote TestingEngine and run the test based on a test Configuration (the coyote command line tool uses this logic under the hood). That way, you can just debug the test with whatever IDE or process you are using until now (e.g. add breakpoints, right click the test and click debug). This is also useful if you want to run Coyote tests on a unit testing framework like xUnit or MSTest, etc. Until we add this info in our docs, you can see my answer on this closed GitHub issue that provides a code snippet how to do this (shows both test and replay): microsoft/coyote#23

    Tomáš Deml
    @tomasdeml
    Hi, I am evaluating Coyote and in the documentation there is no word about the ValueTasktype. I searched the repo and could not find anything related...
    Pantazis Deligiannis
    @pdeligia
    Hi @tomasdeml, thanks for your interest in Coyote! Yes, you are right: we do not support the ValueTask type yet, only the original Task type and other common types like TaskCompletionSource (please note that comparing to our actor/state-machine programming model, our task programming model is still in-preview, so we are keep adding new features and supported types, to make it easier to be consumed, and are prioritizing those based on user demand and feedback). Saying this, supporting ValueTask should not be too hard (as a lot of the logic to control and systematically test those is the same with Task), so we can try to add this asap.
    Rob Landers
    @withinboredom
    Hello all. I was exploring the documentation, but is there a "right" way to serialize/store actors (for program shutdown or resume)?
    Akash Lal
    @akashlal
    @withinboredom One option is to define a Shutdown event, to which an actor will respond by serializing and storing its state, and then halting itself. There should not be any complications here, unless you have special requirements like shutting down actors without pausing client requests.
    Rob Landers
    @withinboredom
    I played a bit with that. I overrode OnHaltAsync(Event e) and called runtime.Stop() which according to the docs "Terminates the runtime and notifies each active actor to halt execution." however OnHaltAsync was never called. Does that seem like a bug or should I move that to an issue?
    Akash Lal
    @akashlal
    The OnHaltAsync method is called when an actor is about to halt. Note that halting of actors has to be done explicitly, by either sending the HaltEvent to it, or the actor itself doing a RaiseHaltEvent. Were you trying to halt an actor, but OnHaltAsync never got invoked? If so, do file an issue.
    Rob Landers
    @withinboredom
    No, I didn't explicitly send the event. https://microsoft.github.io/coyote/learn/ref/Microsoft.Coyote.Runtime/ICoyoteRuntime/Stop seems to say that it will do it for me (which may be needed to halt any actors that may be started by the first actor).
    I'll file an issue later today, need to run a few errands.
    Akash Lal
    @akashlal
    Ah, I see. You called Stop but OnHaltAsync never happened. Do file an issue when you get the time and we can take a closer look.
    Chris Lovett
    @lovettchris

    I checked the code and indeed Runtime Stop was not designed to notify all existing running actors via OnHaltAsync. it just does this :-)

    public void Stop() => this.IsRunning = false;

    But is an interesting idea, but we'd have to change the api to make Stop async, perhaps a new HaltAsync on ICoyoteRuntime would make sense, and this would be implemented by the ActorRuntime, and ignored by the task runtime. So how about this: microsoft/coyote#36

    Brian Chavez
    @bchavez

    I have some questions about using Coyote and Orleans. I think I understand the conceptual programming model with System Task vs Coyote Task; one in which the Task behavior is transparent when running in the runtime/release mode vs under test using the coyote test engine. However, w.r.t. Coyote's Actor model, there doesn't seem to be a 1:1 comparison because there is no Actor object in the .NET Framework. The closest model we have would be something like Akka.net or Orleans.

    So, how does one conceptualize the usage of Coyote's Actor in relation to Orleans Actor/Grain? Is there some way to make Coyote's Actor's behavior become transparent when running under the Orleans runtime?

    Akash Lal
    @akashlal
    @bchavez You're right that Coyote is simply pass-through for Task and for Actors Coyote provides a runtime. However, the runtime is in-memory and very lightweight. One option is the following. Each Orleans Grain hosts a single Coyote Actor. When the grain gets a message, it passes it to the actor. For testing, the Grain is erased away and the actors talk directly to each other. In production, the Grain provides all the distributed goodies. This way, you do incur a small overhead of using the Coyote runtime in production, but it should be very minor. This seems to be a common question -- if you're up for it, perhaps you can prototype a simple solution for "test with coyote, deploy with Orleans" so we can discuss more concretely.
    Brian Chavez
    @bchavez
    @akashlal, thank you for the clarification. It is starting to make much more sense now. I'll try to put together a sample solution over the weekend. Looking forward to it!
    Akash Lal
    @akashlal
    @bchavez excellent, would be happy to help.
    Brian Chavez
    @bchavez

    Hi @akashlal , I have the basic Orleans <-> Coyote grain/actor working. However, the coyote testing part isn't done yet.

    The example below tries to model the operation of a shopping cart:
    https://github.com/bchavez/coyote-samples/tree/master/OrleansActors

    • Items can be added, removed, and a total can be requested.
    • The model can be improved to allow the concept of a quantity, but we can do that later.

    I wanted to pause here because I'm noticing some call semantic issues between the Coyote and Orleans runtimes; I'd like to get some feedback on the call semantics.

    The first thing I notice is this part here:
    https://github.com/bchavez/coyote-samples/blob/0b1d4bb43aa4c8a1549a356840f23913d72802a4/OrleansActors/Grains/CartGrain.cs#L32-L42

    Orleans calls actorRuntime.SendEvent and this operation returns immediately when the item is enqueued for the Coyote Actor's inbox. The ideal semantics I'm looking is "awaiting" for the event to be processed before returning control to the Orleans runtime. I think this is important because if control is returned back to the Orleans runtime (based on Coyote inbox semantics), there's a chance that the Coyote Actor will "lag behind" the Orleans runtime; which I don't think is a good situation to be in.

    The second issue is getting simple values "out of" the Coyote runtime here:
    https://github.com/bchavez/coyote-samples/blob/0b1d4bb43aa4c8a1549a356840f23913d72802a4/OrleansActors/Grains/CartGrain.cs#L65-L76
    Again, I think I need some kind of construct to "await" for the Coyote Actor to "finish its work" before we can return control back to the Orleans runtime. One approach that seemed to work is using the AwaitableEventGroup<T> mechanism, but it feels like a hack. It feels like a hack because I think I should be using something like .SendEvent(backToCaller, response); and instead I'm using a completely different response mechanism this.CurrentEventGroup to acquire the mechanism to respond; where other mechanisms use .SendEvent in the Coyote Actor.

    https://github.com/bchavez/coyote-samples/blob/0b1d4bb43aa4c8a1549a356840f23913d72802a4/OrleansActors/Actors/CartActor.cs#L62-L71

    Ideally, the natural semantics I'm looking for are shown below (when interacting with the Coyote runtime from outside):

    actorRuntime.SendEvent(event);
    await actorRuntime.ReceiveEventAsync(actorId, typeof(Response));

    I guess I could create C# extension methods to hack-in using the this.CurrentEventGroup mechanism, but not sure. Let me know your thoughts. Thanks!

    Brian Chavez
    @bchavez
    image.png
    Chris Lovett
    @lovettchris
    A more general solution to this is the AwaitableEventGroup<T>, see https://microsoft.github.io/coyote/learn/programming-models/actors/event-groups
    Akash Lal
    @akashlal
    @bchavez Great progress. What you need, in some sense, is to synchronously execute the Coyote actor (so its in lockstep with the Orleans Grain). We used to have these APIs for this purpose but we are retiring them in favor of something more general that Chris is referring to.
    Minotaur
    @Psiman62
    I am a Coyote novice, but an actor programming fan. I want to use Coyote actors to express my business logic and conduct systematic testing. I want to continually deploy my actors as Durable Entities in Azure Functions. Your doco seems to suggest that using Coyote StateMachines inside my Durable Entities is the right way to do this. But I have many questions and concerns about this, particularly re systematic testing of multiple concurrent actors. Before I start asking detailed how-to questions re this, is there a guide, or sample or demo, of how a Coyote actor/state machine might be used within other actor-like runtime environments?
    Akash Lal
    @akashlal

    @Psiman62 We don't have specific guidance documented at the moment. The basic strategy follows what we have in our samples: use the distributed machinery in production (e.g., use the "Send" that the distributed actor framework provides), but erase it all away for testing (e.g., use the Coyote Send instead).

    We have some experience doing this with Service Fabric Actors, but it was on an internal codebase. @bchavez is putting together a solution with Orleans Actors that can be a useful guide as it comes together. The "serverless" aspect of Durable Function is perhaps new, so I would be interested in following your progress. And happy to answer questions, of course, as you go along.

    Minotaur
    @Psiman62
    Thanks @akashlal . I'm going to try to write my business actors in a runtime-agnostic way against a minimal abstract actor runtime, then plug-in Coyote for systematic testing. I'll see how far I get with it before trying to implement business actors as state machines - which is really what I ultimately want to do.
    Mark Davies
    @joro550
    Hey there! I'm looking into coyote and am kind of curious about how it interacts with things outside of the system you are developing, say for example I am reading information from a database in the code where i am writing a test around will coyote save the state of the object at the time of testing?
    Akash Lal
    @akashlal
    @joro550 When running in production, coyote is just like any other .NET library. When you're writing a coyote test, you should write it in the style of a unit test. So you won't interact with an actual database (that would be way too slow) but rather a mocked version of it. The mock would then be responsible, say, for saving the state in-memory.
    Mark Davies
    @joro550

    I'm a little confused by that, if you are treating coyote tests as some form of unit tests where you are faking the boundaries between your system and an IO system how are you ever going to find any locking issues.

    For example if I was using files to save and loading information into my application and there was alocking issue in that particular part of my code but then in my coyote tests I replace actual file system with a fake that just talks to in memory file system how would it ever find the locking?

    Akash Lal
    @akashlal
    @joro550 I don't follow your example. How are you associating locks with IO? A coyote test, like a unit test, is naturally designed to find issues in your code when exercised against a model of external dependencies. The difference is that your code, or even the mock, can have concurrency or non-determinism. You need to ensure that the mock is rich enough so that it exercises your code. Can you spell out your example a bit more?
    Chris Lovett
    @lovettchris
    @joro550, you would need to model file level locking in your in-memory file system "mock" if that is something you want tested, something like this: https://gist.github.com/lovettchris/d7ba169a329a51bb0f7571d394e6885b. Would be fun to finish this, I have some apps that could definitely use thorough testing of file system behavior like this...
    Minotaur
    @Psiman62
    Does the Coyote library that replaces Task also support ValueTask? I'd like to extend systematic testing my own async libraries called by my actors that often return ValueTask.
    Pantazis Deligiannis
    @pdeligia
    Thanks for checking @Psiman62, we sadly do not currently support ValueTask, but its on our backlog.
    Hadrien Dorio
    @hdorio
    Hi everyone, can someone tell me what's the differences or links between Coyote, P, P# and Zing? And why Zing's use was dropped? (if it was dropped)
    Akash Lal
    @akashlal
    @hdorio Coyote is an evolution of P#. P is a different project. P is its own language with compiler for code generation. Coyote/P# are .NET. P was using Zing earlier but not anymore. I don't think any one is supporting Zing.
    Hadrien Dorio
    @hdorio
    Thanks @akashlal , so what's the differences between P# and Coyote?
    Akash Lal
    @akashlal
    Coyote supports more things, Actors and `Tasks, better logging and many other things.
    renuIyer
    @renuIyer
    Hi all,
    I am new to Coyote and am still going through the sample example. While going over 'https://microsoft.github.io/coyote/#tutorials/test-failover/', I found that it says "Chances are this will find a bug quickly, one of the safety assertions will fire and you will see that a test output log is produced, like this:
    ".\bin\net5.0\Output\CoffeeMachineTasks.exe\CoyoteOutput\CoffeeMachine_0_0.txt""
    I am actually trying to see the log file and the '.schedule' file to be generated which can be used for replaying the failed case.
    Could someone please help me with this?
    PS - My end goal is to try and integrate this with our dotnet core projects - so any guidance towards that is also welcome
    Akash Lal
    @akashlal
    @renuIyer What is the problem that you're facing? Are you able to run coyote test on the sample? It should output a .txt file, which is the log of the failing execution, and also a .schedule file that can be used for coyote replay.
    renuIyer
    @renuIyer
    @akashlal I am able to get the tracelog and schedule file only when I run it with the option to generate DGML graph.
    Also, I need to write tests for .NetCore APIs, is there a simple sample available for the same?
    renuIyer
    @renuIyer
    I was successful in integrating Coyote in existing .NET APIs. However, logging (WithVerbosityEnabled) doesn't generate any logs. Any ideas anyone?
    I am using integration with xunit.
    Chris Lovett
    @lovettchris
    Hi @renulyer, I see the tutorial recommends "coyote test ./bin/net5.0/CoffeeMachineTasks.dll -i 1000 -ms 500 --sch-pct 10" and I see this does not always catch the bug. So what you can do to increase the chance of finding the bug is to use this command line "coyote test ./bin/net5.0/CoffeeMachineTasks.dll -i 1000 -ms 500 --sch-portfolio -p 10" which will use all scheduling strategies and 10 processes to do a lot more testing at once. See "coyote test --help" for more options. There is also a page here on how to integrate coyote into unit tests: https://microsoft.github.io/coyote/#how-to/unit-testing/.
    Oh, and to connect Coyote logging with XUnit ITestOutputHelper you can use Microsoft.Coyote.Tests.Common.TestOutputLogger as a wrapper, and set that on the TestEngine.Logger property.
    renuIyer
    @renuIyer
    Thanks @lovettchris! If anyone has already tried an integration with xunit, please do share once or let me know if I can share a sample and understand if there are any improvements with the current implementation.
    on the logger - I actually initialized and logger separately in the function however, I still do not get the kind of logs available when running from command line (nothing like iteration #1 .. iteration#2 etc.). Am I missing something with the xunit integration. Any settings specific for logging?
    renuIyer
    @renuIyer
    Could anyone help with this error - Deadlock detected. Task(0) is waiting for a task to complete, but no other controlled tasks are enabled. - Coyote is not helping find the actual deadlock but keeps giving this.
    Akash Lal
    @akashlal
    @renuIyer The error means that Coyote found a deadlock. There is a single Task Task(0) that is blocked, but there is nothing else to unblock it. You can try to repro this run and attach a debugger so that you can step through the trace and see what is going on.
    renuIyer
    @renuIyer
    Thanks @akashlal . Also, Is Coyote supported for .netcore 3.1 as of now? or only for .net 5?
    Pantazis Deligiannis
    @pdeligia
    @renuIyer Coyote is indeed supported for both .NET Core 3.1 and the newer .NET 5.0.
    shaya-pb
    @shaya-pb
    Hi - is the coyote-samples repo up to date? The tests in the ImageGallery solution consistently throw a Deadlock detected. Task(0) is waiting for a task to complete, but no other controlled tasks are enabled. coyote error. And when I debug it seems to be thrown when calling a static method which does almost nothing (the clone method here )
    From my understanding this is not the expected error
    shaya-pb
    @shaya-pb
    running the test from the Coyote CLI directly surfaces the following exception: Unhandled exception 'System.IO.FileNotFoundException' was thrown in 'ImageGalleryTests': Could not load file or assembly 'Microsoft.Extensions.Logging.Abstractions, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'. The system cannot find the file specified. which leads me to believe that something needs to be updated in the coyote-samples repo