Hi @jongoldDB - safe to say I have a way to go before understanding Ambrosia, and will circle back to above API question at some stage.... but already excited by the potential :relaxed: For example from the docs it seems multi producer is already supported via the buffer: we log all incoming requests (which can come from multiple sources)?
On the consumer side though it seems that expensive partitionable workloads need to be sharded, where each shard runs on its own set of cores... wondering if this means even on the same VM/container and if code examples exist for this scenario... how should long running operations (calls to restful APIs, databases, indexing engines for example) be reasoned about & integrated with?
... since serializing and deserializing the context of an awaiting task is pretty experimental, for any sort of service, which runs for a while (as opposed to a job, which runs for a short period of time), and will likely experience upgrades, patches, etc..., we advocate a more message based way of communicating. So in essence, a return value is sent back to the call originator as a separate message/RPC. You could add some sort of an ID to the initial call and the return value RPC which would allow you to connect the return value to the initial call, but it would be your responsibility to squirrel away whatever information you need to complete the action upon receipt of the response on the client. Essentially, you're implementing your own continuation. This would also allow you to evolve your code in a client upgrade scenario by explicitly providing transformations during upgrade of the continuation state, as well as new code to handle the continuation itself as part of the upgrade. In practical terms, this is probably the best you can do in terms of code evolution. Make sense?
Nope, does not make sense at all
So, exactly why should every single developer need to invent a method to get a value back by themselves?
Thanks for your thoughts around this difficult issue :) Since we can no longer support the serialization of Task state in an even experimental way, we will soon be releasing, with Ambrosia, helper libraries that make it easier to author calls which are expected to generate responses, making it much easier to write this sort of code. Note that some of the issues associated with handling upgrades still remain, and are endemic to distributed systems which are long lived, and have evolving APIs. That said, we will keep this in mind when designing these helper libraries, and will hopefully do better in this regard than the fully automatic, but brittle, outcome you get using the old experimental Task serialization approach.
To add some more clarity around this decision, unfortunately, C# just doesn't give us the hook we need to serialize and recover application state that includes Task state. Without this, there is no recourse we know of other than something experimental which makes assumptions about the C# runtime which have already broken. As a result, we can no longer support even the experimental approach we had running before (see the latest version). Note that if C# included this capability, we could immediately make use of it. Also, other languages which support recoverable async state can be supported by Ambrosia with an appropriate language binding, although this is asking a lot of today's languages and runtimes, and I'm not sure if such a language exists. We look forward to your feedback on our future release which addresses these issues for today’s C#.
Hi @jongoldDB - just wanted to share a potentially complementary project has gone live from Bart J.F. De Smet (of Rx fame). Interested in thoughts on how Ambrosia compares, would it make sense to collaborate as both are MS birthed projects 🙂 etc?