This sounds like the behaviour you need? https://www.mongodb.com/blog/post/an-introduction-to-change-streams
I did try the change streams, but its only for replica sets :( Else it would have been perfect. Actually considered what the implications would be of having a replica set just for this reason, but might be overkill.
@ddesmond The reasons why I wanna get away from using actions in Avalon is; clarity and future proofing.
Clarity
If we can map directly from MongoDB to another database like CGWire, it'll be very clear what needs to be implemented for other databases.
Future Proofing
Using actions in Avalon like publishing etc to sync, means we need to either emit a signal or add a "syncing" call when we change something in the database. Granted this could happen at the io.py stage to minimize calls.
But what happens if we built something different in the future that makes changes in the database but does not sync accidentally?
Hooking directly into the changes into the database would allow us to having a syncing mechanism that lasts.
Although if we cant listen to changes in MongoDB, then the io.py method might be better than the brute force method. Sorry, just realising some things while writing :)
For the CGWire side we can listen to events; https://gazu.cg-wire.com/events.html
io (who called it io and named its current context as Session btw?) but that would require thread because watch method is actually while loop that waits for messages from mongo server.
I was thinking that we could discard the whole listening directly to MongoDB ( cause it tricky and a faff), and just have a syncing call before/after writing to the database.
Something like (https://github.com/getavalon/core/blob/master/avalon/io.py#L335):
@auto_reconnect
def insert_one(item):
assert isinstance(item, dict), "item must be of type <dict>"
schema.validate(item)
result = self._database[Session["AVALON_PROJECT"]].insert_one(item)
# Syncing to available registered sync modules here.
return resultYes, I did. I will be looking into getting this prepped for online as soon as possible, and get things online. Only the Autodesk Arnold render talk had some confidential bits which I unfortunately was not allowed to record. But I think I got the crucial bits of it at least since those were allowed.
There were some very impressive talks, so I'm looking forward to sharing it.
