This would be a fair bit of work, because it would require building out a DSL for describing the update clauses
Also, as things currently stand, this would be virtually impossible for the SQL and Cassandra back ends. So as things stand, this would be a Mongo only feature.
I have nothing against a Mongo only feature, but it does make it less useful, and hence lower priority for me
The reason why it's practically impossible in SQL and Cassandra is that as things stand, the persistent is stored as JSON on these back ends, in a text field. RDB and Cassandra have no native support for processing JSON
So there is no way that I can reach into the JSON stored in a text column and tweak an individual field or handful of fields, while remaining on-database. (Maybe we could say in-place instead of on-database)
But there is a way it could work
Instead of storing the persistent as JSON, I could instead break it down into all its individual fields, and store each one in a column
This is more than a little tricky, because we want to allow nested collections, such as lists of lists
This is definitely doable in Cassandra, we would have to use frozen collections and things like that
In SQL, it would require breaking up a persistent across multiple tables in a sort of relational style
Both of these would be doable, but would represent a lot of work
Now, users and potential users might prefer one way, or prefer another. There are pros and cons to each
I'll skip the details of the pros and cons to try to keep this diatribe somewhat contained
It's not clear to me which way users would prefer, or how I would get feedback on the matter.
Getting feedback from users seems to be pretty hard in general :). With Mardo being a glowing exception :)
The two Cassandra back ends might be able to share some code, but my test suite would basically grow in size by 50%
Which is no big deal just on its own
It would definitely be easier to stick to supporting just four back ends (I'm including in-memory here)
So let me try to sum up:
- we could do a Mongo only API for in-place updates. this would represent some work and would be less than desirable for not supporting all back ends
- we could modify SQL and Cassandra back ends to flatten their representation of the persistent out of JSON, and into individual columns. This would represent the same work as the previous bullet point, and then a good deal more on top of that
- we could support existing SQL and Cassandra back ends as well as the new, flattened formats. This would represent a bit more work now, and a continual stream of extra maintenance overhead down the line
We could certainly pursue the first option first, and continue on with the second or third approach at our leisure
But a key idea in all this is "a lot of work"
Okay, I'm done. I guess I got it out all at once! :) @mardo /all Let me know what you think of all that. And we can mull it all over
@sullivan-
WOW!… this was a lot hahaha. Yes, I agree on the amount of work. What I am currently thinking to do for these kind of Mongo-specific things is in fact open my own connection to the database and do custom queries. It is just that I love to keep my whole code type-safe, and this will be a part where it won’t. Not big deal as it is just for some views, not the whole app. Another option I was considering is doing a layer on top of longevity for mongo, but I’m not sure what would be the best way to convert a mongo record to a
WOW!… this was a lot hahaha. Yes, I agree on the amount of work. What I am currently thinking to do for these kind of Mongo-specific things is in fact open my own connection to the database and do custom queries. It is just that I love to keep my whole code type-safe, and this will be a part where it won’t. Not big deal as it is just for some views, not the whole app. Another option I was considering is doing a layer on top of longevity for mongo, but I’m not sure what would be the best way to convert a mongo record to a
case class instance. I am sure there’s a utility in longevity to do this conversion, but what I am not sure is if that’s exposed or not, or if I should be using it or not :)
There is already a
longevity-mongodb artifact. right now it just brings in the mongo dependencies. but i am not averse to putting real code in there. so we could possibly migrate anything you build into there eventually
Ok ok… let me think more about it, and see if I find the time to build such layer
But knowing I can get access to those utilities in Longevity will help a lot
Because I won’t need to manage the connection
I can just reuse Longevity’s one
I do think I should write up some GitHub issues describing all that I laid out here. I'll get to that soon
Thanks for the info :)
Hi @sullivan- ! I was wondering if there’s a way to match a string field with a regex? Right now I’m retrieving all records and doing the filtering in scala code, but this will become a problem when that collection grows. Let me know if there’s any workaround, otherwise I’ll write a custom Mongo query :)