Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
This would be a fair bit of work, because it would require building out a DSL for describing the update clauses
Also, as things currently stand, this would be virtually impossible for the SQL and Cassandra back ends. So as things stand, this would be a Mongo only feature.
I have nothing against a Mongo only feature, but it does make it less useful, and hence lower priority for me
The reason why it's practically impossible in SQL and Cassandra is that as things stand, the persistent is stored as JSON on these back ends, in a text field. RDB and Cassandra have no native support for processing JSON
So there is no way that I can reach into the JSON stored in a text column and tweak an individual field or handful of fields, while remaining on-database. (Maybe we could say in-place instead of on-database)
John Sullivan
@sullivan-
It's possible that using procedural extensions to SQL such as Oracle's PL/SQL, could help. But this is vendor-independent stuff, and a lot of SQL back ends (such as SQLite) will probably never support anything like this
But there is a way it could work
Instead of storing the persistent as JSON, I could instead break it down into all its individual fields, and store each one in a column
This is more than a little tricky, because we want to allow nested collections, such as lists of lists
This is definitely doable in Cassandra, we would have to use frozen collections and things like that
In SQL, it would require breaking up a persistent across multiple tables in a sort of relational style
Both of these would be doable, but would represent a lot of work
Now, users and potential users might prefer one way, or prefer another. There are pros and cons to each
I'll skip the details of the pros and cons to try to keep this diatribe somewhat contained
It's not clear to me which way users would prefer, or how I would get feedback on the matter.
Getting feedback from users seems to be pretty hard in general :). With Mardo being a glowing exception :)
John Sullivan
@sullivan-
We could support both approaches, but that would essentially meaning adding two new back ends
The two Cassandra back ends might be able to share some code, but my test suite would basically grow in size by 50%
Which is no big deal just on its own
It would definitely be easier to stick to supporting just four back ends (I'm including in-memory here)
So let me try to sum up:
  • we could do a Mongo only API for in-place updates. this would represent some work and would be less than desirable for not supporting all back ends
  • we could modify SQL and Cassandra back ends to flatten their representation of the persistent out of JSON, and into individual columns. This would represent the same work as the previous bullet point, and then a good deal more on top of that
  • we could support existing SQL and Cassandra back ends as well as the new, flattened formats. This would represent a bit more work now, and a continual stream of extra maintenance overhead down the line
We could certainly pursue the first option first, and continue on with the second or third approach at our leisure
But a key idea in all this is "a lot of work"
John Sullivan
@sullivan-
I probably should move these ideas into GitHub issues
Okay, I'm done. I guess I got it out all at once! :) @mardo /all Let me know what you think of all that. And we can mull it all over
Mardo Del Cid
@mardo
@sullivan-
WOW!… this was a lot hahaha. Yes, I agree on the amount of work. What I am currently thinking to do for these kind of Mongo-specific things is in fact open my own connection to the database and do custom queries. It is just that I love to keep my whole code type-safe, and this will be a part where it won’t. Not big deal as it is just for some views, not the whole app. Another option I was considering is doing a layer on top of longevity for mongo, but I’m not sure what would be the best way to convert a mongo record to a case class instance. I am sure there’s a utility in longevity to do this conversion, but what I am not sure is if that’s exposed or not, or if I should be using it or not :)
John Sullivan
@sullivan-
There certainly are such marshallers in longevity. They are package-private, but you could work around that by putting your own code in package longevity.persistence.mongo or some such
There is already a longevity-mongodb artifact. right now it just brings in the mongo dependencies. but i am not averse to putting real code in there. so we could possibly migrate anything you build into there eventually
Mardo Del Cid
@mardo
ahhh… I get it
Ok ok… let me think more about it, and see if I find the time to build such layer
But knowing I can get access to those utilities in Longevity will help a lot
John Sullivan
@sullivan-
:)
Mardo Del Cid
@mardo
now that I think of it maybe is even better
Because I won’t need to manage the connection
I can just reuse Longevity’s one
John Sullivan
@sullivan-
longevity.persistence.mongo.BsonToDomainModelTranslator and its sibling
Mardo Del Cid
@mardo
cool
John Sullivan
@sullivan-
yeah you can cast from Repo to MongoRepo to get the connection
I do think I should write up some GitHub issues describing all that I laid out here. I'll get to that soon
Mardo Del Cid
@mardo
Sounds good!
Thanks for the info :)
John Sullivan
@sullivan-
@mardo are you agreeable to putting the longevity idea plugin under Apache 2 license?
Mardo Del Cid
@mardo
Hi @sullivan-, yes apache 2 sounds good!
Mardo Del Cid
@mardo
Hi @sullivan- ! I was wondering if there’s a way to match a string field with a regex? Right now I’m retrieving all records and doing the filtering in scala code, but this will become a problem when that collection grows. Let me know if there’s any workaround, otherwise I’ll write a custom Mongo query :)
John Sullivan
@sullivan-
sorry @mardo , no regex matching yet. I'd love to see your custom mongo code some time - are you building out on top of package longevity.persistence.mongo like I suggested?
Mardo Del Cid
@mardo
Right now I haven’t done any custom queries, but I will definitely need them at some point. And yeah, I’d use longevity.persistence.mongo