Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
Aditya Mukhopadhyay
@adityamukho
You don't need to remove the collection-specific origin events. They can be left intact. In fact, if there are any events belonging to a collection that you want to leave untouched, these origins must remain intact. Same goes for snapshot origins. Like collection-specific event origins, they are safe to remove ONLY when ALL documents for that collection are being purged.
Best to leave all origin events and origin snapshots untouched in all cases. They are not document-specific anyway.
Gary Gendel
@ggendel
From what I saw, they are event specific. I only delete them when the relevant event/command is removed. I originally left them, but then they were not reused.
Aditya Mukhopadhyay
@adityamukho
Ok here is how the event tree is structured:
  1. At the top level there is a single root event from which everything is reachable as a descendant
This is called the super origin
  1. Under this, the 2nd level consists of per-collection origins. All events for all documents of a collection are reachable from the origin event for that collection. These are the ones that i said should be left intact.
From the 3rd level onwards is where document-specific events are recorded. This means the creates, updates and deletes. These are the ones that should be deleted during a purge
Gary Gendel
@ggendel
Thanks. I'll validate that I don't remove collection origins.
Gary Gendel
@ggendel
I'm pretty happy with the result. It removes the collection origin only when the last event for that collection is removed. I have some tests to write and a couple of ArangoDB integration items to work through before I release to the Applications Engineers and Marketing to check it out.
Aditya Mukhopadhyay
@adityamukho
Oh in that case, it is safe to remove. Excited to see the pace at which things are moving ahead at your end!
Aditya Mukhopadhyay
@adityamukho

@/all The new documentation website is live!

Check it out at https://docs.recallgraph.tech/ and do leave your feedback!

Gary Gendel
@ggendel
Great start. How can we help garner more attention to your project?
Aditya Mukhopadhyay
@adityamukho
@ggendel Thank you! I'd be very delighted to see more users onboarded. I'll start on the main website soon enough. When that is ready, there could be a section for testimonials, case studies, etc. In the meantime, any channel through which the word can be spread would be greatly beneficial, including LinkedIn, Twitter and not to underestimate good old word of mouth.
I'm also reaching out to potential individual, academic and enterprise users, particularly those who are using or are likely to use graph databases in their research/analytics/business operations. But this effort could use some up-scaling, and without a doubt the word of a happy enterprise user carries a lot of weight - far more than a developer marketing his own creations ;)
Also, more sections will be added to the doc website over time - guided tutorials with examples, contribution guides, etc.
Always open to feedback
Aditya Mukhopadhyay
@adityamukho

A brand new guide section has been added to the docs, along with minor improvements to other sections:

https://docs.recallgraph.tech/working-with-recallgraph/guide

whyDoesThisWork
@whyDoesThisWork
Hi @adityamukho ! Sorry about the delay on joining glitter
Aditya Mukhopadhyay
@adityamukho
No sorries required absoultely! Welcome to the RecallGraph community @whyDoesThisWork !
Aditya Mukhopadhyay
@adityamukho

PSA: The project homepage on GitHub now has a shiny new Sponsor button.
Enabled transfer methods:

  1. PayPal (active)
  2. GitHub Sponsors (coming soon)

https://github.com/RecallGraph/RecallGraph

Gary Gendel
@ggendel
Good to know. I'm sure I can convince management to contribute once we have officially deployed RecallGraph.
Aditya Mukhopadhyay
@adityamukho
Thank you!
bb-kodexa
@bb-kodexa
Great work Aditya! RecallGraph is a real testament to how Arangodb can be extended and built upon. We are working on our first Arangodb project and near the top of our list is support for versioning, so we are really looking forward to RecallGraph reaching v1.0.
Aditya Mukhopadhyay
@adityamukho
Thank you @bb-kodexa ! I've tried my best to keep it generic and relevant. You may be interested to know that the API for v1.0 is >95% stable, and the code in the dev-raw branch can already be used to start basing your project upon.
I've finished all features on the 1.0 roadmap, what remains is to write all the test cases and ensure it is thoroughly tested and adequately performant.
Aditya Mukhopadhyay
@adityamukho

RecallGraph v1.0 has been released.
https://github.com/RecallGraph/RecallGraph/releases/tag/v1.0.0

Supported ArangoDB versions: 3.5, 3.6

Highlights

  1. Providers for all API endpoints to let other dependent Foxx services invoke RecallGraph's service methods directly, using ArangoDB's service linking mechanism.
  2. Explicit Commits to sync event log with writes that occurred outside of RecallGraph's API methods.
  3. Purge endpoint to remove all history for nodes at a specified path.
  4. Restore endpoint to undelete nodes that were deleted through RecallGraph's API.
  5. Paths are returned in traverse calls, with support for path filters.
  6. k Shortest Paths - Custom-weighted, point-in-time, shortest paths between endpoints.

Changelog: https://docs.recallgraph.tech/working-with-recallgraph/changelog#1-0-0

Gary Gendel
@ggendel
Super news. Congrats.
Aditya Mukhopadhyay
@adityamukho

Here's the HackerNews entry: https://news.ycombinator.com/item?id=23455516

Folks, please show your love with upvotes, shares, and also star the GitHub repository if you haven't already done so. If this project crosses 100 stars, I can then apply to sponsorship programs at OpenCollective and CodeFund. TIA!

Gary Gendel
@ggendel
I'm trying to use the traverse exported function directly, but I must have something wrong. I get the following error:
ValidationError: child \"edges\" fails because [child \"uniqueVertices\" fails because [\"uniqueVertices\" must be one of [inbound, outbound, any]]]
1 reply
I've set uniqueVertices to "path" in the passed object for the edges parameter. That seems to be the right place to pass it.
I'm also not getting the right information back using the web interface. It seems to be ignoring the vFilter but I'd like to get the traverse function call working first.
2 replies
Aditya Mukhopadhyay
@adityamukho
Ok, I think I might have mixed up the validation messages and/or the validation code between input fields. A minor bug either way, though I'm not yet sure why the tests didn't catch it.
I have a presentation on RG to make on the 19th evening, for which I'm a little tied down preparing the material. I hope it is ok if I look into this issue on the 20th.
Gary Gendel
@ggendel
There is no urgency. I've still got a lot of other things on my plate. Good luck with your presentation.
Gary Gendel
@ggendel
Something else for you to investigate when you have some time. A performance degradation seems to be related to the size of the RG tables. My test was to take a graph with a couple million each of objects and edges and "commit" them so there would only be a single create command for each of these. Then I created a small independent graph (10s of vertex and edges) via RG and finally request a traversal of that. It took many seconds to return the request. This is not high priority for me, but it means that I can't use RG calls to do current-time traversals but do it directly on my tables.
Aditya Mukhopadhyay
@adityamukho

Something else for you to investigate when you have some time. A performance degradation seems to be related to the size of the RG tables. My test was to take a graph with a couple million each of objects and edges and "commit" them so there would only be a single create command for each of these. Then I created a small independent graph (10s of vertex and edges) via RG and finally request a traversal of that. It took many seconds to return the request. This is not high priority for me, but it means that I can't use RG calls to do current-time traversals but do it directly on my tables.

Ok I will take a look at this. Can you help me with the input parameters you used for the query?

Gary Gendel
@ggendel
I got pulled in for some top-priority tasks and somehow lost my notes. I'll recreate the problem when I get a little time. I'll take good notes so you don't have to spin your wheels.
Aditya Mukhopadhyay
@adityamukho
That sounds great! Thank you!
Gary Gendel
@ggendel
@adityamukho We're going to have to shelve this until I can recreate it. All my recent tests show no appreciable performance issue. Let's chalk it up to work on three conflicting tasks and messing something up. Once I get the patch for the traverse validation problem, I can do more rigorous testing. I've gotten RG on the official roadmap which means that others will join me in final evaluation and testing.
Aditya Mukhopadhyay
@adityamukho
@ggendel I have run the complete test suite on traverse providers again, and also inspected the validator schema by hand. Nothing seems to be out of place, especially nothing that would give rise to the nested validation error you got. This leads me to believe that this could be an issue with the input provided to the method - perhaps some inadvertent nesting has occurred? I would be able to better diagnose this once I get to see the input provided to the method that gave rise to your particular validation error.
Aditya Mukhopadhyay
@adityamukho
Here's an example input that would yield correct results:
// Method signature:
// function traverseProvider (timestamp, svid, minDepth, maxDepth, edges, options = {}) { ... }

traverseProvider(1581583228.2800217, "vertex_collection/starting_vertex_key", 0, 2, { "edge_collection_1": "inbound", "edge_collection_2": "outbound", "edge_collection_3": "any" }, { "uniqueVertices": "path", "uniqueEdges": "path", "bfs": true, "vFilter": "x == 2 && y < 1", "eFilter": "x == 2 && y < 1", "pFilter": "edges[0].x > 2 && vertices[1].y < y" })
In this example I've been explicit about all options, but as with the API endpoint, it will fill in the usual defaults for fields which are left unspecified.
1 reply
Aditya Mukhopadhyay
@adityamukho

I think in your case you had embedded the uniqueVertices param inside edges, resulting in the nested validation error.

I realize that there is a lot of ground to be covered to make the documentation catch up with all the new exports and I really appreciate your patience in sticking with the product so far. I have it on priority to improve documentation and test coverage as the very next items on my plate, to ensure minimal hiccups in the future.

Gary Gendel
@ggendel
@adityamukho That was a help. Indeed I was sending some attributes in the wrong parameter. However, I have one more puzzle. I want to traverse and return all objects that have their 'type' value set to 'config' (the intermediate nodes. I added "vFilter": "type == 'config'" but it returns nothing. If I remove the vFilter, then the intermediate and leaf nodes are returned (as expected). Am I constructing the filter correctly. In AQL I would filter on v.type=='config'.
Aditya Mukhopadhyay
@adityamukho
Just type=='config'. The filter works by setting the current object under iteration as the this object.
Also please note there are now 2 depth params - minDepth and maxDepth. The vFilter only applies to vertices starting from minDepth.
Aditya Mukhopadhyay
@adityamukho
If it still doesn't work after fixing any depth related errors, i might need to run the query at my end. I think my existing sample database would work. Just need all the query params
Gary Gendel
@ggendel
Yes, the starting vertex is a type=='config' and minDepth is 1. Interestingly, if I set minDepth to 0, then the root node comes out but none of the child nodes which are also type=='config' do.
The information I send to traverse is:
2020-06-24T14:32:42.038Z: RealGraph traverse, {\"svid\":\"pmconfig/453634713\",\"timestamp\":\"1593006183.013\",\"minDepth\":1,\"maxDepth\":50,\"edges\":{\"pm_content\":\"inbound\"},\"options\":{\"uniqueVertices\":\"path\",\"uniqueEdges\":\"path\",\"vFilter\":\"type == 'config'\"}}"