make init-backend should only be run once (or after backend needs reset) so including it into make build would have made it more complex than necessary
make build or make watch should be enough
- on the first run it uses Plone REST API search endpoint to go through every item (sorted in path order to optimize navigation updates later)
- GatsbyJS support caching, so the plugin store the timestamp of the first run and URL of every found page
- on the subsequent runs it still needs to use Plone REST API search endpoint to get all existing paths to be able to: 1) remove pages gone missing, 2) update page with modification date after the last fetch timestamp 3) when folder has been updated, fetch also navigation and breadcrumbs of every children of that folder 4) update all collections, because it is impossible guess if they have been changed or not
@iFlameing About navigation and breadcrumbs. In Plone REST API they are called "expansions". They can be called separately or they can be fetched with the main page results. So, they exists as "expansions" to allow optimization of queries.
When ever a page contents are fetched, I recall, the plugin fetches also navigation and breadcrumb extensions (to allow extra requests). But on subsequent runs, when a folder has been modified, we also need to update breadcrumbs and navigation of its children (because the title of the page may have been changed and might affect those). In this case, we only call navigation and breadcrumbs endpoints directly. (Calling those two separately should be faster than calling the main page with the expansions. That would be only one request, but it could be slower than those two smaller ones combined.)
sourceNodes) to make them support similar lazy updates from websocket as gatsby-source-filesystem supports updates from filesystem events.
@iFlameing And for the websocket, we can define the protocol we need.
Maybe a websocket message could be JSON encoded payload with something like {"event": "created", "id": "http://url/of/the/page", "uuid": "uuid of the page"}
@iFlameing For a development websocket server a minimal chat server could work https://socket.io/get-started/chat/
Just create that chat example separately from gatsby-source-plone
Next make gatsby-source-plone connect that server with websocket (believing that it is Plone)
Finally use the chat client example to publish messages from browser to gatsby-source-plone through that websocket server.
@iFlameing Something else you probably need to learn for this is using node debugger https://nodejs.org/en/docs/guides/debugging-getting-started/
For example, you should be able to run gatsby develop with debugger with node --inspect node_modules/.bin/gatsby develop. With debugging mode you should be able to add "debugger" break points into the code and e.g. have interactive repl to figure things out when websocket connection gets a message.
@iFlameing Please, try to transform the main tasks from your GSOC project plain into tasks or real issues into https://github.com/collective/gatsby-source-plone/projects/2
I'm working on Docker image with Plone and websockets on next Tuesday, but it might still take another week for me to complete it. So you should probably start as you planned.
Let's try to find a time when I would be available here every week so that we could prevent blocking issues. I am on UTC+3 so I assume that we could have that time around you Monday noon or aftertnoon. Something between 6am UTC and 8am UTC should fit for me.
Yet, the best way to ask questions is either using GitHub issues or here. Remember to tag me so that I get notifications.