These are chat archives for orbitjs/orbit.js

11th
Feb 2016
Oliver Searle-Barnes
@opsb
Feb 11 2016 16:19

@dgeb if we assume queries are always executed against the in-memory source (which they need to be if they're to return the combined existing and retrieved results) then I think we're looking at two different apis.

1) A query interface - the existing api on the store, returns filtered results from the memory source
2) A request for data - e.g. fetching data from a remote source

I believe (1) already provides the correct interface, we simply need to acknowledge that it only exists on the Store api. Regarding (2) it seems that you could still use a query to request the information but the results would be returned as transforms(rather than being transmitted along a permanent connector). This is different to the existing connector based setup, where the connectors are permanent. With a query based api they would only exist for the lifetime of the query/live-query.

I've tried to illustrate this a little more concretely in code at https://gist.github.com/opsb/556a3968d8f98b92d5aa.

Dan Gebhardt
@dgeb
Feb 11 2016 16:26
@opsb thanks for putting this together
I suppose that this leaves out querying of the store's cache
i.e. it only covers queries that involve remote sources
Oliver Searle-Barnes
@opsb
Feb 11 2016 16:27
I think the existing Apis cover that ok
Dan Gebhardt
@dgeb
Feb 11 2016 16:27
yes, I agree
Oliver Searle-Barnes
@opsb
Feb 11 2016 16:29
I gave a little thought to graphql as there's a lot of overlap I.e. it expresses queries and mutations in a consistent format
Dan Gebhardt
@dgeb
Feb 11 2016 16:31
ok, so let's talk about returning an array of operations from query
Oliver Searle-Barnes
@opsb
Feb 11 2016 16:31
I thought perhaps a graph based format would make sense to express the updates from queries, but realised that Transforms cover this fine, so long as we don't want to express the query structure. Because we're always running queréis afinar the store though that information is irrelevant.
Ok
Oops, Spanish auto complete was kicking in there ;)
Dan Gebhardt
@dgeb
Feb 11 2016 16:32
:)
so I think that this approach is well tailored to only querying the store
it is lossy in terms of querying a source directly though
Oliver Searle-Barnes
@opsb
Feb 11 2016 16:32
Yeah, it is based in that assumption
Dan Gebhardt
@dgeb
Feb 11 2016 16:33
we can't query a source and get back a sorted resultarray for instance
Oliver Searle-Barnes
@opsb
Feb 11 2016 16:33
Yes, I felt that was ok though because we'd always the results to include the existing information with the remote information
Dan Gebhardt
@dgeb
Feb 11 2016 16:33
yeah, just want to recognize the tradeoff
Oliver Searle-Barnes
@opsb
Feb 11 2016 16:33
Yeah, to do that we would need a result structure closer to the one graphql uses
Yeah, ok
There is a question of efficiency I suppose, for larger queries streaming the results directly would make more sense.
Dan Gebhardt
@dgeb
Feb 11 2016 16:36
right - this approach would mean that you would need to use a store for any queries
so, I'm wondering if the coordinator could do the interpretation of the query results into transforms?
so that sources could still emit resultsets and resultarrays
Oliver Searle-Barnes
@opsb
Feb 11 2016 16:37
Rigg
Hmm
What about compound documents?
That's why I think we'd need a graph structure, or at least multi resource result structurw
Dan Gebhardt
@dgeb
Feb 11 2016 16:39
let's see .... perhaps the result structure could be composed of two parts:
1) a json document including all the data in a normalized form
2) pointers / arrays / sets referencing that data
so the coordinator would simply be merging (1) with the store's cache
and (2) could be used by anyone wanting to query a remote source directly
so (1) makes up all the data that is interpreted into transforms
Oliver Searle-Barnes
@opsb
Feb 11 2016 16:42
Yeah, that is the two sets of information we need. GraphQL does this in a single structure, but if we're focusing on the orbit-common layer then a normalised structure does make sense.
They could be separate though
Dan Gebhardt
@dgeb
Feb 11 2016 16:43
GraphQL can get away with a single structure because it's not trying to map results onto a schema
Oliver Searle-Barnes
@opsb
Feb 11 2016 16:44
I was considering that a query could include the transform id
Dan Gebhardt
@dgeb
Feb 11 2016 16:44
interesting
Oliver Searle-Barnes
@opsb
Feb 11 2016 16:44
That way the store knows when the information is ready for a query to return
Dan Gebhardt
@dgeb
Feb 11 2016 16:45
I see
Oliver Searle-Barnes
@opsb
Feb 11 2016 16:45
It's a looser coupling, which may be useful
Dan Gebhardt
@dgeb
Feb 11 2016 16:45
yeah
so I'm not sure the best way to structure such a query result - but at least we agree on the contents
we could certainly omit (2) to start, but then it might be difficult to retrofit
Oliver Searle-Barnes
@opsb
Feb 11 2016 16:47
The next question is, can we find a symmetrical structure for mutations?
Something else to consider, how would the UI bind to that result structure?
Currently relationships are simply traversed on the Cache. So only the root query results would be used.
It would require a source per query result, and the identity map doesn't make sense.
Dan Gebhardt
@dgeb
Feb 11 2016 16:51
hmmm .... perhaps it would be simpler if a query just returned transforms + results (sorted array, recordset, etc)
Oliver Searle-Barnes
@opsb
Feb 11 2016 16:52
how would you bind to filtered associations?
Or would you only honour the filtering at the root of the results?
Dan Gebhardt
@dgeb
Feb 11 2016 16:53
I'm talking about queries on sources, not stores, to be clear
Oliver Searle-Barnes
@opsb
Feb 11 2016 16:54
Ok, I suppose I'm still trying to work out if we should include the query result information from the sources.
I think orbit's approach is really focused on querying at the store.
Dan Gebhardt
@dgeb
Feb 11 2016 16:55
I see - you're asking if it's pragmatic to return those results at all
vs. transforms alone
Oliver Searle-Barnes
@opsb
Feb 11 2016 16:55
exactly
So the API that I was suggesting only focused on the Transforms because without a rethink at the store/ember-orbit level it doesn't make sense to include from the sources.
Dan Gebhardt
@dgeb
Feb 11 2016 16:56
yeah, I understand what you're saying
Oliver Searle-Barnes
@opsb
Feb 11 2016 16:57
Ok, got to run, keep me posted.
Dan Gebhardt
@dgeb
Feb 11 2016 16:57
ok, still thinking :)
ttys