Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
Martynas Jusevicius
@namedgraph_twitter
can't you package the whole thing into Docker and then run whatever, like Jena/Fuseki?
Tomasz Pluskiewicz
@tpluscode
oh boy that would be such an overkill for tests running jest/jasmine/etc
Martynas Jusevicius
@namedgraph_twitter
might help you in the long run more than you know ;)
Tomasz Pluskiewicz
@tpluscode
I don't know ;)
what you say, I do with end-to-end tests running over actual API
Martynas Jusevicius
@namedgraph_twitter
check the Beautiful Interactions stuff. i haven't looked deeply myself
Tomasz Pluskiewicz
@tpluscode
will do thanks
Martynas Jusevicius
@namedgraph_twitter
i'm also running an HTTP test suite but it's just shell scripts and Docker ;)
Chris Wilkinson
@thewilkybarkid
Hi all :wave: We're starting to build Hydra-based APIs so are adopting linked data. We've started with plain JSON-LD, but are looking at making use of RDF/JS in libero/article-store#64. Would be interested in any feedback, comments or suggestions. Would also be interested to find out about best practices on mixing static (eg Hydra collection) and dynamic (eg items in the collection) statements, as well as access control (eg not exposing parts of the Hydra API doc when you don’t have permission for an operation, or only exposing some statements that make up an item (eg article stored in a database) if you don't have permission to see all of it). Also, advice on persistence would be great, especially in using Postgres rather than a triple store (more details on libero/publisher#292).
(There's background on our project at https://httpapis.slack.com/archives/C1JP575EX/p1572362193080200 in case it's useful.)
Tomasz Pluskiewicz
@tpluscode
not everything strictly related to rdfjs but I can give some of my advice:
Tomasz Pluskiewicz
@tpluscode
  1. avoid URI manipulation at all cost.

for example if you have a /book/foo resource and its chapters, do not mint a /book/foo/chapters URI for a hydra:Collection. this practice will spread like wildfire across resources which are not strictly your domain and also pollute the domain identifiers too.

instead simply create the collection identifier once and persist it somewhere along your data, with a link (reference) to the book it belongs to.

I store RDF so I have </book/foo/chapters> ex:book </book/foo> and use that relation in the queries, thus avoiding the need to explicitly name any resources

an additional benefit is that if you ever change the naming scheme, nothing should break. for example when you migrate from hierarchical URIs like <book/{id}/chapter/{chapter}> to a flat structure </book/{id}> and </chapter/{id}>. the old identifiers should continue to work while the new data has a different naming space

Chris Wilkinson
@thewilkybarkid
Makes sense; so you store a minimal representation of some resources and populate the rest at runtime? (Literally just the ID/link?)
Tomasz Pluskiewicz
@tpluscode
stuff which is dynamic anyway. for a collection I guess you could also store its @type and some constant metadata
Chris Wilkinson
@thewilkybarkid
:+1:
Chris Wilkinson
@thewilkybarkid
Got a couple of RDF/JS-specific questions.
Is the Stream interface not meant to extend NodeJS.ReadableStream on object mode (in @types/rdfjs)? (Refs https://github.com/libero/article-store/pull/64#discussion_r352029306)
And are there details around about when to use Datasets and when to use Streams? The former has a 1.0 version, but with an experimental message inside, and there doesn't seem to be much tooling.
Ruben Taelman
@rubensworks
The Node.JS readable stream interface has more requirements than the RDFJS Stream interface. The former can be used as an implementation of the latter though.
IMO, Datasets should be used for in-memory manipulation of (relatively small) datasets where performance is typically not that much of an issue. Once you are working with larger datasets that do not necessarily fit into memory anymore, RDFJS Sources should be used together with the RDFJS Stream interface.
AFAIK, graphy.js, rdfjs-dataset and rdfjs-dataset-index implement the Dataset interface.
Chris Wilkinson
@thewilkybarkid
Is the Dataset interface still experimental?
Ruben Taelman
@rubensworks
The core should be stable. Dataset itself was marked as experimental as additional methods and fields may be added in the future. It's highly unlikely that any changes/removals will occur.
Chris Wilkinson
@thewilkybarkid
Ok, going to try and use it as we won't be dealing with huge sets and seems simpler. Adding types in DefinitelyTyped/DefinitelyTyped#40794 (found a couple of issues, will comment there to start with).
Chris Wilkinson
@thewilkybarkid
Tried out the various dataset implementations today, unfortunately they all seem incomplete/use (what must be) earlier versions of the spec.
Tomasz Pluskiewicz
@tpluscode
how so? the -indexed package is the latest and greatest and the one to use currently
Chris Wilkinson
@thewilkybarkid
Had problems with some functions not be implemented, and some not adhering to the filter interface.
Did see there’s an unmerged PR to add some of the missing functions.
Tomasz Pluskiewicz
@tpluscode
ok, so I haven't been using the dataset directly. you may give rdf-ext a go
also pinging @bergos
victor felder
@vhf
@thewilkybarkid most dataset implementations you'll find implement 1./1.1/1.2 from the spec. The other parts of https://rdf.js.org/dataset-spec/ aren't part of the spec yet, they are still experimental :)
Chris Wilkinson
@thewilkybarkid
Thanks for the replies. Going to halt my work and have a rethink.
Adrian Gschwend
@ktk
@blake-regalia I get 404 on all the "Features" link from the GitHub readme
Blake Regalia
@blake-regalia
Thanks for the notice Adrian! I'll get that fixed. Here's the main page with those links working: https://graphy.link/#features
Adrian Gschwend
@ktk
yep saw this one thanks
Tomasz Pluskiewicz
@tpluscode
@thewilkybarkid I noticed you added Dataset to rdf-js typings. awesome!
however, I think that the callback signatures are not correct. this doesn't pass the types:
let dataset: Dataset;

dataset.filter(() => true);
anyone willing to comment on that? :)
Tomasz Pluskiewicz
@tpluscode
that's how I think it should be: DefinitelyTyped/DefinitelyTyped#40989
Chris Wilkinson
@thewilkybarkid
While I thought it a little unusual, that's how I read the spec... (Confused by what QuadFilterIteratee etc are otherwise.)
Tomasz Pluskiewicz
@tpluscode
sure, it looks weird indeed. I'm really glad you spearheaded the dataset typings. it was that last missing piece for me to type some higher level stuff
Chris Wilkinson
@thewilkybarkid

Thanks for the replies. Going to halt my work and have a rethink.

DefinitelyTyped/DefinitelyTyped#41016 inspired me to have another crack at using DatasetCore (was aware of the library, but not how to use it), and it seems to solve the problems I was having. Will try and get a draft PR up today, and would appreciate reviews if possible as there's a lot of new concepts/tools that I'm trying to use.

Tomasz Pluskiewicz
@tpluscode
clownface is really useful but takes getting accustomed to. I'll be happy to help with it
Chris Wilkinson
@thewilkybarkid
Took a bit longer than hoped, so haven't had a chance to annotate it. Will try and add something tonight/over the weekend, but the PR is up at libero/article-store#115
Chris Wilkinson
@thewilkybarkid
Added a bit of detail and some comments.
victor felder
@vhf
@thewilkybarkid https://github.com/libero/article-store/blob/e1a9ccefb8e97dd1b190838285be3f015ce536da/package.json#L7 @rdfjs/dataset shouldn't be used, it's only a basic, readable example implementation of the dataset spec. You could replace it with N3 or rdf-dataset-indexed, their perfs are exponentially better
Chris Wilkinson
@thewilkybarkid
:+1: Haven't look at the details of them yet, but will definitely do so.
Thomas Bergwinkl
@bergos
medium-term @rdfjs/dataset will be replaced by a refactored version of the n3 indexed store.
@thewilkybarkid can we shift all the typescript stuff to the https://github.com/DefinitelyTyped/DefinitelyTyped/ repo? i'm not so happy to review/merge PRs i don't understand.