Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Oct 18 16:49
    SolangeUG starred BlueBrain/nexus
  • Oct 18 12:45
    stevenwally commented #757
  • Oct 18 09:21
    pgetta opened #274
  • Oct 18 07:32
    Stafeeva closed #757
  • Oct 18 07:32
    Stafeeva commented #757
  • Oct 18 07:29
    julienmachon commented #757
  • Oct 18 01:22
    stevenwally commented #757
  • Oct 17 15:13
    dhaneshnm synchronize #272
  • Oct 17 15:13

    dhaneshnm on bulk-down

    More formatting fixes (compare)

  • Oct 17 14:26

    kenjinp on 753-fix-paginated-resource-types

    (compare)

  • Oct 17 14:26

    kenjinp on master

    change _result to _results (#27… (compare)

  • Oct 17 14:25
    kenjinp closed #273
  • Oct 17 14:25
    kenjinp closed #753
  • Oct 17 14:11
    kenjinp review_requested #273
  • Oct 17 14:11
    kenjinp review_requested #273
  • Oct 17 14:11
    kenjinp review_requested #273
  • Oct 17 14:11
    kenjinp opened #273
  • Oct 17 14:10
    kenjinp labeled #758
  • Oct 17 14:10
    kenjinp labeled #758
  • Oct 17 14:10
    kenjinp labeled #758
marmaduke woodman
@maedoc
@wwajerowicz thanks for the reply. have there been any audits on ACLs etc with regards to security? do you consider nexus suitable for sensitive data or for non-sensitive/anonymous data only for now?
marmaduke woodman
@maedoc
following the docs, curl localhost/kg works, but following the sandbox URL (https://sandbox.bluebrainnexus.io/web) I tried localhost/web but it doesn't work, and I don't see any URLs in the docs. What should I be looking for?
Wojtek Wajerowicz
@wwajerowicz
just localhost should take you to Nexus Web
regarding security, we haven’t had any security audits on Nexus so far
marmaduke woodman
@maedoc
I've got a 502 there as well, I'll take a look at the logs
nexus router logs show 2019/04/05 13:22:27 [error] 149#149: *2 connect() failed (111: Connection refused) while connecting to upstream, client: 172.18.0.1, server: localhost, request: "GET /web HTTP/1.1", upstream: "http://140.180.220.47:8000/web", host: "localhost"
marmaduke woodman
@maedoc
looking at the kg logs, I see
2019-04-05 13:37:59 INFO  [%PARSER_ERROR[traceToken]] ch.epfl.bluebrain.nexus.kg.Main$ - ==== Cluster is Live ====
2019-04-05 13:38:01 INFO  [%PARSER_ERROR[traceToken]] ch.epfl.bluebrain.nexus.kg.Main$ - Bound to 0.0.0.0: 8080
2019-04-05 13:38:01 INFO  c.e.b.n.s.i.stream.StreamCoordinator - Received initial start value of type 'Offset', with value 'NoOffset' running the indexing function across the element stream
2019-04-05 13:38:01 INFO  c.e.b.n.s.i.stream.StreamCoordinator - Received initial start value of type 'Offset', with value 'NoOffset' running the indexing function across the element stream
2019-04-05 13:38:02 WARN  akka.actor.ActorSystemImpl - Illegal request, responding with status '400 Bad Request': Illegal HTTP message start
2019-04-05 13:38:34 WARN  a.h.impl.engine.client.PoolGateway - [0 (WaitingForEndOfResponseEntity)] Ongoing request [GET /v1/events Empty] was dropped because pool is shutting down
Wojtek Wajerowicz
@wwajerowicz
it seems that there’s some problem with nexus-web docker image, we’re looking into it
Wojtek Wajerowicz
@wwajerowicz
can you run docker pull bluebrain/nexus-web and try running everything again?
marmaduke woodman
@maedoc
yep: I see a page on localhost now, but it states there's an error loading organizations . I see similar error in the kg logs as above
marmaduke woodman
@maedoc
is your dev/ci setup on k8s? should I try that instead?
Wojtek Wajerowicz
@wwajerowicz
are there any logs from nexus-web container? or any error in network requests in developer tools in your browser?
You can try our sandbox environment https://sandbox.bluebrainnexus.io/web/
marmaduke woodman
@maedoc
it looks like a MS Edge problem, I opened in Firefox and it looks OK now
a few more questions: what do you use for schema dev? do the projects act like S3 buckets for the files? is the Python API for putting and getting files the most efficient way to get data in & out? will there be any integration with workflow systems to run some container against data sets? (e.g. Pachyderm like)
Wojtek Wajerowicz
@wwajerowicz
the projects are place to store the files and the knowledge graph which describes their provenance. Python API and CLI are probably the easiest way to get started, but any HTTP client should do.
Ehsan
@ehsanmaag
Hi @bogdanromanx and @MFSY, we are trying SPARQL queries on our projects and some of the projects are showing this error:
Response: b'{"@context":"https://bluebrain.github.io/nexus/contexts/error.json","@type":"NotFound","reason":"Resource \'https://bluebrain.github.io/nexus/vocabulary/defaultSparqlIndex\' not found.","ref":"https://bluebrain.github.io/nexus/vocabulary/defaultSparqlIndex"}'
Is it possible that some of our SPARQL views are corrupted?
Oliver Schmid
@olinux
Hi,
I have a question about the SSEs: I would like to build a reference which allows me to resolve the underlying instance with Nexus (ideally from the primary store). I only get the technical project uuid (which I can resolve to an organization and a project by keeping a lookup table) and it seems as I can't use the resourceId as "id" because the current API expects the ID to be compacted by the project-based namespaces. Would there be an option to add a (technical) API endpoint to fetch instances by "projectUuid/resourceId?rev=revision" where "resourceId" is the full id (as reported in the SSE as "_resourceId")?
Didac
@umbreak
@olinux good point. I think that’ something we are gonna work on. We should be able through the API to use both the user friendly labels and the IDs. Ideally we shuld have:
/v1/resources/{org}/{proj}/….
where
  • {org} could be the UUID or the label
  • {proj} could be the UUID or the label
Would that work for you, @olinux ?
Didac
@umbreak
@olinux however, the resourceId can already be used (without using the project prefix mappings or whatever):
/v1/resources/{org}/{proj}/_/{urlEncode(full_expanded_resourceId)}
Oliver Schmid
@olinux
thanks @umbreak ! sounds very good - I think this would solve my problem. Would it be possible to provide the orgId with the SSE payload as well (so I don't need to keep my lookup tables)? for the resourceId: We're using Apache as the reverse proxy in front of Nexus and I just have noticed that that one messes with the url encoding, so the service does receive the decoded url. I was able to reconfigure it so it leaves the escaped slashes untouched - but there seems to still be an issue (maybe because it still decodes the colon). I will keep you updated if I find a solution for it.
Didac
@umbreak

@olinux Core services had some issues with it too. I will ask @bogdanromanx when he is back what was the solution.

Besides that, I will discuss with the team which is the best approach to allow your use case. For now I created a ticket in order to tackle this issue: BlueBrain/nexus#598

Bogdan Roman
@bogdanromanx
@ehsanmaag , are you still experiencing issues when querying SPARQL endpoints?
Bogdan Roman
@bogdanromanx

@maedoc, thanks for evaluating the software, here are some answers

a few more questions: what do you use for schema dev?

currently we have no dedicated tools to help with that, mostly everyone uses their preferred IDEs

do the projects act like S3 buckets for the files?

yes and no; projects represent a bounding box for the data (both json-ld and files) and considering organisations it’s quite similar to how github organizes code in user/repository
each project currently has a single file storage location, but that will change in the upcoming v1.1 where each project can be configured with multiple storage backends (including s3); but regardless of the physical location where files are stored in terms of the API both json-ld resources and files are managed together in a project

is the Python API for putting and getting files the most efficient way to get data in & out?

if you want to use a python client, yes, but python is not necessarily the fastest runtime out there; the client merely consumes an HTTP based API so the faster the client, the faster the interaction (with a well spec’ed Nexus deployment)
if you’re looking at implementing a different client (in another language for example), the general rules of thumb are:

  • try to avoid closing connections, reusing connections removes the signifficant latency introduced by TLS negociation and the transport layer itself
  • use a connection pool and tune its size to your use
  • avoid using blocking IO
  • investigate the use of pipelining (https://en.wikipedia.org/wiki/HTTP_pipelining) the backend supports it

will there be any integration with workflow systems to run some container against data sets? (e.g. Pachyderm like)

this is something we’ve discussed several times and we find it quite interesting, but currently we haven’t had any usecases; if you have some, please let us know and we’ll see if and how we could accommodate this into our roadmap

Didac
@umbreak
@olinux now you have a way to retrieve resources using the organization and project UUIDs: https://github.com/BlueBrain/nexus/issues/598#issuecomment-492629393
pdehaye
@pdehaye

Hi all, I have been trying to follow the docs and run nexus locally using docker swarm. howeber when i do curl http://localhost/kg as a crude test i get the response curl: (52) Empty reply from server. That's no good.

The command docker service logs nexus_router returns

nexus_router.0.880pzdp27qiq@linuxkit-025000000001    | wait-for-it.sh: waiting for kg:8080 without a timeout
nexus_router.0.nsjhkv89gwbo@linuxkit-025000000001    | wait-for-it.sh: waiting for kg:8080 without a timeout

can someone help me?

Wojtek Wajerowicz
@wwajerowicz
can you paste the logs from other services as well? kg, iam especially
pdehaye
@pdehaye
pdehaye@PaulOliviersMBP nexus $ docker service logs nexus_router
nexus_router.0.hdwdkqbbddjp@linuxkit-025000000001    | wait-for-it.sh: waiting for kg:8080 without a timeout
nexus_router.0.sw5qy7tvkx0a@linuxkit-025000000001    | wait-for-it.sh: waiting for kg:8080 without a timeout
nexus_router.0.ytiatelol8g6@linuxkit-025000000001    | wait-for-it.sh: waiting for kg:8080 without a timeout
pdehaye@PaulOliviersMBP nexus $ docker service logs nexus_kg
nexus_kg.1.jv58pib09wtt@linuxkit-025000000001    | wait-for-it.sh: waiting for admin:8080 without a timeout
nexus_kg.1.5s2ymlfup7im@linuxkit-025000000001    | wait-for-it.sh: waiting for admin:8080 without a timeout
nexus_kg.1.h0mroqehg646@linuxkit-025000000001    | wait-for-it.sh: waiting for admin:8080 without a timeout
nexus_kg.1.axh3kxqe37tu@linuxkit-025000000001    | wait-for-it.sh: waiting for admin:8080 without a timeout
pdehaye@PaulOliviersMBP nexus $ docker service logs nexus_router
nexus_router.0.hdwdkqbbddjp@linuxkit-025000000001    | wait-for-it.sh: waiting for kg:8080 without a timeout
nexus_router.0.ytiatelol8g6@linuxkit-025000000001    | wait-for-it.sh: waiting for kg:8080 without a timeout
nexus_router.0.sw5qy7tvkx0a@linuxkit-025000000001    | wait-for-it.sh: waiting for kg:8080 without a timeout
pdehaye@PaulOliviersMBP nexus $ docker service logs nexus_iam
nexus_iam.1.zw88t7enbr0w@linuxkit-025000000001    | wait-for-it.sh: waiting for cassandra:9042 without a timeout
nexus_iam.1.tp3hrawef4si@linuxkit-025000000001    | wait-for-it.sh: waiting for cassandra:9042 without a timeout
nexus_iam.1.e8qjcu5th6ao@linuxkit-025000000001    | wait-for-it.sh: waiting for cassandra:9042 without a timeout
nexus_iam.1.oggx0aq2y6ii@linuxkit-025000000001    | wait-for-it.sh: waiting for cassandra:9042 without a timeout
nexus_iam.1.lcb7efr3crvh@linuxkit-025000000001    | wait-for-it.sh: waiting for cassandra:9042 without a timeout
pdehaye@PaulOliviersMBP nexus $ docker service logs nexus_cassandra
pdehaye@PaulOliviersMBP nexus $ docker service logs nexus_admin
nexus_admin.1.n76qi8ydrh5g@linuxkit-025000000001    | wait-for-it.sh: waiting for iam:8080 without a timeout
nexus_admin.1.oe5ftou15c8h@linuxkit-025000000001    | wait-for-it.sh: waiting for iam:8080 without a timeout
nexus_admin.1.goglpdxcg1bv@linuxkit-025000000001    | wait-for-it.sh: waiting for iam:8080 without a timeout
nexus_admin.1.0v8htfrm17sh@linuxkit-025000000001    | wait-for-it.sh: waiting for iam:8080 without a timeout
pdehaye@PaulOliviersMBP nexus $
Henry
@hygt
hi @pdehaye it seems cassandra is not running which cause all the other services to wait before starting up
could you delete the entire stack and try again?
and check cassandra's log to see if you get any hint about what's wrong
pdehaye
@pdehaye
I think I did this right:
pdehaye@PaulOliviersMBP nexus $ docker stack rm nexus
Removing service nexus_admin
Removing service nexus_blazegraph
Removing service nexus_cassandra
Removing service nexus_elasticsearch
Removing service nexus_iam
Removing service nexus_kg
Removing service nexus_router
Removing service nexus_web
Removing network nexus_default
pdehaye@PaulOliviersMBP nexus $ docker stack ls
NAME                SERVICES            ORCHESTRATOR
pdehaye@PaulOliviersMBP nexus $ docker stack deploy nexus --compose-file=docker-compose.yaml
Creating service nexus_cassandra
failed to create service nexus_cassandra: Error response from daemon: network nexus_default not found
pdehaye@PaulOliviersMBP nexus $ docker stack deploy nexus --compose-file=docker-compose.yaml
Creating network nexus_default
Creating service nexus_cassandra
Creating service nexus_blazegraph
Creating service nexus_web
Creating service nexus_router
Creating service nexus_kg
Creating service nexus_iam
Creating service nexus_admin
Creating service nexus_elasticsearch
pdehaye@PaulOliviersMBP nexus $ curl http://localhost/kg
curl: (52) Empty reply from server
pdehaye@PaulOliviersMBP nexus $ docker services logs nexus_router
docker: 'services' is not a docker command.
See 'docker --help'
pdehaye@PaulOliviersMBP nexus $ docker service logs nexus_router
nexus_router.0.0y161th7khe5@linuxkit-025000000001    | wait-for-it.sh: waiting for kg:8080 without a timeout
pdehaye@PaulOliviersMBP nexus $ docker service logs nexus_kg
nexus_kg.1.3u5un9lwgx19@linuxkit-025000000001    | wait-for-it.sh: waiting for admin:8080 without a timeout
pdehaye@PaulOliviersMBP nexus $ docker service logs nexus_admin
nexus_admin.1.ypx66c9pf9e0@linuxkit-025000000001    | wait-for-it.sh: waiting for iam:8080 without a timeout
pdehaye@PaulOliviersMBP nexus $ docker service logs nexus_iam
nexus_iam.1.jgg74i5s0tab@linuxkit-025000000001    | wait-for-it.sh: waiting for cassandra:9042 without a timeout
pdehaye@PaulOliviersMBP nexus $ docker service logs nexus_cassandra
pdehaye@PaulOliviersMBP nexus $
in other words, there was this first hiccup on the restart, i tried again and somehow it went further. but then the service is still not running.
pdehaye
@pdehaye
allo allo?
pdehaye
@pdehaye
anyone here to help?
Samuel Kerrien
@samuel-kerrien
hi @pdehaye, our backend team is currently attending a conference, they will be in touch ASAP. Thanks for you patience.
pdehaye
@pdehaye
@samuel-kerrien ok, thanks. good to know. by the way I live in Geneva, so would like to meet the team when possible. got some ideas where Nexus could go.
Samuel Kerrien
@samuel-kerrien
Sounds good, we could arrange to meet for lunch next week if you can manage it ?
pdehaye
@pdehaye
@samuel-kerrien 19th? for lunch near/at Secheron?
Samuel Kerrien
@samuel-kerrien
@pdehaye best send me an email (samuel.kerrien@epfl.ch) so we can arrange something ouside of Gitter.
Ehsan
@ehsanmaag
Hi all, I was wondering if there is an easy way to list resources by Tag through Nexus web? thanks!
Mohameth François SY
@MFSY

Hi @ehsanmaag ,

I’m not aware of that capability in Nexus web or even at the REST API level. This can be a feature request. Please open an issue on the Nexus repository.

Note that listing resources in a given project by rev is available though at the REST API level. This feature can be extended to allow listing of resources by tag.

May be this feature is already in the backlog. @umbreak , @bogdanromanx what do you think ?

Bogdan Roman
@bogdanromanx
the feature is not in the backlog, please create a ticket
Sean Hill
@slhill
You can create a custom view with the parameter resourceTag=blabla and then use that view to do the searches.
Ehsan
@ehsanmaag
Thanks very much @MFSY @bogdanromanx @slhill , I think the solution Sean suggested should work. I will try it and let you know if I find anything else
Oliver Schmid
@olinux
Hi everyone,
I was wondering if the documentation for Nexus v0 is still available somewhere since https://bbp-nexus.epfl.ch/staging/docs/ is not available anymore!?
Anna
@annakristinkaufmann
Hi @olinux ! I am not sure this is the best source, but there is the v0 branch of the kg api: https://github.com/BlueBrain/nexus-kg/tree/v0/docs/src/main/paradox/api-reference
Sheetal Kaktikar
@sheetalkaktikar
Hello everyone, I am an enthusiast currently playing around with Nexus and see that it uses Blazegraph. I am trying to find a strategic roadmap for Blazegraph but cannot seem to find anything concrete online. Given the lack of clarity around the roadmap for Blazegraph, could anyone here help me understand, whether from a strategic perspective, Nexus is looking at alternatives to Blazegraph? Many thanks.
Samuel Kerrien
@samuel-kerrien
Hi Sheetal, thanks for you question ! There is indeed a lack of clarity about where BlazeGraph is going in the future. We are not actively looking for alternatives to BlazeGraph but will very likely spend time evaluating a few somewhere toward the beginning of 2020. Projects like Apache Rya have recently come to our attention and we are considering investigating it further.