Where communities thrive

  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
Repo info
    We have a graph of few million vertices and edges and my current gremlin query is quite long . On profile its traversing over 50k and I see lot of backend_query steps. Can someone suggest what options I have to optimize performance
    3 replies
    Harshit Sharma

    I'm trying to find the all the nodes in a connected component in a graph, which contains around ~130M vertices and ~350M edges.

    Following is the query I'm using to find the count of nodes in connected components -

    Input - starting vertex id/name

    Output - count of nodes in the connected component.

    Query - g.v().has("name", "driver1").repeat(where(without ("a")).store("a").both().simplePath().dedup()).emit().hasLabel("driver").count().fold()

    The above query is taking around ~ 52 sec

    RepeatStep is taking around ~ 29 sec

    Is there a way we can optimize the linear traversal in Repeatstep or lookup in WherePredicateatep?

    Profile output of above query-
    "dur": 29008.200345, "counts": ("traverserCount": 13809,"elementCount": 13809}, name: "RepeatStep ([Where PredicateStep (without ([a])), Profilestep, Storestep (a), Profilestep, JanuaGraphVertexStep(BOTH, vertex), ProfileStep, PathFilterstep(simple), Profilestep, RepeatEndstep, Profilestep], until(false), emit(true))", "annotations":{ "percentDur": 52.557919400750215}, "id": "2.0.0()", "metrics": [ { "dur": 38.137699, "counts":{ traverserCount: 13810, elementCount: 13810}, "name":"WherePredicateStep(without([a]))", "id": "0.1.0 (2.0.0())" }, { "dur": 28628.594393, "counts": { "traverserCount": 252428, "elementCount": 252428 }, name: "JanusGraphVertexStep (BOTH, vertex)", "annotations"

    Natalia Tisseyre
    Hello! Are there any plans to use non-blocking APIs in JanusGraph? For example, CQLStoreManager in org.janusgraph.diskstorage.cql is using a ThreadPoolExecutor with 100 threads at max, and it is blocking them when making a call to Cassandra.
    3 replies
    Matthias Leinweber

    A new day two new questions/problems:
    Is there something in the index which does not exist anymore? Or how can i understand this:

    2021-07-23 11:00:16,135 ERROR (Thread-40) [org.janusgraph.graphdb.olap.job.IndexRemoveJob] - Filtering key   0-  0-  0-  0-  0-  4- 97-  0 due to exception
    java.lang.ArrayIndexOutOfBoundsException: Required size [1] exceeds actual remaining size [0]
        at org.janusgraph.diskstorage.util.StaticArrayBuffer.require(StaticArrayBuffer.java:94)
        at org.janusgraph.diskstorage.util.StaticArrayBuffer.getByte(StaticArrayBuffer.java:170)
        at org.janusgraph.diskstorage.util.ReadArrayBuffer.getByte(ReadArrayBuffer.java:79)
        at org.janusgraph.graphdb.database.idhandling.VariableLong.readUnsigned(VariableLong.java:48)
        at org.janusgraph.graphdb.database.idhandling.VariableLong.readPositive(VariableLong.java:94)
        at org.janusgraph.graphdb.database.IndexSerializer.getIndexIdFromKey(IndexSerializer.java:750)
        at org.janusgraph.graphdb.olap.job.IndexRemoveJob.lambda$getKeyFilter$1(IndexRemoveJob.java:173)
        at org.janusgraph.diskstorage.keycolumnvalue.scan.StandardScannerExecutor$DataPuller.run(StandardScannerExecutor.java:338)

    I have 3 janusgraph instances with ConfigurationManagementGraph and i can only to changes on the indexes if shutdown down the statefullset to 1 replica otherwise the eviction is not acknowldeged altough i see no connections on any of the nodes. What does look a bit weired are the instance names. Could there a communication problem? Is there a keep alive option to automatically close inactice instances?
    its something like this 0a2a0007141-janusgraph-0-janusgraph-headless-kubernetes-namespace-svc-cluster-local2
    as you suggest its a statefullset with a headless service so that i can access the instances with janusgraph-0/1/2.janusgraph-headless

    3 replies
    the index remove job finishes with a ton of this error but the index is not removed
    Oleksandr Porunov

    JanusGraph now accepts donations via LFX Crowdfunding.
    The announcement is available here: https://lists.lfaidata.foundation/g/janusgraph-announce/topic/announcement_janusgraph/84454614
    Donations link: https://crowdfunding.lfx.linuxfoundation.org/projects/janusgraph

    Best regards,
    Oleksandr Porunov
    on behalf of JanusGraph TSC

    Is there a plan to publish current master version on janusgraph as release?
    Or how to create release from current master version? mvn clean pacage build jar filses what about configuration files and scripts
    Florian Hockmann

    0.6.0 will probably be released soon from master, but if you want to build a distribution yourself, then that is described in BUILDING.md:

    To build the distribution archive:

    mvn clean install -Pjanusgraph-release -Dgpg.skip=true -DskipTests=true

    Did anyone tried to use it with Astra? It requires simple change in the configuration. If I succeed I can contribute a code.
    Hi There,
    We are using janusgraph 0.5.2 version and facing high vulnerability issue with commons-compress.jar . How is it handled in JG ? Have you taken exception for it ? Or fixed in new versions ? Can you please suggest how can we go about it. Thanks.
    These are the issues we got in twistlock scans.
    1 reply
    Ryan Quey

    Did anyone tried to use it with Astra? It requires simple change in the configuration. If I succeed I can contribute a code.

    @fmatuszewski If you're asking if anyone has tried to use janus Graph with Astra, I tried before but was unsuccessful. I discussed it on Datastax community here https://community.datastax.com/questions/8775/how-do-you-configure-janusgraph-to-connect-to-data.html

    I'd love to see a working example, if you get it working let me know. If you have a current repo let me know so I can follow it - I ran out of time to keep working on it before, but interested to know how to get it work for future reference
    Ryan Quey
    @fmatuszewski I just found what I recorded from when I tried this before, in case it's useful at all. It corresponds with the Datastax community post given above https://github.com/RyanQuey/janus-graph-tutorial#can-you-connect-janusgraph-to-datastax-astra
    Hi 👋
    @porunov @mbrukman I got our PR's approval on having our company logo up in janusgraph.org. how do i proceed with this?
    Matthias Leinweber
    Am i wrong or do i have to use spark 2.2 for Janusgraph? 3.x is not working?
    6 replies
    This message was deleted
    3 replies
    Hi. I can't find the 0.6.0 release in maven.
    2 replies
    Randeep Singh
    hi all, while using the Cassandra as backend datastore, does addVertex api add it in Cassandra directly (before commit() api call) or in memory, if it only adds it in memory, then I am not able to figure out why it takes about 700 ms for a number of such addVertex calls , though a lot of them complete within 1-5 ms also. Would be great if someone knows and can help understanding about this
    4 replies
    hello there, i would like to ask if there is any good entry point of documentation if i want to implement my own storage backend.
    ok, found so far the tinkerpop api to implement a gremlin backend
    does JanusGraph support GraphQL?
    Fred Eisele

    I have created a variant of janusgraph-docker https://github.com/babeloff/janusgraph-docker2
    It is not quite ready for prime time just yet (but you are welcome to look at it and give me feedback (preferably via issues)).
    That said I am having issues creating indexes for my sample.
    0) install docker
    1) create the server: cql and es

    ./gradlew upJanusgraph2DynamicServer

    2) create the client

    ./gradlew upJanusgraph2DynamicServer

    3) create the schema

    :remote connect tinkerpop.server conf/remote.yaml session
    :remote console
    :load schema/ensample.groovy
    :load schema/ensample-index.groovy
    :load schema/ensample-index-create.groovy

    The indexes never progress from INSTALLED.

    Vertex Index Name              | Type        | Unique    | Backing        | Key:           Status |
    byIdComposite                  | Composite   | true      | internalindex  | []ID:       INSTALLED |
    byName_Index                   | Composite   | false     | internalindex  | []Name:     INSTALLED |
    Graph_Name_Index               | Composite   | true      | internalindex  | graph.graphname:    ENABLED |
    Template_Index                 | Composite   | false     | internalindex  | Template_Configuration:    ENABLED |
    Created_Using_Template_Index   | Composite   | false     | internalindex  | Created_Using_Template:    ENABLED |
    3 replies
    Harshit Sharma
    I was reading about Janusgraph Cache in janusgraph documentation. I have few doubts -
    1. Regarding transaction cache, If I'm only doing a read query for eg. - g. V().has("name","ABC") using gremlin HTTP endpoint, so will this value be cached in transaction cache or database level cache? because here I'm not opening any transaction.
    2. If it is stored in the transaction cache, how updated values will be fetched for this vertex if I have multi-node deployment?
    Jens Geyer

    Hi, I just had a docker container crashing (power outage) and now it does not start the gremlin server anymore. It worked before. This is what I get:

    1735 [main] WARN org.apache.tinkerpop.gremlin.server.util.DefaultGraphManager - Graph [graph] configured at [/etc/opt/janusgraph/janusgraph.properties] could not be instantiated and will not be available in Gremlin Server. GraphFactory message: GraphFactory could not instantiate this Graph implementation [class org.janusgraph.core.JanusGraphFactory]

    How can I resol,ve thgat w/o having to import all data again?

    Brian Miller
    Hi, does anyone know how to override the Gremlin v3.3.1 Java driver's default org.apache.tinkerpop.gremlin.driver.hostssetting at JVM runtime? I thought passing in something like -Dorg.apache.tinkerpop.gremlin.driver.hosts= IP per https://tinkerpop.apache.org/docs/current/reference/#gremlin-java-configuration would have worked but no dice :confused:
    Hi There, we are using JanusGraph-FoundationDB, what is the best possible approach for a robust implementation for bulk read (export) and bulk write (Import) operations? any documentation would be more helpful. FYI - I already gone through this https://docs.janusgraph.org/operations/bulk-loading/
    3 replies
    Natalia Tisseyre

    Hello! Is there any efficient way to search vertices by vertex index + edge filter?
    I have a use case of about 65k vertexes, they can be quickly retrived using Graph Index (property "name") but then i need to filter the result using edge value.

    Using standard JanusGraph/Gremlin synatxes doesn't seem to be efficient enough. This query runs about 65 seconds:

    dbCtx.traversal.V() .hasLabel("VertexLabel1") .has("name", "hello") .where(__.outE("EdgeLabel1).has("edgeId", "6789")) .range(100, 200)

    While this query runs about 2 seconds:

    val iterator = dbCtx.traversal.V() .hasLabel("VertexLabel1") .has("name", "hello")
    //Do parallel traversal of the stream to filter vertexes by edge property
    inParallel(iterator) { vertex => filterByEdgeId}

    ViVek Raghuwanshi
    Hi All
    Is there is any wany to use storage backend a flat file
    instead of cassandra or in memory
    @viveksraghuwanshi: there is an in-memory graph storage afaik
    Hi There, we are using JanusGraph-FoundationDB, what is the best possible approach for a robust implementation for bulk read (export) and bulk write (Import) operations? any documentation would be more helpful. FYI - I already gone through this https://docs.janusgraph.org/operations/bulk-loading/
    posting it for the 2nd time, any input would be of a great help

    Hi, I am seeing this weird exception while traversing vertices with a given label:

    ",throwable.class="java.lang.IllegalArgumentException",throwable.msg="expected one element but was: <Artifact, Artifact>",throwable.stack="
        at com.google.common.collect.Iterators.getOnlyElement(Iterators.java:315)
        at com.google.common.collect.Iterators.getOnlyElement(Iterators.java:327)
        at com.google.common.collect.Iterables.getOnlyElement(Iterables.java:268)
        at org.janusgraph.graphdb.vertices.AbstractVertex.getVertexLabelInternal(AbstractVertex.java:125)
        at org.janusgraph.graphdb.vertices.AbstractVertex.vertexLabel(AbstractVertex.java:130)
        at org.janusgraph.graphdb.vertices.AbstractVertex.label(AbstractVertex.java:121)
        at org.janusgraph.graphdb.types.system.ImplicitKey.computeProperty(ImplicitKey.java:83)
        at org.janusgraph.graphdb.query.vertex.BasicVertexCentricQueryBuilder.executeImplicitKeyQuery(BasicVertexCentricQueryBuilder.java:211)
        at org.janusgraph.graphdb.query.vertex.VertexCentricQueryBuilder.properties(VertexCentricQueryBuilder.java:99)
        at org.janusgraph.graphdb.util.ElementHelper.getValues(ElementHelper.java:41)
        at org.janusgraph.graphdb.query.condition.PredicateCondition.evaluate(PredicateCondition.java:68)
        at org.janusgraph.graphdb.query.condition.And.evaluate(And.java:55)
        at org.janusgraph.graphdb.query.graph.GraphCentricQuery.matches(GraphCentricQuery.java:153)
        at com.google.common.collect.Iterators$5.computeNext(Iterators.java:637)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136)
        at org.janusgraph.graphdb.query.QueryProcessor.getUnfoldedIterator(QueryProcessor.java:98)
        at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:67)
        at org.janusgraph.graphdb.query.graph.GraphCentricQueryBuilder$1.iterator(GraphCentricQueryBuilder.java:204)
        at org.janusgraph.graphdb.query.graph.GraphCentricQueryBuilder$1.iterator(GraphCentricQueryBuilder.java:201)
        at org.janusgraph.graphdb.tinkerpop.optimize.JanusGraphStep.executeGraphCentricQuery(JanusGraphStep.java:160)
        at org.janusgraph.graphdb.tinkerpop.optimize.JanusGraphStep.lambda$new$1(JanusGraphStep.java:95)
        at java.lang.Iterable.forEach(Iterable.java:75)
        at org.janusgraph.graphdb.tinkerpop.optimize.JanusGraphStep.lambda$new$2(JanusGraphStep.java:95)
        at org.apache.tinkerpop.gremlin.process.traversal.step.map.GraphStep.processNextStart(GraphStep.java:157)
        at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:144)
        at org.apache.tinkerpop.gremlin.process.traversal.util.DefaultTraversal.hasNext(DefaultTraversal.java:196)
        at org.apache.tinkerpop.gremlin.process.traversal.Traversal.tryNext(Traversal.java:97)

    Any pointers as to how to understand the root-cause here?

    15 replies
    Lucas Loboda
    any idea why recent docker elastic/cassandra just fails to build?
    I had to downgrade to 0.5.3 just to keep any working version of janusgraph
    I spent 2 days of my life trying to make it work on 2 different hosts
    but containers just keep going down after they get built
    I must say that I did consider to drop this whole idea about janus and use something else but community seem to be just too good
    am I the only person with such issues?
    1 reply
    Lewis Chan
    Hi, all. Anyone who does performance comparison with different storage backends such as Cassandra, BigTable, Foundationdb ? Or can we just use backends that I and my team are familiar with ?
    1 reply
    Can anyone help me to some docs on how to handle this WARN log:
    Local setting storage.cql.replication-strategy-options=... (Type: FIXED) is overridden by globally managed value (...). Use the ManagementSystem interface instead of the local configuration to control this setting.
    1 reply
    Lewis Chan
    gitter seems not easier to use than slack, discord. No sub-channel, inconvenient to search. Community consider to change ?
    5 replies
    neo4j has migrated to discord. https://neo4j.com/blog/neo4j-community-is-migrating-from-slack-to-discord/ . I don't know whether JG could do that.
    @FlorianHockmann -- we have Janus-Graph with back-end store Foundation DB and index back-end as Elastic Search. Please let me know what is the best way to export/read Millions of Records from JaunusGraph by keeping performance in mind. We don't have the option of using Spark in our environment. I have seen 100s of articles on Bulk Loading but not bulk export/Read. Any suggestion would be of a great help here.
    7 replies
    Hi all, can anyone advise on https://github.com/JanusGraph/janusgraph/discussions/2820, what is best way to reliably move data from janusgraph to another store/queue