by

Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Florian Hockmann
    @FlorianHockmann
    Wow, we now don't have any janusgraph-cassandra module in the main repo after #2095 is merged :tada:
    Thanks a lot, @farodin91, for cleaning this up :+1:
    Jan Jansen
    @farodin91

    Hi
    Started to work on Grpc based replacement of the ManagementServer, I want to show you my current state to get some feedback. I have to add "how to use".
    Repo: https://github.com/farodin91/janusgraph-grpc

    This Code also wrappers the GremlinServer and GremlinServer.Settings. In this way, I'm able to start the gremlin server and grpc server at same. Moreover, I can ensure default settings values for serializers. If you update the JanusGraph Server and never changed any of those settings you get a correct default with all serializers of JanusGraph.

    The Grpc server allows you to access multiple graphs using a context which can be helpful, if your are using the ConfigureGraphFactory.

    Next step add more functionally to server, write a client and add some examples.

    Any thoughts?

    31 replies
    Debasish Kanhar
    @debasishdebs
    image.png

    image.png

    When I ran gradle build --info

    Jan Jansen
    @farodin91

    image.png

    I have to fix this issue.

    Oleksandr Porunov
    @porunov
    Hi, I would like to prepare 0.5.2 short after JanusGraph/janusgraph#2103 . Let me know if you think we should include some other contributions in 0.5.2 or if there is a reason to delay this release.
    Jan Jansen
    @farodin91

    Hi, I would like to prepare 0.5.2 short after JanusGraph/janusgraph#2103 . Let me know if you think we should include some other contributions in 0.5.2 or if there is a reason to delay this release.

    I don't think we have any blocker for this release.

    Oleksandr Porunov
    @porunov
    Committers approval is needed for 0.5.2 release commit. All TP tests passed. JanusGraph/janusgraph#2108
    Oleksandr Porunov
    @porunov
    Sonatype405.png
    Oleksandr Porunov
    @porunov

    Unable to deploy artifacts to Sonatype. Unable to open staging repositories. The error returns: Nexus returned an error: ERROR 405: Method Not Allowed. Attached screenshot above. When trying to deploy artifacts with the command mvn clean javadoc:jar deploy -Pjanusgraph-release -DskipTests=true it returns:

    [ERROR] Failed to execute goal org.apache.maven.plugins:maven-deploy-plugin:2.7:deploy (default-deploy) on project janusgraph: Failed to deploy artifacts: Could not transfer artifact org.janusgraph:janusgraph:pom:0.5.2 from/to ossrh (https://oss.sonatype.org/service/local/staging/deploy/maven2/): Failed to transfer file https://oss.sonatype.org/service/local/staging/deploy/maven2/org/janusgraph/janusgraph/0.5.2/janusgraph-0.5.2.pom with status code 405 -> [Help 1]
    [ERROR] 
    [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
    [ERROR] Re-run Maven using the -X switch to enable full debug logging.
    [ERROR] 
    [ERROR] For more information about the errors and possible solutions, please read the following articles:
    [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

    If anyone knows what could cause the problem, please, share your thoughts. I didn't change the releasing flow. As seen from the screenshot, I cannot access staging repositories using UI right now.
    I guess either something changed in Sonatype or I lost permissions to access repository or there is a temporary problem in Sonatype OSS.

    Oleksandr Porunov
    @porunov
    Looks like it was a temporary problem on Sonatype side.
    The issue is now resolved and is not reproducible.
    The artifacts are now uploaded correctly and staging repositories can be accessed.
    Oleksandr Porunov
    @porunov
    JanusGraph 0.5.2 release VOTE is started here: https://groups.google.com/forum/#!topic/janusgraph-dev/T9d20Hx9Kgc
    Debasish Kanhar
    @debasishdebs

    @farodin91 : Quick question. I'm planning to connect the server to a BerkleyDB instance with GraphOfGods loaded already. Do all GET opeartions on it, make it work then move to PUT operation from Client side. (Accordingly we can also find out which feature is missing from Server).

    The current grpc server which embeds gremlin server, does it also expose multiple contexts/janusgraphs defined in .YAML file or does it take single (the last key:value pair in Map) ?

    So, if I expose g and g1 from gremlin server based on graphs map property, I can initialize the context as request.set_context('g1'); request.set_context('g') .. and so on similarly while querying the grpc server to do different set of operations on different Graphs right?
    Debasish Kanhar
    @debasishdebs
    Because I'm looking at core.server.DefaultJanusGraphManager and core.server.JanusGraphServer and not seeing any Map<String, String> to initialize StandardJanusGraph
    Debasish Kanhar
    @debasishdebs

    Okay @farodin91 : The issue is resolved. Loaded GraphOfGods and I'm able to do GET on on existing VertexLabels and EdgeLabels. Able to add more VertexLabel using EnsureVertexLabelRequest. I'll be expanding client to support more operations. Currently it supports only following:

    1: GET - VertexLabel - ALL
    2: GET - VertexLabel - Name
    3: GET - EdgeLabel - ALL
    4: GET - EggeLabel - Name
    5: PUT - VertexLabel - Name [Only with defaults]

    WIll be adding support for variations as well :-)

    Jan Jansen
    @farodin91
    Cool to hear that you solved most of your question. My client was just to test that I'm able to connect with client.
    ((name)string, (config)string) can be found https://github.com/farodin91/janusgraph-grpc/blob/master/src/main/kotlin/org/janusgraph/core/server/DefaultJanusGraphManager.kt#L238 here.
    I'll improve the ConfiguredGraphFactory as next. So, I'm able to contribute the JanusGraphServer with default configuration.
    so on similarly while querying the grpc server to do different set of operations on different Graphs right? If you use ConfiguredGraphFactory, you'll able to access an manage multiple graph at same time.
    1 reply
    Jan Jansen
    @farodin91
    @debasishdebs I improved the index support.
    Debasish Kanhar
    @debasishdebs

    Great @farodin91 . Well I was able to test following scenarios now:
    1: GET CompositeIndex on VertexLabel w/o INDEX_NAME
    2: PUT CompositeIndex on VertexLabel

    I've few concerns like I don't think server supports scenario like
    1: GET CompositeIndex ALL_LABELS (Vertex/Edge) w/o INDEX_NAME
    2: GET CompositeIndex ALL_LABELS (Vertex/Edge) with INDEX_NAME
    3: PUT CompositeIndex ALL_LABELS (Vertex/Edge)

    11 replies
    I'll verify if the above mentioned scenarios are covered, and if not, how can we cover those. I'll check out the code and let you know. Accordingly either you can add the support on Server side or I can whichever we decide. :-) Let me pull your recent version of master and verify if the support is still absent or not
    Debasish Kanhar
    @debasishdebs
    @farodin91 : Added support for directed in EdgeLabel message. Added support for recoginzing when multiplicity and directed is passed while creating EdgeLabel in ManagementForEdgeLabels.kt files. You can have a look at my branch index-support in my fork and if you plan to develop those, you can sync up with my branch :-)
    Jan Jansen
    @farodin91
    @debasishdebs Cool, I'll checkout your changes.
    Debasish Kanhar
    @debasishdebs

    @farodin91 : Have created a PR. It contains Readme on how to invoke APIs from Python client. Quick question is, how do you plan to integrate this with Client? Do we plan to create a GraphQL on top of it for API Access?

    Or, we want to create a DAO type access layer in native language (like python for eg) which shares the same syntax as we have in Java management system, and it calls underlaying gRPC Clients and its Services in backend based on what management query is constructed?

    Something like:

    mgmt = JanusGraph().connect().openManagement()
    mgmt.makeVertexLabel().static().make()
    mgmt.makePropertyKey().dataType().make()
    
    and so on

    As you can see above those are proposed syntax for Python client which shared a highly similar format to Java client. Since we don't expect users to call the gRPC Client stub manually to create Schema elements, having a similar syntax structure to existing JVM systems will help them port easily across languages.

    If you also agree, I can next get started working on these DAOs with the existing implementations of Client we have. What do you say? Also looping in @FlorianHockmann

    In end we will have 2 possible ways for non JVM users to access ManagementSystem either using a GraphQL [On top of gRPC as proposed in [ https://groups.google.com/forum/#!topic/janusgraph-dev/kf21jGgWNn8 ]] and via GLV/DAO from native language mentioned above.

    Jan Jansen
    @farodin91

    Quick question is, how do you plan to integrate this with Client? Do we plan to create a GraphQL on top of it for API Access?

    I don't will build an graphql on top of it from JanusGraph site. It would prefer client lib to integrate with this api.

    Or, we want to create a DAO type access layer in native language (like python for eg) which shares the same syntax as we have in Java management system, and it calls underlaying gRPC Clients and its Services in backend based on what management query is constructed?

    I don't think we should use the same syntax in each language. each language has a different styling.

    As you can see above those are proposed syntax for Python client which shared a highly similar format to Java client. Since we don't expect users to call the gRPC Client stub manually to create Schema elements, having a similar syntax structure to existing JVM systems will help them port easily across languages.

    My plan is to remove this part after grpc client and server is integrate because the ManagementSystem is not the best code.

    GLV part as mention from me in the group would be hard would take a long time.
    Debasish Kanhar
    @debasishdebs

    Or, we want to create a DAO type access layer in native language (like python for eg) which shares the same syntax as we have in Java management system, and it calls underlaying gRPC Clients and its Services in backend based on what management query is constructed?

    I don't think we should use the same syntax in each language. each language has a different styling.

    Makes sense. But if we plan to use diff syntax for different languages, then won't that be another huge discussion forum and we will need to design the synax from scratch up. That would be lots of work I feel. If we have something like existing ManagementSystem based styling, we have some ground work already and we can use some elements from there while add few from our own so that saves up lot of time in designing aspect. What do you think?

    As you can see above those are proposed syntax for Python client which shared a highly similar format to Java client. Since we don't expect users to call the gRPC Client stub manually to create Schema elements, having a similar syntax structure to existing JVM systems will help them port easily across languages.

    My plan is to remove this part after grpc client and server is integrate because the ManagementSystem is not the best code.

    So the current client in Rust is just for testing purpose is it? Cool. Well my Python client I'm developing I want to re-use this later while building management system / schema system in Python. Hence wanted some sort of dicsussion on that.

    Debasish Kanhar
    @debasishdebs

    GLV part as mention from me in the group would be hard would take a long time.

    Yes, correct as it would include lot of components :-) But we can get started with some sort of DAO/GLV.
    So, I've been able to complete following syntax today from Python and verified it works through unit tests. :

    mgmt = JanusGraphManagement().connect(self.HOST, self.PORT, self.GRAPH)
    edges = mgmt.makeEdgelabel(default_label).make()
    edges = mgmt.makeEdgelabel(default_label).undirected().make()
    edges = mgmt.makeEdgelabel(default_label).multiplicity("Many2One").undirected().make()
    edge.getLabel()
    edge.getMultiplicity()
    edge.getDirected()
    edge.getDirection()
    
    vertex = mgmt.makeVertexLabel(default_label).make()
    vertex = mgmt.makeVertexLabel(default_label).setStatic().make()
    vertex = mgmt.makeVertexLabel(default_label).partition().make()
    vertex.getLabel()
    vertex.getStatic()
    vertex.getPartitioned()

    Ultimately I want all of us to decide on the set of Syntax we want to follow, so that some similar DAOs can be created for user to do Schema actions from their native language. Reason for keeping all syntax acoss language close to similar is for a user to achieve interportability without going through much of docs and making life easier :-)

    Jan Jansen
    @farodin91

    Reason for keeping all syntax acoss language close to similar is for a user to achieve interportability without going through much of docs and making life easier :-)

    Sure. From my point of view, we should start again recreating a api which is shared cross system, more like a green field.

    The code the of ManagementSystem so hardly couple with internal code, so I can't change it without creating huge breaking changes. Therefore, i started to create a new schema interaction endpoint (grpc). So, we can refactor the code much easier in the backend without making breaking changes on the user side.

    Debasish Kanhar
    @debasishdebs

    Yea. And per my view, as a good starting point we would need to enable following set of features as initial point so as to make it usable with bigger community in my view:

    1: Create Property Key bound and unbound to any vertex/edge label
    2: Get property key, either bound to labels (Get with respect to label) or unbound.
    3: Create Vertex/Edge labels with or without constrained property keys
    4: Get Vertex/Edge labels
    6: Create composite index, either constrained to single label (Single or multi properties) or un constrained to any label
    7: Get Composite indices.

    So, I would suggest, as course of action, let's create grpc server to support above as those set of features would cover lot of gerenic use case. What do you say? I can get working on fill up the missing gaps from ur grpc server (like 1 and 2/6) and you can get working on getting grpc server into official Janus. That way we both can work in parallel and make this happen. What do you suggest @farodin91 ?
    3 replies

    The code the of ManagementSystem so hardly couple with internal code, so I can't change it without creating huge breaking changes. Therefore, i started to create a new schema interaction endpoint (grpc). So, we can refactor the code much easier in the backend without making breaking changes on the user side

    Yea, that's what I though as well the reason for creating grpc server that the current ManagementSystem APIs are really complicated. My question is, as I went through existing server code, the Server makes use of JanusGraphManagement APIs itself to do any sort of Schema action. How do you plan to replace that? Or we want to make accessibility of JanusGraphManagement system internal, not expose it to outside world?

    1 reply
    Debasish Kanhar
    @debasishdebs

    And, as you mentioned, any changes to ManagementSystem APIs beings in breaking changes, is the following course of action you think we need to take?
    1: Create server,
    2: Make users start migrating to usage of grpc instead of creating schema directly using ManagementSystem
    3: Slowly start changing the Managemen system while also at same time changing the usage/APIs in grpc?
    4: So that we can slowly keep updating grpc server without user knowing what changes happening inside. Is that what you envision?

    Also, we will need to start a post [DISCUSS] I guess to decide on query framework from scratch up for our new client to be built don't you think? I think all these things will take long time, maybe months :-)

    1 reply
    Boxuan Li
    @li-boxuan
    Hi, does anyone know the difference between cla:yes and cla:external labels in PRs? Does this have anything to do with ICLA vs CCLA?
    3 replies
    GeorgeHon
    @GeorgeHon
    Hi, i had fix an serializer issues , when user set accept application/vnd.gremlin-v2.0+json it throws org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerGremlinV2d0 cannot be cast to org.apache.tinkerpop.gremlin.driver.ser.MessageTextSerializer, cloud anyone give me a review?
    this is the [detail link] (JanusGraph/janusgraph#2151)