1. Topics
  2. Adding a bunch of Vertices and edges takes for ever

Adding a bunch of Vertices and edges takes for ever

yair...@...
 

I am writing an app that loads an existing RDBMS based graph implementation into JanusGraph.

I create this index:

        JanusGraphManagement mgmt = javaGraph.openManagement();
        if (mgmt.containsPropertyKey("nodeId")) {
            nodeId = mgmt.getPropertyKey("nodeId");
        } else {
            nodeId = mgmt.makePropertyKey("nodeId").dataType(Integer.class).make();
        }

        if(!mgmt.containsGraphIndex("nodeIdInd")){
            JanusGraphManagement.IndexBuilder nodeIdIndexBuilder = mgmt.buildIndex("nodeIdInd", Vertex.class).addKey(nodeId);
            nodeIdIndexBuilder.unique();
            JanusGraphIndex nodeIdIdx = nodeIdIndexBuilder.buildCompositeIndex();
        }


        mgmt.commit();


Creating the vertices and edges (~6K vertices and ~40K edges) takes a very long time.

Am I missing something?
yair...@...
 

I should add I do this query (to eliminate duplicate vertices) before adding each Vertex as I know that in the existing RDBMS same nodes exist multiple times:

javaGraph.traversal().V().hasLabel(instanceId).has("nodeId", nodeInfo.getId());

        if (vertexNode.hasNext()) {
            return vertexNode.next();
        } else {
            JanusGraphVertex node = javaGraph.addVertex(instanceId/*, T.id,((Integer)id).longValue()*/);
            node.property("nodeId", id);
            return node;
        }

Do I need to index the label as well?

Any other ideas?


On Monday, September 11, 2017 at 11:03:07 AM UTC+3, ya...@... wrote:
I am writing an app that loads an existing RDBMS based graph implementation into JanusGraph.

I create this index:

        JanusGraphManagement mgmt = javaGraph.openManagement();
        if (mgmt.containsPropertyKey("nodeId")) {
            nodeId = mgmt.getPropertyKey("nodeId");
        } else {
            nodeId = mgmt.makePropertyKey("nodeId").dataType(Integer.class).make();
        }

        if(!mgmt.containsGraphIndex("nodeIdInd")){
            JanusGraphManagement.IndexBuilder nodeIdIndexBuilder = mgmt.buildIndex("nodeIdInd", Vertex.class).addKey(nodeId);
            nodeIdIndexBuilder.unique();
            JanusGraphIndex nodeIdIdx = nodeIdIndexBuilder.buildCompositeIndex();
        }


        mgmt.commit();


Creating the vertices and edges (~6K vertices and ~40K edges) takes a very long time.

Am I missing something?
Ankur Goel <ankur...@...>
 

You are using unique(), this takes lock for each operation. Try to handle uniqueness through application instead of Janus.

~AnkurG


On Monday, September 11, 2017 at 1:33:07 PM UTC+5:30, ya...@... wrote:
I am writing an app that loads an existing RDBMS based graph implementation into JanusGraph.

I create this index:

        JanusGraphManagement mgmt = javaGraph.openManagement();
        if (mgmt.containsPropertyKey("nodeId")) {
            nodeId = mgmt.getPropertyKey("nodeId");
        } else {
            nodeId = mgmt.makePropertyKey("nodeId").dataType(Integer.class).make();
        }

        if(!mgmt.containsGraphIndex("nodeIdInd")){
            JanusGraphManagement.IndexBuilder nodeIdIndexBuilder = mgmt.buildIndex("nodeIdInd", Vertex.class).addKey(nodeId);
            nodeIdIndexBuilder.unique();
            JanusGraphIndex nodeIdIdx = nodeIdIndexBuilder.buildCompositeIndex();
        }


        mgmt.commit();


Creating the vertices and edges (~6K vertices and ~40K edges) takes a very long time.

Am I missing something?
Yair Ogen <yair...@...>
 

good to know. Thanks

On Mon, Sep 11, 2017 at 4:34 PM, Ankur Goel <ankur...@...> wrote:
You are using unique(), this takes lock for each operation. Try to handle uniqueness through application instead of Janus.

~AnkurG


On Monday, September 11, 2017 at 1:33:07 PM UTC+5:30, ya...@... wrote:
I am writing an app that loads an existing RDBMS based graph implementation into JanusGraph.

I create this index:

        JanusGraphManagement mgmt = javaGraph.openManagement();
        if (mgmt.containsPropertyKey("nodeId")) {
            nodeId = mgmt.getPropertyKey("nodeId");
        } else {
            nodeId = mgmt.makePropertyKey("nodeId").dataType(Integer.class).make();
        }

        if(!mgmt.containsGraphIndex("nodeIdInd")){
            JanusGraphManagement.IndexBuilder nodeIdIndexBuilder = mgmt.buildIndex("nodeIdInd", Vertex.class).addKey(nodeId);
            nodeIdIndexBuilder.unique();
            JanusGraphIndex nodeIdIdx = nodeIdIndexBuilder.buildCompositeIndex();
        }


        mgmt.commit();


Creating the vertices and edges (~6K vertices and ~40K edges) takes a very long time.

Am I missing something?





--
You received this message because you are subscribed to a topic in the Google Groups "JanusGraph users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/janusgraph-users/wnRpz_C5Nk0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to janusgraph-users+unsubscribe@googlegroups.com.
 To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/f5f44081-c7a0-4336-ad7c-30b56a372356%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.
1 - 4 of 4