Sharing detail on how i am creating node/edges to make sure nothing wrong with that which is resulting in ID allocation failures.
I am creating one static instance JanusGraph object on each spark worker box and using that i am creating multiple transaction and commit.
pairRDD.foreachPartition(partIterator -> {
partIterator.forEachRemaining( tuple -> {
createNodeAndEdge(tuple, JanusGraphConfig.getJanusGraph(janusConfig));
});
}); where JanusGraphConfig.getJanusGraph returns static instance.
In createNodeAndEdge() method i am creating GraphTraversalSource using static janusGraph, creating node, edge, committing and then closing GraphTraversalSource object, as shown below in pseudo code:
createNodeAndEdge(Tuple2<K, V> pair, JanusGraph janusGraph)
{
GraphTraversalSource g = janusGraph.buildTransaction().start().traversal();
try{
create node;
create edge;
g.tx().commit();
} catch ( Exception) {
g.tx().rollback();
} finally() {
g.tx().close();
g.close();
}
}
Thanks,
Anjani