dynamic graphics, limits and global index


Matthew Nguyen
 

Hi, if we're creating graphs dynamically in JG are the limits like 2^60 edges and Global V and E indexes bound/scoped to the graph or bound across all graphs in the set of graphs managed by ConfigurationManagementGraph?


hadoopmarc@...
 

Hi Matthew,

I suppose you mean identifiers instead of indices? The identifier space is per graph. The storage backends make separate files per graph.

Marc


Matthew Nguyen
 

Hi Marc, just to be clear.

If I create Graph A and build an index on Vertex P via something like:

JanusGraphManagement mgmt = graph.openManagement();
PropertyKey propP = mgmt.getPropertyKey("propertyP");  // assume 'propertyP' has been created via makeProperty()
mgmt.buildIndex("byP", Vertex.class).addKey(propP).buildCompositeIndex();

and I create another Graph B and build the same index and I insert "Hello World" via

GraphA.V().addV().property("propertyP", "Hello World") and 
GraphB.V().addV().property("propertyP", "Hello World")

There should be no conflict btwn the two graphs on propertyP correct?  I assume not but something I want to be absolutely clear on.  Also, are the limits on number of vertices (2^59?) and edges (2^60) per graph or per storage installation?

thx, matt


hadoopmarc@...
 

Hi Matt,

Correct, but you should really try this out and see for yourself. Also check the janusgraph db folder after having created two graphs and see what files are created.

Per graph, but you realize that these number would require exabytes of storage?

Best wishes,   Marc


Matthew Nguyen
 

Hi Marc, I follow what you're saying but will point out it doesn't have to play out to exabytes. Reason is that IDs are non recyclable and there are losses due to bulk load reservations and deletions. I'm looking at JG from a 3store pov where a trillion triples isn't out of reach when it comes to knowledge graphs. Hearing that limits are scoped to the graph makes it much more palatable. 

thx, matt


hadoopmarc@...
 

Hi Matt,

Adding to what I stated above about independent composite indices for separate graphs on the same storage backend, the issue turns out to more nuanced for mixed indices on an indexing bakend, see the recent question:

https://lists.lfaidata.foundation/g/janusgraph-users/topic/88879391

I though it useful to add it to this thread too.

Marc

PS Good to hear that JanusGraph can possibly support your usecase!


Matthew Nguyen
 

Thanks Marc.  Currently triplestore/LPG is on hold awaiting streaming incidental edge queries in order to play some more.  Hoping we will see a day when LPG/3store harmonize.  


-----Original Message-----
From: hadoopmarc@...
To: janusgraph-users@...
Sent: Sun, Feb 6, 2022 1:56 pm
Subject: Re: [janusgraph-users] dynamic graphics, limits and global index

Hi Matt,

Adding to what I stated above about independent composite indices for separate graphs on the same storage backend, the issue turns out to more nuanced for mixed indices on an indexing bakend, see the recent question:

https://lists.lfaidata.foundation/g/janusgraph-users/topic/88879391

I though it useful to add it to this thread too.

Marc

PS Good to hear that JanusGraph can possibly support your usecase!