I tried to use custom ids for created vertices setting
graph.set-vertex-id option.

Now the most time is consumed by code that creates indices and calls
ManagementSystem.getOrCreateVertexLabel() (it takes about 20% of total
time for 1000 vertices and 3000 edges creation).

StandardJanusGraphTx.makeSchemaVertex() calls assignID() through
addProperty() and waits for waitForIDBlockGetter() 565ms total time
and it calls assignID() directly which waits for waitForIDBlockGetter()
334ms total time on my laptop.

Does it mean that each type vertex and property waits for the id separately?

Is there a way to provide custom ids for vertices and properties during
index creation (e.g for ManagementSystem.getOrCreateVertexLabel() call)?

Thanks,
Alexander.

On 24.09.2019 20:02, Pavel Ershov wrote:
> All entities needs assigned id: edges, vertices, properties and same for
> schema entities too. But this pause occurred not so frequently for each
> 10 000 ids by default
> You can trace pool size and gather frequency here
> <https://github.com/JanusGraph/janusgraph/blob/master/janusgraph-core/src/main/java/org/janusgraph/graphdb/database/idassigner/StandardIDPool.java#L191>
> That's pool may be increased for bulk loading, see
> https://github.com/JanusGraph/janusgraph/blob/master/janusgraph-core/src/main/java/org/janusgraph/graphdb/configuration/GraphDatabaseConfiguration.java#L694
>
> I try speed up block acquisition for local storages here
> <https://github.com/mad/janusgraph/commit/342a1814b26a8b63f9337f88145c0dcb8f628a09>,
> it may increase graph initialization when schema is too big. With
> current fix both tests executed for 200ms
>
> вторник, 24 сентября 2019 г., 17:37:28 UTC+3 пользователь
> st...@... написал:
>
>
>     I used JanusGraph with Berkeley DB and found that adding even 10
>     vertices takes 6 seconds on my laptop.
>
>     I narrowed down the problem to a simple program which use JanusGraph
>     with inmemory backend storage.
>
>     Profiler showed that the code 'g.addV(LABEL).property(KEY,
>     value).next()' takes 87% of time in main thread
>     and both methods   StandardJanusGraphTx.addProperty() and
>     StandardJanusGraphTx.addVertex() spends most time in
>     StandardJanusGraph.assignID() which just waits for
>     StandardIDPool.waitForIDBlockGetter().
>
>     I updated my program to use custom generated IDs
>     `g.addV(LABEL).property(T.id, id).property(KEY, value).next()'
>     and the adding vertices took about 1 second and about 59% from the
>     main thread time.
>
>     However, the second approach requires custom id management.
>
>     Is it possible to use automatically generated IDs so time to adding
>     vertices does not take too much time?
>
>     I used JanusGraph version 0.4.0.
>
>     Thanks,
>     Alexander.
>
>     The code used for testing:
>     -----------  JanusGraphSample.java -----------------
>     package sample;
>
>     import org.apache.commons.io.FileUtils;
>     import
>     org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversalSource;
>     import org.apache.tinkerpop.gremlin.structure.T;
>     import org.janusgraph.core.JanusGraph;
>     import org.janusgraph.core.JanusGraphFactory;
>     import org.janusgraph.core.JanusGraphTransaction;
>     import org.janusgraph.graphdb.database.StandardJanusGraph;
>     import org.janusgraph.graphdb.idmanagement.IDManager;
>
>     import java.io.File;
>     import java.io.IOException;
>
>     public class JanusGraphSample {
>
>          private static final String LABEL = "SampleLabel";
>          private static final String KEY = "SampleKey";
>          private static final String VALUE = "SampleValue";
>          private static final int N = 10;
>
>          public static void main(String[] args) {
>              addVerticesWithGeneratedIds();
>              addVerticesWithCustomIds();
>          }
>
>          private static JanusGraph getJanusGraph(boolean customIds) {
>
>              JanusGraphFactory.Builder builder = JanusGraphFactory.build()
>                      .set("storage.backend", "inmemory");
>
>              if (customIds) {
>                  builder = builder.set("graph.set-vertex-id", "true");
>              }
>
>              return builder.open();
>          }
>
>          static void clearGraph(JanusGraph graph) {
>              try (JanusGraphTransaction tx = graph.newTransaction()) {
>                  GraphTraversalSource g = tx.traversal();
>                  g.V().drop().iterate();
>                  tx.commit();
>              }
>          }
>
>          static long verticesCount(JanusGraph graph) {
>              try (JanusGraphTransaction tx = graph.newTransaction()) {
>                  GraphTraversalSource g = tx.traversal();
>                  return g.V().count().next();
>              }
>          }
>
>          public static void addVerticesWithGeneratedIds() {
>
>              System.out.printf("Add vertices with generated ids%n");
>
>              try (JanusGraph graph = getJanusGraph(false)) {
>
>                  clearGraph(graph);
>
>                  try (JanusGraphTransaction tx = graph.newTransaction()) {
>                      GraphTraversalSource g = tx.traversal();
>
>                      System.out.printf("N: %d%n", N);
>                      long time = System.currentTimeMillis();
>                      for (int i = 1; i <= N; i++) {
>                          String value = String.format("%s-%d", VALUE, i);
>                          g.addV(LABEL).property(KEY, value).next();
>                      }
>                      tx.commit();
>                      System.out.printf("elapsed time: %d(ms)%n",
>     System.currentTimeMillis() - time);
>                  }
>
>                  long vertices = verticesCount(graph);
>                  System.out.printf("added vertices: %d%n", vertices);
>              }
>          }
>
>          public static void addVerticesWithCustomIds() {
>
>              System.out.printf("Add vertices with custom ids%n");
>
>              try (JanusGraph graph = getJanusGraph(true)) {
>
>                  clearGraph(graph);
>
>                  IDManager idManager = ((StandardJanusGraph)
>     graph).getIDManager();
>
>                  try (JanusGraphTransaction tx = graph.newTransaction()) {
>                      GraphTraversalSource g = tx.traversal();
>
>                      System.out.printf("N: %d%n", N);
>                      long time = System.currentTimeMillis();
>                      for (int i = 1; i <= N; i++) {
>                          long id = idManager.toVertexId(i);
>                          String value = String.format("%s-%d", VALUE, i);
>                          g.addV(LABEL).property(T.id, id).property(KEY,
>     value).next();
>
>                      }
>                      tx.commit();
>                      System.out.printf("elapsed time: %d(ms)%n",
>     System.currentTimeMillis() - time);
>                  }
>
>                  long vertices = verticesCount(graph);
>                  System.out.printf("added vertices: %d%n", vertices);
>              }
>          }
>     }
>     --------------------------
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "JanusGraph users" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/janusgraph-users/ME9U6-0Xn1Y/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> janusgra...@...
> <mailto:janusgra...@...>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/janusgraph-users/aa5daeac-ae57-4a30-ad14-91bd0e3a31b4%40googlegroups.com
> <https://groups.google.com/d/msgid/janusgraph-users/aa5daeac-ae57-4a30-ad14-91bd0e3a31b4%40googlegroups.com?utm_medium=email&utm_source=footer>.

--
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgra...@....
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/51caccd9-8364-cb9f-0610-ee4ae11bbd73%40gmail.com.