https://docs.janusgraph.org/advanced-topics/bulk-loading/#optimizing-id-allocation

Thanks for response Marc. Below is the method i am using to create janus connection:

public JanusGraph createJanusConnection(HashMap<String, Object> janusConfig) {

    JanusGraphFactory.Builder configProps = JanusGraphFactory.build();

  configProps.set(GREMLIN_GRAPH, “org.janusgraph.core.JanusGraphFactory”);

    configProps.set(STORAGE_BACKEND, “cql”);

    configProps.set(STORAGE_HOSTNAME, janusConfig.get("storage.hostname"));

    configProps.set(STORAGE_CQL_KEYSPACE, janusConfig.get("storage.keyspace"));

    configProps.set(CACHE_DB_CACHE, “false”);

    configProps.set(CACHE_DB_CACHE_SIZE, “0.5”);

    configProps.set(CACHE_DB_CACHE_TIME, “180000”);

    configProps.set(CACHE_DB_CACHE_CLEAN_WAIT, “20”);

    configProps.set(STORAGE_CQL_LOCAL_DATACENTER, janusConfig.get("local-datacenter"));

     configProps.set(STORAGE_CQL_WRITE_CONSISTENCY_LEVEL, “LOCAL_ONE”);

    configProps.set(STORAGE_CQL_READ_CONSISTENCY_LEVEL, “LOCAL_ONE”);

    configProps.set(STORAGE_CQL_SSL_ENABLED, janusConfig.get("cql.ssl.enabled"));

    configProps.set(STORAGE_CQL_SSL_TRUSTSTORE_LOCATION, janusConfig.get("truststore.location"));

    configProps.set(STORAGE_CQL_SSL_TRUSTSTORE_PASSWORD, janusConfig.get("truststore.password"));

    configProps.set(STORAGE_USERNAME, janusConfig.get("cassandra.username"));

    configProps.set(STORAGE_PASSWORD, janusConfig.get("cassandra.password"));

    configProps.set("storage.read-time", "120000");

    configProps.set("storage.write-time", "120000");

    configProps.set("storage.connection-timeout", "120000");

    // added to fix ID block allocation exceptions

    configProps.set("renew-timeout", "240000");

    configProps.set("write-time", "1000");

    configProps.set("read-time", "100");

    configProps.set("renew-percentage", "0.4");

    configProps.set(METRICS_ENABLED, “true”);

    configProps.set(METRICS_JMX_ENABLED, “true”);

    configProps.set(INDEX_SEARCH_BACKEND, “elasticsearch”);

    configProps.set(INDEX_SEARCH_HOSTNAME, janusConfig.get("elasticsearch.hostname"));

   configProps.set(INDEX_SEARCH_ELASTICSEARCH_HTTP_AUTH_TYPE,”basic”);

    }

    configProps.set(INDEX_SEARCH_ELASTICSEARCH_HTTP_AUTH_BASIC_USERNAME, janusConfig.get("elasticsearch.username"));

    configProps.set(INDEX_SEARCH_ELASTICSEARCH_HTTP_AUTH_BASIC_PASSWORD, janusConfig.get("elasticsearch.password"));

    configProps.set(INDEX_SEARCH_ELASTICSEARCH_SSL_ENABLED, janusConfig.get("elasticsearch.ssl.enabled")

    );

    configProps.set(IDS_BLOCK_SIZE, “1000000000”);

    configProps.set(IDS_RENEW_PERCENTAGE, “0.3”);

    logger.info("JanusGraph config initialization!!");

    return configProps.open();

}

Hi All,

I am creating vertex and edges in bulk and getting below error while creating edge. Below is the exception log:

Cluster edge creation failed between guest node 20369408030929128 and identifier node 16891904008515712. Exception : org.janusgraph.core.JanusGraphException: ID block allocation on partition(29)-namespace(3) failed with an exception in 12.23 ms

I tired increasing value of "ids.block-size" but still no luck, even set to 1B also for testing purpose but still no luck, getting above error. I am creating around 4 - 5M nodes per hours.

Could you please share some pointers to fix it? Appreciate your help and time.

Thanks,
Anjani

• the repetition of as('v1') is unusual. Can you try what happens if you use the aggegate('v1')..............cap('v1', e, 'v2') mechanism instead? Or, simpler, what happens if you use neither the as() nor the aggregate() steps, omitting the formatting of the output?
• are you sure there are no memory constraints, even if this seems unlikely given the limit(100) steps applied. You can check by increasing memory for gremlin console:
  export JAVA_OPTIONS="-Xmx4g"