Many organizations use JanusGraph on this scale. Insertion of data is slow so you need massive parallel operations to do an entire bulk load overnight. Most people use tools like Apache Spark for

+ I want to add one more question. What about ScyllaDB as storage backend? Is it better to use this in terms of performance?

Hi everyone, We are working on a project that we would like to use JanusGraph. Our system will consist of 100MM nodes and 1B edges between those nodes. We are going to work with last 90 days

If your use case can handle the downtime, stopping writes and waiting until all changes are propagated to both the storage and the index backend sounds like a viable solution. However, I have no idea

Yeah, good point, it's a bit hairy. Having potentially inconsistent index backups makes them much less attractive. Though I guess I could run a reindex job on just the delta since last Scylla write

Although the solution presented by Marc is also the closest to a consistent backup that I can think of, there are obviously caveats to it. Updates of values which were written after the time of the

Awesome, yes, that's very similar to what I was planning! It's not perfect and definitely needs to tested thoroughly, but it should be much faster and reasonably scriptable. I'll let you all know how

In theory (not used in practice) the following should be possible: make a snapshot of the ScyllaDB keyspace after the ScyllaDB snapshot is written, make a snapshot of corresponding ES mixed

Thanks again. Yeah, might end up doing that, but it seems like a complicated solution.. hmm. Regarding the feature request, I'll dig into the code and ask around the janusgraph-dev group :)

Yeah, reindexing can be slow in that case. You could try the transaction recovery mechanism as described in https://docs.janusgraph.org/advanced-topics/recovery/#transaction-failure which makes use of

Hi Sai, In your last post a line with ConfiguredGraphFactory.createConfiguration(new MapConfiguration(map)); is missing. A complete working transcript that works out of the box from the

Hi Boxuan, thank you for the detailed response. What if inconsistency between my primary storage and my indexing backend is not tolerable? Can I somehow make Janusgraph respect that and fail the

Hi Florian, JanusGraph's philosophy is that your primary storage (ScyllaDB in your case) is the primary and authoritative source of truth, and inconsistency between your mixed index backend and

Hi Marc, I have mentioned all the properties in the config file, i am not sure why the configurations are not applied when grem server is restarted. gremlin> :remote connect tinkerpop.server

Hi Marc, I tried commenting it out and setting it to false but i got the same error message. gremlin> ConfiguredGraphFactory.createConfiguration(new MapConfiguration(map)); Must provide vertex id Type

Hi, what is the recommended approach for backing up the Janusgraph storage layer (ScyllaDB in my case) together with a mixed index backend (Elasticsearch)? I know I can back up & restore them

Hi Sai, I suspect this is related to your setting: #do not auto generate graph vertex id graph.set-vertex-id=true Can you try without? Best wishes, Marc

Hi Joe, just as Boxuan already said, the cache size is crucial for this task. But assuming your graph is large, only a fraction of the vertices will fit into the cache even if scaled appropriately.

Hi, This is the gremlin-server.yaml file # Copyright 2019 JanusGraph Authors # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with

Hi Joe, Vertex properties are indeed cached both in DB cache and transaction cache. If you check out https://docs.janusgraph.org/advanced-topics/data-model/, you will find that the doc says, Thus, I