|
Re: Support for DB cache for Multi Node Janus Server Setup
Hi Pasan,
The multiple janusgraph nodes share the same storage backend, so the common caching is done in the storage backend.
Best wishes, Marc
Hi Pasan,
The multiple janusgraph nodes share the same storage backend, so the common caching is done in the storage backend.
Best wishes, Marc
|
By
hadoopmarc@...
·
#5877
·
|
|
Support for DB cache for Multi Node Janus Server Setup
Hi All,
I would like to understand the possibility of horizontal scaling of janusgraph servers while keeping the cache enabled. Based on the janusgraph document -
Hi All,
I would like to understand the possibility of horizontal scaling of janusgraph servers while keeping the cache enabled. Based on the janusgraph document -
|
By
pasansumanathilake@...
·
#5876
·
|
|
Re: Not able to run queries using spark graph computer from java
Hi Marc,
Sorry my bad I have posted the wrong code.
I used Graph graph = GraphFactory.open("read-cql.properties");
and i got the above error.
Thanks
Sai
Hi Marc,
Sorry my bad I have posted the wrong code.
I used Graph graph = GraphFactory.open("read-cql.properties");
and i got the above error.
Thanks
Sai
|
By
Sai Supraj R
·
#5875
·
|
|
Re: Not able to run queries using spark graph computer from java
Hi Sai,
The calling code you present is not complete.
The first line should read (because HadoopGraph does not derive from JanusGraph):
Graph graph = GraphFactory.open("read-cql.properties");Best
Hi Sai,
The calling code you present is not complete.
The first line should read (because HadoopGraph does not derive from JanusGraph):
Graph graph = GraphFactory.open("read-cql.properties");Best
|
By
hadoopmarc@...
·
#5874
·
|
|
Re: olap connection with spark standalone cluster
Hi Sai,
This exception is not really related to this thread.
JanusGraph with SparkGraphComputer can only be used with the TinkerPop HadoopGraph. Therefore, the example in the JanusGraph ref docs has a
Hi Sai,
This exception is not really related to this thread.
JanusGraph with SparkGraphComputer can only be used with the TinkerPop HadoopGraph. Therefore, the example in the JanusGraph ref docs has a
|
By
hadoopmarc@...
·
#5873
·
|
|
Not able to run queries using spark graph computer from java
Hi,
I am getting the following error when running queries using spark graph computer from java.
Caused by: java.lang.RuntimeException: java.lang.IllegalArgumentException: Edge with id already exists:
Hi,
I am getting the following error when running queries using spark graph computer from java.
Caused by: java.lang.RuntimeException: java.lang.IllegalArgumentException: Edge with id already exists:
|
By
Sai Supraj R
·
#5872
·
|
|
Re: olap connection with spark standalone cluster
HI I tried with the above solution but it is still throwing error :
java.lang.Throwable: Hook creation trace
at org.janusgraph.graphdb.database.StandardJanusGraph.<init>(StandardJanusGraph.java:185)
HI I tried with the above solution but it is still throwing error :
java.lang.Throwable: Hook creation trace
at org.janusgraph.graphdb.database.StandardJanusGraph.<init>(StandardJanusGraph.java:185)
|
By
Sai Supraj R
·
#5871
·
|
|
Re: Any advice on performance concern of JanusGraph with Cassandra&Elastic Search?
Many organizations use JanusGraph on this scale. Insertion of data is slow so you need massive parallel operations to do an entire bulk load overnight. Most people use tools like Apache Spark for
Many organizations use JanusGraph on this scale. Insertion of data is slow so you need massive parallel operations to do an entire bulk load overnight. Most people use tools like Apache Spark for
|
By
hadoopmarc@...
·
#5870
·
|
|
Re: Any advice on performance concern of JanusGraph with Cassandra&Elastic Search?
+ I want to add one more question.
What about ScyllaDB as storage backend? Is it better to use this in terms of performance?
+ I want to add one more question.
What about ScyllaDB as storage backend? Is it better to use this in terms of performance?
|
By
hazalkecoglu@...
·
#5869
·
|
|
Any advice on performance concern of JanusGraph with Cassandra&Elastic Search?
Hi everyone,
We are working on a project that we would like to use JanusGraph. Our system will consist of 100MM nodes and 1B edges between those nodes.
We are going to work with last 90 days
Hi everyone,
We are working on a project that we would like to use JanusGraph. Our system will consist of 100MM nodes and 1B edges between those nodes.
We are going to work with last 90 days
|
By
hazalkecoglu@...
·
#5868
·
|
|
Re: Backup & Restore of Janusgraph Data with Mixed Index Backend (Elastisearch)
If your use case can handle the downtime, stopping writes and waiting until all changes are propagated to both the storage and the index backend sounds like a viable solution. However, I have no idea
If your use case can handle the downtime, stopping writes and waiting until all changes are propagated to both the storage and the index backend sounds like a viable solution. However, I have no idea
|
By
rngcntr
·
#5867
·
|
|
Re: Backup & Restore of Janusgraph Data with Mixed Index Backend (Elastisearch)
Yeah, good point, it's a bit hairy. Having potentially inconsistent index backups makes them much less attractive. Though I guess I could run a reindex job on just the delta since last Scylla write
Yeah, good point, it's a bit hairy. Having potentially inconsistent index backups makes them much less attractive. Though I guess I could run a reindex job on just the delta since last Scylla write
|
By
florian.caesar
·
#5866
·
|
|
Re: Backup & Restore of Janusgraph Data with Mixed Index Backend (Elastisearch)
Although the solution presented by Marc is also the closest to a consistent backup that I can think of, there are obviously caveats to it. Updates of values which were written after the time of the
Although the solution presented by Marc is also the closest to a consistent backup that I can think of, there are obviously caveats to it. Updates of values which were written after the time of the
|
By
rngcntr
·
#5865
·
|
|
Re: Backup & Restore of Janusgraph Data with Mixed Index Backend (Elastisearch)
Awesome, yes, that's very similar to what I was planning!
It's not perfect and definitely needs to tested thoroughly, but it should be much faster and reasonably scriptable.
I'll let you all know how
Awesome, yes, that's very similar to what I was planning!
It's not perfect and definitely needs to tested thoroughly, but it should be much faster and reasonably scriptable.
I'll let you all know how
|
By
florian.caesar
·
#5864
·
|
|
Re: Backup & Restore of Janusgraph Data with Mixed Index Backend (Elastisearch)
In theory (not used in practice) the following should be possible:
make a snapshot of the ScyllaDB keyspace
after the ScyllaDB snapshot is written, make a snapshot of corresponding ES mixed
In theory (not used in practice) the following should be possible:
make a snapshot of the ScyllaDB keyspace
after the ScyllaDB snapshot is written, make a snapshot of corresponding ES mixed
|
By
hadoopmarc@...
·
#5863
·
|
|
Re: Backup & Restore of Janusgraph Data with Mixed Index Backend (Elastisearch)
Thanks again. Yeah, might end up doing that, but it seems like a complicated solution.. hmm.
Regarding the feature request, I'll dig into the code and ask around the janusgraph-dev group :)
Thanks again. Yeah, might end up doing that, but it seems like a complicated solution.. hmm.
Regarding the feature request, I'll dig into the code and ask around the janusgraph-dev group :)
|
By
florian.caesar
·
#5862
·
|
|
Re: Backup & Restore of Janusgraph Data with Mixed Index Backend (Elastisearch)
Yeah, reindexing can be slow in that case. You could try the transaction recovery mechanism as described in https://docs.janusgraph.org/advanced-topics/recovery/#transaction-failure which makes use of
Yeah, reindexing can be slow in that case. You could try the transaction recovery mechanism as described in https://docs.janusgraph.org/advanced-topics/recovery/#transaction-failure which makes use of
|
By
Boxuan Li
·
#5861
·
|
|
Re: Configured graph factory not working after making changes to gremlin-server.yaml
Hi Sai,
In your last post a line with ConfiguredGraphFactory.createConfiguration(new MapConfiguration(map)); is missing.
A complete working transcript that works out of the box from the
Hi Sai,
In your last post a line with ConfiguredGraphFactory.createConfiguration(new MapConfiguration(map)); is missing.
A complete working transcript that works out of the box from the
|
By
hadoopmarc@...
·
#5860
·
|
|
Re: Backup & Restore of Janusgraph Data with Mixed Index Backend (Elastisearch)
Hi Boxuan,
thank you for the detailed response.
What if inconsistency between my primary storage and my indexing backend is not tolerable? Can I somehow make Janusgraph respect that and fail the
Hi Boxuan,
thank you for the detailed response.
What if inconsistency between my primary storage and my indexing backend is not tolerable? Can I somehow make Janusgraph respect that and fail the
|
By
florian.caesar
·
#5859
·
|
|
Re: Backup & Restore of Janusgraph Data with Mixed Index Backend (Elastisearch)
Hi Florian,
JanusGraph's philosophy is that your primary storage (ScyllaDB in your case) is the primary and authoritative source of truth, and inconsistency between your mixed index backend and
Hi Florian,
JanusGraph's philosophy is that your primary storage (ScyllaDB in your case) is the primary and authoritative source of truth, and inconsistency between your mixed index backend and
|
By
Boxuan Li
·
#5858
·
|