|
[Meetup] JanusGraph Meetup May 18 covering JG OLAP approaches
Hello,
We will be hosting a community meetup next week on Tuesday, May 18th at 9:30 central/10:30 eastern. We have a great set of speakers who will be discussing all things JanusGraph OLAP:
* Hadoop
Hello,
We will be hosting a community meetup next week on Tuesday, May 18th at 9:30 central/10:30 eastern. We have a great set of speakers who will be discussing all things JanusGraph OLAP:
* Hadoop
|
By
Ted Wilmes
·
#5908
·
|
|
Re: MapReduce reindexing with authentication
Hi Marc, you are right, we are indeed using this -files option :)
Hi Marc, you are right, we are indeed using this -files option :)
|
By
Boxuan Li
·
#5907
·
|
|
Re: MapReduce reindexing with authentication
Hi Boxuan,
Using existing mechanisms for configuring mapreduce would be nicer, indeed.
Upon reading this hadoop command, I see a GENERIC_OPTIONS env variable read by the mapreduce client, that can
Hi Boxuan,
Using existing mechanisms for configuring mapreduce would be nicer, indeed.
Upon reading this hadoop command, I see a GENERIC_OPTIONS env variable read by the mapreduce client, that can
|
By
hadoopmarc@...
·
#5906
·
|
|
MapReduce reindexing with authentication
We have been using a yarn cluster to run MapReduce reindexing (Cassandra + Elasticsearch) for a long time. Recently, we introduced Kerberos-based authentication to the Elasticsearch cluster, meaning
We have been using a yarn cluster to run MapReduce reindexing (Cassandra + Elasticsearch) for a long time. Recently, we introduced Kerberos-based authentication to the Elasticsearch cluster, meaning
|
By
Boxuan Li
·
#5905
·
|
|
Multiple entries with same key on mixed index
Hello,
I encounter the error describe in the issue #1916 (https://github.com/JanusGraph/janusgraph/issues/1916) on a mixed index (lucene). When I list property keys of the index, they are all
Hello,
I encounter the error describe in the issue #1916 (https://github.com/JanusGraph/janusgraph/issues/1916) on a mixed index (lucene). When I list property keys of the index, they are all
|
By
toom@...
·
#5904
·
|
|
Re: Support for DB cache for Multi Node Janus Server Setup
Hi Marc,
Thanks for the reply. Yeah, it's true that a multi-node setup uses the same storage backend. However, I am referring to here is the Janusgraph database cache -
Hi Marc,
Thanks for the reply. Yeah, it's true that a multi-node setup uses the same storage backend. However, I am referring to here is the Janusgraph database cache -
|
By
pasansumanathilake@...
·
#5903
·
|
|
Storing and reading connected component RDD through OutputFormatRDD & InputFormatRDD
Hi All,
I am using connected component vertex program to find all the connected nodes in graph and then using that RDD for further processing in graph. I want to store that RDD at some output location
Hi All,
I am using connected component vertex program to find all the connected nodes in graph and then using that RDD for further processing in graph. I want to store that RDD at some output location
|
By
anjanisingh22@...
·
#5902
·
Edited
|
|
Re: ID block allocation exception while creating edge
Thanks for help Marc, i will try updating value of ids.num-partitions = number of executors.
Thanks for help Marc, i will try updating value of ids.num-partitions = number of executors.
|
By
anjanisingh22@...
·
#5901
·
|
|
Re: ID block allocation exception while creating edge
Hi Anjani,
It is a while ago I did this myself. I interpret ids.num-partitions as a stock of reserved id blocks that can be delegated to a janugraph instance. It does not have a large value to not
Hi Anjani,
It is a while ago I did this myself. I interpret ids.num-partitions as a stock of reserved id blocks that can be delegated to a janugraph instance. It does not have a large value to not
|
By
hadoopmarc@...
·
#5900
·
|
|
Re: ID block allocation exception while creating edge
We have 250 parallel spark task running for creating node/edges.
I didn't get parallel tasks - (for setting ids.num-partitions)? Could you please help me on it?
We have 250 parallel spark task running for creating node/edges.
I didn't get parallel tasks - (for setting ids.num-partitions)? Could you please help me on it?
|
By
anjanisingh22@...
·
#5899
·
|
|
Re: ID block allocation exception while creating edge
What is the number of parallel tasks? (for setting ids.num-partitions)
You have the ids.authority.wait-time still on its default value of 300 ms, so that seems worthwhile experimenting with.
Best
What is the number of parallel tasks? (for setting ids.num-partitions)
You have the ids.authority.wait-time still on its default value of 300 ms, so that seems worthwhile experimenting with.
Best
|
By
hadoopmarc@...
·
#5898
·
|
|
Re: ID block allocation exception while creating edge
Thanks for response Marc. Below is the method i am using to create janus connection:
public JanusGraph createJanusConnection(HashMap<String, Object> janusConfig) {
JanusGraphFactory.Builder
Thanks for response Marc. Below is the method i am using to create janus connection:
public JanusGraph createJanusConnection(HashMap<String, Object> janusConfig) {
JanusGraphFactory.Builder
|
By
anjanisingh22@...
·
#5897
·
|
|
Re: ID block allocation exception while creating edge
Hi Anjani,
Please show the properties file you use to open janusgraph.
I assume you also saw the other recommendations in
Hi Anjani,
Please show the properties file you use to open janusgraph.
I assume you also saw the other recommendations in
|
By
hadoopmarc@...
·
#5896
·
|
|
ID block allocation exception while creating edge
Hi All,
I am creating vertex and edges in bulk and getting below error while creating edge. Below is the exception log:I tired increasing value of "ids.block-size" but still no luck, even set to 1B
Hi All,
I am creating vertex and edges in bulk and getting below error while creating edge. Below is the exception log:I tired increasing value of "ids.block-size" but still no luck, even set to 1B
|
By
anjanisingh22@...
·
#5895
·
|
|
Re: Query Optimisation
Hi Vinayak,
Actually, query 4 was easier to rework. It could read somewhat like:
g.V().has('property1', 'vertex1').as('v1').outE().has('property1', 'edge1').limit(100).as('e').inV().has('property1',
Hi Vinayak,
Actually, query 4 was easier to rework. It could read somewhat like:
g.V().has('property1', 'vertex1').as('v1').outE().has('property1', 'edge1').limit(100).as('e').inV().has('property1',
|
By
hadoopmarc@...
·
#5894
·
|
|
Re: Query Optimisation
Hi Marc,
Thank you for your reply. I will try to report this issue on janusgraph repository. Regarding the work around you suggested, if possible please share the updated query with work around for
Hi Marc,
Thank you for your reply. I will try to report this issue on janusgraph repository. Regarding the work around you suggested, if possible please share the updated query with work around for
|
By
Vinayak Bali
·
#5893
·
|
|
Re: Query Optimisation
Hi Vinayak,
If you would bother to demonstrate this behavior with a reproducible, generated graph, you can report it as an issue on github.
For now, you can only look for workarounds:
- combine the
Hi Vinayak,
If you would bother to demonstrate this behavior with a reproducible, generated graph, you can report it as an issue on github.
For now, you can only look for workarounds:
- combine the
|
By
hadoopmarc@...
·
#5892
·
|
|
Re: Query Optimisation
Hi Marc,
This query takes 18 sec to run by changing as to aggregate and select to project. But still, 99% of the time is taken to compute union. There is no memory issue, it already set to
Hi Marc,
This query takes 18 sec to run by changing as to aggregate and select to project. But still, 99% of the time is taken to compute union. There is no memory issue, it already set to
|
By
Vinayak Bali
·
#5891
·
|
|
Re: Query Optimisation
Hi Vinayak,
Your last remark explains it well: it seems that in JanusGraph a union of multiple clauses can take much longer than the sum of the individual clauses. There are still two things that we
Hi Vinayak,
Your last remark explains it well: it seems that in JanusGraph a union of multiple clauses can take much longer than the sum of the individual clauses. There are still two things that we
|
By
hadoopmarc@...
·
#5890
·
|
|
Re: Query Optimisation
Hi Marc,
That works as expected. Union also works as expected as in Query1 but when I add limit to all edge the performance degrades.
Thanks
Hi Marc,
That works as expected. Union also works as expected as in Query1 but when I add limit to all edge the performance degrades.
Thanks
|
By
Vinayak Bali
·
#5889
·
|