Re: Strange behaviors for Janusgraph 0.5.3 on AWS EMR


hadoopmarc@...
 

Hi Alessandro,

I assume Amazon EMR uses hadoop-yarn, so you need to specify spark.master  = yarn, see:
https://tinkerpop.apache.org/docs/current/recipes/#olap-spark-yarn

Once you can run the TinkerPop example, you can try and switch to janusgraph. You have to realize that janusgraph does not do a good job (yet) in partitioning the input data from a storage backend. Basically, when using cql, you get the partitions as used by Cassandra. So with 1 or 2 spark partitions, there is no need to fire 90 executors.

Best wishes,    Marc

Join janusgraph-users@lists.lfaidata.foundation to automatically receive all group messages.