Re: Strange behaviors for Janusgraph 0.5.3 on AWS EMR
hadoopmarc@...
Hi Alessandro,
I assume Amazon EMR uses hadoop-yarn, so you need to specify spark.master = yarn, see: https://tinkerpop.apache.org/docs/current/recipes/#olap-spark-yarn Once you can run the TinkerPop example, you can try and switch to janusgraph. You have to realize that janusgraph does not do a good job (yet) in partitioning the input data from a storage backend. Basically, when using cql, you get the partitions as used by Cassandra. So with 1 or 2 spark partitions, there is no need to fire 90 executors. Best wishes, Marc
|
|