Re: Strange behaviors for Janusgraph 0.5.3 on AWS EMR


Hi Alessandro,

I assume Amazon EMR uses hadoop-yarn, so you need to specify spark.master  = yarn, see:

Once you can run the TinkerPop example, you can try and switch to janusgraph. You have to realize that janusgraph does not do a good job (yet) in partitioning the input data from a storage backend. Basically, when using cql, you get the partitions as used by Cassandra. So with 1 or 2 spark partitions, there is no need to fire 90 executors.

Best wishes,    Marc

Join to automatically receive all group messages.