Re: Can BulkLoaderVertexProgram also add mixed indexes
mystic m <mita...@...>
toggle quoted messageShow quoted text
Thanks Marc, your blog post is helpful.
I started the set-up from scratch but I did replace/added distribution specific jars for hadoop and hbase to be able to interact with maprfs and mapr-db.
Also I was able to get rid of MapR spark-assembly from gremlin CLASSPATH by placing it in hdfs and adding spark-yarn jar to gremlin CLASSPATH. This lets me submit the spark job on yarn. I added the janusgraph-hbase jar and spark-gremlin jars as you have specified in the blog, when spark job starts the jars are copied appropriately to the staging area in hdfs but still I get below listed exception, in last setup I had copied hadoop-gremlin-libs in SPARK_LIB directory across the cluster to resolve the issue, I am not sure why they are not picked from hdfs directory, I will debug this more tomorrow and post back.
java.lang.ClassCastException: cannot assign instance of java.lang.invoke.SerializedLambda to field org.apache.spark.api.java.JavaPairRDD$$anonfun$pairFunToScalaFun$1.x$330 of type org.apache.spark.api.java.function.PairFunction in instance of org.apache.spark.api.java.JavaPairRDD$$anonfun$pairFunToScalaFun$1
On Thursday, August 24, 2017 at 12:30:58 AM UTC+5:30, HadoopMarc wrote: