about janusgraph use spark(yarn-client) compute


liuzhip...@...
 

1、configuration " conf/hadoop-graph/read-cassandra.properties "  file ,as follow:
---------------------------------------------------------------------------------------------------
gremlin.graph=org.apache.tinkerpop.gremlin.hadoop.structure.HadoopGraph
gremlin.hadoop.graphInputFormat=org.janusgraph.hadoop.formats.hbase.HBaseInputFormat
gremlin.hadoop.graphOutputFormat=org.apache.tinkerpop.gremlin.hadoop.structure.io.gryo.GryoOutputFormat

gremlin.hadoop.jarsInDistributedCache=true
gremlin.hadoop.inputLocation=none
gremlin.hadoop.outputLocation=output

janusgraphmr.ioformat.conf.storage.backend=hbase
janusgraphmr.ioformat.conf.storage.hostname=testhadoop001.ppdapi.com,testhadoop002.ppdapi.com,testhadoop003.ppdapi.com
janusgraphmr.ioformat.conf.storage.keyspace=janusgraph

spark.master=yarn-client
spark.serializer=org.apache.spark.serializer.KryoSerializer
spark.yarn.services=org.apache.spark.deploy.yarn.history.YarnHistoryService
spark.yarn.historyServer.address=http://testhadoop002.ppdapi.com:18088
spark.history.provider=org.apache.spark.deploy.yarn.history.YarnHistoryProvider
spark.history.ui.port=18088
gremlin.spark.persistContext=true

2、start gremlin.sh,problem is here





view the spark task by web-ui  
it seems to miss a jar package, but i put guava-16.0.1.jar into the lib




Join janusgraph-users@lists.lfaidata.foundation to automatically receive all group messages.