Date
1 - 2 of 2
ERROR org.apache.spark.scheduler.cluster.StandaloneSchedulerBackend - Application has been killed. Reason: All masters are unresponsive! Giving up.
Vinayak Bali
Hi All, gremlin> graph = GraphFactory.open('conf/hadoop-graph/read-cql-standalone-cluster.properties') ==>hadoopgraph[cqlinputformat->nulloutputformat] gremlin> g = graph.traversal().withComputer(SparkGraphComputer) ==>graphtraversalsource[hadoopgraph[cqlinputformat->nulloutputformat], sparkgraphcomputer] gremlin> g.V().has('title','Plant').count() 11:09:18 WARN org.apache.tinkerpop.gremlin.spark.process.computer.SparkGraphComputer - class org.apache.hadoop.mapreduce.lib.output.NullOutputFormat does not implement PersistResultGraphAware and thus, persistence options are unknown -- assuming all options are possible 11:09:20 WARN org.apache.spark.util.Utils - Your hostname, ip-xx-xx-xx-xx resolves to a loopback address: 127.0.0.1; using xx.xx.xx.xx instead (on interface ens5) 11:09:20 WARN org.apache.spark.util.Utils - Set SPARK_LOCAL_IP if you need to bind to another address 11:10:25 ERROR org.apache.spark.scheduler.cluster.StandaloneSchedulerBackend - Application has been killed. Reason: All masters are unresponsive! Giving up. 11:10:25 WARN org.apache.spark.scheduler.cluster.StandaloneSchedulerBackend - Application ID is not initialized yet. 11:10:25 WARN org.apache.spark.deploy.client.StandaloneAppClient$ClientEndpoint - Drop UnregisterApplication(null) because has not yet connected to master 11:10:25 WARN org.apache.spark.metrics.MetricsSystem - Stopping a MetricsSystem that is not running 11:10:26 ERROR org.apache.spark.SparkContext - Error initializing SparkContext. java.lang.NullPointerException at org.apache.spark.SparkContext.<init>(SparkContext.scala:560) at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2520) at org.apache.spark.SparkContext.getOrCreate(SparkContext.scala) at org.apache.tinkerpop.gremlin.spark.structure.Spark.create(Spark.java:52) at org.apache.tinkerpop.gremlin.spark.structure.Spark.create(Spark.java:60) at org.apache.tinkerpop.gremlin.spark.process.computer.SparkGraphComputer.lambda$submitWithExecutor$1(SparkGraphComputer.java:313) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 11:10:26 ERROR org.apache.spark.scheduler.AsyncEventQueue - Listener AppStatusListener threw an exception java.lang.NullPointerException at org.apache.spark.status.AppStatusListener.onApplicationEnd(AppStatusListener.scala:157) at org.apache.spark.scheduler.SparkListenerBus$class.doPostEvent(SparkListenerBus.scala:57) at org.apache.spark.scheduler.AsyncEventQueue.doPostEvent(AsyncEventQueue.scala:37) at org.apache.spark.scheduler.AsyncEventQueue.doPostEvent(AsyncEventQueue.scala:37) at org.apache.spark.util.ListenerBus$class.postToAll(ListenerBus.scala:91) at org.apache.spark.scheduler.AsyncEventQueue.org$apache$spark$scheduler$AsyncEventQueue$$super$postToAll(AsyncEventQueue.scala:92) at org.apache.spark.scheduler.AsyncEventQueue$$anonfun$org$apache$spark$scheduler$AsyncEventQueue$$dispatch$1.apply$mcJ$sp(AsyncEventQueue.scala:92) at org.apache.spark.scheduler.AsyncEventQueue$$anonfun$org$apache$spark$scheduler$AsyncEventQueue$$dispatch$1.apply(AsyncEventQueue.scala:87) at org.apache.spark.scheduler.AsyncEventQueue$$anonfun$org$apache$spark$scheduler$AsyncEventQueue$$dispatch$1.apply(AsyncEventQueue.scala:87) at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58) at org.apache.spark.scheduler.AsyncEventQueue.org$apache$spark$scheduler$AsyncEventQueue$$dispatch(AsyncEventQueue.scala:87) at org.apache.spark.scheduler.AsyncEventQueue$$anon$1$$anonfun$run$1.apply$mcV$sp(AsyncEventQueue.scala:83) at org.apache.spark.util.Utils$.tryOrStopSparkContext(Utils.scala:1302) at org.apache.spark.scheduler.AsyncEventQueue$$anon$1.run(AsyncEventQueue.scala:82) java.lang.NullPointerException Type ':help' or ':h' for help. Display stack trace? [yN] Hadoop: 3.3.0 Spark: 2.2.2 Scala: 2.11.2 Janusgraph: 0.5.2 Referred the below documentation for configuration files: Thanks & Regards, Vinayak |
|
hadoopmarc@...
Hi Vinayak,
Your properties file says: spark.master=spark://127.0.0.1:7077 Do you have a spark standalone cluster running? Does the spark master reside on 127.0.0.1 and does it listen on 7077? With spark on localhost, you can also simply take the "read-cql.properties¨ which uses all cores on localhost for running spark executors. Best wishes, Marc |
|