Date
1 - 2 of 2
GryoRegistrator class not found exception
|
Real Life Adventure <srinu....@...>
Hi All, when i am running spark query g.V().count() from gremlin console facing below error. gremlin> :remote connect tinkerpop.server conf/remote.yaml session ==>Configured jce-janusgraph/172.19.0.2:8182-[bf331c15-12fb-4283-90da-ef3d81311022] gremlin> :remote console ==>All scripts will now be sent to Gremlin Server - [jce-janusgraph/172.19.0.2:8182]-[bf331c15-12fb-4283-90da-ef3d81311022] - type ':remote console' to return to local mode gremlin> graph1 = GraphFactory.open('conf/hadoop-graph/hadoop-load.properties') ==>hadoopgraph[gryoinputformat->nulloutputformat] gremlin> g1 = graph1.traversal().withComputer(org.apache.tinkerpop.gremlin.spark.process.computer.SparkGraphComputer) ==>graphtraversalsource[hadoopgraph[gryoinputformat->nulloutputformat], sparkgraphcomputer] gremlin> g1.V() org.apache.tinkerpop.gremlin.spark.structure.io.gryo.GryoRegistrator Type ':help' or ':h' for help. Display stack trace? [yN] gremlin> g1.V() org.apache.tinkerpop.gremlin.spark.structure.io.gryo.GryoRegistrator Type ':help' or ':h' for help. Display stack trace? [yN]y java.lang.ClassNotFoundException: org.apache.tinkerpop.gremlin.spark.structure.io.gryo.GryoRegistrator at java.net.URLClassLoader.findClass(URLClassLoader.java:382) at java.lang.ClassLoader.loadClass(ClassLoader.java:419) at java.lang.ClassLoader.loadClass(ClassLoader.java:352) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:348) at org.apache.spark.serializer.KryoSerializer$$anonfun$newKryo$5.apply(KryoSerializer.scala:135) at org.apache.spark.serializer.KryoSerializer$$anonfun$newKryo$5.apply(KryoSerializer.scala:135) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234) at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33) at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:186) at scala.collection.TraversableLike$class.map(TraversableLike.scala:234) at scala.collection.mutable.ArrayOps$ofRef.map(ArrayOps.scala:186) at org.apache.spark.serializer.KryoSerializer.newKryo(KryoSerializer.scala:135) at org.apache.spark.serializer.KryoSerializerInstance.borrowKryo(KryoSerializer.scala:328) at org.apache.spark.serializer.KryoSerializerInstance.<init>(KryoSerializer.scala:313) at org.apache.spark.serializer.KryoSerializer.newInstance(KryoSerializer.scala:222) at org.apache.spark.broadcast.TorrentBroadcast$.unBlockifyObject(TorrentBroadcast.scala:305) at org.apache.spark.broadcast.TorrentBroadcast$$anonfun$readBroadcastBlock$1$$anonfun$apply$2.apply(TorrentBroadcast.scala:235) at scala.Option.getOrElse(Option.scala:121) at org.apache.spark.broadcast.TorrentBroadcast$$anonfun$readBroadcastBlock$1.apply(TorrentBroadcast.scala:211) at org.apache.spark.util.Utils$.tryOrIOException(Utils.scala:1326) at org.apache.spark.broadcast.TorrentBroadcast.readBroadcastBlock(TorrentBroadcast.scala:207) at org.apache.spark.broadcast.TorrentBroadcast._value$lzycompute(TorrentBroadcast.scala:66) at org.apache.spark.broadcast.TorrentBroadcast._value(TorrentBroadcast.scala:66) at org.apache.spark.broadcast.TorrentBroadcast.getValue(TorrentBroadcast.scala:96) at org.apache.spark.broadcast.Broadcast.value(Broadcast.scala:70) at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:89) at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:55) at org.apache.spark.scheduler.Task.run(Task.scala:123) at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408) at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) gremlin> Any help Appreciated. Thanks, RLA. |
|
|
|
HadoopMarc <bi...@...>
Funny thing is: I replayed the scenario you described above on a freshly unzipped janusgraph-0.5.2 (with bin/gremlin_server.sh and bin/gremlin.sh in two different terminals) and ... it just worked. That is, this used spark.master = local[*] and had hadoop fall back to the local file system. In what way does your config differ from this one? Somehow, you do not have the spark-gremlin jar on the gremlin server or spark executor classpath. If you have a stand-alone Spark cluster, see the notes in the ref docs on distributing the required jars and updating the executor classpath: https://docs.janusgraph.org/advanced-topics/hadoop/#olap-traversals-with-spark-standalone-cluster HTH, Marc Op vrijdag 26 juni 2020 om 13:39:44 UTC+2 schreef sri...@...:
|
|
|