- Compatiblity with Spark 2.3
Re: Compatiblity with Spark 2.3
Juraj Polačok <polaco...@...>
toggle quoted messageShow quoted text
Currently, I am getting this error:
java.lang.ClassCastException: org.apache.hadoop.yarn.proto.YarnServiceProtos$GetNewApplicationRequestProto cannot be cast to org.apache.hadoop.hbase.shaded.com.google.protobuf.Message
TinkerPop 3.4+ should support Apache Spark 2.3, so I think it should work. Have you tried running some queries using Spark 2.3?
On Wednesday, 24 April 2019 15:11:26 UTC+2, HadoopMarc wrote:
JanusGraph/TinkerPop have all necessary Spark dependencies included in their distributions, so Spark compatibility on Spark/Yarn is not an issue as long as you keep the Spark jars of the Yarn cluster from the various CLASSPATHS involved. You can get the general idea by combining:
What storage backend do you use for JanusGraph? SparkGraphComputer is known to be painfully slow on JanusGraph/HBase (although I am not sure whether reading from the HBase Snaphot feature was ever implemented in JanusGraph, have not ever tried that).
Op woensdag 24 april 2019 12:42:17 UTC+2 schreef Evgeniy Ignatiev:
As far as I remember, Spark compatibility is dictated solely by
the corresponding spark-gremlin module from Tinkerpop and 3.3.x
release train officially supports only 2.2.
Maybe it is worth duplicating this question to the gremlin-users
list? The change that lifted Spark support to 2.3 wasn't large
though - https://github.com/apache/tinkerpop/pull/886
it will probably work out of the box with properly fixed Netty
is it possible to run olap queries using SparkGraphComputer
via YARN, with Apache Spark 2.3.2 ?
You received this message because you are subscribed to the Google
Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to janu...@....
To post to this group, send email to janu...@....
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/1a4414e8-abde-4432-8a18-94d9a8ed65c4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Join email@example.com to automatically receive all group messages.