Re: Janusgraph with YARN and HBASE

Fábio Dapper <fda...@...>

That's it!
Thank you, very much!!!

Em qui., 23 de jul. de 2020 às 05:20, Petr Stentor <kiri...@...> escreveu:


Try this

четверг, 23 июля 2020 г., 1:57:38 UTC+3 пользователь Fábio Dapper написал:
Hello, we have a Cluster with CLOUDERA CDH 6.3.2 and I'm trying to run Janusgraph on the Cluster with YARN and HBASE, but without success.
(it's OK with SPARK Local)

Version SPARK 2.4.2
HBASE: 2.1.0-cdh6.3.2
Janusgraph (v 0.5.2 and v0.4.1)

I did a lot of searching, but I didn't find any recent references, and they all use older versions of SPARK and Janusgraph.

Some examples:

According to these references, I followed the following steps:

  1. Copy the following files to the Janusgraph "lib" directory:
    1. spark-yarn-2.11-2.4.0.jar
    2. scala-reflect-2.10.5.jar
    3. hadoop-yarn-server-web-proxy-2.7.2.jar
    4. guice-servlet-3.0.jar
  2. Generate a "/tmp/" file containing all the .jar files from "janusgraph / lib /".
  3. Create a configuration file called '' from conf/hadoop-graph/ by adding (or modifying) the properties below: 
spark.master= yarn


Then I ran the following commands:
    graph =
    = graph.traversal().withComputer(SparkGraphComputer)
Can someone help me?
a) Are these problems related to version incompatibility?
b) Has anyone successfully used similar infrastructure?
c) Would anyone know how to determine a correct version of the necessary libraries?
d) Any suggestion?

Thank you all !!!

 Below is a copy of the Yarn Log from my last attempt.

ERROR org.apache.spark.scheduler.TaskSetManager  - Task 0 in stage 0.0 failed 4 times; aborting job
.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 0.0 failed 4 times, most recent failure: Lost task 0.3 in stage 0.0 (TID 3, [SERVER_NAME], executor 1): java.lang.NoSuchMethodError: net.jpountz.lz4.LZ4BlockInputStream.<init>(Ljava/io/InputStream;Z)V
at org
at org
at org
at scala
at org
at org
at scala
at org
at org
at org
at org
at org
at org
at org
at org
at org
at org
at org
at org
at org
at java
at java
at java

Thank you!!

You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgra...@....
To view this discussion on the web visit

Join to automatically receive all group messages.