Re: Tinkerpop 3.4.1 with Hadoop3
Hi Anjani,
JanusGraph receives the hadoop dependency from Apache TinkerPop, so Apache TinkerPop will be in the driver seat regarding the upgrade of hadoop. Making a custom build of janusgraph with hadoop3 will be very time consuming because many library conflicts exist, which need to be resolved manually and tested. If you try, be sure to start from a janusgraph branch that uses TinkerPop 3.5.x with spark 3.0 (so janusgraph master or the future 0.6.0) to minimize the chances of library conflicts.
After a quick googling session, I find that hadoop3 services (resource manager, name node) support hadoop2 clients to a certain extent. What errors do you get when trying janusgraph-0.5.3 on a hadoop3 cluster? Note that TinkerPop/Janusgraph are not shipped with hadoop-yarn, see: https://tinkerpop.apache.org/docs/current/recipes/#olap-spark-yarn
Best wishes, Marc
[Edited] Adding missing spark-yarn and hadoop-yarn jars is described here.
JanusGraph receives the hadoop dependency from Apache TinkerPop, so Apache TinkerPop will be in the driver seat regarding the upgrade of hadoop. Making a custom build of janusgraph with hadoop3 will be very time consuming because many library conflicts exist, which need to be resolved manually and tested. If you try, be sure to start from a janusgraph branch that uses TinkerPop 3.5.x with spark 3.0 (so janusgraph master or the future 0.6.0) to minimize the chances of library conflicts.
After a quick googling session, I find that hadoop3 services (resource manager, name node) support hadoop2 clients to a certain extent. What errors do you get when trying janusgraph-0.5.3 on a hadoop3 cluster? Note that TinkerPop/Janusgraph are not shipped with hadoop-yarn, see: https://tinkerpop.apache.org/docs/current/recipes/#olap-spark-yarn
Best wishes, Marc
[Edited] Adding missing spark-yarn and hadoop-yarn jars is described here.