Any Thoughts on Hadoop Distribution Builds?


TheAjitator <sam...@...>
 

Is there any interest in building versions that support some of the popular Hadoop distributions?

I ask because the Cloudera version 5.x.x for HBase 1.0.x doesn't have the same binary interface as the Apache version.  Specifically there are some calls that now return the object in a "fluent" style in the Apache version that aren't doing so in the Cloudera versions.  At runtime this causes an exception to be raised by the JVM since the signature of the methods aren't actually the same (different returns, one is void the other is the object).

Thanks.


Jerry He <jerr...@...>
 

Could you be more specific to the issue you are facing?

We normally would not want to adapt to support a specific vendor distribution. 
But vendor distributions are welcome to adapt and include JanusGraph in their distributions.

Thanks.

Jerry


On Friday, April 14, 2017 at 2:31:54 PM UTC-7, TheAjitator wrote:
Is there any interest in building versions that support some of the popular Hadoop distributions?

I ask because the Cloudera version 5.x.x for HBase 1.0.x doesn't have the same binary interface as the Apache version.  Specifically there are some calls that now return the object in a "fluent" style in the Apache version that aren't doing so in the Cloudera versions.  At runtime this causes an exception to be raised by the JVM since the signature of the methods aren't actually the same (different returns, one is void the other is the object).

Thanks.


HadoopMarc <m.c.d...@...>
 

Hi Jerry and others,

The problem is that JanusGraph test suites only use local Spark and backend installations. Connecting to an existing cluster gives all kinds of version conflicts. I did not try for JanusGraph rcx releases yet, but for the Titan graben1437 PR you had to exclude some jars from the HBase libs and I have never had SparkGraphComputer working in combination with the spark-assembly jar for Yarn from Hortonworks.

While I agree that JanusGraph cannot be responsible for maintaining compatibility with Hadoop disitributions, it will be certainly necessary to make JanusGraph "installation friendly for Hadoop distributions" and to have users share knowledge about particulat solutions for particular distributions.

Cheers,    Marc

Op zaterdag 15 april 2017 00:19:10 UTC+2 schreef Jerry He:

Could you be more specific to the issue you are facing?

We normally would not want to adapt to support a specific vendor distribution. 
But vendor distributions are welcome to adapt and include JanusGraph in their distributions.

Thanks.

Jerry


On Friday, April 14, 2017 at 2:31:54 PM UTC-7, TheAjitator wrote:
Is there any interest in building versions that support some of the popular Hadoop distributions?

I ask because the Cloudera version 5.x.x for HBase 1.0.x doesn't have the same binary interface as the Apache version.  Specifically there are some calls that now return the object in a "fluent" style in the Apache version that aren't doing so in the Cloudera versions.  At runtime this causes an exception to be raised by the JVM since the signature of the methods aren't actually the same (different returns, one is void the other is the object).

Thanks.