1. Janusgraph-Users
  2. Topics

spark operations with dymic graphs

Real Life Adventure <srinu....@...>
 

Hi,
               How to do spark traversal on dynamically created graphs created with configuredgraphfactory .
               I dont find any documentation on that.
               i found only with GrapgFactory 
graph = GraphFactory.open('conf/hadoop-graph/read-cql-standalone-cluster.properties')
g = graph.traversal().withComputer(SparkGraphComputer)
g.V().count()
Any help Appreciated.

Thanks,
RLA.
HadoopMarc <bi...@...>
 


This describes how to obtain a GraphTraversalSource instance, like g1, for a configured graph, like graph1:

Best wishes,     Marc

Op vrijdag 19 juni 2020 08:14:24 UTC+2 schreef Real Life Adventure:

Hi,
               How to do spark traversal on dynamically created graphs created with configuredgraphfactory .
               I dont find any documentation on that.
               i found only with GrapgFactory 
graph = GraphFactory.open('conf/hadoop-graph/read-cql-standalone-cluster.properties')
g = graph.traversal().withComputer(SparkGraphComputer)
g.V().count()
Any help Appreciated.

Thanks,
RLA.
Real Life Adventure <srinu....@...>
 

Thanks for the reply.
          iam able to get instance.but how to set spark config in configuredgraphfactory template.


On Saturday, June 20, 2020 at 2:22:33 PM UTC+5:30, HadoopMarc wrote:

This describes how to obtain a GraphTraversalSource instance, like g1, for a configured graph, like graph1:

Best wishes,     Marc

Op vrijdag 19 juni 2020 08:14:24 UTC+2 schreef Real Life Adventure:
Hi,
               How to do spark traversal on dynamically created graphs created with configuredgraphfactory .
               I dont find any documentation on that.
               i found only with GrapgFactory 
graph = GraphFactory.open('conf/hadoop-graph/read-cql-standalone-cluster.properties')
g = graph.traversal().withComputer(SparkGraphComputer)
g.V().count()
Any help Appreciated.

Thanks,
RLA.
HadoopMarc <bi...@...>
 


I see what you mean. The problem is that Gremlin Server has to serve a HadoopGraph to be able to do an OLAP query with SparkGraphComputer. However, as the name suggests, JanusGraphFactory.open() always returns a StandardJanusGraph instance. So, for doing a remote OLAP query on JanusGraph you are left with the following choices:
  1. configure a HadoopGraph on Gremlin Server in a static way
  2. use withComputer() on a configured JanusGraph, that is without SparkGraphComputer. This is only useful if Gremlin Server runs on a host with a lot of cores and memory. You will also have to scan the TinkerPop ref docs how to set the number of workers on the DefaultGraphComputer from the remote client.
HTH,    Marc

Op zondag 21 juni 2020 11:22:16 UTC+2 schreef Real Life Adventure:

Thanks for the reply.
          iam able to get instance.but how to set spark config in configuredgraphfactory template.


On Saturday, June 20, 2020 at 2:22:33 PM UTC+5:30, HadoopMarc wrote:

This describes how to obtain a GraphTraversalSource instance, like g1, for a configured graph, like graph1:

Best wishes,     Marc

Op vrijdag 19 juni 2020 08:14:24 UTC+2 schreef Real Life Adventure:
Hi,
               How to do spark traversal on dynamically created graphs created with configuredgraphfactory .
               I dont find any documentation on that.
               i found only with GrapgFactory 
graph = GraphFactory.open('conf/hadoop-graph/read-cql-standalone-cluster.properties')
g = graph.traversal().withComputer(SparkGraphComputer)
g.V().count()
Any help Appreciated.

Thanks,
RLA.

1 - 4 of 4