Thanks a lot for responding Marc. Yes, I had initially tried setting spark.yarn.archive with the path to spark-gremlin.zip. However with this approach, the containers were failing with the message

Hi Manabu, Yes, providing an example graph works much better in exploring the problem space. I am afraid, though, that I did not find much that will help you out. on a single machine with cassandra

In the mean time I found that the difference between profile() and currentTimeMillis can be much larger. Apparently, the profile() step takes into account that for real queries, vertices are not

Hi, Can anyone explain why the total duration displayed by the profile() step is more than twice as large as the time difference clocked with System.currentTimeMillis? see below, For those who wonder,

Hi Roy, I think I would first check whether the skew is absent if you count the rows reading the HBase table directly from spark (so, without using janusgraph),

Hi Varun, Good job. However, your last solution will only work with everything running on a single machine. So, indeed, there is something wrong with the contents of spark-gremlin.zip or with the way

Thank you Marc. I was able to reduce the tasks by adjusting the `num_tokens` settings on Cassandra. Still unsure about why each task takes so long though. Hoping that this a per-task overhead that

Answering my own question. I was able fix the above error and successfully run the count job after explicitly adding /Users/my_comp/Downloads/janusgraph-0.5.2/lib/* to

An update on this, I tried setting the env var below: export HADOOP_GREMLIN_LIBS=$GREMLIN_HOME/lib After doing this I was able to successfully run the tinkerpop-modern.kryo example from the Recipes

Thanks for sharing! I personally only use MapReduce and not sure if there is an existing solution for Spark. > if there is any danger in opening multiple separate graph instances and using them to

A slight correction and clarification of my previous post - the total number of partitions/splits is exactly equal to total_number_of_tokens + 1. In a 3-node cassandra cluster where each node has 256

Hello, I am trying to run SparkGraphComputer on a JanusGraph backed by Cassandra and ElasticSearch. I have previously verified that I am able to run SparkGraphComputer on a local Spark standalone

JanusGraph documentation: https://docs.janusgraph.org/index-management/index-performance/ is describing usage of Vertex Centrix Index [edge=battled +

Hello Boxuan, I need to support reindexing very large graphs. To my knowledge, the only feasible way that's supported is via the `MapReduceIndexManagement` class. This is not ideal for me as I'd like

Hi Marc, The parameter hbase.mapreduce.tableinput.mappers.per.region can be effective. I set it to 40, and there are 40 tasks processing every region. But here comes the new promblem--the data skew.

Hi Mladen, Agree with Marc, that's something you could try. If possible, could you share the reason why you have to open the same graph multiple times with different graph objects? If there is no

Hello, I am currently using Janusgraph version 0.5.2. I have a graph with about 18 million vertices and 25 million edges. I have two versions of this graph, one backed by a 3 node Cassandra cluster

Hi Sandeep, I think I have already added below line to indicate that it should pull the detail from now onwords in processor. Is it not working? "setStartTimeNow()" Is anyone other face the same

You could end your script with: System.exit(0) HTH, Marc Op woensdag 9 december 2020 om 04:16:43 UTC+1 schreef Phate:

Hi Mladen, The constructor of StandardJanusGraph seems worth a