1. Janusgraph-Dev
  2. Topics

The memory can only be add() during vertex program execute

anjani...@...
 

Hi All,

We have a custom vertex program which find connected nodes. We have some unique requirement to remove some duplicate data, for that we are collecting all data using collectAsMap() method.
But getting below mentioned error when load is more. For low load it works fine. 


Data Store: Cassandra
Index Store: Elasticsearch

Data :  ~300M Nodes & ~371M Edges. 
Spark Cluster: 100 Executors, 7 Cores



Job aborted due to stage failure: Task 199 in stage 14.0 failed 4 times, most recent failure: Lost task 199.3 in stage 14.0 (TID 9825, <ip>, executor 97): java.lang.IllegalArgumentException: The memory can only be add() during vertex program execute: gremlin.connectedNodeVertexProgram.voteToHalt
	at org.apache.tinkerpop.gremlin.process.computer.Memory$Exceptions.memoryAddOnlyDuringVertexProgramExecute(Memory.java:161)



I am not able to figure out why this exception is thrown, i don't see any OOM error. Please share thought/pointers.


Appreciate you help.


Thanks,

Anjani
Oleksandr Porunov <alexand...@...>
 

Hi Anjani,

This is JanusGraph developers channel to discuss JanusGraph development process. You can get help with your issue in the next channel: https://groups.google.com/forum/#!forum/janusgraph-users


On Wednesday, April 29, 2020 at 10:27:53 AM UTC-7, anj...@... wrote:
Hi All,

We have a custom vertex program which find connected nodes. We have some unique requirement to remove some duplicate data, for that we are collecting all data using collectAsMap() method.
But getting below mentioned error when load is more. For low load it works fine. 


Data Store: Cassandra
Index Store: Elasticsearch

Data :  ~300M Nodes & ~371M Edges. 
Spark Cluster: 100 Executors, 7 Cores



Job aborted due to stage failure: Task 199 in stage 14.0 failed 4 times, most recent failure: Lost task 199.3 in stage 14.0 (TID 9825, <ip>, executor 97): java.lang.IllegalArgumentException: The memory can only be add() during vertex program execute: gremlin.connectedNodeVertexProgram.voteToHalt
	at org.apache.tinkerpop.gremlin.process.computer.Memory$Exceptions.memoryAddOnlyDuringVertexProgramExecute(Memory.java:161)



I am not able to figure out why this exception is thrown, i don't see any OOM error. Please share thought/pointers.


Appreciate you help.


Thanks,

Anjani

1 - 2 of 2