Date
1 - 2 of 2
The memory can only be add() during vertex program execute
anjani...@...
Hi All,
We have a custom vertex program which find connected nodes. We have some unique requirement to remove some duplicate data, for that we are collecting all data using collectAsMap() method.
But getting below mentioned error when load is more. For low load it works fine.
Data Store: Cassandra
Index Store: Elasticsearch
Data : ~300M Nodes & ~371M Edges.
Spark Cluster: 100 Executors, 7 Cores
Job aborted due to stage failure: Task 199 in stage 14.0 failed 4 times, most recent failure: Lost task 199.3 in stage 14.0 (TID 9825, <ip>, executor 97): java.lang.IllegalArgumentException: The memory can only be add() during vertex program execute: gremlin.connectedNodeVertexProgram.voteToHalt at org.apache.tinkerpop.gremlin.process.computer.Memory$Exceptions.memoryAddOnlyDuringVertexProgramExecute(Memory.java:161)
I am not able to figure out why this exception is thrown, i don't see any OOM error. Please share thought/pointers.
Appreciate you help.
Thanks,
Anjani
Oleksandr Porunov <alexand...@...>
Hi Anjani,
This is JanusGraph developers channel to discuss JanusGraph development process. You can get help with your issue in the next channel: https://groups.google.com/forum/#!forum/janusgraph-users
On Wednesday, April 29, 2020 at 10:27:53 AM UTC-7, anj...@... wrote:
Hi All,We have a custom vertex program which find connected nodes. We have some unique requirement to remove some duplicate data, for that we are collecting all data using collectAsMap() method.But getting below mentioned error when load is more. For low load it works fine.
Data Store: Cassandra
Index Store: Elasticsearch
Data : ~300M Nodes & ~371M Edges.
Spark Cluster: 100 Executors, 7 Cores
Job aborted due to stage failure: Task 199 in stage 14.0 failed 4 times, most recent failure: Lost task 199.3 in stage 14.0 (TID 9825, <ip>, executor 97): java.lang.IllegalArgumentException: The memory can only be add() during vertex program execute: gremlin. connectedNodeVertexProgram. voteToHalt at org.apache.tinkerpop.gremlin. process.computer.Memory$ Exceptions. memoryAddOnlyDuringVertexProgr amExecute(Memory.java:161)
I am not able to figure out why this exception is thrown, i don't see any OOM error. Please share thought/pointers.
Appreciate you help.
Thanks,
Anjani