1. Janusgraph-Users
  2. Topics

Potential thread leakage in JanusGraphTransaction?

L TY <n0b0...@...>
 

Hi,

I'm using Elasticsearch and HBase as backend of JanusGraph and I write a multithread import tool. When using multithread mode, the program hit the ulimit of process easily. 

Then I check the thread consumption of JanusGraphTransaction in single thread mode.
The logic of my code:
```
JanusGraph graph = JanusGraphFactory.open(this.janusProp);
JanusGraphTransaction transaction = graph.newTransaction();
int count = 0;
while (line = readline()) {
transaction.addVertex(processLine(line));
count += 1;
if (count % VERTEX_BATCH_SIZE == 0) {
logger.info(String.format("Threads in this thread group before: %d", Thread.activeCount()));
transaction.commit();
transaction.close();
logger.info(String.format("Check transaction closed: %b %b", transaction.isOpen(), transaction.isClosed()));
transaction = graph.newTransaction();
logger.info(String.format("Threads in this thread group after: %d", Thread.activeCount()));
}
}
transaction.commit();
transaction.close();
graph.close();
```

The batch size is 10000.
And I got the log as below:
```
[INFO ] [main] 2020-08-28 17:41:28,391 Threads in this thread group before: 62  
[INFO ] [main] 2020-08-28 17:41:31,301 Check transaction closed: false true  
[INFO ] [main] 2020-08-28 17:41:31,301 Threads in this thread group after: 109  
[INFO ] [main] 2020-08-28 17:41:31,638 Threads in this thread group before: 110  
[INFO ] [main] 2020-08-28 17:41:34,242 Check transaction closed: false true  
[INFO ] [main] 2020-08-28 17:41:34,242 Threads in this thread group after: 159  
[INFO ] [main] 2020-08-28 17:41:34,487 Threads in this thread group before: 159  
[INFO ] [main] 2020-08-28 17:41:37,213 Check transaction closed: false true  
[INFO ] [main] 2020-08-28 17:41:37,213 Threads in this thread group after: 208  
[INFO ] [main] 2020-08-28 17:41:37,441 Threads in this thread group before: 208  
[INFO ] [main] 2020-08-28 17:41:39,968 Check transaction closed: false true  
[INFO ] [main] 2020-08-28 17:41:39,969 Threads in this thread group after: 257  
[INFO ] [main] 2020-08-28 17:41:40,188 Threads in this thread group before: 258  
[INFO ] [main] 2020-08-28 17:41:43,477 Check transaction closed: false true  
[INFO ] [main] 2020-08-28 17:41:43,478 Threads in this thread group after: 307  
[INFO ] [main] 2020-08-28 17:41:43,706 Threads in this thread group before: 307  
[INFO ] [main] 2020-08-28 17:41:47,096 Check transaction closed: false true  
[INFO ] [main] 2020-08-28 17:41:47,097 Threads in this thread group after: 313  
[INFO ] [main] 2020-08-28 17:41:47,333 Threads in this thread group before: 313  
[INFO ] [main] 2020-08-28 17:41:50,962 Check transaction closed: false true  
[INFO ] [main] 2020-08-28 17:41:50,963 Threads in this thread group after: 313  
[INFO ] [main] 2020-08-28 17:41:51,156 Threads in this thread group before: 313  
[INFO ] [main] 2020-08-28 17:41:53,897 Check transaction closed: false true  
[INFO ] [main] 2020-08-28 17:41:53,898 Threads in this thread group after: 313  
[INFO ] [main] 2020-08-28 17:41:54,089 Threads in this thread group before: 313  
[INFO ] [main] 2020-08-28 17:41:56,739 Check transaction closed: false true  
[INFO ] [main] 2020-08-28 17:41:56,740 Threads in this thread group after: 313
```

The thread number is stablized at 313.

Anyone has idea what should I do to reduce the thread consumption when using JanusGraphTransaction?

1 - 1 of 1