1. Janusgraph-Users
  2. Messages

Re: Janusgraph 0.5.3 potential memory leak

rngcntr
 
Edited

Hey @mad, thanks for your benchmark code! I ran a few experiments with it today and figured out that creating an Iterable from the Iterator seems to already solve the problem. I added the following function to the benchmark:

@Benchmark
public void iterator_iterable(Blackhole bh) {
    EntryList result = StaticArrayEntryList.ofStaticBuffer(() -> entries.iterator(), StaticArrayEntry.ENTRY_GETTER);
    bh.consume(result);
}


And the results look very promising:

Benchmark                                        (size)  (valueSize)   Mode  Cnt     Score   Error  Units
StaticArrayEntryListBenchmark.iterable            10000           50  thrpt    2  3954.258          ops/s
StaticArrayEntryListBenchmark.iterable            10000         1000  thrpt    2   305.872          ops/s
StaticArrayEntryListBenchmark.iterable            10000         5000  thrpt    2    85.734          ops/s
StaticArrayEntryListBenchmark.iterable           100000           50  thrpt    2   224.861          ops/s
StaticArrayEntryListBenchmark.iterable           100000         1000  thrpt    2    19.816          ops/s
StaticArrayEntryListBenchmark.iterable           100000         5000  thrpt    2     7.058          ops/s
StaticArrayEntryListBenchmark.iterator            10000           50  thrpt    2  1619.764          ops/s
StaticArrayEntryListBenchmark.iterator            10000         1000  thrpt    2   142.065          ops/s
StaticArrayEntryListBenchmark.iterator            10000         5000  thrpt    2    27.785          ops/s
StaticArrayEntryListBenchmark.iterator           100000           50  thrpt    2   181.209          ops/s
StaticArrayEntryListBenchmark.iterator           100000         1000  thrpt    2    17.115          ops/s
StaticArrayEntryListBenchmark.iterator           100000         5000  java.lang.OutOfMemoryError: Java heap space
StaticArrayEntryListBenchmark.iterator_iterable   10000           50  thrpt    2  3557.666          ops/s
StaticArrayEntryListBenchmark.iterator_iterable   10000         1000  thrpt    2   331.978          ops/s
StaticArrayEntryListBenchmark.iterator_iterable   10000         5000  thrpt    2    87.827          ops/s
StaticArrayEntryListBenchmark.iterator_iterable  100000           50  thrpt    2   241.963          ops/s
StaticArrayEntryListBenchmark.iterator_iterable  100000         1000  thrpt    2    20.257          ops/s
StaticArrayEntryListBenchmark.iterator_iterable  100000         5000  thrpt    2     7.278          ops/s

 

 

The throughput is almost as high as using Iterable and even the OOM does not occur anymore. If that also fixes the original problem stated at the beginning of this thread, the solution is just a  () ->  away!
Join janusgraph-users@lists.lfaidata.foundation to automatically receive all group messages.