Re: Replacing EntryList by an iterable type


Jerry He <jerr...@...>
 

Hi, Hieu

I am not aware of any special reason it is such way as you point out. Feel free to propose any improvement or new design.  
The thing I can think of is that you probably would need to maintain a mini cache and mini batch (going to the backend storage) to feed into the Iterator.

Thanks.

Jerry

Thanks.

Jerry

On Fri, Jul 26, 2019 at 11:24 AM Hieu Nguyen <ntrh...@...> wrote:
When I look at the OrderedKeyValueStoreAdapter class, in methods getSlice() or getSlices(), objects of type RecordIterator<KeyValueEntry>  are converted to objects of type EntryList (StaticArrayEntryList). This, basically, breaks the iterator fashion since all data are collected and materialized. In many cases, this is very inefficient. For example, with multi-query enabled and I want to do g.V().has("name", "xxx").out().limit(50) and assume g.V().has("name", "xxx") result has 1k vertices, out() would trigger 1k concurrent requests and may return thousands of vertices although it is likely that with much fewer requests it can satisfy the limit(50). 

With an iterator fashion, it may try to do next() until it reaches the limit (50) and then stops.

I believe replacing EntryList by an iterable type to enable fully iterator-way of retrieving and processing data will benefit JanusGraph efficiency. My question is is there some reasons that EntryList was designed at the first place or there are some fundamental drawbacks of using iterator instead of EntryList? This would save time a lot since I plan to spend my effort to improve this.

--
You received this message because you are subscribed to the Google Groups "JanusGraph developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgr...@....
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-dev/1d6bd615-d355-4618-9a7e-5a3a687939ef%40googlegroups.com.

Join janusgraph-dev@lists.lfaidata.foundation to automatically receive all group messages.