Re: Multiple writers cause inconsistent vertices
Ted Wilmes <twi...@...>
toggle quoted messageShow quoted text
If at all possible, I usually try to remove the need for unique constraints on indexes. You mentioned that eventually
you'll have about a 99:1 r/w ratio. Does this mean that you'll do a big bulk load up front? If so, could you structure your load
so that you do not need to have the unique index enabled and you can instead build it after the load? For example, maybe
you could load all of your vertices first, and then load the edges. This would require some preprocessing but would speed
things up greatly. This is how the TinkerPop BulkLoaderVertexProgram  that can be run against Janus works, granted, you
must put your data in one of the support adjacency list formats first or provide a custom reader.
If you can't load the vertices separate, maybe you could partition your input data so that you could isolate
reads and writes for any specific vertex to the same thread, this would let you safely perform a read before write
to check for existence without having to worry about race conditions. Combine this with an in-thread cache of
what vertices have already been inserted and their corresponding Janus IDs and you'll speed things up.
On Sunday, September 24, 2017 at 9:56:17 AM UTC-5, Ajay Srivastava wrote: