Re: Transactions and uploading a lot of vertices


Alexander Scherbatiy <stell...@...>
 

I have a graph that I need to upload which has about 1000 vertices and 3000 edges.

I tried to load it using commit after
a) each iteration
b) every 100th iteration
c) after all data uploading

It took about a) 2.3s b) 2.3s c) 9.8 sec on my system.

I used 'graph.tx().commit()' code to commit uploaded data in transaction.

My graph parameters are:
JanusGraph graph = JanusGraphFactory.build()
.set("storage.backend", "inmemory")
.set("graph.set-vertex-id", "true")
.set("ids.block-size", "100000")
.set("ids.authority.wait-time", "5")
.set("ids.renew-timeout", "50")
.open();

The profiler shows that the most time in c) which uses only one commit after whole data uploading takes the method JanusGraphStep.executeGraphCentryQuery where it creates Sets.newHashSet() and then takes a lot time over iterating elements.

By the way, what is the right way to call transactions in JanusGraph?
The Transactions chapter in JanusGraph doc describes only examples of 'graph.tx().commit()' and 'graph.tx().createThreadedTx()'.

There is also method 'graph.newTransaction()'.
Is there drawbacks to explicitly create transactions by 'graph.newTransaction()' over using the default one and committing by 'graph.tx().commit()'?

Thanks,
Alexander.

On 26.09.2019 20:16, Pavel Ershov wrote:
It depends on graph structure how many properties/edges
Good starting point do commit every 10k vertices. Addition info about bulk loading https://docs.janusgraph.org/advanced-topics/bulk-loading/
среда, 25 сентября 2019 г., 23:14:14 UTC+3 пользователь Alexander Scherbatiy написал:
I am trying to upload several thousands vertices into JanusGraph for
my testing.
Is there any performance benefits to commit the transaction after
adding all vertices, after adding each vertex or may be there is an
optimal vertices batch size that can be used to make the commit
after uploading a certain amount of vertices?
Thanks,
Alexander.
--
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgra...@... <mailto:janusgra...@...>.
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/3ecea828-b9a7-4e98-9376-01a66bb07eb5%40googlegroups.com <https://groups.google.com/d/msgid/janusgraph-users/3ecea828-b9a7-4e98-9376-01a66bb07eb5%40googlegroups.com?utm_medium=email&utm_source=footer>.

Join janusgraph-users@lists.lfaidata.foundation to automatically receive all group messages.