Hi All, I am using connected component vertex program to find all the connected nodes in graph and then using that RDD for further processing in graph. I want to store that RDD at some output location

On Tue, May 11, 2021 at 04:56 PM, <hadoopmarc@...> wrote: ids.num-partitionsThanks for help Marc, i will try updating value of ids.num-partitions = number of executors.

Hi Anjani, It is a while ago I did this myself. I interpret ids.num-partitions as a stock of reserved id blocks that can be delegated to a janugraph instance. It does not have a large value to not

We have 250 parallel spark task running for creating node/edges. I didn't get parallel tasks - (for setting ids.num-partitions)? Could you please help me on it?

What is the number of parallel tasks? (for setting ids.num-partitions) You have the ids.authority.wait-time still on its default value of 300 ms, so that seems worthwhile experimenting with. Best

On Tue, May 11, 2021 at 11:54 AM, <hadoopmarc@...> wrote: https://docs.janusgraph.org/advanced-topics/bulk-loading/#optimizing-id-allocation Thanks for response Marc. Below is the method i am using

Hi Anjani, Please show the properties file you use to open janusgraph. I assume you also saw the other recommendations in

Hi All, I am creating vertex and edges in bulk and getting below error while creating edge. Below is the exception log:I tired increasing value of "ids.block-size" but still no luck, even set to 1B

Hi Vinayak, Actually, query 4 was easier to rework. It could read somewhat like: g.V().has('property1', 'vertex1').as('v1').outE().has('property1', 'edge1').limit(100).as('e').inV().has('property1',

Hi Marc, Thank you for your reply. I will try to report this issue on janusgraph repository. Regarding the work around you suggested, if possible please share the updated query with work around for

Hi Vinayak, If you would bother to demonstrate this behavior with a reproducible, generated graph, you can report it as an issue on github. For now, you can only look for workarounds: - combine the

Hi Marc, This query takes 18 sec to run by changing as to aggregate and select to project. But still, 99% of the time is taken to compute union. There is no memory issue, it already set to

Hi Vinayak, Your last remark explains it well: it seems that in JanusGraph a union of multiple clauses can take much longer than the sum of the individual clauses. There are still two things that we

Hi Marc, That works as expected. Union also works as expected as in Query1 but when I add limit to all edge the performance degrades. Thanks

Hi Vinayak, What happens with a single clause, so without the union: g.V().has('property1', 'vertex3').outE().has('property1', 'edge3').inV().has('property1',

Hi Marc, Yes, all the index are made available and no warning is thrown while executing the query. I tried debugging using profile step. 99% of time is taken by the union query. Thanks &

Hi Vinayak, To be sure: we are dealing here with a large graph, so all V().has('property1', 'vertex...') steps do hit the index (no index log warnings)? For one, it would be interesting to see the

Hi Marc, Tried the approach you suggested. There is some improvement. Earlier it took 2 mins, now it's taking 1min 50sec. Is there any other way to optimize this further may to ms or seconds?? Thank

Hi Vinayak, My answer already contains a concrete suggestion. Replace all union subclauses starting with outE with the alternate form that has a local(................limit1)) construct, as

Hi Marc, Thank you for your reply. I understand the queries are big, so there is a problem viewing them. Actually I am not interested in either of v1 or v2. I want to apply limit on edges, and don't