janusgraph-dev@lists.lfaidata.foundation | Slow index lookup on property key

Home Messages Hashtags Subgroups Calendar

Date Date 1 - 9 of 9

Slow index lookup on property key

Calvin Lei <ckp...@...> #423 Hi, I am trying to test out the performance on index lookup on property key. Below is my setup: JanusGraph version: 0.1.1 Storage backend: Cassandra cluster running in EC2 Index backend Elastic Search Number of vertices: 1000 Each vertex has the property "customId" with some random integer. The test perform two queries. One with a single key lookup and another one with 500 keys lookup with the following code: Single key lookup: tx.query().has("customId", "saturnClone100").vertices() Time taken: 500 ms 500 keys lookup: tx.query().has("customId", Contain.IN, customIds) Time taken: 36131ms Did I misconfigure anything? The performance is rather bad. Thanks in advance Cal
More
Calvin Lei <ckp...@...> #424 Forgot to mention I am using Elastic Search v1.7.6 toggle quoted message Show quoted text On Friday, October 13, 2017 at 11:24:20 AM UTC-7, Calvin Lei wrote: Hi, I am trying to test out the performance on index lookup on property key. Below is my setup: JanusGraph version: 0.1.1 Storage backend: Cassandra cluster running in EC2 Index backend Elastic Search Number of vertices: 1000 Each vertex has the property "customId" with some random integer. The test perform two queries. One with a single key lookup and another one with 500 keys lookup with the following code: Single key lookup: tx.query().has("customId", "saturnClone100").vertices() Time taken: 500 ms 500 keys lookup: tx.query().has("customId", Contain.IN, customIds) Time taken: 36131ms Did I misconfigure anything? The performance is rather bad. Thanks in advance Cal
More
Jason Plurad <plu...@...> #425 If you're doing exact match lookups, you'd get better performance from a composite index. http://docs.janusgraph.org/latest/indexes.html#_composite_versus_mixed_indexes Also, you can do the lookup with a regular Gremlin traversal: `g.V().has("customId", within(customIds)).toList()` toggle quoted message Show quoted text On Friday, October 13, 2017 at 2:26:29 PM UTC-4, Calvin Lei wrote: Forgot to mention I am using Elastic Search v1.7.6 On Friday, October 13, 2017 at 11:24:20 AM UTC-7, Calvin Lei wrote: Hi, I am trying to test out the performance on index lookup on property key. Below is my setup: JanusGraph version: 0.1.1 Storage backend: Cassandra cluster running in EC2 Index backend Elastic Search Number of vertices: 1000 Each vertex has the property "customId" with some random integer. The test perform two queries. One with a single key lookup and another one with 500 keys lookup with the following code: Single key lookup: tx.query().has("customId", "saturnClone100").vertices() Time taken: 500 ms 500 keys lookup: tx.query().has("customId", Contain.IN, customIds) Time taken: 36131ms Did I misconfigure anything? The performance is rather bad. Thanks in advance Cal
More
Calvin Lei <ckp...@...> #427 thanks Jason. I doubled check the code, I was already using a composite index when I got the numbers i posted JanusGraphManagement.IndexBuilder customIdIndexBuilder = mgmt.buildIndex("customId", Vertex.class).addKey(customId); customIdIndexBuilder.buildCompositeIndex(); toggle quoted message Show quoted text On Friday, October 13, 2017 at 11:53:12 AM UTC-7, Jason Plurad wrote: If you're doing exact match lookups, you'd get better performance from a composite index. http://docs.janusgraph.org/latest/indexes.html#_composite_versus_mixed_indexes Also, you can do the lookup with a regular Gremlin traversal: `g.V().has("customId", within(customIds)).toList()` On Friday, October 13, 2017 at 2:26:29 PM UTC-4, Calvin Lei wrote: Forgot to mention I am using Elastic Search v1.7.6 On Friday, October 13, 2017 at 11:24:20 AM UTC-7, Calvin Lei wrote: Hi, I am trying to test out the performance on index lookup on property key. Below is my setup: JanusGraph version: 0.1.1 Storage backend: Cassandra cluster running in EC2 Index backend Elastic Search Number of vertices: 1000 Each vertex has the property "customId" with some random integer. The test perform two queries. One with a single key lookup and another one with 500 keys lookup with the following code: Single key lookup: tx.query().has("customId", "saturnClone100").vertices() Time taken: 500 ms 500 keys lookup: tx.query().has("customId", Contain.IN, customIds) Time taken: 36131ms Did I misconfigure anything? The performance is rather bad. Thanks in advance Cal
More
ankur...@... #456 Any update on this, i am also facing the same. ~ toggle quoted message Show quoted text On Friday, October 13, 2017 at 11:54:20 PM UTC+5:30, Calvin Lei wrote: Hi, I am trying to test out the performance on index lookup on property key. Below is my setup: JanusGraph version: 0.1.1 Storage backend: Cassandra cluster running in EC2 Index backend Elastic Search Number of vertices: 1000 Each vertex has the property "customId" with some random integer. The test perform two queries. One with a single key lookup and another one with 500 keys lookup with the following code: Single key lookup: tx.query().has("customId", "saturnClone100").vertices() Time taken: 500 ms 500 keys lookup: tx.query().has("customId", Contain.IN, customIds) Time taken: 36131ms Did I misconfigure anything? The performance is rather bad. Thanks in advance Cal
More
Calvin Lei <ckp...@...> #458 Hi Are you also using Cassandra as storage backend? toggle quoted message Show quoted text On Monday, October 23, 2017 at 7:25:23 AM UTC-7, an...@... wrote: Any update on this, i am also facing the same. ~ On Friday, October 13, 2017 at 11:54:20 PM UTC+5:30, Calvin Lei wrote: Hi, I am trying to test out the performance on index lookup on property key. Below is my setup: JanusGraph version: 0.1.1 Storage backend: Cassandra cluster running in EC2 Index backend Elastic Search Number of vertices: 1000 Each vertex has the property "customId" with some random integer. The test perform two queries. One with a single key lookup and another one with 500 keys lookup with the following code: Single key lookup: tx.query().has("customId", "saturnClone100").vertices() Time taken: 500 ms 500 keys lookup: tx.query().has("customId", Contain.IN, customIds) Time taken: 36131ms Did I misconfigure anything? The performance is rather bad. Thanks in advance Cal
More
Ankur Goel <ankur...@...> #462 Yes, i am using cassandra as backend. ~ toggle quoted message Show quoted text On Mon, Oct 23, 2017 at 10:23 PM, Calvin Lei <ckp...@...> wrote: Hi Are you also using Cassandra as storage backend? On Monday, October 23, 2017 at 7:25:23 AM UTC-7, an...@... wrote: Any update on this, i am also facing the same. ~ On Friday, October 13, 2017 at 11:54:20 PM UTC+5:30, Calvin Lei wrote: Hi, I am trying to test out the performance on index lookup on property key. Below is my setup: JanusGraph version: 0.1.1 Storage backend: Cassandra cluster running in EC2 Index backend Elastic Search Number of vertices: 1000 Each vertex has the property "customId" with some random integer. The test perform two queries. One with a single key lookup and another one with 500 keys lookup with the following code: Single key lookup: tx.query().has("customId", "saturnClone100").vertices() Time taken: 500 ms 500 keys lookup: tx.query().has("customId", Contain.IN, customIds) Time taken: 36131ms Did I misconfigure anything? The performance is rather bad. Thanks in advance Cal -- You received this message because you are subscribed to a topic in the Google Groups "JanusGraph developers" group. To unsubscribe from this topic, visit https://groups.google.com/d/topic/janusgraph-dev/gbGVZ9eZqH8/unsubscribe. To unsubscribe from this group and all its topics, send an email to janusgraph-dev+unsubscribe@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-dev/95611956-b2f4-484c-852f-17f44ce5ec04%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
More
Calvin Lei <ckp...@...> #463 It is still a work in progress. I do notice a small performance boost when using the Gremlin traversal instead of using the JanusGraph core api since there is no need to open a transaction. Haven't dug in to why yet. May be someone here can shed some lights toggle quoted message Show quoted text On Monday, October 23, 2017 at 9:56:45 PM UTC-7, Ankur Goel wrote: Yes, i am using cassandra as backend. ~ On Mon, Oct 23, 2017 at 10:23 PM, Calvin Lei <c...@...> wrote: Hi Are you also using Cassandra as storage backend? On Monday, October 23, 2017 at 7:25:23 AM UTC-7, an...@... wrote: Any update on this, i am also facing the same. ~ On Friday, October 13, 2017 at 11:54:20 PM UTC+5:30, Calvin Lei wrote: Hi, I am trying to test out the performance on index lookup on property key. Below is my setup: JanusGraph version: 0.1.1 Storage backend: Cassandra cluster running in EC2 Index backend Elastic Search Number of vertices: 1000 Each vertex has the property "customId" with some random integer. The test perform two queries. One with a single key lookup and another one with 500 keys lookup with the following code: Single key lookup: tx.query().has("customId", "saturnClone100").vertices() Time taken: 500 ms 500 keys lookup: tx.query().has("customId", Contain.IN, customIds) Time taken: 36131ms Did I misconfigure anything? The performance is rather bad. Thanks in advance Cal -- You received this message because you are subscribed to a topic in the Google Groups "JanusGraph developers" group. To unsubscribe from this topic, visit https://groups.google.com/d/topic/janusgraph-dev/gbGVZ9eZqH8/unsubscribe. To unsubscribe from this group and all its topics, send an email to janusgraph-de...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-dev/95611956-b2f4-484c-852f-17f44ce5ec04%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
More
Hacking J <hyeonm...@...> #464 I am also facing same. In my case, I have two property that "category", "data". 'data' is serialized object(HashMap) and not indexed. 'category' is string and indexed(composite) and I use embedded janusgraph in java. below query profile result is too long. It just only count of vertice `g.V().has("category", "Broadcast").count().profile().next()` profile result Traversal Metrics Step Count Traversers Time (ms) % Dur ============================================================================================================= JanusGraphStep([],[category.eq(Broadcast)]) 24518 24518 17002.297 97.86 \_condition=(category = Broadcast) \_isFitted=true \_query=multiKSQ[1]@2147483647 \_index=category \_orders=[] \_isOrdered=true optimization 274.814 backend-query 0.000 \_query=category:multiKSQ[1]@2147483647 backend-query 0.000 \_query=category:multiKSQ[1]@2147483647 backend-query 0.000 \_query=category:multiKSQ[1]@2147483647 backend-query 0.000 \_query=category:multiKSQ[1]@2147483647 backend-query 24518 4458.324 \_query=category:multiKSQ[1]@2147483647 CountGlobalStep 1 1 371.796 2.14 >TOTAL - - 17374.094 - 2017년 10월 14일 토요일 오전 3시 24분 20초 UTC+9, Calvin Lei 님의 말: toggle quoted message Show quoted text Hi, I am trying to test out the performance on index lookup on property key. Below is my setup: JanusGraph version: 0.1.1 Storage backend: Cassandra cluster running in EC2 Index backend Elastic Search Number of vertices: 1000 Each vertex has the property "customId" with some random integer. The test perform two queries. One with a single key lookup and another one with 500 keys lookup with the following code: Single key lookup: tx.query().has("customId", "saturnClone100").vertices() Time taken: 500 ms 500 keys lookup: tx.query().has("customId", Contain.IN, customIds) Time taken: 36131ms Did I misconfigure anything? The performance is rather bad. Thanks in advance Cal
More

1 - 9 of 9

1

Previous Topic Next Topic

Home Hashtags Subgroups Calendar Terms