Mixed index is not used by Gremlin query


simone...@...
 

@Dale,

I retried the procedure following the guide on Chapter 29

this is the script executed

import org.janusgraph.graphdb.database.management.ManagementSystem;


System.out.println('START');
graph
.tx().rollback();


System.out.println('STEP-0');
mgmt
= graph.openManagement();
attribute
= mgmt.getPropertyKey('title'); mgmt.buildIndex('titleAttributeSearch', Vertex.class).addKey(attribute, Mapping.TEXTSTRING.asParameter()).buildMixedIndex('search');
mgmt
.commit();
graph
.tx().commit();


System.out.println('STEP-1');
ManagementSystem.awaitGraphIndexStatus(graph, 'titleAttributeSearch').status(SchemaStatus.REGISTERED).call();


System.out.println('STEP-2');
m
= graph.openManagement();
i
= m.getGraphIndex('titleAttributeSearch');
m
.updateIndex(i, SchemaAction.REINDEX);
m
.commit();


System.out.println('STEP-3');
ManagementSystem.awaitGraphIndexStatus(graph, 'titleAttributeSearch').status(SchemaStatus.ENABLED).call();


System.out.println('END');"

And this is the log produced


janus_1                  
| START
janus_1                  
| STEP-0
janus_1                  
| STEP-1
janus_1                  
| 2017-09-29 09:40:58,858 [gremlin-server-exec-1] INFO  org.janusgraph.graphdb.database.management.GraphIndexStatusWatcher  - Some key(s) on index titleAttributeSearch do not currently have status REGISTERED: title=INSTALLED
janus_1                  
| 2017-09-29 09:40:59,361 [gremlin-server-exec-1] INFO  org.janusgraph.graphdb.database.management.GraphIndexStatusWatcher  - Some key(s) on index titleAttributeSearch do not currently have status REGISTERED: title=INSTALLED
janus_1                  
| 2017-09-29 09:40:59,866 [gremlin-server-exec-1] INFO  org.janusgraph.graphdb.database.management.GraphIndexStatusWatcher  - Some key(s) on index titleAttributeSearch do not currently have status REGISTERED: title=INSTALLED
                           
...
janus_1                  
| 2017-09-29 09:41:58,099 [gremlin-server-exec-1] INFO  org.janusgraph.graphdb.database.management.GraphIndexStatusWatcher  - Some key(s) on index titleAttributeSearch do not currently have status REGISTERED: title=INSTALLED
janus_1                  
| 2017-09-29 09:41:58,601 [gremlin-server-exec-1] INFO  org.janusgraph.graphdb.database.management.GraphIndexStatusWatcher  - Some key(s) on index titleAttributeSearch do not currently have status REGISTERED: title=INSTALLED
janus_1                  
| 2017-09-29 09:41:59,102 [gremlin-server-exec-1] INFO  org.janusgraph.graphdb.database.management.GraphIndexStatusWatcher  - Some key(s) on index titleAttributeSearch do not currently have status REGISTERED: title=INSTALLED
janus_1                  
| 2017-09-29 09:41:59,103 [gremlin-server-exec-1] INFO  org.janusgraph.graphdb.database.management.GraphIndexStatusWatcher  - Timed out (PT1M) while waiting for index titleAttributeSearch to converge on status REGISTERED
janus_1                  
| STEP-2
janus_1                  
| STEP-3
janus_1                  
| 2017-09-29 09:41:59,117 [gremlin-server-exec-1] INFO  org.janusgraph.graphdb.database.management.GraphIndexStatusWatcher  - Some key(s) on index titleAttributeSearch do not currently have status ENABLED: title=INSTALLED
janus_1                  
| 2017-09-29 09:41:59,620 [gremlin-server-exec-1] INFO  org.janusgraph.graphdb.database.management.GraphIndexStatusWatcher  - Some key(s) on index titleAttributeSearch do not currently have status ENABLED: title=INSTALLED
janus_1                  
| 2017-09-29 09:42:00,003 [Thread-2] ERROR org.janusgraph.graphdb.database.management.ManagementLogger  - Evicted [1@ac12000470-4e1a79a68c521] from cache but waiting too long for transactions to close. Stale transaction alert on: [standardjanusgraphtx[0x41f686af]]
janus_1                  
| 2017-09-29 09:42:00,125 [gremlin-server-exec-1] INFO  org.janusgraph.graphdb.database.management.GraphIndexStatusWatcher  - Some key(s) on index titleAttributeSearch do not currently have status ENABLED: title=INSTALLED
janus_1                  
| 2017-09-29 09:42:00,626 [gremlin-server-exec-1] INFO  org.janusgraph.graphdb.database.management.GraphIndexStatusWatcher  - Some key(s) on index titleAttributeSearch do not currently have status ENABLED: title=INSTALLED
janus_1                  
| 2017-09-29 09:42:01,128 [gremlin-server-exec-1] INFO  org.janusgraph.graphdb.database.management.GraphIndexStatusWatcher  - Some key(s) on index titleAttributeSearch do not currently have status ENABLED: title=INSTALLED
                           
...
janus_1                  
| 2017-09-29 09:42:58,380 [gremlin-server-exec-1] INFO  org.janusgraph.graphdb.database.management.GraphIndexStatusWatcher  - Some key(s) on index titleAttributeSearch do not currently have status ENABLED: title=INSTALLED
janus_1                  
| 2017-09-29 09:42:58,881 [gremlin-server-exec-1] INFO  org.janusgraph.graphdb.database.management.GraphIndexStatusWatcher  - Some key(s) on index titleAttributeSearch do not currently have status ENABLED: title=INSTALLED
janus_1                  
| 2017-09-29 09:42:59,382 [gremlin-server-exec-1] INFO  org.janusgraph.graphdb.database.management.GraphIndexStatusWatcher  - Some key(s) on index titleAttributeSearch do not currently have status ENABLED: title=INSTALLED
janus_1                  
| 2017-09-29 09:42:59,382 [gremlin-server-exec-1] INFO  org.janusgraph.graphdb.database.management.GraphIndexStatusWatcher  - Timed out (PT1M) while waiting for index titleAttributeSearch to converge on status ENABLED
janus_1                  
| END



Il giorno venerdì 29 settembre 2017 10:55:57 UTC+2, sim...@... ha scritto:
Hi Dale,

the index was originally defined before data ingestion. Actually, some vertices (for schema description) were already existing.
After ingesting the nodes (real repository ones), ElasticSearch index was populated as expected (direct index queries are working)

Anyway, I tried to apply the REINDEX command you proposed, but it throws a NullPointerException 

janus_1              | 2017-09-29 08:50:00,108 [gremlin-server-exec-10] WARN  org.apache.tinkerpop.gremlin.server.op.AbstractEvalOpProcessor  - Exception processing a script on request [RequestMessage{, requestId=39402196-be36-48e2-9eb4-43e8477f985d, op='eval', processor='', args={gremlin=mgmt = graph.openManagement(); mgmt.updateIndex(mgmt.getGraphIndex("titleAttributeSearch"), SchemaAction.REINDEX).get(); mgmt.commit(), bindings={}, batchSize=64}}].
janus_1              
| java.lang.NullPointerException: Cannot invoke method get() on null object
janus_1              
| at org.codehaus.groovy.runtime.NullObject.invokeMethod(NullObject.java:91)
janus_1              
| at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:48)
janus_1              
| at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
janus_1              
| at org.codehaus.groovy.runtime.callsite.NullCallSite.call(NullCallSite.java:35)
janus_1              
| at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
janus_1              
| at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
janus_1              
| at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:117)
janus_1              
| at Script11.run(Script11.groovy:1)
janus_1              
| at org.apache.tinkerpop.gremlin.groovy.jsr223.GremlinGroovyScriptEngine.eval(GremlinGroovyScriptEngine.java:619)
janus_1              
| at org.apache.tinkerpop.gremlin.groovy.jsr223.GremlinGroovyScriptEngine.eval(GremlinGroovyScriptEngine.java:448)
janus_1              
| at javax.script.AbstractScriptEngine.eval(AbstractScriptEngine.java:233)
janus_1              
| at org.apache.tinkerpop.gremlin.groovy.engine.ScriptEngines.eval(ScriptEngines.java:119)
janus_1              
| at org.apache.tinkerpop.gremlin.groovy.engine.GremlinExecutor.lambda$eval$2(GremlinExecutor.java:287)
janus_1              
| at java.util.concurrent.FutureTask.run(FutureTask.java:266)
janus_1              
| at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
janus_1              
| at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
janus_1              
| at java.lang.Thread.run(Thread.java:745)

  

Il giorno giovedì 28 settembre 2017 19:39:29 UTC+2, Robert Dale ha scritto:

Since you have data and the index wasn't enabled, you'll need to reindex.  This will also move the index into enabled when complete.

mgmt = graph.openManagement()
mgmt.updateIndex(mgmt.getGraphIndex("titleAttributeSearch"), SchemaAction.REINDEX).get()
mgmt.commit()


Robert Dale

On Thu, Sep 28, 2017 at 12:16 PM, <sim...@...> wrote:
I have just noticed that the key title for the index is not ENABLED 

mgmt = graph.openManagement()
mgmt
.getGraphIndex("titleAttributeSearch").getIndexStatus(mgmt.getGraphIndex("titleAttributeSearch").getFieldKeys()[0])
==>INSTALLED

how can I enable it?

Il giorno giovedì 28 settembre 2017 15:24:09 UTC+2, sim...@... ha scritto:
Following the Janus documentation, I defined a mixed index on my graph

mgmt = graph.openManagement()
attribute
= mgmt.getPropertyKey("title")
mgmt
.buildIndex("titleAttributeSearch", Vertex.class).addKey(attribute, Mapping.TEXTSTRING.asParameter()).buildMixedIndex("search")
mgmt
.commit()

After indexes definition, I ingested data, now the graph contains 120181 nodes.
Executing a direct index query searching for a string that should not return results the response is immediate

:> graph.indexQuery("titleAttributeSearch", "v.title:(sghjhgfds)").vertices()
2017-09-28 13:13:06,211 [gremlin-server-exec-36] INFO  org.janusgraph.graphdb.database.IndexSerializer  - Converted query string with 1 replacements: [v.title:(sghjhgfds)] => [title:(sghjhgfds)]

When I try to use a Gremlin query, the response requires many seconds and log reports that I'm not using indexes

:> g.V().has('title', Text.textContainsPrefix('sghjhgfds'))
2017-09-28 13:13:55,813 [gremlin-server-exec-8] WARN  org.janusgraph.graphdb.transaction.StandardJanusGraphTx  - Query requires iterating over all vertices [(title CONTAINS_PREFIX sghjhgfds)]. For better performance, use indexes

Am I doing some error?

Before the mixed index definition I set up a property index for versioning management

mgmt = graph.openManagement()
mgmt
.makePropertyKey("title").dataType(String.class).cardinality(LIST).make()
attribute
= mgmt.getPropertyKey("title")
mgmt
.buildPropertyIndex(attribute, "titleAttributeByVersion", mgmt.getPropertyKey("from"));
mgmt
.commit()

--
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/b4bd9346-9dcd-4315-bcbf-efa67015b2b0%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.


simone...@...
 

Hi Dale,

the index was originally defined before data ingestion. Actually, some vertices (for schema description) were already existing.
After ingesting the nodes (real repository ones), ElasticSearch index was populated as expected (direct index queries are working)

Anyway, I tried to apply the REINDEX command you proposed, but it throws a NullPointerException 

janus_1              | 2017-09-29 08:50:00,108 [gremlin-server-exec-10] WARN  org.apache.tinkerpop.gremlin.server.op.AbstractEvalOpProcessor  - Exception processing a script on request [RequestMessage{, requestId=39402196-be36-48e2-9eb4-43e8477f985d, op='eval', processor='', args={gremlin=mgmt = graph.openManagement(); mgmt.updateIndex(mgmt.getGraphIndex("titleAttributeSearch"), SchemaAction.REINDEX).get(); mgmt.commit(), bindings={}, batchSize=64}}].
janus_1              
| java.lang.NullPointerException: Cannot invoke method get() on null object
janus_1              
| at org.codehaus.groovy.runtime.NullObject.invokeMethod(NullObject.java:91)
janus_1              
| at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:48)
janus_1              
| at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
janus_1              
| at org.codehaus.groovy.runtime.callsite.NullCallSite.call(NullCallSite.java:35)
janus_1              
| at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
janus_1              
| at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
janus_1              
| at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:117)
janus_1              
| at Script11.run(Script11.groovy:1)
janus_1              
| at org.apache.tinkerpop.gremlin.groovy.jsr223.GremlinGroovyScriptEngine.eval(GremlinGroovyScriptEngine.java:619)
janus_1              
| at org.apache.tinkerpop.gremlin.groovy.jsr223.GremlinGroovyScriptEngine.eval(GremlinGroovyScriptEngine.java:448)
janus_1              
| at javax.script.AbstractScriptEngine.eval(AbstractScriptEngine.java:233)
janus_1              
| at org.apache.tinkerpop.gremlin.groovy.engine.ScriptEngines.eval(ScriptEngines.java:119)
janus_1              
| at org.apache.tinkerpop.gremlin.groovy.engine.GremlinExecutor.lambda$eval$2(GremlinExecutor.java:287)
janus_1              
| at java.util.concurrent.FutureTask.run(FutureTask.java:266)
janus_1              
| at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
janus_1              
| at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
janus_1              
| at java.lang.Thread.run(Thread.java:745)

  

Il giorno giovedì 28 settembre 2017 19:39:29 UTC+2, Robert Dale ha scritto:

Since you have data and the index wasn't enabled, you'll need to reindex.  This will also move the index into enabled when complete.

mgmt = graph.openManagement()
mgmt.updateIndex(mgmt.getGraphIndex("titleAttributeSearch"), SchemaAction.REINDEX).get()
mgmt.commit()


Robert Dale

On Thu, Sep 28, 2017 at 12:16 PM, <sim...@...> wrote:
I have just noticed that the key title for the index is not ENABLED 

mgmt = graph.openManagement()
mgmt
.getGraphIndex("titleAttributeSearch").getIndexStatus(mgmt.getGraphIndex("titleAttributeSearch").getFieldKeys()[0])
==>INSTALLED

how can I enable it?

Il giorno giovedì 28 settembre 2017 15:24:09 UTC+2, sim...@... ha scritto:
Following the Janus documentation, I defined a mixed index on my graph

mgmt = graph.openManagement()
attribute
= mgmt.getPropertyKey("title")
mgmt
.buildIndex("titleAttributeSearch", Vertex.class).addKey(attribute, Mapping.TEXTSTRING.asParameter()).buildMixedIndex("search")
mgmt
.commit()

After indexes definition, I ingested data, now the graph contains 120181 nodes.
Executing a direct index query searching for a string that should not return results the response is immediate

:> graph.indexQuery("titleAttributeSearch", "v.title:(sghjhgfds)").vertices()
2017-09-28 13:13:06,211 [gremlin-server-exec-36] INFO  org.janusgraph.graphdb.database.IndexSerializer  - Converted query string with 1 replacements: [v.title:(sghjhgfds)] => [title:(sghjhgfds)]

When I try to use a Gremlin query, the response requires many seconds and log reports that I'm not using indexes

:> g.V().has('title', Text.textContainsPrefix('sghjhgfds'))
2017-09-28 13:13:55,813 [gremlin-server-exec-8] WARN  org.janusgraph.graphdb.transaction.StandardJanusGraphTx  - Query requires iterating over all vertices [(title CONTAINS_PREFIX sghjhgfds)]. For better performance, use indexes

Am I doing some error?

Before the mixed index definition I set up a property index for versioning management

mgmt = graph.openManagement()
mgmt
.makePropertyKey("title").dataType(String.class).cardinality(LIST).make()
attribute
= mgmt.getPropertyKey("title")
mgmt
.buildPropertyIndex(attribute, "titleAttributeByVersion", mgmt.getPropertyKey("from"));
mgmt
.commit()

--
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/b4bd9346-9dcd-4315-bcbf-efa67015b2b0%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.


Robert Dale <rob...@...>
 


Since you have data and the index wasn't enabled, you'll need to reindex.  This will also move the index into enabled when complete.

mgmt = graph.openManagement()
mgmt.updateIndex(mgmt.getGraphIndex("titleAttributeSearch"), SchemaAction.REINDEX).get()
mgmt.commit()


Robert Dale

On Thu, Sep 28, 2017 at 12:16 PM, <simone...@...> wrote:
I have just noticed that the key title for the index is not ENABLED 

mgmt = graph.openManagement()
mgmt
.getGraphIndex("titleAttributeSearch").getIndexStatus(mgmt.getGraphIndex("titleAttributeSearch").getFieldKeys()[0])
==>INSTALLED

how can I enable it?

Il giorno giovedì 28 settembre 2017 15:24:09 UTC+2, sim...@... ha scritto:
Following the Janus documentation, I defined a mixed index on my graph

mgmt = graph.openManagement()
attribute
= mgmt.getPropertyKey("title")
mgmt
.buildIndex("titleAttributeSearch", Vertex.class).addKey(attribute, Mapping.TEXTSTRING.asParameter()).buildMixedIndex("search")
mgmt
.commit()

After indexes definition, I ingested data, now the graph contains 120181 nodes.
Executing a direct index query searching for a string that should not return results the response is immediate

:> graph.indexQuery("titleAttributeSearch", "v.title:(sghjhgfds)").vertices()
2017-09-28 13:13:06,211 [gremlin-server-exec-36] INFO  org.janusgraph.graphdb.database.IndexSerializer  - Converted query string with 1 replacements: [v.title:(sghjhgfds)] => [title:(sghjhgfds)]

When I try to use a Gremlin query, the response requires many seconds and log reports that I'm not using indexes

:> g.V().has('title', Text.textContainsPrefix('sghjhgfds'))
2017-09-28 13:13:55,813 [gremlin-server-exec-8] WARN  org.janusgraph.graphdb.transaction.StandardJanusGraphTx  - Query requires iterating over all vertices [(title CONTAINS_PREFIX sghjhgfds)]. For better performance, use indexes

Am I doing some error?

Before the mixed index definition I set up a property index for versioning management

mgmt = graph.openManagement()
mgmt
.makePropertyKey("title").dataType(String.class).cardinality(LIST).make()
attribute
= mgmt.getPropertyKey("title")
mgmt
.buildPropertyIndex(attribute, "titleAttributeByVersion", mgmt.getPropertyKey("from"));
mgmt
.commit()

--
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/b4bd9346-9dcd-4315-bcbf-efa67015b2b0%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.


simone...@...
 

I have just noticed that the key title for the index is not ENABLED 

mgmt = graph.openManagement()
mgmt
.getGraphIndex("titleAttributeSearch").getIndexStatus(mgmt.getGraphIndex("titleAttributeSearch").getFieldKeys()[0])
==>INSTALLED

how can I enable it?


Il giorno giovedì 28 settembre 2017 15:24:09 UTC+2, sim...@... ha scritto:
Following the Janus documentation, I defined a mixed index on my graph

mgmt = graph.openManagement()
attribute
= mgmt.getPropertyKey("title")
mgmt
.buildIndex("titleAttributeSearch", Vertex.class).addKey(attribute, Mapping.TEXTSTRING.asParameter()).buildMixedIndex("search")
mgmt
.commit()

After indexes definition, I ingested data, now the graph contains 120181 nodes.
Executing a direct index query searching for a string that should not return results the response is immediate

:> graph.indexQuery("titleAttributeSearch", "v.title:(sghjhgfds)").vertices()
2017-09-28 13:13:06,211 [gremlin-server-exec-36] INFO  org.janusgraph.graphdb.database.IndexSerializer  - Converted query string with 1 replacements: [v.title:(sghjhgfds)] => [title:(sghjhgfds)]

When I try to use a Gremlin query, the response requires many seconds and log reports that I'm not using indexes

:> g.V().has('title', Text.textContainsPrefix('sghjhgfds'))
2017-09-28 13:13:55,813 [gremlin-server-exec-8] WARN  org.janusgraph.graphdb.transaction.StandardJanusGraphTx  - Query requires iterating over all vertices [(title CONTAINS_PREFIX sghjhgfds)]. For better performance, use indexes

Am I doing some error?

Before the mixed index definition I set up a property index for versioning management

mgmt = graph.openManagement()
mgmt
.makePropertyKey("title").dataType(String.class).cardinality(LIST).make()
attribute
= mgmt.getPropertyKey("title")
mgmt
.buildPropertyIndex(attribute, "titleAttributeByVersion", mgmt.getPropertyKey("from"));
mgmt
.commit()


simone...@...
 

Following the Janus documentation, I defined a mixed index on my graph

mgmt = graph.openManagement()
attribute
= mgmt.getPropertyKey("title")
mgmt
.buildIndex("titleAttributeSearch", Vertex.class).addKey(attribute, Mapping.TEXTSTRING.asParameter()).buildMixedIndex("search")
mgmt
.commit()

After indexes definition, I ingested data, now the graph contains 120181 nodes.
Executing a direct index query searching for a string that should not return results the response is immediate

:> graph.indexQuery("titleAttributeSearch", "v.title:(sghjhgfds)").vertices()
2017-09-28 13:13:06,211 [gremlin-server-exec-36] INFO  org.janusgraph.graphdb.database.IndexSerializer  - Converted query string with 1 replacements: [v.title:(sghjhgfds)] => [title:(sghjhgfds)]

When I try to use a Gremlin query, the response requires many seconds and log reports that I'm not using indexes

:> g.V().has('title', Text.textContainsPrefix('sghjhgfds'))
2017-09-28 13:13:55,813 [gremlin-server-exec-8] WARN  org.janusgraph.graphdb.transaction.StandardJanusGraphTx  - Query requires iterating over all vertices [(title CONTAINS_PREFIX sghjhgfds)]. For better performance, use indexes

Am I doing some error?

Before the mixed index definition I set up a property index for versioning management

mgmt = graph.openManagement()
mgmt
.makePropertyKey("title").dataType(String.class).cardinality(LIST).make()
attribute
= mgmt.getPropertyKey("title")
mgmt
.buildPropertyIndex(attribute, "titleAttributeByVersion", mgmt.getPropertyKey("from"));
mgmt
.commit()