Date
1 - 5 of 5
Serious Issue on traversal with query-batch=true
|
marchi...@...
Hi all, I think I found a bug on Janusgraph 0.5.2. I attach here the discussion on stack overflow with Stephen Mallette. Basically, during the upgrade from Janusgraph from 0.2.0 to 0.5.x, I encounter some problems with previously working queries. I isolated the issue with a sample graph and after checking the configurations I found that the parameter query-batch=true was creating the issue. I tested this on an empty Janusgraph instance created from the default latest Janusgraph docker image, with the standard configuration I started the image with this command: docker run --name janusgraph-default janusgraph/janusgraph:latest Test node I used: if(g.tx().isOpen()) { g.tx().rollback() }g.tx().open() v = [:] v["test"]=graph.addVertex('test_vertex') v["test"].property("uri", "test")v["test"].property("type", "test_type")v["test"].property("id", "123")v["test"].property("test_prop", 1593189100420,"from",1593189100420,"to",-1)v["test"].property("test_prop", 1593189099710,"from",1593189099710,"to",-1)g.tx().commit() Query used:
If I start the docker image enabling the query batch, an exception occurs: docker run -e janusgraph.query.batch=true --name janusgraph-default janusgraph/janusgraph:latest Exception:
It seems something related to the multi-query optimization, (found this issues and PRs on Janusgraph JIRA -> #71 #1000 PR #1000 ) Disabling query batch makes my system works fine, but since this param should be an optimization, I cannot optimize my Janusgraph cluster. In the StackOverflow discussion, Stephen Mallette tested this example on the TinkerGraph and reported the same issue. It seems like a bug. Do someone else has encountered this or have some guesses? Cheers, Nicolò |
|
|
|
Nicolò Marchi <marchi...@...>
Hi all,
Anyone have some guesses about it? The contributing guide says that to open a bug I need feedbacks from here. This bug is giving us problem for productio Upgrade of janusgraph. Thank you, cheers Nicolò |
|
|
|
HadoopMarc <bi...@...>
Hi Nicolò, The upgrade instructions from JanusGraph v0.4.1 state: "Adding multiple values in the same query to a new vertex property without explicitly defined type
(i.e. using Automatic Schema Maker to create a property type) requires explicit usage of VertexProperty.Cardinality
for each call (only for the first query which defines a property) if the VertexProperty.Cardinality is different than
VertexProperty.Cardinality.single." What does this mean for you? Where Stephen Mallette uses for TinkerGraph: ...property(list,"test_prop", 1593189100420,"from",1593189100420,"to",-1). you have to use for JanusGraph: ...property(Cardinality.LIST,"test_prop", 1593189100420,"from",1593189100420,"to",-1). See also: https://javadoc.io/doc/org.janusgraph/janusgraph-core/latest/index.html https://javadoc.io/doc/org.janusgraph/janusgraph-core/latest/org/janusgraph/core/Cardinality.html Best wishes, Marc Op maandag 3 augustus 2020 om 10:57:26 UTC+2 schreef mar...@...: Hi all, |
|
|
|
Nicolò Marchi <nicolo...@...>
Hi Marc, thank you for your answer. Before starting the upgrade procedures I read the upgrading guide on Janusgraph documentation and stepped on that particular point related to Tinkerpop upgrade, but it seems that we weren't affected by it (looking in our groovy scripts). Anyway, I tried to explicit the cardinality in every property call, but it seems still not working with the query.batch parameter set to true. What I'm trying to highlight here is that there are some problems with the query.batch parameter. Without it, my example works fine. The problem arise when I set the query.batch to true. It is supposed to be an optimization parameter, but it seems not working. Do you have any other hint? Or I did it something wrong? Run container with query.batch=true ✔ 11:35:23 docker run -e janusgraph.query.batch=true --name janusgraph-default janusgraph/janusgraph:latest Run the example with cardinality explicited ✔ 11:40:34 ~ $ docker exec -e COLUMNS="tput cols" -e LINES="tput lines" -itu0 $(docker container ls | grep "janusgraph-default" | cut -d' ' -f1) /opt/janusgraph/bin/gremlin.sh Aug 11, 2020 9:41:26 AM java.util.prefs.FileSystemPreferences$1 run INFO: Created user preferences directory. \,,,/ (o o) -----oOOo-(3)-oOOo----- SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/opt/janusgraph/lib/slf4j-log4j12-1.7.12.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/opt/janusgraph/lib/logback-classic-1.1.3.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] plugin activated: tinkerpop.server plugin activated: tinkerpop.tinkergraph 09:41:39 WARN org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable plugin activated: tinkerpop.hadoop plugin activated: tinkerpop.spark plugin activated: tinkerpop.utilities plugin activated: janusgraph.imports gremlin> :remote connect tinkerpop.server conf/remote.yaml session ==>Configured localhost/127.0.0.1:8182-[77adc164-7496-46a4-a5d9-ecc2ceffa6fb] gremlin> :remote console ==>All scripts will now be sent to Gremlin Server - [localhost/127.0.0.1:8182]-[77adc164-7496-46a4-a5d9-ecc2ceffa6fb] - type ':remote console' to return to local mode gremlin> gremlin> gremlin> gremlin> gremlin> if(g.tx().isOpen()) { g.tx().rollback() } ==>null gremlin> gremlin> g.tx().open() ==>null gremlin> v = [:] gremlin> v["test"]=graph.addVertex('test_vertex') ==>v[4176] gremlin> v["test"].property(single, "uri", "test") ==>vp[uri->test] gremlin> v["test"].property(single, "type", "test_type") ==>vp[type->test_type] gremlin> v["test"].property(single, "id", "123") ==>vp[id->123] gremlin> v["test"].property(list, "test_prop", 1593189100420,"from",1593189100420,"to",-1) ==>vp[test_prop->1593189100420] gremlin> v["test"].property(list, "test_prop", 1593189099710,"from",1593189099710,"to",-1) ==>vp[test_prop->1593189099710] gremlin> gremlin> gremlin> gremlin> gremlin> gremlin> g.V().has("uri", "test").match( ......1> __.as('n').properties().has('to', -1).as('p'), ......2> __.as('p').label().as('label'), ......3> __.as('p').values('from').as('from'), ......4> __.as('n').properties().has('to', -1) ......5> .where(eq('p')).by(T.label) ......6> .values('from').max().as('max') ......7> ).where('from', neq('max')).select('p').property('to', __.select('max')).iterate(); org.janusgraph.graphdb.relations.StandardVertexProperty cannot be cast to org.apache.tinkerpop.gremlin.structure.Vertex Type ':help' or ':h' for help. Display stack trace? [yN]y java.lang.ClassCastException: org.janusgraph.graphdb.relations.StandardVertexProperty cannot be cast to org.apache.tinkerpop.gremlin.structure.Vertex at org.janusgraph.graphdb.tinkerpop.optimize.JanusGraphPropertiesStep.lambda$initializeMultiQuery$1(JanusGraphPropertiesStep.java:110) at java.util.Collections$SingletonList.forEach(Collections.java:4824) at org.janusgraph.graphdb.tinkerpop.optimize.JanusGraphPropertiesStep.initializeMultiQuery(JanusGraphPropertiesStep.java:110) at org.janusgraph.graphdb.tinkerpop.optimize.JanusGraphPropertiesStep.flatMap(JanusGraphPropertiesStep.java:132) at org.apache.tinkerpop.gremlin.process.traversal.step.map.FlatMapStep.processNextStart(FlatMapStep.java:49) at org.janusgraph.graphdb.tinkerpop.optimize.JanusGraphPropertiesStep.processNextStart(JanusGraphPropertiesStep.java:125) at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:143) at org.apache.tinkerpop.gremlin.process.traversal.step.util.ExpandableStepIterator.next(ExpandableStepIterator.java:50) at org.apache.tinkerpop.gremlin.process.traversal.step.map.MatchStep$MatchEndStep.processNextStart(MatchStep.java:569) at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:143) at org.apache.tinkerpop.gremlin.process.traversal.util.DefaultTraversal.hasNext(DefaultTraversal.java:197) at org.apache.tinkerpop.gremlin.process.traversal.step.map.MatchStep.standardAlgorithm(MatchStep.java:371) at org.apache.tinkerpop.gremlin.process.traversal.step.util.ComputerAwareStep.processNextStart(ComputerAwareStep.java:46) at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:143) at org.apache.tinkerpop.gremlin.process.traversal.step.util.ExpandableStepIterator.next(ExpandableStepIterator.java:50) at org.apache.tinkerpop.gremlin.process.traversal.step.map.MapStep.processNextStart(MapStep.java:36) at org.apache.tinkerpop.gremlin.process.traversal.step.map.SelectOneStep.processNextStart(SelectOneStep.java:131) at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:143) at org.apache.tinkerpop.gremlin.process.traversal.step.util.ExpandableStepIterator.hasNext(ExpandableStepIterator.java:42) at org.apache.tinkerpop.gremlin.process.traversal.step.map.NoOpBarrierStep.processAllStarts(NoOpBarrierStep.java:66) at org.apache.tinkerpop.gremlin.process.traversal.step.map.NoOpBarrierStep.processNextStart(NoOpBarrierStep.java:55) at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:143) at org.apache.tinkerpop.gremlin.process.traversal.step.util.ExpandableStepIterator.next(ExpandableStepIterator.java:50) at org.apache.tinkerpop.gremlin.process.traversal.step.sideEffect.SideEffectStep.processNextStart(SideEffectStep.java:38) at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:143) at org.apache.tinkerpop.gremlin.process.traversal.step.util.ExpandableStepIterator.next(ExpandableStepIterator.java:50) at org.apache.tinkerpop.gremlin.process.traversal.step.filter.FilterStep.processNextStart(FilterStep.java:37) at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.next(AbstractStep.java:128) at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.next(AbstractStep.java:38) at org.apache.tinkerpop.gremlin.process.traversal.Traversal.iterate(Traversal.java:207) at org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversal.iterate(GraphTraversal.java:2931) at org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversal$Admin.iterate(GraphTraversal.java:189) at org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.DefaultGraphTraversal.iterate(DefaultGraphTraversal.java:48) at org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversal$Admin$iterate.call(Unknown Source) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:115) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:119) at Script13.run(Script13.groovy:8) at org.apache.tinkerpop.gremlin.groovy.jsr223.GremlinGroovyScriptEngine.eval(GremlinGroovyScriptEngine.java:674) at org.apache.tinkerpop.gremlin.groovy.jsr223.GremlinGroovyScriptEngine.eval(GremlinGroovyScriptEngine.java:376) at javax.script.AbstractScriptEngine.eval(AbstractScriptEngine.java:233) at org.apache.tinkerpop.gremlin.groovy.engine.GremlinExecutor.lambda$eval$0(GremlinExecutor.java:267) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) gremlin> Thank you again, cheers. Nicolò Il giorno lunedì 3 agosto 2020 alle 21:09:56 UTC+2 HadoopMarc ha scritto:
|
|
|
|
HadoopMarc <bi...@...>
Hi Nicolò, Thanks for your patience. This has turned out to be a rather involved issue, because there were multiple causes:
This made us look into the wrong direction. See my docker exec session below which ran with your config with query-batch=true. I cannot conclude from this whether there is a bug. At least my alternative gives a working solution that also works for the config without query-batch. Happy graphing, Marc plugin activated: tinkerpop.utilities plugin activated: janusgraph.imports gremlin> g = traversal().withRemote(DriverRemoteConnection.using("localhost",8182,"g")) ==>graphtraversalsource[emptygraph[empty], standard] gremlin> gremlin> g.addV('test_vertex'). ......1> property("uri", "test"). ......2> property("type", "test_type"). ......3> property("id", "123"). ......4> property(list, "test_prop", 1593189100420,"from",1593189100420,"to",-1L). ......5> property(list, "test_prop", 1593189099710,"from",1593189099710,"to",-1L) ==>v[8416] gremlin> gremlin> g.V().properties('test_prop').values('from') ==>1593189100420 ==>1593189099710 gremlin> g.V().properties('test_prop').values('to') ==>-1 ==>-1 gremlin> gremlin> g.V().has("uri", "test"). ......1> match( ......2> __.as('n').properties().has('to', -1L).as('p'), ......3> __.as('p').label().as('label'), ......4> __.as('p').values('from').fold().as('from'), ......5> __.as('n').properties().has('to', -1L). ......6> where(eq('p')).by(T.label). ......7> values('from').max().as('max') ......8> ).where('max', not(within('from'))).select('p').values('from', 'to') ==>1593189099710 ==>-1 gremlin> gremlin> g.V().has("uri", "test"). ......1> match( ......2> __.as('n').properties().has('to', -1L).as('p'), ......3> __.as('p').label().as('label'), ......4> __.as('p').values('from').fold().as('from'), ......5> __.as('n').properties().has('to', -1L). ......6> where(eq('p')).by(T.label). ......7> values('from').max().as('max') ......8> ).where('max', not(within('from'))).select('p'). ......9> property('to', __.select('max')).valueMap('from', 'to') ==>[from:1593189099710,to:1593189100420] gremlin> gremlin> g.V().properties('test_prop').values('from') ==>1593189100420 ==>1593189099710 gremlin> g.V().properties('test_prop').values('to') ==>-1 ==>1593189100420 gremlin> Op vrijdag 21 augustus 2020 om 16:28:36 UTC+2 schreef Nicolò Marchi:
|
|
|