Local lock contention on edges


Carlos Cheung <cheung...@...>
 

So I understand that if I open up separate transactions and modify the schema, a commit on one of the transactions will fail. What I don't understand is why adding edges of the same name between different nodes in different transactions occasionally cause a local lock contention. Sometimes it goes through fine, other times I will encounter an error. A schema has been defined for everything.

Strangely I am unable to reproduce this in groovy, but attached is a test case that causes the issue in Python utilizing GremlinClient library. At the bottom is the stack trace from the server when the error occurs.

from gremlinpython import PythonGraphTraversalSource

g1 = PythonGraphTraversalSource("ws://localhost:8182/", "g")
g2 = PythonGraphTraversalSource("ws://localhost:8182/", "g")

head1 = g1.addV("head").property("number", 1).toList()[0]
head2 = g2.addV("head").property("number", 2).toList()[0]

g1.commit()
g2.commit()
g1.close()
g2.close()

g1 = PythonGraphTraversalSource("ws://localhost:8182/", "g")
g2 = PythonGraphTraversalSource("ws://localhost:8182/", "g")


for i in range(0, 10):
dummy = g1.addV("dummy").toList()[0]
g1.V(head1['id']).as_("a").V(dummy['id']).addE("from").from_("a").toList() # modify the edge label to something other than "from" will fix this issue

for i in range(0, 10):
dummy = g2.addV("dummy2").toList()[0]
g2.V(head2['id']).as_("a").V(dummy['id']).addE("from").from_("a").toList()

print g1.commit()
print g2.commit() # local lock contention here

g1.close()
g2.close()




59071 [gremlin-server-worker-2] WARN  org.apache.tinkerpop.gremlin.server.op.AbstractEvalOpProcessor  - Exception processing a script on request [RequestMessage{, requestId=08a56980-b362-4f80-9aee-745d969543de, op='eval', processor='session', args={gremlin=g.tx().commit(), session=9f1f55cb-5bf7-4654-8e0e-fc9b47540d44, language=gremlin-groovy, bindings={}, scriptEvaluationTimeout=30000, aliases={}}}].
org.janusgraph.diskstorage.locking.PermanentLockingException: Local lock contention
at org.janusgraph.diskstorage.locking.AbstractLocker.writeLock(AbstractLocker.java:327)
at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStore.acquireLock(ExpectedValueCheckingStore.java:103)
at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.acquireLock(KCVSProxy.java:54)
at org.janusgraph.diskstorage.BackendTransaction.acquireIndexLock(BackendTransaction.java:255)
at org.janusgraph.graphdb.database.StandardJanusGraph.prepareCommit(StandardJanusGraph.java:568)
at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:697)
at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1366)
at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:277)
at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:105)
at org.apache.tinkerpop.gremlin.structure.Transaction$commit.call(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:117)
at Script4.run(Script4.groovy:1)
at org.apache.tinkerpop.gremlin.groovy.jsr223.GremlinGroovyScriptEngine.eval(GremlinGroovyScriptEngine.java:619)
at org.apache.tinkerpop.gremlin.groovy.jsr223.GremlinGroovyScriptEngine.eval(GremlinGroovyScriptEngine.java:448)
at javax.script.AbstractScriptEngine.eval(AbstractScriptEngine.java:233)
at org.apache.tinkerpop.gremlin.groovy.engine.ScriptEngines.eval(ScriptEngines.java:119)
at org.apache.tinkerpop.gremlin.groovy.engine.GremlinExecutor.lambda$eval$2(GremlinExecutor.java:287)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)




Austin Sharp <austins...@...>
 

My guess would be that it's because you have some kind of index that is being updated when you add those edges (note 'BackendTransaction.acquireIndexLock' in your call stack). I would see if you can repro the problem when adding edges that aren't indexed in any way. I also have a distant memory of unique indices sometimes being more of a hassle, so that might be something to look into as well.


On Monday, March 20, 2017 at 4:25:19 PM UTC-7, Carlos Cheung wrote:
So I understand that if I open up separate transactions and modify the schema, a commit on one of the transactions will fail. What I don't understand is why adding edges of the same name between different nodes in different transactions occasionally cause a local lock contention. Sometimes it goes through fine, other times I will encounter an error. A schema has been defined for everything.

Strangely I am unable to reproduce this in groovy, but attached is a test case that causes the issue in Python utilizing GremlinClient library. At the bottom is the stack trace from the server when the error occurs.

from gremlinpython import PythonGraphTraversalSource

g1 = PythonGraphTraversalSource("ws://localhost:8182/", "g")
g2 = PythonGraphTraversalSource("ws://localhost:8182/", "g")

head1 = g1.addV("head").property("number", 1).toList()[0]
head2 = g2.addV("head").property("number", 2).toList()[0]

g1.commit()
g2.commit()
g1.close()
g2.close()

g1 = PythonGraphTraversalSource("ws://localhost:8182/", "g")
g2 = PythonGraphTraversalSource("ws://localhost:8182/", "g")


for i in range(0, 10):
dummy = g1.addV("dummy").toList()[0]
g1.V(head1['id']).as_("a").V(dummy['id']).addE("from").from_("a").toList() # modify the edge label to something other than "from" will fix this issue

for i in range(0, 10):
dummy = g2.addV("dummy2").toList()[0]
g2.V(head2['id']).as_("a").V(dummy['id']).addE("from").from_("a").toList()

print g1.commit()
print g2.commit() # local lock contention here

g1.close()
g2.close()




59071 [gremlin-server-worker-2] WARN  org.apache.tinkerpop.gremlin.server.op.AbstractEvalOpProcessor  - Exception processing a script on request [RequestMessage{, requestId=08a56980-b362-4f80-9aee-745d969543de, op='eval', processor='session', args={gremlin=g.tx().commit(), session=9f1f55cb-5bf7-4654-8e0e-fc9b47540d44, language=gremlin-groovy, bindings={}, scriptEvaluationTimeout=30000, aliases={}}}].
org.janusgraph.diskstorage.locking.PermanentLockingException: Local lock contention
at org.janusgraph.diskstorage.locking.AbstractLocker.writeLock(AbstractLocker.java:327)
at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStore.acquireLock(ExpectedValueCheckingStore.java:103)
at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.acquireLock(KCVSProxy.java:54)
at org.janusgraph.diskstorage.BackendTransaction.acquireIndexLock(BackendTransaction.java:255)
at org.janusgraph.graphdb.database.StandardJanusGraph.prepareCommit(StandardJanusGraph.java:568)
at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:697)
at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1366)
at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:277)
at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:105)
at org.apache.tinkerpop.gremlin.structure.Transaction$commit.call(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:117)
at Script4.run(Script4.groovy:1)
at org.apache.tinkerpop.gremlin.groovy.jsr223.GremlinGroovyScriptEngine.eval(GremlinGroovyScriptEngine.java:619)
at org.apache.tinkerpop.gremlin.groovy.jsr223.GremlinGroovyScriptEngine.eval(GremlinGroovyScriptEngine.java:448)
at javax.script.AbstractScriptEngine.eval(AbstractScriptEngine.java:233)
at org.apache.tinkerpop.gremlin.groovy.engine.ScriptEngines.eval(ScriptEngines.java:119)
at org.apache.tinkerpop.gremlin.groovy.engine.GremlinExecutor.lambda$eval$2(GremlinExecutor.java:287)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)