Can't add a graph when the number of nodes are high - ERROR Handler$GremlinResponseHandler


Dilan Ranasinghe <dila...@...>
 

Hello


I'm using janusgraph/gremlin server and connect to it via gremlin driver 3.2.3.

The gremlin server and the application which connect to it runs on the same machine.

I need to create a graph with thousands of nodes and edges (More than hundred thousand).

What i did is create a gremlin query at once for the creation of the whole graph and send to the server.

As the graph grows larger, the query i create for generating the graph is also getting larger and it seems the gremlin server fails to process the request.
I got the following error.

2017-09-26 07:26:54 ERROR Handler$GremlinResponseHandler:246 - Could not process the response
java.io.IOException: Connection reset by peer
at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
at sun.nio.ch.IOUtil.read(IOUtil.java:192)
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380)
at io.netty.buffer.PooledUnsafeDirectByteBuf.setBytes(PooledUnsafeDirectByteBuf.java:221)
at io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:899)
at io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:241)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:119)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:572)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:513)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:427)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:399)
at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:140)
at java.lang.Thread.run(Thread.java:745)

So i increase the content length in the gremlin-server.yaml. After that i'm getting the following error in creating large graphs.

java.lang.RuntimeException: Method code too large!
at groovyjarjarasm.asm.MethodWriter.a(Unknown Source)
at groovyjarjarasm.asm.ClassWriter.toByteArray(Unknown Source)
at org.codehaus.groovy.control.CompilationUnit$16.call(CompilationUnit.java:815)
at org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1053)
at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:591)
at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:569)
at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:546)
at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298)
at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268)
at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:254)
at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:211)
at org.apache.tinkerpop.gremlin.groovy.jsr223.GremlinGroovyScriptEngine.getScriptClass(GremlinGroovyScriptEngine.java:527)
at org.apache.tinkerpop.gremlin.groovy.jsr223.GremlinGroovyScriptEngine.eval(GremlinGroovyScriptEngine.java:446)
at javax.script.AbstractScriptEngine.eval(AbstractScriptEngine.java:233)
at org.apache.tinkerpop.gremlin.groovy.engine.ScriptEngines.eval(ScriptEngines.java:119)
at org.apache.tinkerpop.gremlin.groovy.engine.GremlinExecutor.lambda$eval$2(GremlinExecutor.java:287)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

1 error

at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)
at tsn.poc.GraphConnector.submitQuery(GraphConnector.java:40)
at tsn.poc.OptimizedGraphBuilder.buildGraph(OptimizedGraphBuilder.java:157)
at tsn.poc.Poc.main(Poc.java:44)
Caused by: org.apache.tinkerpop.gremlin.driver.exception.ResponseException: startup failed:
General error during class generation: Method code too large!



I need to know
what is the recommended way to add a large graph?
Do i need to add the graph part by part?
Or is there any configuration in the driver or the server to resolve the above issue?

Thanks
Dilan

Join {janusgraph-users@lists.lfaidata.foundation to automatically receive all group messages.