I'm using janusgraph/gremlin server and connect to it via gremlin driver 3.2.3.
The gremlin server and the application which connect to it runs on the same machine.
I need to create a graph with thousands of nodes and edges (More than hundred thousand).
What i did is create a gremlin query at once for the creation of the whole graph and send to the server.
As the graph grows larger, the query i create for generating the graph is also getting larger and it seems the gremlin server fails to process the request. I got the following error.
2017-09-26 07:26:54 ERROR Handler$GremlinResponseHandler:246 - Could not process the response java.io.IOException: Connection reset by peer at sun.nio.ch.FileDispatcherImpl.read0(Native Method) at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223) at sun.nio.ch.IOUtil.read(IOUtil.java:192) at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380) at io.netty.buffer.PooledUnsafeDirectByteBuf.setBytes(PooledUnsafeDirectByteBuf.java:221) at io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:899) at io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:241) at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:119) at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:572) at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:513) at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:427) at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:399) at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:140) at java.lang.Thread.run(Thread.java:745)
So i increase the content length in the gremlin-server.yaml. After that i'm getting the following error in creating large graphs.
java.lang.RuntimeException: Method code too large! at groovyjarjarasm.asm.MethodWriter.a(Unknown Source) at groovyjarjarasm.asm.ClassWriter.toByteArray(Unknown Source) at org.codehaus.groovy.control.CompilationUnit$16.call(CompilationUnit.java:815) at org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1053) at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:591) at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:569) at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:546) at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298) at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268) at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:254) at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:211) at org.apache.tinkerpop.gremlin.groovy.jsr223.GremlinGroovyScriptEngine.getScriptClass(GremlinGroovyScriptEngine.java:527) at org.apache.tinkerpop.gremlin.groovy.jsr223.GremlinGroovyScriptEngine.eval(GremlinGroovyScriptEngine.java:446) at javax.script.AbstractScriptEngine.eval(AbstractScriptEngine.java:233) at org.apache.tinkerpop.gremlin.groovy.engine.ScriptEngines.eval(ScriptEngines.java:119) at org.apache.tinkerpop.gremlin.groovy.engine.GremlinExecutor.lambda$eval$2(GremlinExecutor.java:287) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745)
1 error
at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357) at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895) at tsn.poc.GraphConnector.submitQuery(GraphConnector.java:40) at tsn.poc.OptimizedGraphBuilder.buildGraph(OptimizedGraphBuilder.java:157) at tsn.poc.Poc.main(Poc.java:44) Caused by: org.apache.tinkerpop.gremlin.driver.exception.ResponseException: startup failed: General error during class generation: Method code too large!
I need to know what is the recommended way to add a large graph? Do i need to add the graph part by part? Or is there any configuration in the driver or the server to resolve the above issue?