Hi Vinayak,

What actually is the "desired operation", it is not clear to me? In other words, what is the result after you have run the desired operation?

Best wishes,    Marc

Hi Marc,

While trying to execute a query using HTTP throws an out memory error. The number of records returned is more hence can't get the entire output. I want an entire output of the query in the file JSON format. Size of graph nodes: 7 lakh and edges: 3 lakh.

Thanks & Regards,

Vinayak 

On Mon, Mar 8, 2021 at 5:52 PM <hadoopmarc@...> wrote:

Hi Vinayak,

What actually is the "desired operation", it is not clear to me? In other words, what is the result after you have run the desired operation?

Best wishes,    Marc

Vinayak,

What is the query?  It sounds like whatever it is requires instantiating a lot of objects in order to process the results and construct the JSON.

What are your memory settings for the Janusgraph server?

Kevin

On Mon, Mar 8, 2021 at 6:11 AM Vinayak Bali <vinayakbali16@...> wrote:

Hi Marc,

While trying to execute a query using HTTP throws an out memory error. The number of records returned is more hence can't get the entire output. I want an entire output of the query in the file JSON format. Size of graph nodes: 7 lakh and edges: 3 lakh.

Thanks & Regards,

Vinayak 

On Mon, Mar 8, 2021 at 5:52 PM <hadoopmarc@...> wrote:

Hi Vinayak,

What actually is the "desired operation", it is not clear to me? In other words, what is the result after you have run the desired operation?

Best wishes,    Marc

Hi Kevin,

Query: g.V().as('a').outE().as('b').inV().as('c').select('a','b','c').by(valueMap().by(unfold()))

Memory Settings:

maxInitialLineLength: 4096
maxHeaderSize: 8192
maxChunkSize: 8192
maxContentLength: 2000000000
maxAccumulationBufferComponents: 1024
resultIterationBatchSize: 64
writeBufferLowWaterMark: 32768
writeBufferHighWaterMark: 65536

Thanks & Regards,

Vinayak

On Mon, Mar 8, 2021 at 8:01 PM Kevin Schmidt <ktschmidt@...> wrote:

Vinayak,

What is the query?  It sounds like whatever it is requires instantiating a lot of objects in order to process the results and construct the JSON.

What are your memory settings for the Janusgraph server?

Kevin

On Mon, Mar 8, 2021 at 6:11 AM Vinayak Bali <vinayakbali16@...> wrote:

Hi Marc,

While trying to execute a query using HTTP throws an out memory error. The number of records returned is more hence can't get the entire output. I want an entire output of the query in the file JSON format. Size of graph nodes: 7 lakh and edges: 3 lakh.

Thanks & Regards,

Vinayak 

On Mon, Mar 8, 2021 at 5:52 PM <hadoopmarc@...> wrote:

Hi Vinayak,

What actually is the "desired operation", it is not clear to me? In other words, what is the result after you have run the desired operation?

Best wishes,    Marc

Vinayak,

What about the memory settings, specifically the max heap, for the JVM Janusgraph is running in?

Your traversal is effectively retrieving your entire graph.  Is that really what you need to do?  Or is this just a test and your real scenarios wouldn't be doing that?  Doing a retrieval like this is going to require a lot of memory and your JVM simply isn't configured with enough to complete it, thus the out of memory error.

Kevin

On Mon, Mar 8, 2021 at 6:35 AM Vinayak Bali <vinayakbali16@...> wrote:

Hi Kevin,

Query: g.V().as('a').outE().as('b').inV().as('c').select('a','b','c').by(valueMap().by(unfold()))

Memory Settings:

maxInitialLineLength: 4096
maxHeaderSize: 8192
maxChunkSize: 8192
maxContentLength: 2000000000
maxAccumulationBufferComponents: 1024
resultIterationBatchSize: 64
writeBufferLowWaterMark: 32768
writeBufferHighWaterMark: 65536

Thanks & Regards,

Vinayak

On Mon, Mar 8, 2021 at 8:01 PM Kevin Schmidt <ktschmidt@...> wrote:

Vinayak,

What is the query?  It sounds like whatever it is requires instantiating a lot of objects in order to process the results and construct the JSON.

What are your memory settings for the Janusgraph server?

Kevin

On Mon, Mar 8, 2021 at 6:11 AM Vinayak Bali <vinayakbali16@...> wrote:

Hi Marc,

While trying to execute a query using HTTP throws an out memory error. The number of records returned is more hence can't get the entire output. I want an entire output of the query in the file JSON format. Size of graph nodes: 7 lakh and edges: 3 lakh.

Thanks & Regards,

Vinayak 

On Mon, Mar 8, 2021 at 5:52 PM <hadoopmarc@...> wrote:

Hi Vinayak,

What actually is the "desired operation", it is not clear to me? In other words, what is the result after you have run the desired operation?

Best wishes,    Marc

Hi Kevin,

JVM size at the server where janusgraph and Cassandra (same machine) is set to 24GB. On the API side, we tried setting it to 8GB. Is there any other way using which we can write the query result  to a file in json format ??

Thanks & Regards,

Vinayak

On Mon, Mar 8, 2021 at 8:20 PM Kevin Schmidt <ktschmidt@...> wrote:

Vinayak,

What about the memory settings, specifically the max heap, for the JVM Janusgraph is running in?

Your traversal is effectively retrieving your entire graph.  Is that really what you need to do?  Or is this just a test and your real scenarios wouldn't be doing that?  Doing a retrieval like this is going to require a lot of memory and your JVM simply isn't configured with enough to complete it, thus the out of memory error.

Kevin

On Mon, Mar 8, 2021 at 6:35 AM Vinayak Bali <vinayakbali16@...> wrote:

Hi Kevin,

Query: g.V().as('a').outE().as('b').inV().as('c').select('a','b','c').by(valueMap().by(unfold()))

Memory Settings:

maxInitialLineLength: 4096
maxHeaderSize: 8192
maxChunkSize: 8192
maxContentLength: 2000000000
maxAccumulationBufferComponents: 1024
resultIterationBatchSize: 64
writeBufferLowWaterMark: 32768
writeBufferHighWaterMark: 65536

Thanks & Regards,

Vinayak

On Mon, Mar 8, 2021 at 8:01 PM Kevin Schmidt <ktschmidt@...> wrote:

Vinayak,

What is the query?  It sounds like whatever it is requires instantiating a lot of objects in order to process the results and construct the JSON.

What are your memory settings for the Janusgraph server?

Kevin

On Mon, Mar 8, 2021 at 6:11 AM Vinayak Bali <vinayakbali16@...> wrote:

Hi Marc,

While trying to execute a query using HTTP throws an out memory error. The number of records returned is more hence can't get the entire output. I want an entire output of the query in the file JSON format. Size of graph nodes: 7 lakh and edges: 3 lakh.

Thanks & Regards,

Vinayak 

On Mon, Mar 8, 2021 at 5:52 PM <hadoopmarc@...> wrote:

Hi Vinayak,

What actually is the "desired operation", it is not clear to me? In other words, what is the result after you have run the desired operation?

Best wishes,    Marc

Vinayak,

Is the out of memory error in the server or the client?  And are Janusgraph and Cassandra running in the same JVM or are you running Cassandra separately?

And what is your use case for your client needing to retrieve your entire graph?

Kevin

On Mon, Mar 8, 2021 at 7:01 AM Vinayak Bali <vinayakbali16@...> wrote:

Hi Kevin,

JVM size at the server where janusgraph and Cassandra (same machine) is set to 24GB. On the API side, we tried setting it to 8GB. Is there any other way using which we can write the query result  to a file in json format ??

Thanks & Regards,

Vinayak

On Mon, Mar 8, 2021 at 8:20 PM Kevin Schmidt <ktschmidt@...> wrote:

Vinayak,

What about the memory settings, specifically the max heap, for the JVM Janusgraph is running in?

Your traversal is effectively retrieving your entire graph.  Is that really what you need to do?  Or is this just a test and your real scenarios wouldn't be doing that?  Doing a retrieval like this is going to require a lot of memory and your JVM simply isn't configured with enough to complete it, thus the out of memory error.

Kevin

On Mon, Mar 8, 2021 at 6:35 AM Vinayak Bali <vinayakbali16@...> wrote:

Hi Kevin,

Query: g.V().as('a').outE().as('b').inV().as('c').select('a','b','c').by(valueMap().by(unfold()))

Memory Settings:

maxInitialLineLength: 4096
maxHeaderSize: 8192
maxChunkSize: 8192
maxContentLength: 2000000000
maxAccumulationBufferComponents: 1024
resultIterationBatchSize: 64
writeBufferLowWaterMark: 32768
writeBufferHighWaterMark: 65536

Thanks & Regards,

Vinayak

On Mon, Mar 8, 2021 at 8:01 PM Kevin Schmidt <ktschmidt@...> wrote:

Vinayak,

What is the query?  It sounds like whatever it is requires instantiating a lot of objects in order to process the results and construct the JSON.

What are your memory settings for the Janusgraph server?

Kevin

On Mon, Mar 8, 2021 at 6:11 AM Vinayak Bali <vinayakbali16@...> wrote:

Hi Marc,

While trying to execute a query using HTTP throws an out memory error. The number of records returned is more hence can't get the entire output. I want an entire output of the query in the file JSON format. Size of graph nodes: 7 lakh and edges: 3 lakh.

Thanks & Regards,

Vinayak 

On Mon, Mar 8, 2021 at 5:52 PM <hadoopmarc@...> wrote:

Hi Vinayak,

What actually is the "desired operation", it is not clear to me? In other words, what is the result after you have run the desired operation?

Best wishes,    Marc

Hi Kevin,

To verify whether the issue is from the server or client-side, we are trying to execute the same query on the server and output the data in the file. But still it's not possible. Janusgraph is using Cassandra Dameon, hence both are on the same jvm. We are testing for further use-cases which can contain more data than now.

Thanks & Regards,

Vinayak

On Mon, Mar 8, 2021 at 8:37 PM Kevin Schmidt <ktschmidt@...> wrote:

Vinayak,

Is the out of memory error in the server or the client?  And are Janusgraph and Cassandra running in the same JVM or are you running Cassandra separately?

And what is your use case for your client needing to retrieve your entire graph?

Kevin

On Mon, Mar 8, 2021 at 7:01 AM Vinayak Bali <vinayakbali16@...> wrote:

Hi Kevin,

JVM size at the server where janusgraph and Cassandra (same machine) is set to 24GB. On the API side, we tried setting it to 8GB. Is there any other way using which we can write the query result  to a file in json format ??

Thanks & Regards,

Vinayak

On Mon, Mar 8, 2021 at 8:20 PM Kevin Schmidt <ktschmidt@...> wrote:

Vinayak,

What about the memory settings, specifically the max heap, for the JVM Janusgraph is running in?

Your traversal is effectively retrieving your entire graph.  Is that really what you need to do?  Or is this just a test and your real scenarios wouldn't be doing that?  Doing a retrieval like this is going to require a lot of memory and your JVM simply isn't configured with enough to complete it, thus the out of memory error.

Kevin

On Mon, Mar 8, 2021 at 6:35 AM Vinayak Bali <vinayakbali16@...> wrote:

Hi Kevin,

Query: g.V().as('a').outE().as('b').inV().as('c').select('a','b','c').by(valueMap().by(unfold()))

Memory Settings:

maxInitialLineLength: 4096
maxHeaderSize: 8192
maxChunkSize: 8192
maxContentLength: 2000000000
maxAccumulationBufferComponents: 1024
resultIterationBatchSize: 64
writeBufferLowWaterMark: 32768
writeBufferHighWaterMark: 65536

Thanks & Regards,

Vinayak

On Mon, Mar 8, 2021 at 8:01 PM Kevin Schmidt <ktschmidt@...> wrote:

Vinayak,

What is the query?  It sounds like whatever it is requires instantiating a lot of objects in order to process the results and construct the JSON.

What are your memory settings for the Janusgraph server?

Kevin

On Mon, Mar 8, 2021 at 6:11 AM Vinayak Bali <vinayakbali16@...> wrote:

Hi Marc,

While trying to execute a query using HTTP throws an out memory error. The number of records returned is more hence can't get the entire output. I want an entire output of the query in the file JSON format. Size of graph nodes: 7 lakh and edges: 3 lakh.

Thanks & Regards,

Vinayak 

On Mon, Mar 8, 2021 at 5:52 PM <hadoopmarc@...> wrote:

Hi Vinayak,

What actually is the "desired operation", it is not clear to me? In other words, what is the result after you have run the desired operation?

Best wishes,    Marc

Vinayak,

If your goal is simply to get your entire graph into a JSON-based format, you may want to consider GraphSON (http://tinkerpop.apache.org/docs/3.4.1/dev/io/#graphson).

Kevin

On Mon, Mar 8, 2021 at 7:13 AM Vinayak Bali <vinayakbali16@...> wrote:

Hi Kevin,

To verify whether the issue is from the server or client-side, we are trying to execute the same query on the server and output the data in the file. But still it's not possible. Janusgraph is using Cassandra Dameon, hence both are on the same jvm. We are testing for further use-cases which can contain more data than now.

Thanks & Regards,

Vinayak

On Mon, Mar 8, 2021 at 8:37 PM Kevin Schmidt <ktschmidt@...> wrote:

Vinayak,

Is the out of memory error in the server or the client?  And are Janusgraph and Cassandra running in the same JVM or are you running Cassandra separately?

And what is your use case for your client needing to retrieve your entire graph?

Kevin

On Mon, Mar 8, 2021 at 7:01 AM Vinayak Bali <vinayakbali16@...> wrote:

Hi Kevin,

JVM size at the server where janusgraph and Cassandra (same machine) is set to 24GB. On the API side, we tried setting it to 8GB. Is there any other way using which we can write the query result  to a file in json format ??

Thanks & Regards,

Vinayak

On Mon, Mar 8, 2021 at 8:20 PM Kevin Schmidt <ktschmidt@...> wrote:

Vinayak,

What about the memory settings, specifically the max heap, for the JVM Janusgraph is running in?

Your traversal is effectively retrieving your entire graph.  Is that really what you need to do?  Or is this just a test and your real scenarios wouldn't be doing that?  Doing a retrieval like this is going to require a lot of memory and your JVM simply isn't configured with enough to complete it, thus the out of memory error.

Kevin

On Mon, Mar 8, 2021 at 6:35 AM Vinayak Bali <vinayakbali16@...> wrote:

Hi Kevin,

Query: g.V().as('a').outE().as('b').inV().as('c').select('a','b','c').by(valueMap().by(unfold()))

Memory Settings:

maxInitialLineLength: 4096
maxHeaderSize: 8192
maxChunkSize: 8192
maxContentLength: 2000000000
maxAccumulationBufferComponents: 1024
resultIterationBatchSize: 64
writeBufferLowWaterMark: 32768
writeBufferHighWaterMark: 65536

Thanks & Regards,

Vinayak

On Mon, Mar 8, 2021 at 8:01 PM Kevin Schmidt <ktschmidt@...> wrote:

Vinayak,

What is the query?  It sounds like whatever it is requires instantiating a lot of objects in order to process the results and construct the JSON.

What are your memory settings for the Janusgraph server?

Kevin

On Mon, Mar 8, 2021 at 6:11 AM Vinayak Bali <vinayakbali16@...> wrote:

Hi Marc,

While trying to execute a query using HTTP throws an out memory error. The number of records returned is more hence can't get the entire output. I want an entire output of the query in the file JSON format. Size of graph nodes: 7 lakh and edges: 3 lakh.

Thanks & Regards,

Vinayak 

On Mon, Mar 8, 2021 at 5:52 PM <hadoopmarc@...> wrote:

Hi Vinayak,

What actually is the "desired operation", it is not clear to me? In other words, what is the result after you have run the desired operation?

Best wishes,    Marc