Two things to check come up:

• am I right in assuming that your remote connection to JanusGraph-Server does not use Sessions (indeed you do no not want ths because this would prevent auto commit of individual insertion traversals)?
• are their any signs of stress of the Cassandra container? You mention "no exceptions", but how do the CPU usage and write delay develop?

Retry with a longer backoff period is certainly a good idea as long as average insertion times are reasonable. You can check the following blog for reference:
https://www.experoinc.com/expero-resources/janusgraph-nuts-and-bolts-part-1-write-performance

I'm trying to test ingestion of data locally using Docker. I've got a Janus Server (1.0.0-rc2) and Cassandra (latest) container running. I'm running a Java TinkerPop (3.6.4) application locally that's uploading data using the remote graph interface.

It's initially just adding vertexes and after 31000+ inserts, I get this exception in the application.

java.util.concurrent.CompletionException: org.apache.tinkerpop.gremlin.driver.exception.ResponseException: Could not commit transaction due to exception during persistence
... 
Caused by: org.apache.tinkerpop.gremlin.driver.exception.ResponseException: Could not commit transaction due to exception during persistence

In the Janus server logs it has (see attachment for full stack trace):

6:28:28 WARN  org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.checkSingleLock - Discarded expired claim on KeyColumn [k=0x0x6000000001376882, c=0x0x02] with timestamp 2023-05-26T16:10:46.504156Z
16:28:28 WARN  org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.checkSingleLock - Discarded expired claim on KeyColumn [k=0x0xE800000001376902, c=0x0x02] with timestamp 2023-05-26T16:10:46.505566Z
16:28:28 ERROR org.janusgraph.graphdb.database.StandardJanusGraph.commit - Could not commit transaction [229736] due to storage exception in commit
org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception
...
Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT1M40S
...
Caused by: org.janusgraph.diskstorage.locking.consistentkey.ExpiredLockException: Expired lock on KeyColumn [k=0x0x6000000001376882, c=0x0x02]: lock timestamp 2023-05-26T16:10:46.504156Z Micros is older than storage.lock.expiry-time=PT5M

There are no exceptions in the Cassandra container.
There are no consistency settings, but one index for one of the vertex fields. 
There's only one process interacting with the graph, which is just inserting vertexes. There's no dependencies on anything stored in the graph. I know the indexed value is unique, because it's inserting nodes from a HashSet keyed by the indexed value. 
It retries 10 times and always gets the same error. The status code is SERVER_ERROR.

Not sure what's causing the lock to expire. Maybe things are running to slowly? I don't believe this could be caused by data consistency issues.

More importantly, how can I recover from this error? Does the application need to retry with a longer backoff period?

If I restart the application without restarting Janus Server or Cassandra, it just starts inserting nodes again. That leads me to believe it's a temporary issue, but retrying isn't working.

Thanks for any pointers.

Dear Marc,

I see! Thanks for your reply. But TinkerGraph would support this syntax in their next major version update: https://issues.apache.org/jira/browse/TINKERPOP-2910. Maybe we can also consider this kind of reasonable features.

Best regards

No, the query syntax is not correct.

See https://javadoc.io/doc/org.apache.tinkerpop/gremlin-core/latest/index-all.html#I:V
and you not that V() exists for both the GraphTraversalSource, the GraphTraversal and for __.

Look again at https://javadoc.io/doc/org.apache.tinkerpop/gremlin-core/latest/index-all.html#I:E
and you see that E() only exists for the GraphTraversalSource.

In other words: a mid-traversal E() is not allowed and you get the error that more or less says so.

Best wishes,     Marc

Could it be because I docker compose down the backend and removed the volumes for data and index, before restart the janusgraph server. If so, how should I clean the db.

I am attaching the log for your reference.

Thanks,
 

Hello,

I am looking for a tool in janusgraph1.0.0.rc2 (local install), that I can wipe out my data and schema quickly.  The tool that is  similar to "janusgraph.sh clean"  in the janusgraph-full version.  Thanks,

Currently if I do it from gremlin, I got evaluation timeout. 

Thanks Florian. Can you please help me with 

1. the exact ENV code  that I can add to my Docker to enable ConfigurationManagementGraph so that I can apply my own schema (using ConfiguredGraphFactory to create graph and create index) when docker start. 

2. the exact janusgprah env of the above if I use docker-compose to compose 3 containers janusgraph1.0.0-rc2, cassandra, and OpenSearch(ElasticSearch). I put the below lines in the env as you suggested in other email chain, and unfortunately they are not working except the last one. So what am missing here?

gremlinserver.graphs.ConfigurationManagementGraph=/etc/opt/janusgraph/janusgraph.properties

janusgraph.gremlin.graph=org.janusgraph.core.ConfiguredGraphFactory

janusgraph.graph.graphname=ConfigurationManagementGraph

gremlinserver%d.graphs.graph=

gremlinserver.evaluationTimeout=300000

I know the general concept,and I can set other env like gremlinserver.evaluationTimeout. I just can not get ConfiguredGraphFactory work me using docker or docker-compose. 

For now, yes, I get the local installation of janugraph1.0.0-rc2 work (e.g., use ConfiguredGraphFactory  to create graph etc) with the other two containers  by simply invoking janusgprah server with config: 
./bin/janusgraph-server.sh start conf/gremlin-server/gremlin-server-configuration.yaml and initialize it with my own schema,

i was able to inject near million nodes/edges into the graph (3-5 faster than the jansugraph-full version), but I would perfer to not creating my image from source code if there is docker image we can leverage.

Thanks 

Thank you

Yingjie

To report back, If I directly use janusgraph-1.0.0-rc local install,
after starting the server with  ./bin/janusgraph-server.sh start conf/gremlin-server/gremlin-server-configuration.yaml
i can create graph use ConfigureGraphFactory, and initize it with my own schema with composite and mxied index. 

To be able to traveral, I need to restart the server. then I can do the followings sucessfully: 

gremlin> :remote connect tinkerpop.server conf/remote.yaml session

08:41:02 INFO  com.jcabi.log.Logger.infoForced - 0 attributes loaded from 328 stream(s) in 56ms, 104 saved, 5165 ignored: []

08:41:03 INFO  org.apache.tinkerpop.gremlin.driver.Connection.<init> - Created new connection for ws://localhost:8182/gremlin

08:41:03 INFO  org.apache.tinkerpop.gremlin.driver.ConnectionPool.<init> - Opening connection pool on Host{address=localhost/127.0.0.1:8182, hostUri=ws://localhost:8182/gremlin} with core size of 1

==>Configured localhost/127.0.0.1:8182-[e732c251-4d85-4a3e-89b6-f5c8db565b47]

gremlin> :remote console

==>All scripts will now be sent to Gremlin Server - [localhost/127.0.0.1:8182]-[e732c251-4d85-4a3e-89b6-f5c8db565b47] - type ':remote console' to return to local mode

gremlin> ConfiguredGraphFactory.getGraphNames()

==>graph2

==>graph1

gremlin> graph2_traversal.V().count()

==>0

gremlin> graph1_traversal.V().count()

==>0

 

But I still will need to know how to use the docker unless there is no choice but to build the docker from source code.

 

Thanks,
Yingjei

Hello,

Sorry if this post is kind of a continuation of my last  post. As I have not received any suggestion, so i would like to start a new thread to surface up the problem. 

I would like to know the exactly code for me to be able to configure janusgraph-1.0.0-docker (either rebuild a new image from this base) or docker-compose it with env variable settings.  So that I can use ConfigurationManagementGraph and graph traversal for query.  I will need it to be initialized with my own schema groovy (where I define the graph name,  set up other configurations and created composite/mixed index into Cassandra/ElasticSearch( I used OpenSearch instead and it kind of worked).  A sample schema (init_graph_sample1.groovy) is attached, This is similar to the schema I have been using in Januagraph prepcakged versions.

From the previous response I got, it seems all these should have been supported already, but unfortunately it does not work for me with my various tries. I feel I am on the right track, but not exactly there yet. 

I attached a Docker file  (please ignore the jar removal part for now)  that I would like your help to adding the necessary code to use ConfigurationManagementGraph. Only after I get the configuration right, I can initize it with my own schema as attached, 

 

If you have a docker compose that shows the env settings for the above, I am glad to try that as well. I have been trying getting these settings in a env file of docker compose, but it did not work except for the last line. 

 

I have been using jansguraph prepackage versions, where I know exactly what yaml file to replace/change  to enable the above. But it becomes kind of mystery of how to do it in the unprepackage version.

I also tried having a local install of janusgraph-1.0.0-rc2, and invoke server by providing a configuration file which i think should be used: 

./bin/janusgraph-server.sh start conf/gremlin-server/gremlin-server-configuration-cql-es.yaml

And after starting gremlin.sh, I can create my own graph graph1 using ConfiguredGraphFactory, and can create mixed index, and I can see the graph1 is created in the backend cassandra and my indexes (not exactly the same name)  are created in the backend OpenSearch (ElasticSearch), pictures are attached, but I am not able to use graph1_traversal to query the graph. 

 

My request is kind of urgent as Janusgraph has been the core of our project that has been going on for two/three years, and we really need your help to bring it to the next level  for potential production. We have been used prepackaged version starting from 0.5.3, and recently we need to replace the all in one (janusgraph1.0.0-rc1-ful) with the lastest (janusgraph1.0.0-rc2) and use Cassandra and OpenSearch.  

Due to a tight deadline, would appreciate somebody can help me as soon as possible.  

I did some resource cleaning up and now the docker compose up of all these component are successful without below addtional env setup. What I need help is to confirm whether there are anything wrong with these configs and if so, shall I build a docker from based by manually change the config files like what I did before with janusgprah-full version. 

Thanks

 

Hello Florian,

Is below the right env setup  for janusgraph docker-compose  if I want to change the default? Thank you.



I think I was able to start Janusgraph 1.1.0CR2 without these configs when docker compose cassandra and opensearch.  There are some issues like  out of memeory  for cassandra and opensearch that I need to take care of seperately. But when I add these configures to the janusgraph.env. I was not abe to start janusgprah. Any suggestions? I am attaching the complete code with verbose output from both runs. 

3.janus1.0rc2-config-cs4.0.0-opensearch.zip  is the one with added configuration
2.janus1.0rc2-cs4.0.0-opensearch.zip is the one without.

I tried with https://github.com/JanusGraph/janusgraph-docker/blob/master/docker-compose-cql-es.yml by replacing the images with Janusgraph1.0.0RC2 and   Cassandra (4.0.0) and ElasticSearch *8.6.0) that are closest to the versions that are listed in the tested pltform of the Annoucement . So just to cofirm. the test were done with local installation but not at container level.  If so, Fwhat are the most recent versions that have been tested?

Thanks,
Yingjie

Hello Florian,

Thanks for the pointers. I started with this one as I need to replace Elastic Search With other index backends that have Apache license, prefer to be OpenSearch. I found an example (the one and only) that worked  (I tried) with  older images (janusgraph/janusgraph:0.6.2, cassandra:3,docker.io/opensearchproject/opensearch:2.4.0)  from  https://gist.github.com/Phate334/9fd90ddce0eaea74086c1a2e5d419402    that was  emebed in an earlier discussion in Janusgraph git:  https://github.com/JanusGraph/janusgraph/discussions/2837

As old images contains security issues such as log4j etc, so I swapped old images with the latest  in the docker compose yml,  but kept the env files the same, and encounter the error. 

As for "Your log shows that JanusGraph tries to connect to Cassandra under 127.0.0.1 which doesn’t make much sense if you are using Docker compose (and don’t use the host network)"  can you elaborate on it? 

I will also try the docker comopose yml link you share and see.

Thanks

 

 

 

In the  Janusgraph 1.0.0RC2 annoucement, it is said that "support Csssandra 4", So there should be a way to docker compose them. Is there something wrong/missing  in my code?


Thanks

Sorry, the simple case seems that it does not work.
Here case with a little complex after-check that can cause the exception.
g.addV("v1")
g.addV("v2").property("name", "joye")
g.V().order().by("name")

Sorry for any inconvenience it caused.

Best regards,
Joye

\u in a java/groovy string means that a unicode character follows. If that is not your intention, you have to escape the backslash, like:

gremlin> println '\\u河ⴛ)'
\u河ⴛ)
==>null

Best wishes,    Marc