Date   

Re: Caused by: org.janusgraph.core.JanusGraphException: A JanusGraph graph with the same instance id [0a000439355-0b2b58ca5c222] is already open. Might required forced shutdown.

hadoopmarc@...
 

Hi Srinivas,

Do I understand right that you get this error on restarting of gremlin server?
How does the yaml config file for gremlin server look like?

Best wishes,    Marc


Re: Script16.groovy: 2: unable to resolve class StandardJanusGraph

hadoopmarc@...
 

Hi Vinayak,

If I understand it right (did not test it), you have to use two gremlin consoles:

  1. Use one gremlin console to connect to your server and use ConfiguredGraphFactory. This will manage which graphs are available to to other gremlin console instances.
  2. Use the second gremlin console to connect to the graph that you activated in the first console. In the second console you can use a sessioned remote connection to do any imports you need to run your script remotely.
Best wishes,    Marc


Caused by: org.janusgraph.core.JanusGraphException: A JanusGraph graph with the same instance id [0a000439355-0b2b58ca5c222] is already open. Might required forced shutdown.

Real Life Adventure
 

Hi,

getting janusgraph same instance  issue even if i set graph.replace-instance-if-exists=true .please respond to below issue.any help appreciated.

graph.replace-instance-if-exists=true
jnausgraph version 0.5.2
 
 
- Version:0.5.2
- Storage Backend:cql
- Mixed Index Backend:elasticsearch
 
```
10689 [main] ERROR org.apache.tinkerpop.gremlin.server.util.ServerGremlinExecutor  - Could not invoke constructor on class org.janusgraph.graphdb.management.JanusGraphManager (defined by the 'graphManager' setting) with one argument of class Settings
 
 
Exception in thread "main" java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
 
 
at org.apache.tinkerpop.gremlin.server.util.ServerGremlinExecutor.<init>(ServerGremlinExecutor.java:91)
 
 
at org.apache.tinkerpop.gremlin.server.GremlinServer.<init>(GremlinServer.java:122)
 
 
at org.apache.tinkerpop.gremlin.server.GremlinServer.<init>(GremlinServer.java:86)
 
 
at org.apache.tinkerpop.gremlin.server.GremlinServer.main(GremlinServer.java:345)
 
 
Caused by: java.lang.reflect.InvocationTargetException
 
 
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
 
 
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
 
 
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
 
 
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
 
 
at org.apache.tinkerpop.gremlin.server.util.ServerGremlinExecutor.<init>(ServerGremlinExecutor.java:80)
 
 
... 3 more
 
 
Caused by: org.janusgraph.core.JanusGraphException: A JanusGraph graph with the same instance id [0a000439355-0b2b58ca5c222] is already open. Might required forced shutdown.
 
 
at org.janusgraph.graphdb.database.StandardJanusGraph.<init>(StandardJanusGraph.java:173)
 
 
at org.janusgraph.core.JanusGraphFactory.lambda$open$0(JanusGraphFactory.java:150)
 
 
at org.janusgraph.graphdb.management.JanusGraphManager.openGraph(JanusGraphManager.java:243)
 
 
at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:150)
 
 
at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:100)
 
 
at org.janusgraph.graphdb.management.JanusGraphManager.lambda$new$0(JanusGraphManager.java:75)
 
 
at java.util.LinkedHashMap.forEach(LinkedHashMap.java:684)
 
 
at org.janusgraph.graphdb.management.JanusGraphManager.<init>(JanusGraphManager.java:74)
 
 
... 8 more
```

Thanks,
Srinivas


Re: Not able to reindex with bigtable as backend

owner.mad.epa@...
 

Reindex process may stuck on ghost vertices, see https://github.com/JanusGraph/janusgraph/issues/1750

Try to remove ghost vertices by GhostVertexRemover

JanusGraphManagement.IndexJobFuture ghostRemover =
graph.getBackend().buildEdgeScanJob()
.setJob(new GhostVertexRemover(graph))
.execute();
try {
logger.info("GhostVertexRemover statistics: {},{},{}",
ghostRemover.get().getCustom(REMOVED_VERTEX_COUNT),
ghostRemover.get().getCustom(REMOVED_RELATION_COUNT),
ghostRemover.get().getCustom(SKIPPED_GHOST_LIMIT_COUNT));
} catch (InterruptedException | ExecutionException e) {
throw new RuntimeException(e);
}


Re: JMX authentication for cassandra

Vinayak Bali
 

Hi Marc,

How can I enable JMX authentication for the Cassandra daemon ?? 

Thanks & Regards,
Vinayak

On Tue, Mar 9, 2021 at 10:01 PM <hadoopmarc@...> wrote:
Hi Vinayak,

Please review the change you made in the cassandra-env.sh, it does not make sense.  I do not know about JMX settings for Cassandra either, but the cassandra-env.sh is written such that you do not have to.

Best wishes,  Marc


Re: Multiple vertices generated for the same index value and vertex properties missing with RF3

sauverma
 

Hi Marc

==>------------------------------------------------------------------------------------------------
Vertex Label Name              | Partitioned | Static                                             |
---------------------------------------------------------------------------------------------------
node                           | false       | false                                              |
---------------------------------------------------------------------------------------------------
Edge Label Name                | Directed    | Unidirected | Multiplicity                         |
---------------------------------------------------------------------------------------------------
link                           | true        | false       | SIMPLE                               |
---------------------------------------------------------------------------------------------------
Property Key Name              | Cardinality | Data Type                                          |
---------------------------------------------------------------------------------------------------
country_LUX                    | SET         | class java.lang.Integer                            |
country_FRA                    | SET         | class java.lang.Integer                            |
country_CHE                    | SET         | class java.lang.Integer                            |
country_ESP                    | SET         | class java.lang.Integer                            |
dpts_0                         | SINGLE      | class java.lang.Long                               |
dpts_100                       | SINGLE      | class java.lang.Long                               |
dpts_101                       | SINGLE      | class java.lang.Long                               |
dpts_1011                      | SINGLE      | class java.lang.Long                               |
dpts_103                       | SINGLE      | class java.lang.Long                               |
dpts_1038                      | SINGLE      | class java.lang.Long                               |
dpts_104                       | SINGLE      | class java.lang.Long                               |
dpts_1045                      | SINGLE      | class java.lang.Long                               |
dpts_105                       | SINGLE      | class java.lang.Long                               |
dpts_1055                      | SINGLE      | class java.lang.Long                               |
dpts_1076                      | SINGLE      | class java.lang.Long                               |
dpts_1077                      | SINGLE      | class java.lang.Long                               |
dpts_1081                      | SINGLE      | class java.lang.Long                               |
dpts_1085                      | SINGLE      | class java.lang.Long                               |
dpts_1099                      | SINGLE      | class java.lang.Long                               |
dpts_1110                      | SINGLE      | class java.lang.Long                               |
dpts_1129                      | SINGLE      | class java.lang.Long                               |
dpts_1141                      | SINGLE      | class java.lang.Long                               |
dpts_1146                      | SINGLE      | class java.lang.Long                               |
dpts_1147                      | SINGLE      | class java.lang.Long                               |
dpts_1148                      | SINGLE      | class java.lang.Long                               |
dpts_1168                      | SINGLE      | class java.lang.Long                               |
dpts_118                       | SINGLE      | class java.lang.Long                               |
dpts_12                        | SINGLE      | class java.lang.Long                               |
dpts_120                       | SINGLE      | class java.lang.Long                               |
dpts_1201                      | SINGLE      | class java.lang.Long                               |
dpts_121                       | SINGLE      | class java.lang.Long                               |
dpts_1222                      | SINGLE      | class java.lang.Long                               |
dpts_123                       | SINGLE      | class java.lang.Long                               |
dpts_1249                      | SINGLE      | class java.lang.Long                               |
dpts_1252                      | SINGLE      | class java.lang.Long                               |
dpts_1257                      | SINGLE      | class java.lang.Long                               |
dpts_1258                      | SINGLE      | class java.lang.Long                               |
dpts_1263                      | SINGLE      | class java.lang.Long                               |
dpts_1271                      | SINGLE      | class java.lang.Long                               |
dpts_1290                      | SINGLE      | class java.lang.Long                               |
dpts_1292                      | SINGLE      | class java.lang.Long                               |
dpts_1295                      | SINGLE      | class java.lang.Long                               |
dpts_13                        | SINGLE      | class java.lang.Long                               |
dpts_1302                      | SINGLE      | class java.lang.Long                               |
dpts_1314                      | SINGLE      | class java.lang.Long                               |
dpts_1316                      | SINGLE      | class java.lang.Long                               |
dpts_1325                      | SINGLE      | class java.lang.Long                               |
dpts_1326                      | SINGLE      | class java.lang.Long                               |
dpts_1332                      | SINGLE      | class java.lang.Long                               |
dpts_1344                      | SINGLE      | class java.lang.Long                               |
dpts_1346                      | SINGLE      | class java.lang.Long                               |
dpts_1348                      | SINGLE      | class java.lang.Long                               |
dpts_135                       | SINGLE      | class java.lang.Long                               |
dpts_1353                      | SINGLE      | class java.lang.Long                               |
dpts_1354                      | SINGLE      | class java.lang.Long                               |
dpts_1357                      | SINGLE      | class java.lang.Long                               |
dpts_1361                      | SINGLE      | class java.lang.Long                               |
dpts_1363                      | SINGLE      | class java.lang.Long                               |
dpts_1380                      | SINGLE      | class java.lang.Long                               |
dpts_1382                      | SINGLE      | class java.lang.Long                               |
dpts_1396                      | SINGLE      | class java.lang.Long                               |
dpts_1401                      | SINGLE      | class java.lang.Long                               |
dpts_1406                      | SINGLE      | class java.lang.Long                               |
dpts_1408                      | SINGLE      | class java.lang.Long                               |
country_NLD                    | SET         | class java.lang.Integer                            |
dpts_1409                      | SINGLE      | class java.lang.Long                               |
dpts_1410                      | SINGLE      | class java.lang.Long                               |
dpts_1415                      | SINGLE      | class java.lang.Long                               |
dpts_1429                      | SINGLE      | class java.lang.Long                               |
dpts_1441                      | SINGLE      | class java.lang.Long                               |
dpts_1443                      | SINGLE      | class java.lang.Long                               |
dpts_1479                      | SINGLE      | class java.lang.Long                               |
dpts_1480                      | SINGLE      | class java.lang.Long                               |
dpts_1491                      | SINGLE      | class java.lang.Long                               |
dpts_18                        | SINGLE      | class java.lang.Long                               |
dpts_1801                      | SINGLE      | class java.lang.Long                               |
dpts_21                        | SINGLE      | class java.lang.Long                               |
dpts_212                       | SINGLE      | class java.lang.Long                               |
dpts_213                       | SINGLE      | class java.lang.Long                               |
dpts_215                       | SINGLE      | class java.lang.Long                               |
dpts_216                       | SINGLE      | class java.lang.Long                               |
dpts_22                        | SINGLE      | class java.lang.Long                               |
dpts_222                       | SINGLE      | class java.lang.Long                               |
dpts_226                       | SINGLE      | class java.lang.Long                               |
dpts_229                       | SINGLE      | class java.lang.Long                               |
dpts_233                       | SINGLE      | class java.lang.Long                               |
dpts_237                       | SINGLE      | class java.lang.Long                               |
dpts_239                       | SINGLE      | class java.lang.Long                               |
dpts_24                        | SINGLE      | class java.lang.Long                               |
dpts_255                       | SINGLE      | class java.lang.Long                               |
dpts_258                       | SINGLE      | class java.lang.Long                               |
dpts_26                        | SINGLE      | class java.lang.Long                               |
dpts_27                        | SINGLE      | class java.lang.Long                               |
dpts_28                        | SINGLE      | class java.lang.Long                               |
dpts_29                        | SINGLE      | class java.lang.Long                               |
dpts_307                       | SINGLE      | class java.lang.Long                               |
dpts_31                        | SINGLE      | class java.lang.Long                               |
dpts_313                       | SINGLE      | class java.lang.Long                               |
dpts_314                       | SINGLE      | class java.lang.Long                               |
dpts_320                       | SINGLE      | class java.lang.Long                               |
dpts_322                       | SINGLE      | class java.lang.Long                               |
dpts_329                       | SINGLE      | class java.lang.Long                               |
dpts_33                        | SINGLE      | class java.lang.Long                               |
dpts_331                       | SINGLE      | class java.lang.Long                               |
dpts_335                       | SINGLE      | class java.lang.Long                               |
dpts_336                       | SINGLE      | class java.lang.Long                               |
dpts_338                       | SINGLE      | class java.lang.Long                               |
dpts_34                        | SINGLE      | class java.lang.Long                               |
dpts_346                       | SINGLE      | class java.lang.Long                               |
dpts_35                        | SINGLE      | class java.lang.Long                               |
dpts_362                       | SINGLE      | class java.lang.Long                               |
dpts_37                        | SINGLE      | class java.lang.Long                               |
dpts_370                       | SINGLE      | class java.lang.Long                               |
dpts_372                       | SINGLE      | class java.lang.Long                               |
dpts_374                       | SINGLE      | class java.lang.Long                               |
dpts_376                       | SINGLE      | class java.lang.Long                               |
dpts_391                       | SINGLE      | class java.lang.Long                               |
dpts_395                       | SINGLE      | class java.lang.Long                               |
dpts_41                        | SINGLE      | class java.lang.Long                               |
dpts_416                       | SINGLE      | class java.lang.Long                               |
dpts_45                        | SINGLE      | class java.lang.Long                               |
dpts_50                        | SINGLE      | class java.lang.Long                               |
dpts_516                       | SINGLE      | class java.lang.Long                               |
dpts_521                       | SINGLE      | class java.lang.Long                               |
dpts_522                       | SINGLE      | class java.lang.Long                               |
dpts_525                       | SINGLE      | class java.lang.Long                               |
dpts_529                       | SINGLE      | class java.lang.Long                               |
dpts_532                       | SINGLE      | class java.lang.Long                               |
dpts_533                       | SINGLE      | class java.lang.Long                               |
dpts_556                       | SINGLE      | class java.lang.Long                               |
dpts_56                        | SINGLE      | class java.lang.Long                               |
dpts_566                       | SINGLE      | class java.lang.Long                               |
dpts_568                       | SINGLE      | class java.lang.Long                               |
dpts_575                       | SINGLE      | class java.lang.Long                               |
dpts_58                        | SINGLE      | class java.lang.Long                               |
dpts_59                        | SINGLE      | class java.lang.Long                               |
dpts_598                       | SINGLE      | class java.lang.Long                               |
dpts_611                       | SINGLE      | class java.lang.Long                               |
dpts_613                       | SINGLE      | class java.lang.Long                               |
dpts_620                       | SINGLE      | class java.lang.Long                               |
dpts_625                       | SINGLE      | class java.lang.Long                               |
dpts_627                       | SINGLE      | class java.lang.Long                               |
dpts_63                        | SINGLE      | class java.lang.Long                               |
dpts_634                       | SINGLE      | class java.lang.Long                               |
dpts_637                       | SINGLE      | class java.lang.Long                               |
dpts_648                       | SINGLE      | class java.lang.Long                               |
dpts_654                       | SINGLE      | class java.lang.Long                               |
dpts_667                       | SINGLE      | class java.lang.Long                               |
dpts_67                        | SINGLE      | class java.lang.Long                               |
dpts_678                       | SINGLE      | class java.lang.Long                               |
dpts_68                        | SINGLE      | class java.lang.Long                               |
dpts_685                       | SINGLE      | class java.lang.Long                               |
dpts_69                        | SINGLE      | class java.lang.Long                               |
dpts_697                       | SINGLE      | class java.lang.Long                               |
dpts_7123                      | SINGLE      | class java.lang.Long                               |
dpts_735                       | SINGLE      | class java.lang.Long                               |
dpts_738                       | SINGLE      | class java.lang.Long                               |
dpts_75                        | SINGLE      | class java.lang.Long                               |
dpts_751                       | SINGLE      | class java.lang.Long                               |
dpts_754                       | SINGLE      | class java.lang.Long                               |
dpts_76                        | SINGLE      | class java.lang.Long                               |
dpts_768                       | SINGLE      | class java.lang.Long                               |
dpts_78                        | SINGLE      | class java.lang.Long                               |
dpts_784                       | SINGLE      | class java.lang.Long                               |
dpts_799                       | SINGLE      | class java.lang.Long                               |
dpts_800                       | SINGLE      | class java.lang.Long                               |
dpts_801                       | SINGLE      | class java.lang.Long                               |
dpts_806                       | SINGLE      | class java.lang.Long                               |
dpts_807                       | SINGLE      | class java.lang.Long                               |
dpts_808                       | SINGLE      | class java.lang.Long                               |
dpts_809                       | SINGLE      | class java.lang.Long                               |
dpts_810                       | SINGLE      | class java.lang.Long                               |
dpts_82                        | SINGLE      | class java.lang.Long                               |
dpts_820                       | SINGLE      | class java.lang.Long                               |
dpts_824                       | SINGLE      | class java.lang.Long                               |
dpts_84                        | SINGLE      | class java.lang.Long                               |
dpts_86                        | SINGLE      | class java.lang.Long                               |
dpts_87                        | SINGLE      | class java.lang.Long                               |
dpts_871                       | SINGLE      | class java.lang.Long                               |
dpts_876                       | SINGLE      | class java.lang.Long                               |
dpts_90901                     | SINGLE      | class java.lang.Long                               |
dpts_946                       | SINGLE      | class java.lang.Long                               |
dpts_95                        | SINGLE      | class java.lang.Long                               |
dpts_96                        | SINGLE      | class java.lang.Long                               |
dpts_963                       | SINGLE      | class java.lang.Long                               |
dpts_979                       | SINGLE      | class java.lang.Long                               |
dpts_98                        | SINGLE      | class java.lang.Long                               |
dpts_984                       | SINGLE      | class java.lang.Long                               |
dpts_988                       | SINGLE      | class java.lang.Long                               |
dpts_99                        | SINGLE      | class java.lang.Long                               |
device_deviceos                | SET         | class java.lang.String                             |
device_ipaddress               | SET         | class java.lang.String                             |
id                             | SINGLE      | class java.lang.String                             |
country_AUT                    | SET         | class java.lang.Integer                            |
type                           | SINGLE      | class java.lang.String                             |
soft_del                       | SINGLE      | class java.lang.Boolean                            |
country_IRL                    | SET         | class java.lang.Integer                            |
country_ITA                    | SET         | class java.lang.Integer                            |
country_GBR                    | SET         | class java.lang.Integer                            |
country_BEL                    | SET         | class java.lang.Integer                            |
country_DEU                    | SET         | class java.lang.Integer                            |
country_GEO                    | SET         | class java.lang.Integer                            |
---------------------------------------------------------------------------------------------------
Vertex Index Name              | Type        | Unique    | Backing        | Key:           Status |
---------------------------------------------------------------------------------------------------
idIndex                        | Composite   | true      | internalindex  | id:           ENABLED |
                               |             |           |                | type:         ENABLED |
---------------------------------------------------------------------------------------------------
Edge Index (VCI) Name          | Type        | Unique    | Backing        | Key:           Status |
---------------------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------------------
Relation Index                 | Type        | Direction | Sort Key       | Order    |     Status |
---------------------------------------------------------------------------------------------------

After modifying index and properties to ConsistenceModifier.LOCK as per https://docs.janusgraph.org/advanced-topics/eventual-consistency/
the issue seems to be fixed

Thanks


Re: How to circumvent transaction cache?

Boxuan Li
 

My thoughts are:

1) As you said, you wanted to be able to disable the transaction cache so that you can read from database again. I have provided a solution here: https://lists.lfaidata.foundation/g/janusgraph-users/message/5668 A PR is available here: https://github.com/JanusGraph/janusgraph/pull/2502
2) An alternative approach (apart from your external locking approach) is to use JanusGraph built-in locking mechanism, as we have discussed here: https://groups.google.com/g/janusgraph-users/c/WzsO78ndobA/m/e6GzFXI5CQAJ

Although the above approaches will likely work most of the time, they are not guaranteed to be robust due to the eventual consistency nature of HBase. If you need ACID you should switch to FoundationDB backend. To my best knowledge, there is no way that JanusGraph can provide ACID on top of an eventual consistent storage backend, because the graph instances can only “communicate" with each other via the underlying storage backend.

On Mar 8, 2021, at 9:40 PM, timon.schneider@... wrote:

Currently using HBase.

Consider the following:
User A decides to set isPublished of Vertex X from false to true, does not commit yet.
User B changes isPublished of Vertex X from false to true and commits immediately.
User A commits and will get an error because the property value is not the same anymore as at the start of the transaction.

Why wouldn't it be possible for JG to provide user A with select for update functionality that allows user A to select vertex X for update, do edits to other elements, commit and get the same message as in the example above if the property on vertex X is changed?


Re: Multiple vertices generated for the same index value and vertex properties missing with RF3

hadoopmarc@...
 

Hi Saurabh,

To be sure, in the valueMap(true) return value I see both the id token and an "id" property key appear (I am not even sure whether an "id" property key is allowed). Is this "id" property key something that was part of the input data of the graph? Can you also show the graph schema using:
mgmt = graph.openManagement()
mgmt.printSchema()
Best wishes,    Marc


Re: Script16.groovy: 2: unable to resolve class StandardJanusGraph

Vinayak Bali
 

Hi Marc,

The same code works for JanusGraphFactory but not for ConfiguredGraphFactory. I investigated issue further. For ConfiguredGraphFactory we establish remote connection using following commands:
bin/gremlin.sh
:remote connect tinkerpop.server conf/remote.yaml session
:remote console

The imports are not working after establishing remote connection. Imports works without remote connection. How can we solve this problem ???

Thanks & Regards,
Vinayak

On Tue, 9 Mar 2021, 9:30 pm , <hadoopmarc@...> wrote:
Hi Vinayak,

I only replayed exactly your imports, not the rest of your code. Downloading dependencies may depend on the grape configs of your system, see:

https://docs.janusgraph.org/basics/server/#using-tinkerpop-gremlin-server-with-janusgraph
https://tinkerpop.apache.org/docs/current/reference/#gremlin-applications

Best wishes,    Marc


Re: JMX authentication for cassandra

hadoopmarc@...
 

Hi Vinayak,

Please review the change you made in the cassandra-env.sh, it does not make sense.  I do not know about JMX settings for Cassandra either, but the cassandra-env.sh is written such that you do not have to.

Best wishes,  Marc


Re: [Performance Issue] Large partitions formed on janusgraph_ids table leading to read perf issues (throughput reduces to 1/3rd of original)

hadoopmarc@...
 
Edited

I learned from your other thread that you use many spark executors that have their own janusgraph instance. I remember that I used a similar scheme many years ago with janusgraph-0.1.1. At that time I simply stored the janusgraph id's at load time in a partitioned file on hdfs, so that I could later use them for analytics queries with spark.

Could you elaborate on what you call the janusgraph_ids table and how you distribute vertex id's to your spark executors? According to the janusgraph data model there is no separate id  table, but vertex id's are encoded in the row key.

Best wishes,     Marc


Re: [Performance Issue] Large partitions formed on janusgraph_ids table leading to read perf issues (throughput reduces to 1/3rd of original)

hadoopmarc@...
 


Re: Script16.groovy: 2: unable to resolve class StandardJanusGraph

hadoopmarc@...
 

Hi Vinayak,

I only replayed exactly your imports, not the rest of your code. Downloading dependencies may depend on the grape configs of your system, see:

https://docs.janusgraph.org/basics/server/#using-tinkerpop-gremlin-server-with-janusgraph
https://tinkerpop.apache.org/docs/current/reference/#gremlin-applications

Best wishes,    Marc


[Performance Issue] Large partitions formed on janusgraph_ids table leading to read perf issues (throughput reduces to 1/3rd of original)

sauverma
 

Hi all

We are using janusgraph at zeotap at humongous scale (~70B V and 50B E) backed by scylla.

Right now I am facing an issue in janusgraph_ids table, wherein there are large partitions created in the scylla DB, and this is leading to huge read performance issues. The queries hitting janusgraph_ids table are range queries and with large partitions, the reads are becoming super slow.  

I would like to know if anyone else has observed similar issue, is there a set of configurations that need to be checked or something else you would suggest.

In scylla grafana board, this issue is seen with high number of foreground read tasks.



Thanks
Saurabh


Multiple vertices generated for the same index value and vertex properties missing with RF3

sauverma
 

We are facing a weird issue with Replication factor 3 with scylla DB as backend.  

- we are using index for looking up V in the graph
- observation is that for many vertices with the same index value, multiple vertex IDs are generated as seen below



Has anyone else encountered the same issue with RF3, with RF1 the issue is not encountered

Thanks
Saurabh Verma


Re: Script16.groovy: 2: unable to resolve class StandardJanusGraph

Vinayak Bali
 

Hi Marc,

If possible please share the code used, let me check if it can be used to modify my code.

Thanks & Regards,
Vinayak

On Tue, 9 Mar 2021, 12:54 pm , <hadoopmarc@...> wrote:
Hi Vinayak,

When I replayed your imports I got the same error, but when I tried again it worked. I guess the delay for downloading dependencies was too high and second time the cache could be used.

Best wishes,    Marc


Re: Script16.groovy: 2: unable to resolve class StandardJanusGraph

hadoopmarc@...
 

Hi Vinayak,

When I replayed your imports I got the same error, but when I tried again it worked. I guess the delay for downloading dependencies was too high and second time the cache could be used.

Best wishes,    Marc


Re: Script16.groovy: 2: unable to resolve class StandardJanusGraph

Vinayak Bali
 

Hi Marc,

Explicitly, imported StandardJanusGraph using the following import command which is not working:
org.janusgraph.graphdb.database.StandardJanusGraph

Tried Graph approach too, it throws the following error:

Script158.groovy: 1: unable to resolve class LinkedBlockingQueue
 @ line 1, column 1.
   class WaitingQueue<E> extends LinkedBlockingQueue<E>
   ^

1 error
Type ':help' or ':h' for help.
Display stack trace? [yN]y
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
Script158.groovy: 1: unable to resolve class LinkedBlockingQueue
 @ line 1, column 1.
   class WaitingQueue<E> extends LinkedBlockingQueue<E>
   ^

1 error

at org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:311)
at org.codehaus.groovy.control.CompilationUnit.applyToSourceUnits(CompilationUnit.java:980)
at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:647)
at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:596)
at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:390)
at groovy.lang.GroovyClassLoader.access$300(GroovyClassLoader.java:89)
at groovy.lang.GroovyClassLoader$5.provide(GroovyClassLoader.java:330)
at groovy.lang.GroovyClassLoader$5.provide(GroovyClassLoader.java:327)
at org.codehaus.groovy.runtime.memoize.ConcurrentCommonCache.getAndPut(ConcurrentCommonCache.java:147)
at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:325)
at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:309)
at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:251)
at org.apache.tinkerpop.gremlin.groovy.jsr223.GremlinGroovyScriptEngine$GroovyCacheLoader.lambda$load$0(GremlinGroovyScriptEngine.java:819)
at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604)
at java.util.concurrent.CompletableFuture.asyncSupplyStage(CompletableFuture.java:1618)
at java.util.concurrent.CompletableFuture.supplyAsync(CompletableFuture.java:1843)
at org.apache.tinkerpop.gremlin.groovy.jsr223.GremlinGroovyScriptEngine$GroovyCacheLoader.load(GremlinGroovyScriptEngine.java:817)
at org.apache.tinkerpop.gremlin.groovy.jsr223.GremlinGroovyScriptEngine$GroovyCacheLoader.load(GremlinGroovyScriptEngine.java:812)
at com.github.benmanes.caffeine.cache.BoundedLocalCache$BoundedLocalLoadingCache.lambda$new$0(BoundedLocalCache.java:3117)
at com.github.benmanes.caffeine.cache.LocalCache.lambda$statsAware$0(LocalCache.java:144)
at com.github.benmanes.caffeine.cache.BoundedLocalCache.lambda$doComputeIfAbsent$16(BoundedLocalCache.java:1968)
at java.util.concurrent.ConcurrentHashMap.compute(ConcurrentHashMap.java:1853)
at com.github.benmanes.caffeine.cache.BoundedLocalCache.doComputeIfAbsent(BoundedLocalCache.java:1966)
at com.github.benmanes.caffeine.cache.BoundedLocalCache.computeIfAbsent(BoundedLocalCache.java:1949)
at com.github.benmanes.caffeine.cache.LocalCache.computeIfAbsent(LocalCache.java:113)
at com.github.benmanes.caffeine.cache.LocalLoadingCache.get(LocalLoadingCache.java:67)
at org.apache.tinkerpop.gremlin.groovy.jsr223.GremlinGroovyScriptEngine.getScriptClass(GremlinGroovyScriptEngine.java:567)
at org.apache.tinkerpop.gremlin.groovy.jsr223.GremlinGroovyScriptEngine.eval(GremlinGroovyScriptEngine.java:374)
at javax.script.AbstractScriptEngine.eval(AbstractScriptEngine.java:233)
at org.apache.tinkerpop.gremlin.groovy.engine.GremlinExecutor.lambda$eval$0(GremlinExecutor.java:267)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

All the import statements are:
import org.janusgraph.core.ConfiguredGraphFactory
//import org.janusgraph.core.Cardinality
//import org.apache.tinkerpop.gremlin.structure.Vertex

import java.util.concurrent.Executors
import java.util.concurrent.LinkedBlockingQueue
import java.util.concurrent.ThreadPoolExecutor
import java.util.concurrent.TimeUnit

import org.janusgraph.graphdb.database.StandardJanusGraph
import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversalSource
import com.opencsv.CSVReaderBuilder
import com.opencsv.CSVReader
import com.opencsv.enums.CSVReaderNullFieldIndicator

Based on your experience, please suggest an approach using which I can solve the issue.

Thanks & Regards,
Vinayak

On Mon, Mar 8, 2021 at 5:50 PM <hadoopmarc@...> wrote:
Hi Vinayak,

What may confuse you is that the gremlin console does a lot of under the hood imports, but it doesn't import all janusgraph classes. So, you can solve this in two ways:
  1. Preferred: rather use the underlying interface for specifiying the type, so line 3 private Graph graph;
  2. Optional: do an explicit import of StandardJanusGraph
gremlin> StandardJanusGraph
No such property: StandardJanusGraph for class: groovysh_evaluate
Type ':help' or ':h' for help.
Display stack trace? [yN]n
gremlin> Graph
==>interface org.apache.tinkerpop.gremlin.structure.Graph


Best wishes,    Marc


Re: Driver's for java connectivity

Vinayak Bali
 

Hi,

The goal is to execute a query which retrieves all the data from database and store it to a file to analyse the results 

Thanks 
Vinayak 

On Mon, 8 Mar 2021, 8:49 pm Kevin Schmidt, <ktschmidt@...> wrote:
Vinayak,

If your goal is simply to get your entire graph into a JSON-based format, you may want to consider GraphSON (http://tinkerpop.apache.org/docs/3.4.1/dev/io/#graphson).

Kevin

On Mon, Mar 8, 2021 at 7:13 AM Vinayak Bali <vinayakbali16@...> wrote:
Hi Kevin,

To verify whether the issue is from the server or client-side, we are trying to execute the same query on the server and output the data in the file. But still it's not possible. Janusgraph is using Cassandra Dameon, hence both are on the same jvm. We are testing for further use-cases which can contain more data than now.

Thanks & Regards,
Vinayak

On Mon, Mar 8, 2021 at 8:37 PM Kevin Schmidt <ktschmidt@...> wrote:
Vinayak,

Is the out of memory error in the server or the client?  And are Janusgraph and Cassandra running in the same JVM or are you running Cassandra separately?

And what is your use case for your client needing to retrieve your entire graph?

Kevin

On Mon, Mar 8, 2021 at 7:01 AM Vinayak Bali <vinayakbali16@...> wrote:
Hi Kevin,

JVM size at the server where janusgraph and Cassandra (same machine) is set to 24GB. On the API side, we tried setting it to 8GB. Is there any other way using which we can write the query result  to a file in json format ??

Thanks & Regards,
Vinayak

On Mon, Mar 8, 2021 at 8:20 PM Kevin Schmidt <ktschmidt@...> wrote:
Vinayak,

What about the memory settings, specifically the max heap, for the JVM Janusgraph is running in?

Your traversal is effectively retrieving your entire graph.  Is that really what you need to do?  Or is this just a test and your real scenarios wouldn't be doing that?  Doing a retrieval like this is going to require a lot of memory and your JVM simply isn't configured with enough to complete it, thus the out of memory error.

Kevin

On Mon, Mar 8, 2021 at 6:35 AM Vinayak Bali <vinayakbali16@...> wrote:
Hi Kevin,

Query: g.V().as('a').outE().as('b').inV().as('c').select('a','b','c').by(valueMap().by(unfold()))

Memory Settings:
maxInitialLineLength: 4096
maxHeaderSize: 8192
maxChunkSize: 8192
maxContentLength: 2000000000
maxAccumulationBufferComponents: 1024
resultIterationBatchSize: 64
writeBufferLowWaterMark: 32768
writeBufferHighWaterMark: 65536

Thanks & Regards,
Vinayak

On Mon, Mar 8, 2021 at 8:01 PM Kevin Schmidt <ktschmidt@...> wrote:
Vinayak,

What is the query?  It sounds like whatever it is requires instantiating a lot of objects in order to process the results and construct the JSON.

What are your memory settings for the Janusgraph server?

Kevin

On Mon, Mar 8, 2021 at 6:11 AM Vinayak Bali <vinayakbali16@...> wrote:
Hi Marc,

While trying to execute a query using HTTP throws an out memory error. The number of records returned is more hence can't get the entire output. I want an entire output of the query in the file JSON format. Size of graph nodes: 7 lakh and edges: 3 lakh.

Thanks & Regards,
Vinayak 

On Mon, Mar 8, 2021 at 5:52 PM <hadoopmarc@...> wrote:
Hi Vinayak,

What actually is the "desired operation", it is not clear to me? In other words, what is the result after you have run the desired operation?

Best wishes,    Marc


Re: Driver's for java connectivity

Kevin Schmidt
 

Vinayak,

If your goal is simply to get your entire graph into a JSON-based format, you may want to consider GraphSON (http://tinkerpop.apache.org/docs/3.4.1/dev/io/#graphson).

Kevin

On Mon, Mar 8, 2021 at 7:13 AM Vinayak Bali <vinayakbali16@...> wrote:
Hi Kevin,

To verify whether the issue is from the server or client-side, we are trying to execute the same query on the server and output the data in the file. But still it's not possible. Janusgraph is using Cassandra Dameon, hence both are on the same jvm. We are testing for further use-cases which can contain more data than now.

Thanks & Regards,
Vinayak

On Mon, Mar 8, 2021 at 8:37 PM Kevin Schmidt <ktschmidt@...> wrote:
Vinayak,

Is the out of memory error in the server or the client?  And are Janusgraph and Cassandra running in the same JVM or are you running Cassandra separately?

And what is your use case for your client needing to retrieve your entire graph?

Kevin

On Mon, Mar 8, 2021 at 7:01 AM Vinayak Bali <vinayakbali16@...> wrote:
Hi Kevin,

JVM size at the server where janusgraph and Cassandra (same machine) is set to 24GB. On the API side, we tried setting it to 8GB. Is there any other way using which we can write the query result  to a file in json format ??

Thanks & Regards,
Vinayak

On Mon, Mar 8, 2021 at 8:20 PM Kevin Schmidt <ktschmidt@...> wrote:
Vinayak,

What about the memory settings, specifically the max heap, for the JVM Janusgraph is running in?

Your traversal is effectively retrieving your entire graph.  Is that really what you need to do?  Or is this just a test and your real scenarios wouldn't be doing that?  Doing a retrieval like this is going to require a lot of memory and your JVM simply isn't configured with enough to complete it, thus the out of memory error.

Kevin

On Mon, Mar 8, 2021 at 6:35 AM Vinayak Bali <vinayakbali16@...> wrote:
Hi Kevin,

Query: g.V().as('a').outE().as('b').inV().as('c').select('a','b','c').by(valueMap().by(unfold()))

Memory Settings:
maxInitialLineLength: 4096
maxHeaderSize: 8192
maxChunkSize: 8192
maxContentLength: 2000000000
maxAccumulationBufferComponents: 1024
resultIterationBatchSize: 64
writeBufferLowWaterMark: 32768
writeBufferHighWaterMark: 65536

Thanks & Regards,
Vinayak

On Mon, Mar 8, 2021 at 8:01 PM Kevin Schmidt <ktschmidt@...> wrote:
Vinayak,

What is the query?  It sounds like whatever it is requires instantiating a lot of objects in order to process the results and construct the JSON.

What are your memory settings for the Janusgraph server?

Kevin

On Mon, Mar 8, 2021 at 6:11 AM Vinayak Bali <vinayakbali16@...> wrote:
Hi Marc,

While trying to execute a query using HTTP throws an out memory error. The number of records returned is more hence can't get the entire output. I want an entire output of the query in the file JSON format. Size of graph nodes: 7 lakh and edges: 3 lakh.

Thanks & Regards,
Vinayak 

On Mon, Mar 8, 2021 at 5:52 PM <hadoopmarc@...> wrote:
Hi Vinayak,

What actually is the "desired operation", it is not clear to me? In other words, what is the result after you have run the desired operation?

Best wishes,    Marc

941 - 960 of 6651