Date   

Re: Unconfigured table exceptions in Janusgraph 0.6.0

Oleksandr Porunov
 

Great to hear that worked for you.

Caused by: com.datastax.oss.driver.api.core.DriverTimeoutException: Query timed out after PT2S
As seen from this PR: https://github.com/JanusGraph/janusgraph/pull/2812 the default cql request timeout of 10 seconds isn't applied if you don't provide it explicitly. Thus, it now default to 2 seconds (as seen from DataStax driver default configurations). You can set it explicitly by providing the next config: `storage.cql.request-timeout = 10000` (to make it 10 seconds timeout).

Best regards,
Oleksandr Porunov


Re: Can we find connected nodes without using gremlin query

anjanisingh22@...
 

Thanks Marc for response.

Our graph is very large around 1TB, so can not hold in memory.
Subgraphs that we want to retrieve can have 30 -40 connected nodes.
We have requirement of around 200 TPS.
We are using JanusGraph only,  i meant i am using gremlin query to search in Janus graph.

Thanks,
Anjani
 


Re: JanusGraph 0.6.0 Binary Driver failing on serialization

hadoopmarc@...
 

Hi Chris,

Hard to interpret your report. It maybe worthwhile to try the documented way of remote connection with traversal.withRemote(...) or Cluster.open(...):

https://tinkerpop.apache.org/docs/current/reference/#gremlin-java-connecting

There should be no need to instantiate serializer objects yourself.

Best wishes,    Marc


Template configuration parameters with comma-separated lists in Janusgraph 0.6.0

Mladen Marović
 

Hello,

I wanted to try out the new Janusgraph 0.6.0 release and I encountered some unexpected issues while trying to deploy it on Cassandra 3.11.5.

One of the issues I came across seems to be connected to the switch to commons-configuration2. In previous Janusgraph versions, I used a startup script to create a configuration template for graphs and a single default_ graph:

def globals = [:]

globals << [hook : [
    onStartUp: { ctx ->
        def map = new HashMap<String, Object>();

        map.put('schema.default', 'none');
        map.put('schema.constraints', 'true');

        map.put('graph.allow-upgrade', 'false');

        map.put('storage.backend', 'cql');
        map.put('storage.hostname', 'test-master,test-worker1');
        map.put('storage.cql.replication-factor', '1');
        map.put('storage.cql.read-consistency-level', 'LOCAL_QUORUM');
        map.put('storage.cql.write-consistency-level', 'LOCAL_QUORUM');
        map.put('storage.cql.only-use-local-consistency-for-system-operations', 'true');
        map.put('storage.cql.local-datacenter', 'dc1');
        map.put('storage.cql.replication-strategy-options', 'dc1,1');
        map.put('storage.cql.replication-strategy-class', 'NetworkTopologyStrategy');

        map.put('index.es.backend', 'elasticsearch');
        map.put('index.es.hostname', 'test-dev-elasticsearch');
        map.put('index.es.elasticsearch.client-only', "true");
        map.put('index.es.elasticsearch.create.ext.index.number_of_shards', '5');
        map.put('index.es.elasticsearch.create.ext.index.number_of_replicas', '1');

        def conf = new MapConfiguration(map);
        conf.setDelimiterParsingDisabled(true);

        if (ConfiguredGraphFactory.getTemplateConfiguration() == null) {
            ConfiguredGraphFactory.createTemplateConfiguration(conf);
            ctx.logger.info("Successfully created the template configuration");
        } else {
            ConfiguredGraphFactory.updateTemplateConfiguration(conf);
            ctx.logger.info("Successfully updated the template configuration");
        }

        if (ConfiguredGraphFactory.getConfiguration('default_') == null) {
            ConfiguredGraphFactory.create('default_');
            ctx.logger.info("Successfully created the graph 'default_'");
        }
    }
] as LifeCycleHook]

An important piece of this was the line conf.setDelimiterParsingDisabled(true); which was a workaround to disable automatically parsing multiple comma-separated hostnames and other similar parameters as lists and keep them as strings. In commons-configuration2 this method does not exist and the docs (https://commons.apache.org/proper/commons-configuration/userguide/upgradeto2_0.html, https://commons.apache.org/proper/commons-configuration/apidocs/org/apache/commons/configuration2/MapConfiguration.html) state that list splitting is disabled by default. This is also confirmed here: https://github.com/JanusGraph/janusgraph/issues/1447#issuecomment-851119479.

After installing Janusgraph 0.6.0 I used a similar script, but with conf.setDelimiterParsingDisabled(true); commented out:

def globals = [:]

globals << [hook : [
    onStartUp: { ctx ->
        def map = new HashMap<String, Object>();

        map.put('schema.default', 'none');
        map.put('schema.constraints', 'true');

        map.put('graph.allow-upgrade', 'false');

        map.put('storage.backend', 'cql');
        map.put('storage.hostname', 'test-master,test-worker1');
        map.put('storage.cql.replication-factor', '1');
        map.put('storage.cql.read-consistency-level', 'LOCAL_QUORUM');
        map.put('storage.cql.write-consistency-level', 'LOCAL_QUORUM');
        map.put('storage.cql.only-use-local-consistency-for-system-operations', 'true');
        map.put('storage.cql.local-datacenter', 'dc1');
        map.put('storage.cql.replication-strategy-options', 'dc1,1');
        map.put('storage.cql.replication-strategy-class', 'NetworkTopologyStrategy');
        map.put('storage.cql.local-max-connections-per-host', 5)
        map.put('storage.cql.max-requests-per-connection', 1024)
        map.put('storage.cql.executor-service.enabled', 'false')
        map.put('storage.parallel-backend-executor-service.core-pool-size', 100)

        map.put('index.es.backend', 'elasticsearch');
        map.put('index.es.hostname', 'test-dev-elasticsearch');
        map.put('index.es.elasticsearch.client-only', "true");
        map.put('index.es.elasticsearch.create.ext.index.number_of_shards', '5');
        map.put('index.es.elasticsearch.create.ext.index.number_of_replicas', '1');

        map.put('storage.cql.internal.string-configuration', 'datastax-java-driver { advanced.metadata.schema.debouncer.window = 1 second }');

        map.put('query.smart-limit', 'false')

        def conf = new MapConfiguration(map);
        // conf.setDelimiterParsingDisabled(true);

        if (ConfiguredGraphFactory.getTemplateConfiguration() == null) {
            ConfiguredGraphFactory.createTemplateConfiguration(conf);
            ctx.logger.info("Successfully created the template configuration");
        } else {
            ConfiguredGraphFactory.updateTemplateConfiguration(conf);
            ctx.logger.info("Successfully updated the template configuration");
        }

        if (ConfiguredGraphFactory.getConfiguration('default_') == null) {
            ConfiguredGraphFactory.create('default_');
            ctx.logger.info("Successfully created the graph 'default_'");
        }
    }
] as LifeCycleHook]

The script executed without errors, but the graph default_ cannot be opened. I get the following exception:

2021-09-21 14:11:15,347 [pool-12-thread-1] com.datastax.oss.driver.internal.core.ContactPoints - WARN  - Ignoring invalid contact point [test-master:9042 (unknown host [test-master)
2021-09-21 14:11:15,348 [pool-12-thread-1] com.datastax.oss.driver.internal.core.ContactPoints - WARN  - Ignoring invalid contact point test-worker1]:9042 (unknown host test-worker1])
2021-09-21 14:11:15,352 [JanusGraph Session-admin-0] com.datastax.oss.driver.internal.core.time.Clock - INFO  - Using native clock for microsecond precision
2021-09-21 14:11:15,352 [JanusGraph Session-admin-0] com.datastax.oss.driver.internal.core.metadata.MetadataManager - INFO  - [JanusGraph Session] No contact points provided, defaulting to /127.0.0.1:9042
2021-09-21 14:11:15,355 [JanusGraph Session-admin-1] com.datastax.oss.driver.internal.core.control.ControlConnection - WARN  - [JanusGraph Session] Error connecting to Node(endPoint=/127.0.0.1:9042, hostId=null, hashCode=74fef982), trying next node (ConnectionInitException: [JanusGraph Session|control|connecting...] Protocol initialization request, step 1 (OPTIONS): failed to send request (io.netty.channel.StacklessClosedChannelException))
2021-09-21 14:11:17,434 [pool-12-thread-1] org.janusgraph.graphdb.management.JanusGraphManager - ERROR - Failed to open graph default_ with the following error:
 java.lang.IllegalArgumentException: Could not instantiate implementation: org.janusgraph.diskstorage.cql.CQLStoreManager.
Thus, it and its traversal will not be bound on this server.

This indicates that the string was split somewhere along the way, and the resulting array [test-master, test-worker1] was stored as the property value. Inspecting the configuration in the gremlin console also points to this:

gremlin> ConfiguredGraphFactory.getConfiguration('default_').get('storage.hostname')
==>[test-master, test-worker1]
gremlin> 
gremlin> ConfiguredGraphFactory.getConfiguration('default_').get('storage.hostname').getClass()
==>class java.lang.String
gremlin> 

I even tried to manually set the delimiter in my script to be something other than ',' (with something like conf.setListDelimiterHandler(new org.apache.commons.configuration2.convert.DefaultListDelimiterHandler(';' as char));), but that didn't help either.

Did anyone else come across this issue? Did I miss something else in the changelog about this?

Kind regards,

Mladen Marović


Re: Can we find connected nodes without using gremlin query

hadoopmarc@...
 

Hi Anjani,

How large is your graph, can you hold it in memory in its entirety?
How large are the subgraphs you want to retrieve in terms of width and size?
What are your requirements regarding TPS?
When you refer to "gremlin" what graph system did you use instead of JanusGraph?

Best wishes,    Marc


Re: Unconfigured table exceptions in Janusgraph 0.6.0

Mladen Marović
 

Hello,

thanks for the responses. I attempted to add storage.cql.internal.string-configuration = "datastax-java-driver { advanced.metadata.schema.debouncer.window = 1 second }" to my configuration-management-graph.properties, but got the following exception after starting Janusgraph:

java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
        at org.apache.tinkerpop.gremlin.server.util.ServerGremlinExecutor.<init>(ServerGremlinExecutor.java:95)
        at org.apache.tinkerpop.gremlin.server.GremlinServer.<init>(GremlinServer.java:124)
        at org.apache.tinkerpop.gremlin.server.GremlinServer.<init>(GremlinServer.java:87)
        at org.janusgraph.graphdb.server.JanusGraphServer.start(JanusGraphServer.java:85)
        at org.janusgraph.graphdb.server.JanusGraphServer.main(JanusGraphServer.java:53)
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at org.apache.tinkerpop.gremlin.server.util.ServerGremlinExecutor.<init>(ServerGremlinExecutor.java:84)
        ... 4 more
Caused by: java.lang.IllegalArgumentException: Could not instantiate implementation: org.janusgraph.diskstorage.cql.CQLStoreManager
        at org.janusgraph.util.system.ConfigurationUtil.instantiate(ConfigurationUtil.java:79)
        at org.janusgraph.diskstorage.Backend.getImplementationClass(Backend.java:525)
        at org.janusgraph.diskstorage.Backend.getStorageManager(Backend.java:489)
        at org.janusgraph.graphdb.configuration.builder.GraphDatabaseConfigurationBuilder.build(GraphDatabaseConfigurationBuilder.java:64)
        at org.janusgraph.core.JanusGraphFactory.lambda$open$0(JanusGraphFactory.java:165)
        at org.janusgraph.graphdb.management.JanusGraphManager.openGraph(JanusGraphManager.java:239)
        at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:165)
        at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:115)
        at org.janusgraph.graphdb.management.JanusGraphManager.lambda$new$0(JanusGraphManager.java:73)
        at java.util.LinkedHashMap.forEach(LinkedHashMap.java:684)
        at org.janusgraph.graphdb.management.JanusGraphManager.<init>(JanusGraphManager.java:72)
        ... 9 more
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at org.janusgraph.util.system.ConfigurationUtil.instantiate(ConfigurationUtil.java:73)
        ... 19 more
Caused by: com.typesafe.config.ConfigException$Parse: String: 1: Key '"datastax-java-driver { advanced.metadata.schema.debouncer.window = 1 second }"' may not be followed by token: end of file
        at com.typesafe.config.impl.ConfigDocumentParser$ParseContext.parseError(ConfigDocumentParser.java:201)
        at com.typesafe.config.impl.ConfigDocumentParser$ParseContext.parseError(ConfigDocumentParser.java:197)
        at com.typesafe.config.impl.ConfigDocumentParser$ParseContext.parseObject(ConfigDocumentParser.java:461)
        at com.typesafe.config.impl.ConfigDocumentParser$ParseContext.parse(ConfigDocumentParser.java:648)
        at com.typesafe.config.impl.ConfigDocumentParser.parse(ConfigDocumentParser.java:14)
        at com.typesafe.config.impl.Parseable.rawParseValue(Parseable.java:262)
        at com.typesafe.config.impl.Parseable.rawParseValue(Parseable.java:250)
        at com.typesafe.config.impl.Parseable.parseValue(Parseable.java:180)
        at com.typesafe.config.impl.Parseable.parseValue(Parseable.java:174)
        at com.typesafe.config.impl.Parseable.parse(Parseable.java:301)
        at com.typesafe.config.ConfigFactory.parseString(ConfigFactory.java:1187)
        at com.typesafe.config.ConfigFactory.parseString(ConfigFactory.java:1197)
        at com.datastax.oss.driver.internal.core.config.typesafe.DefaultDriverConfigLoader.lambda$fromString$4(DefaultDriverConfigLoader.java:125)
        at com.datastax.oss.driver.internal.core.config.typesafe.DefaultDriverConfigLoader.<init>(DefaultDriverConfigLoader.java:196)
        at com.datastax.oss.driver.internal.core.config.typesafe.DefaultDriverConfigLoader.fromString(DefaultDriverConfigLoader.java:120)
        at com.datastax.oss.driver.api.core.config.DriverConfigLoader.fromString(DriverConfigLoader.java:196)
        at org.janusgraph.diskstorage.cql.builder.CQLSessionBuilder.build(CQLSessionBuilder.java:74)
        at org.janusgraph.diskstorage.cql.CQLStoreManager.<init>(CQLStoreManager.java:135)
        at org.janusgraph.diskstorage.cql.CQLStoreManager.<init>(CQLStoreManager.java:116)
        ... 24 more

I'm guessing something's wrong with the syntax, but I couldn't get it right. Instead, I set storage.cql.internal.file-configuration=/home/test/janusgraph/conf/datastax_java_driver.conf and created the following file:

datastax-java-driver {
  advanced.metadata {
    schema {
      debouncer {
        window = 1 second
      }
    }
  }
}

This seems to give acceptable results because out of about 20-25 attempts at creating different graphs, only one resulted in the same error as before.

For completeness, i also went to the extreme by setting the same parameter to 5 seconds which resulted in:

...
Caused by: com.datastax.oss.driver.api.core.DriverTimeoutException: Query timed out after PT2S
...

so I'm guessing there are some other timeouts to consider as well, but did not look further into that.

Additionally, I also tested advanced.metadata.schema.debouncer.max-events with the following values:

  • 1 - often fails; same behaviour as setting window to 0
  • 3 - occasionally fails
  • 9 - occasionally fails
  • 10 - seems to be about the same as setting window to 1 second

Right now setting window to 1 seconds seems good enough to me, but I'll let you know if this issue persists even with that setting on different environments.

Kind regards,

Mladen Marović

P.S. I managed to get the string configuration to work as well by adding storage.cql.internal.string-configuration = datastax-java-driver { advanced.metadata.schema.debouncer.window = 1 second } (no double quotes).


Can we find connected nodes without using gremlin query

anjanisingh22@...
 

Hello Everyone,

We have a requirement to build a search functionality to find connected nodes based on some identifiers with high performance and TPS.
While doing analysis we noticed gremlin queries are not able to support required performance. So i was thinking it there any other way to get connected node.

For ex , Is it possible to do via janus ES?

Please share your thoughts and suggestions.

Appreciate your help.

Thanks,
Anjani


Re: Unconfigured table exceptions in Janusgraph 0.6.0

Oleksandr Porunov
 

Also, notice, if you want to use multiple configuration options, you will need to split them using new line character. I.e. `\n` and not `;`.
See example here: https://github.com/JanusGraph/janusgraph/blob/26a06a0d27931d5e97ca7870493336fe21b522be/janusgraph-cql/src/test/java/org/janusgraph/diskstorage/cql/CQLConfigTest.java#L98


Re: Unconfigured table exceptions in Janusgraph 0.6.0

Oleksandr Porunov
 

You can also play with `advanced.metadata.schema.debouncer.max-events`. For more configurations see: https://docs.datastax.com/en/developer/java-driver/4.13/manual/core/configuration/reference/


Re: Unconfigured table exceptions in Janusgraph 0.6.0

Oleksandr Porunov
 

Hi,

Mladen that issue is most likely related to this discussion of how the new CQL driver updates schema view: https://groups.google.com/a/lists.datastax.com/g/java-driver-user/c/Bc0gQuOVVL0

As suggested in that thread, to improve performance we set metadata_schema_window to 0 (to keep schema creation performance similar to the previous driver). That said, we probably made a mistake when hardcoded that value to 0. We probably should make it configurable in the next minor release. (you can see that configuration set here: https://github.com/JanusGraph/janusgraph/blob/abe67055a23c729d55e0e4ac1508189d93f9820c/janusgraph-cql/src/main/java/org/janusgraph/diskstorage/cql/builder/CQLProgrammaticConfigurationLoaderBuilder.java#L144)

Luckily, there is a way to overwrite JanusGraph default CQL Driver configurations.
See explanation here (configuration options under `storage.cql.internal`): https://docs.janusgraph.org/configs/configuration-reference/#storagecqlinternal

Probably the easiest way to overwrite that value as for now would be to use `storage.cql.internal.string-configuration`. I.e. something like:
```
storage.cql.internal.string-configuration = "datastax-java-driver { advanced.metadata.schema.debouncer.window = 1 second }"
```
The DataStax default value is "1 second" thus, I would recommend to try 1 second as they recommend. In case that isn't enough you could try to increase that value but be aware that your schema creation (i.e. new graphs initial creation) will be longer due to this window delay.

I have opened the issue to tack here: https://github.com/JanusGraph/janusgraph/issues/2809
Let me know if increasing `advanced.metadata.schema.debouncer.window` helps with your issue.

Best regards,
Oleksandr Porunov


Unconfigured table exceptions in Janusgraph 0.6.0

Mladen Marović
 

Hello,

I wanted to try out the new Janusgraph 0.6.0 release and I encountered some unexpected issues while trying to deploy it on Cassandra 3.11.5.

The first issue occurs during graph creation. I'm trying to set up a ConfiguredGraphFactory. When starting Janusgraph for the first time, it attempts to create its internal tables ('graphindex', 'janusgraph_ids', ...), but sometimes fails to create a table and throws the following exception:

java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
        at org.apache.tinkerpop.gremlin.server.util.ServerGremlinExecutor.<init>(ServerGremlinExecutor.java:95)
        at org.apache.tinkerpop.gremlin.server.GremlinServer.<init>(GremlinServer.java:124)
        at org.apache.tinkerpop.gremlin.server.GremlinServer.<init>(GremlinServer.java:87)
        at org.janusgraph.graphdb.server.JanusGraphServer.start(JanusGraphServer.java:85)
        at org.janusgraph.graphdb.server.JanusGraphServer.main(JanusGraphServer.java:53)
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at org.apache.tinkerpop.gremlin.server.util.ServerGremlinExecutor.<init>(ServerGremlinExecutor.java:84)
        ... 4 more
Caused by: com.datastax.oss.driver.api.core.servererrors.InvalidQueryException: unconfigured table janusgraph_ids
        at com.datastax.oss.driver.api.core.servererrors.InvalidQueryException.copy(InvalidQueryException.java:48)
        at com.datastax.oss.driver.internal.core.util.concurrent.CompletableFutures.getUninterruptibly(CompletableFutures.java:149)
        at com.datastax.oss.driver.internal.core.cql.CqlPrepareSyncProcessor.process(CqlPrepareSyncProcessor.java:59)
        at com.datastax.oss.driver.internal.core.cql.CqlPrepareSyncProcessor.process(CqlPrepareSyncProcessor.java:31)
        at com.datastax.oss.driver.internal.core.session.DefaultSession.execute(DefaultSession.java:230)
        at com.datastax.oss.driver.api.core.cql.SyncCqlSession.prepare(SyncCqlSession.java:206)
        at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.<init>(CQLKeyColumnValueStore.java:168)
        at org.janusgraph.diskstorage.cql.CQLStoreManager.lambda$openDatabase$7(CQLStoreManager.java:267)
        at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1660)
        at org.janusgraph.diskstorage.cql.CQLStoreManager.openDatabase(CQLStoreManager.java:267)
        at org.janusgraph.diskstorage.keycolumnvalue.KeyColumnValueStoreManager.openDatabase(KeyColumnValueStoreManager.java:43)
        at org.janusgraph.diskstorage.Backend.initialize(Backend.java:312)
        at org.janusgraph.graphdb.configuration.GraphDatabaseConfiguration.getBackend(GraphDatabaseConfiguration.java:1356)
        at org.janusgraph.graphdb.database.StandardJanusGraph.<init>(StandardJanusGraph.java:184)
        at org.janusgraph.core.JanusGraphFactory.lambda$open$0(JanusGraphFactory.java:165)
        at org.janusgraph.graphdb.management.JanusGraphManager.openGraph(JanusGraphManager.java:239)
        at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:165)
        at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:115)
        at org.janusgraph.graphdb.management.JanusGraphManager.lambda$new$0(JanusGraphManager.java:73)
        at java.util.LinkedHashMap.forEach(LinkedHashMap.java:684)
        at org.janusgraph.graphdb.management.JanusGraphManager.<init>(JanusGraphManager.java:72)

Inspecting the keyspaces via cqlsh shows that the keyspace is created, as well as some (but not all) of the tables:

cqlsh> describe keyspaces;

configuration_management_graph  system_auth  system_distributed
system_schema                   system       system_traces     

cqlsh> select table_name from system_schema.tables where keyspace_name = 'configuration_management_graph';

 table_name
-------------------
    janusgraph_ids
 system_properties

Upon restarting, Janusgraph continues from where it last failed and tries to create the other missing tables. Eventually, cqlsh shows that all tables are created.

cqlsh> select table_name from system_schema.tables where keyspace_name = 'configuration_management_graph';

 table_name
-------------------------
               edgestore
         edgestore_lock_
              graphindex
        graphindex_lock_
          janusgraph_ids
       system_properties
 system_properties_lock_
               systemlog
                   txlog

I tried to test this multiple times by shutting down Janusgraph, dropping its keyspaces in Cassandra, and starting it again. Sometimes it successfully finishes from the first try, sometimes it fails once, and sometimes multiple times, on different tables.

As for the relevant configuration, I've updated it according to the 0.6.0 changelog. I'm using the following gremlin-server.yaml:

host: 0.0.0.0

port: 8183

evaluationTimeout: 60000

channelizer: org.apache.tinkerpop.gremlin.server.channel.WsAndHttpChannelizer

graphManager: org.janusgraph.graphdb.management.JanusGraphManager

graphs:
  ConfigurationManagementGraph: /home/test/janusgraph/conf/configuration-management-graph.properties

scriptEngines:
  gremlin-groovy:
    plugins:
      org.janusgraph.graphdb.tinkerpop.plugin.JanusGraphGremlinPlugin: {}
      org.apache.tinkerpop.gremlin.server.jsr223.GremlinServerGremlinPlugin: {}
      org.apache.tinkerpop.gremlin.tinkergraph.jsr223.TinkerGraphGremlinPlugin: {}
      org.apache.tinkerpop.gremlin.jsr223.ImportGremlinPlugin:
        classImports:
          - java.lang.Math
        methodImports:
          - java.lang.Math#*
      org.apache.tinkerpop.gremlin.jsr223.ScriptFileGremlinPlugin:
        files:
          - /home/test/janusgraph/storage/scripts/globals.groovy

processors:
  - className: org.apache.tinkerpop.gremlin.server.op.session.SessionOpProcessor
    config:
      sessionTimeout: 28800000
  - className: org.apache.tinkerpop.gremlin.server.op.traversal.TraversalOpProcessor
    config:
      cacheExpirationTime: 600000
      cacheMaxSize: 1000

metrics:
  consoleReporter:
    enabled: true
    interval: 180000

maxInitialLineLength: 4096

maxHeaderSize: 32768

maxChunkSize: 32768

maxContentLength: 2097152

maxAccumulationBufferComponents: 2048

resultIterationBatchSize: 2048

writeBufferLowWaterMark: 131072

writeBufferHighWaterMark: 2097152

threadPoolWorker: 4

gremlinPool: 8

and configuration-management-graph.properties is as follows:

graph.graphname=configuration_management_graph
gremlin.graph=org.janusgraph.core.ConfiguredGraphFactory

storage.backend=cql
storage.hostname=test-master,test-worker1
storage.cql.keyspace=configuration_management_graph
storage.cql.replication-factor=1
storage.cql.read-consistency-level=LOCAL_QUORUM
storage.cql.write-consistency-level=LOCAL_QUORUM
storage.cql.only-use-local-consistency-for-system-operations=true
storage.cql.local-datacenter=dc1
storage.cql.replication-strategy-options=dc1,1
storage.cql.replication-strategy-class=NetworkTopologyStrategy
storage.cql.local-max-connections-per-host=5
storage.cql.max-requests-per-connection=1024
storage.cql.executor-service.enabled=false
storage.parallel-backend-executor-service.core-pool-size=100

query.smart-limit=false

# disable this property as soon as the upgrade is done to avoid unwanted upgrades
graph.allow-upgrade=false

Since this is my first look at Janusgraph 0.6.0, I'm testing it on two local VMs. I'm using Ansible scripts to automate the deployment for Janusgraph, Cassandra and Elasticsearch, and the entire setup has previously been tested on multiple different Janusgraph versions (0.2.0, 0.3.1, 0.4.0, 0.5.1, 0.5.3) and on several different environments. Therefore, I do not expect there to be any issues with Cassandra, Elasticsearch, the deployment scripts, or the setup itself, unless I missed some breaking change in the 0.6.0 changelog that I'm not aware of.

Did anyone else have a similar issue?

Kind regards,

Mladen Marović

P.S. Since this turned out to be a lengthy post, I'll create another post about the other issue I mentioned.


Re: JanusGraph 0.6.0 Binary Driver failing on serialization

cmilowka
 

Thank you Marc,

I was able to reproduce your sample by console:  :remote connect tinkerpop.server conf/remote.yaml session  # it is GryoMessageSerializerV3d0 - no problem here, but must have serializeResultToString: true 

:remote connect tinkerpop.server conf/remote-graph-binary.yaml session

:> g.V().properties("some")    ==> Server error - Error during serialization: java.lang.IllegalStateException. 

:> g.V().properties("some").hasNext() => true, it is actually working in this context, where server testing the results, not the client after serialization.

It is fine when "remote-graph-binary.yaml" configuration is equipped with serializeResultToString: true. 

 

That hint should indicate that our server is different that the driver we use in java to remote access, but with more debugging I cannot say where is it... We run JanusGraph services through "bin/gremlin-server-websoc.sh", that suppose to run new Binary Driver with gremlin-server-websoc.yaml (+serializeResultToString: true). Then we getting remote access from Java by connection with 

    MessageSerializer<?> serializer = new GraphBinaryMessageSerializerV1(typeSerializerRegistry);

To be sure, we have true for serializeResultToString in the configuration, we even modified that on fly:

    final Map<String, Object> cfg = new HashMap<>();  cfg.put("serializeResultToString",true);     serializer.configure(cfg, null);

With serializeResultToString(true) we have the same exception IllegalStageException as initially described, with (false) it is faling everywhere, on simple command like:

    GraphTraversal<Vertex, Vertex> t = g.V().hasLabel(label).has(property, value);

if (t.hasNext()) {  // java.lang.ClassCastException: java.lang.String cannot be cast to org.apache.tinkerpop.gremlin.process.remote.traversal.RemoteTraverser

so for serializeResultToString(true), all is fine, except "gv().properties()",...  somehow we have good driver config for remote access.

Regards, Chris

 

 

 


Re: Berkeley JE information

kosulin@...
 

Here is how I see this:

- the current public release with direct download is 4 years old 7.5.11, compiled with Java 8 and supporting Java 9 according to its and build script;
- the public Oracle maven repo release is 3 years old 18.3.2, which is according to Oracle a fork from Berkeley JE created specifically for Oracle NoSQL Database, compiled with Java 10;
- the most recent download release of Oracle NoSQL Database includes je version 20.3.17 which is compiled with Java 11 but is not present in public Oracle maven repo (both LATEST and RELEASE return 18.3.12).

Therefore there are 3 different releases of je, where direct download and maven versions are outdated, and Oracle has unofficial plans to replace je with unknown future product.
Too much unsertainty IMHO.


Re: Berkeley JE information

hadoopmarc@...
 
Edited

Thanks for this information [I mean the facts, this forum is not intended to spread opinions on third parties].

After, checking, I was surprised myself. The deviation from the JE has occurred since v0.5.0. However, the following file:

https://github.com/JanusGraph/janusgraph/blob/v0.6.0/NOTICE.txt

Still mentions:
* Oracle BerkeleyDB Java Edition [https://www.oracle.com/technetwork/products/berkeleydb/] (see license below)
This does not seem correct. Can anyone clarify?

Also relevant:
https://www.oracle.com/database/technologies/related/berkeleydb/berkeleydb-licensing.html

Best wishes,    Marc


Re: JanusGraph 0.6.0 Binary Driver failing on serialization

hadoopmarc@...
 


Re: JanusGraph 0.6.0 Binary Driver failing on serialization

hadoopmarc@...
 

Hi Chris,

Thanks for reporting.  I could confirm the exception you got by using the console and server files from the distribution. I got the right results, though,  when I changed the contents of conf/remote-graph-binary.yaml to:

hosts: [localhost]
port: 8182
serializer: { className: org.apache.tinkerpop.gremlin.driver.ser.GraphBinaryMessageSerializerV1, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry], serializeResultToString: true }}

in this way, it matches the configs of gremlin-server in conf/gremlin-server/gremin-server.yaml

I will post a ticket for it.

Best wishes,      Marc


Berkeley JE information

kosulin@...
 

I found the following information that might be important for those who (myself included) use or plan to use Bekeley JE as embedded storage:

"The current public release of BDB JE is 7.5.11. What you are referring to as 18.3.12 is actually from Oracle NoSQL Database Server. This is a fork of BDB JE and has diverged significantly from the 7.5.11 public version. From time to time, we will release updates to the 7.5.x code line but additional public major releases are not planned at this time. While we have not tested internally, it makes sense to me that the 7.5.11 test suite does not run with 18.3.12 (code line has diverged). We do have plans in CY21 to release a new product that will serve as a replacement for BDB JE. Feel free to reach out to me later this spring and I will hopefully have more details (michael.brey@...)"

https://community.oracle.com/tech/developers/discussion/4478434/what-is-the-current-berkeley-db-java-edition-release-is-there-a-public-source-code-repository

I looked inside the Oracle NoSQL Database Server 20.3.18, and it includes je version 20.3.17.

IMHO this is so cumbersome in the usual Oracle way, I start looking for a way to get rid of berkeley in my product.


JanusGraph 0.6.0 Binary Driver failing on serialization

cmilowka
 

We have fully integrated our backend with JS:0.6 as recommended, including DB connection using new driver: GraphBinaryMessageSerializerV1.

All seams o work as before, except:

GraphTraversal<Vertex, ? extends Property<Object>> p = g.V(12345).properties("myproperty");

if (p.hasNext()) {...}

 

12:02:47.609 [https-jsse-nio-127.0.0.1-8443-exec-8] ERROR o.a.c.c.C.[.[.[.[dispatcherServlet] [175] - Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is java.util.concurrent.CompletionException: org.apache.tinkerpop.gremlin.driver.exception.ResponseException: Error during serialization: java.lang.IllegalStateException] with root cause

org.apache.tinkerpop.gremlin.driver.exception.ResponseException: Error during serialization: java.lang.IllegalStateException

at org.apache.tinkerpop.gremlin.driver.Handler$GremlinResponseHandler.channelRead0(Handler.java:245)

at org.apache.tinkerpop.gremlin.driver.Handler$GremlinResponseHandler.channelRead0(Handler.java:200)

at io.shaded.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)

/.../

 

Above was, and still is fine on p.hasNext() with previously used GryoMessageSerializerV3d0 driver. Workaround we have is to use:

GraphTraversal<Vertex, ? extends Property<Object>> p = g.V(12345).values("myproperty");

 

if (p.hasNext()) {...}  // now is ok

 

Why above is failing, is the question....

Regards, Chris 

 

 


Re: [DISCUSS] Dropping HBase 1 support

hadoopmarc@...
 

Hi Jan,

When you say "We stopped testing HBase 1 in our CI solution already in branch 0.3" does that mean you are not sure whether HBase1 tests would run on a custom build for 0.4+? If so, the code was already abandoned and dropping it is the only reasonable way to go. Which does not mean such a decision could not be reversed under any circumstances.

Best wishes,   Marc


Re: [DISCUSS] Dropping HBase 1 support

Jansen, Jan
 

Will the support for BigTable persist when dropping HBase 1? With 0.5, JG uses the HBase 2 as client for Bigtable.


Von: janusgraph-users@... <janusgraph-users@...> im Auftrag von schwartz@... <schwartz@...>
Gesendet: Montag, 13. September 2021 08:05:25
An: janusgraph-users@...
Betreff: Re: [janusgraph-users] [DISCUSS] Dropping HBase 1 support
 
Will the support for BigTable persist when dropping HBase 1?

We are working with BigTable, and in the days of 0.4 (or was it 0.4.1), BT didn't work, so we had to compile 0.4.x with HBase 1 support.

Thanks!

481 - 500 of 6651