Date   

Re: ConfiguredGraphFactory and Authentication not working

hadoopmarc@...
 

Hi Vinayak,

The information you provide is still a puzzle of little pieces which is hard to make sense of. Do you mean either of the following:

  1. there is some behaviour of janusgraph that is undocumented. Please provide steps to reproduce the issue.
  2. you created a graph with v.0.4.0 and now you have trouble reading it with janusgraph v0.5.2 . According to the upgrade instuctions in the changelog, https://docs.janusgraph.org/changelog/, there are no specific instructions to read v0.4.x graphs with janusgraph v0.5.x, in other words this is not expected to be an issue.
Or is it something else? Please be very specific.

You state "The difference which I saw between the two was when I start 0.4.0 automatically configuredgraphfactory schema was created in Cassandra, but in 0.5.2 janusgraph schema is created by default. This may be the reason for it."  Can you elaborate on that. How do the different schema's look like. What are the differences in the yaml and properties config files?

Best wishes,    Marc


Delete label is very difficult

vamsi.lingala@...
 
Edited

Deleting all edges in a label/all vertices in a label from gremlin is almost impossible..
g.E().hasLabel('MAID-BRAND-012021').drop()
g.V().hasLabel('BRAND').drop()


it gets timeout even thought you increase timeout to larger limits.
I think it scans all vertices/edges and then finds which are belong to that label..which is very inefficient


Re: Not able to reindex with bigtable as backend

liqingtaobkd@...
 

Sorry for multiple emails. My found the final error from Janusgraph. Not sure it's a bigtable issue or a janusgraph/bigtable compatibility issue. Can anybody help to take a look?

3035627 [Thread-8] ERROR org.janusgraph.graphdb.olap.job.IndexRepairJob - Transaction commit threw runtime exception:

org.janusgraph.core.JanusGraphException: Could not commit transaction due to exception during persistence at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1449) at org.janusgraph.graphdb.olap.job.IndexUpdateJob.workerIterationEnd(IndexUpdateJob.java:136) at org.janusgraph.graphdb.olap.job.IndexRepairJob.workerIterationEnd(IndexRepairJob.java:208) at org.janusgraph.graphdb.olap.VertexJobConverter.workerIterationEnd(VertexJobConverter.java:118) at org.janusgraph.diskstorage.keycolumnvalue.scan.StandardScannerExecutor$Processor.run(StandardScannerExecutor.java:285) Caused by: org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:133) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commit(BackendTransaction.java:150) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1440) ... 4 more Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT1M40S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 9 more Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at org.janusgraph.diskstorage.hbase.HBaseStoreManager.mutateMany(HBaseStoreManager.java:460) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 10 more Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed 1 action: IllegalStateException: 1 time, servers with issues: bigtable.googleapis.com at com.google.cloud.bigtable.hbase.BatchExecutor.batchCallback(BatchExecutor.java:288) at com.google.cloud.bigtable.hbase.BatchExecutor.batch(BatchExecutor.java:207) at com.google.cloud.bigtable.hbase.AbstractBigtableTable.batch(AbstractBigtableTable.java:185) at org.janusgraph.diskstorage.hbase.HTable1_0.batch(HTable1_0.java:51) at org.janusgraph.diskstorage.hbase.HBaseStoreManager.mutateMany(HBaseStoreManager.java:455) ... 14 more


Re: Not able to reindex with bigtable as backend

liqingtaobkd@...
 

Found follow error in the log. janusgraph version 0.5.3 with bigtable as backend. Any suggestions pls? I have been stuck with it for a few days...

"org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at org.janusgraph.diskstorage.hbase.HBaseStoreManager.mutateMany(HBaseStoreManager.java:460) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:133) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commit(BackendTransaction.java:150) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1440) at org.janusgraph.graphdb.olap.job.IndexUpdateJob.workerIterationEnd(IndexUpdateJob.java:136) at org.janusgraph.graphdb.olap.job.IndexRepairJob.workerIterationEnd(IndexRepairJob.java:208) at org.janusgraph.graphdb.olap.VertexJobConverter.workerIterationEnd(VertexJobConverter.java:118) at org.janusgraph.diskstorage.keycolumnvalue.scan.StandardScannerExecutor$Processor.run(StandardScannerExecutor.java:285) Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed 1 action: IllegalStateException: 1 time, servers with issues: bigtable.googleapis.com at com.google.cloud.bigtable.hbase.BatchExecutor.batchCallback(BatchExecutor.java:288) at com.google.cloud.bigtable.hbase.BatchExecutor.batch(BatchExecutor.java:207) at com.google.cloud.bigtable.hbase.AbstractBigtableTable.batch(AbstractBigtableTable.java:185) at org.janusgraph.diskstorage.hbase.HTable1_0.batch(HTable1_0.java:51) at org.janusgraph.diskstorage.hbase.HBaseStoreManager.mutateMany(HBaseStoreManager.java:455) ... 14 more "


Re: ConfiguredGraphFactory and Authentication not working

Vinayak Bali
 

Hi Marc,

It was working with 0.4.0. After the update to 0.5.2, it is not working. The difference which I saw between the two was when I start 0.4.0 automatically configuredgraphfactory schema was created in Cassandra, but in 0.5.2 janusgraph schema is created by default. This may be the reason for it. Please let me know how to solve the issue.

Thanks & Regards,
Vinayak

On Sun, Feb 28, 2021 at 9:26 PM <hadoopmarc@...> wrote:
Hi Vinayak,

I played around a bit with the ConfigurationManagementGraph myself. The client error you describe occurs when you run ":remote console" twice. It works as a toggle switch, in other words, the command you run was probably interpreted as for a locally embedded janusgraph instance (which was not present).

Best wishes,    Marc


Re: JanusGraphIndex how to retrieve constraint (indexOnly) specified for the global index?

cmilowka
 

Works like a charm, thank you Bo Xuan Li.


Re: Not able to reindex with bigtable as backend

liqingtaobkd@...
 

Thanks for the reply. I carefully followed each step described in the doc. Before the reindex, I closed all the open transactions and management instance. I sent the reindex command from the console and it never returns (at least for 10h+):
mgmt.updateIndex(mgmt.getRelationIndex(flowsTo, "flowsToByTimestamp"), SchemaAction.REINDEX).get()

So I don't have a chance to commit.

But from my monitoring of janusgraph and bigtable, there is no activity after 30min.

Do you have any further suggestion?


Re: ConfiguredGraphFactory and Authentication not working

hadoopmarc@...
 

Hi Vinayak,

I played around a bit with the ConfigurationManagementGraph myself. The client error you describe occurs when you run ":remote console" twice. It works as a toggle switch, in other words, the command you run was probably interpreted as for a locally embedded janusgraph instance (which was not present).

Best wishes,    Marc


Re: Not able to reindex with bigtable as backend

hadoopmarc@...
 

Please be sure to run all the steps (including a graph.tx().rollback() before index creation and a mgmt.commit() after update of the index) from the example in:

https://docs.janusgraph.org/index-management/index-performance/#vertex-centric-indexes

Best wishes,   Marc


Not able to reindex with bigtable as backend

liqingtaobkd@...
 

Hi community,

I am new to Janusgraph and am trying to build a POC. I ran into an issue that I really need some help. I am running Janusgraph on GCP with bigtable as storage backend. I am trying to create. a new vertex centric index. Since I already have some data in the database, I will need to do a re-index. The index is in "REGISTERED" status right now. 
Here are the steps that I took:

mgmt = graph.openManagement()
flowsTo = mgmt.getEdgeLabel('flowsTo')
mgmt.updateIndex(mgmt.getRelationIndex(flowsTo, "flowsToByTimestamp"), SchemaAction.REINDEX).get()
 
I am not sure if there is any way to check the status of the reindex process. I just check my janusgraph/bigtable's cpu/memory, etc, and it shows that the reindex should have finished in half an hour. However the reindex command never returned for hours, and the status is still "registered". If I try to reindex again, it shows
"Another job with the same id is already running: flowsToByTimestamp[flowsTo]"

I don't see any error from server side so I have no clue right now. Any suggestion will be greatly appreciated. Thanks!


Re: JanusGraphIndex how to retrieve constraint (indexOnly) specified for the global index?

Boxuan Li
 

You can do this:
JanusGraphSchemaType constraint = ManagementSystem.getGraphIndexDirect("indexName", ((ManagementSystem) mgmt).getWrappedTx()).getSchemaTypeConstraint();
If (constraint == null) then there is no constraint. Otherwise constraint.name() returns your indexOnly label.

Best regards,
Boxuan

On Feb 25, 2021, at 2:43 PM, hadoopmarc@... wrote:

Thanks. Still, my earlier answer remains unsatisfactory. Somehow, a new JanusGraph instance must be able to load an existing schema from the system table and have the information about label constraints available. Otherwise, the instance would not be able to index newly added vertices and edges correctly.

Best wishes,   Marc


Re: ConfiguredGraphFactory and Authentication not working

hadoopmarc@...
 

Hi Vinayak,

You showed the error message from the remote console. Does gremlin server itself log anything particular about the ConfigurationManagementGraph after a restart?

Marc


Re: ConfiguredGraphFactory and Authentication not working

Vinayak Bali
 

Hi Marc,

Tried the approach you mentioned, still facing the same issue. In one of the blogs, I read that it is due to the Cassandra cluster is first configured with janusgraphfactory and it works for the new Cassandra cluster. But this is not an acceptable solution. 

Thanks & Regards,
Vinayak

On Fri, Feb 26, 2021 at 1:04 PM <hadoopmarc@...> wrote:
Hi Vinayak,

Could you try in the gremlin-server.yaml with just:
graphs: {
  ConfigurationManagementGraph: conf/janusgraph-cql-configurationgraph.properties
}
So, without the graph, graph1 and graph2 keys.

This is a speculative answer, but this is the only thing that seems to be different from the ref docs, so we need to test the assumption that these additional keys are not harmful.

Best wishes,    Marc


Re: ConfiguredGraphFactory and Authentication not working

hadoopmarc@...
 

Hi Vinayak,

Could you try in the gremlin-server.yaml with just:
graphs: {
  ConfigurationManagementGraph: conf/janusgraph-cql-configurationgraph.properties
}
So, without the graph, graph1 and graph2 keys.

This is a speculative answer, but this is the only thing that seems to be different from the ref docs, so we need to test the assumption that these additional keys are not harmful.

Best wishes,    Marc


Re: ConfiguredGraphFactory and Authentication not working

Vinayak Bali
 

Hi All,

Investigated the issue further. We I start janusgraph by default janusgraph schema is created, but I need configuredmanagementgraph. Request you to please help regarding it .

Thanks & Regards,
Vinayak

On Thu, Feb 25, 2021 at 11:44 AM Vinayak Bali <vinayakbali16@...> wrote:
Hi All,

Followed the following blogs for configuring authentication and ConfiguredGraphFactory respectively.



Still, both of them are not working. Sharing the property files. 

gremlin-server.yaml

# Copyright 2019 JanusGraph Authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#      http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

host: 0.0.0.0
port: 8182
scriptEvaluationTimeout: 30000
channelizer: org.apache.tinkerpop.gremlin.server.channel.WsAndHttpChannelizer
graphManager: org.janusgraph.graphdb.management.JanusGraphManager
graphs: {
  ConfigurationManagementGraph: conf/janusgraph-cql-configurationgraph.properties,
 graph: conf/graph.properties,
 graph1: conf/graph1.properties,
 graph2: conf/graph2.properties
}

authentication: {
  authenticator: org.janusgraph.graphdb.tinkerpop.gremlin.server.auth.SaslAndHMACAuthenticator,
  authenticationHandler: org.janusgraph.graphdb.tinkerpop.gremlin.server.handler.SaslAndHMACAuthenticationHandler,
  config: {
    #defaultUsername: user,
    #defaultPassword: password,
    hmacSecret: secret,
    credentialsDb: conf/janusgraph-credentials-server.properties
  }
}
scriptEngines: {
  gremlin-groovy: {
    plugins: { org.janusgraph.graphdb.tinkerpop.plugin.JanusGraphGremlinPlugin: {},
               org.apache.tinkerpop.gremlin.server.jsr223.GremlinServerGremlinPlugin: {},
               org.apache.tinkerpop.gremlin.tinkergraph.jsr223.TinkerGraphGremlinPlugin: {},
               org.apache.tinkerpop.gremlin.jsr223.ImportGremlinPlugin: {classImports: [java.lang.Math], methodImports: [java.lang.Math#*]},
               org.apache.tinkerpop.gremlin.jsr223.ScriptFileGremlinPlugin: {files: []}}}}
serializers:
  - { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV3d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
  - { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV3d0, config: { serializeResultToString: true }}
  - { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV3d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
 # Older serialization versions for backward compatibility:
  - { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV1d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
  - { className: org.apache.tinkerpop.gremlin.driver.ser.GryoLiteMessageSerializerV1d0, config: {ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
  - { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV1d0, config: { serializeResultToString: true }}
  - { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerGremlinV2d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
  - { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerGremlinV1d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistryV1d0] }}
  - { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV1d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistryV1d0] }}
processors:
  - { className: org.apache.tinkerpop.gremlin.server.op.session.SessionOpProcessor, config: { sessionTimeout: 28800000 }}
  - { className: org.apache.tinkerpop.gremlin.server.op.traversal.TraversalOpProcessor, config: { cacheExpirationTime: 600000, cacheMaxSize: 1000 }}
metrics: {
  consoleReporter: {enabled: true, interval: 180000},
  csvReporter: {enabled: true, interval: 180000, fileName: /tmp/gremlin-server-metrics.csv},
  jmxReporter: {enabled: true},
  slf4jReporter: {enabled: true, interval: 180000},
  gangliaReporter: {enabled: false, interval: 180000, addressingMode: MULTICAST},
  graphiteReporter: {enabled: false, interval: 180000}}
maxInitialLineLength: 4096
maxHeaderSize: 8192
maxChunkSize: 8192
maxContentLength: 65536
maxAccumulationBufferComponents: 1024
resultIterationBatchSize: 64
writeBufferLowWaterMark: 32768
writeBufferHighWaterMark: 65536

janusgraph-cql-configurationgraph.properties

# Copyright 2019 JanusGraph Authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#      http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# JanusGraph configuration sample: Cassandra over a socket
#
# This file connects to a Cassandra daemon running on localhost via
# CQL.  Cassandra must already be started before starting JanusGraph
# with this file.

gremlin.graph=org.janusgraph.core.ConfiguredGraphFactory

# The primary persistence provider used by JanusGraph.  This is required.
# It should be set one of JanusGraph's built-in shorthand names for its
# standard storage backends (shorthands: berkeleyje, cql, cassandrathrift,
# cassandra, astyanax, embeddedcassandra, hbase, inmemory) or to the full
# package and classname of a custom/third-party StoreManager
# implementation.
#
# Default:    (no default value)
# Data Type:  String
# Mutability: LOCAL
storage.backend=cql

# The configuration management graph reserved for use with JanusGraphManager
# and ConfiguredGraphFactory
graph.graphname=ConfigurationManagementGraph
# The hostname or comma-separated list of hostnames of storage backend
# servers.  This is only applicable to some storage backends, such as
# cassandra and hbase.
#
# Default:    127.0.0.1
# Data Type:  class java.lang.String[]
# Mutability: LOCAL
storage.hostname=127.0.0.1

# Whether to enable JanusGraph's database-level cache, which is shared
# across all transactions. Enabling this option speeds up traversals by
# holding hot graph elements in memory, but also increases the likelihood
# of reading stale data.  Disabling it forces each transaction to
# holding hot graph elements in memory, but also increases the likelihood
# of reading stale data.  Disabling it forces each transaction to
# independently fetch graph elements from storage before reading/writing
# them.
#
# Default:    false
# Data Type:  Boolean
# Mutability: MASKABLE
cache.db-cache = true

# How long, in milliseconds, database-level cache will keep entries after
# flushing them.  This option is only useful on distributed storage
# backends that are capable of acknowledging writes without necessarily
# making them immediately visible.
#
# Default:    50
# Data Type:  Integer
# Mutability: GLOBAL_OFFLINE
#
# Settings with mutability GLOBAL_OFFLINE are centrally managed in
# JanusGraph's storage backend.  After starting the database for the first
# time, this file's copy of this setting is ignored.  Use JanusGraph's
# Management System to read or modify this value after bootstrapping.
cache.db-cache-clean-wait = 20

# Default expiration time, in milliseconds, for entries in the
# database-level cache. Entries are evicted when they reach this age even
# if the cache has room to spare. Set to 0 to disable expiration (cache
# entries live forever or until memory pressure triggers eviction when set
# to 0).
#
# Default:    10000
# Data Type:  Long
# Mutability: GLOBAL_OFFLINE
#
# Settings with mutability GLOBAL_OFFLINE are centrally managed in
# JanusGraph's storage backend.  After starting the database for the first
# time, this file's copy of this setting is ignored.  Use JanusGraph's
# Management System to read or modify this value after bootstrapping.
cache.db-cache-time = 180000

# Size of JanusGraph's database level cache.  Values between 0 and 1 are
# interpreted as a percentage of VM heap, while larger values are
# interpreted as an absolute size in bytes.
#
# Default:    0.3
# Data Type:  Double
# Mutability: MASKABLE
cache.db-cache-size = 0.5

Error:

gremlin> :remote connect tinkerpop.server conf/remote.yaml session
==>Configured localhost/127.0.0.1:8182-[8671c17b-e724-484a-aa9f-6b0d255b5ded]
gremlin> :remote console
==>All scripts will now be sent to Gremlin Server - [localhost/127.0.0.1:8182]-[8671c17b-e724-484a-aa9f-6b0d255b5ded] - type ':remote console' to return to local mode
gremlin> ConfiguredGraphFactory.getGraphNames()
Please add a key named "ConfigurationManagementGraph" to the "graphs" property in your YAML file and restart the server to be able to use the functionality of the ConfigurationManagementGraph class.

Request you to check and help me to solve the issue.

Thanks & Regards,
Vinayak


Re: JanusGraphIndex how to retrieve constraint (indexOnly) specified for the global index?

hadoopmarc@...
 

Thanks. Still, my earlier answer remains unsatisfactory. Somehow, a new JanusGraph instance must be able to load an existing schema from the system table and have the information about label constraints available. Otherwise, the instance would not be able to index newly added vertices and edges correctly.

Best wishes,   Marc


ConfiguredGraphFactory and Authentication not working

Vinayak Bali
 

Hi All,

Followed the following blogs for configuring authentication and ConfiguredGraphFactory respectively.



Still, both of them are not working. Sharing the property files. 

gremlin-server.yaml

# Copyright 2019 JanusGraph Authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#      http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

host: 0.0.0.0
port: 8182
scriptEvaluationTimeout: 30000
channelizer: org.apache.tinkerpop.gremlin.server.channel.WsAndHttpChannelizer
graphManager: org.janusgraph.graphdb.management.JanusGraphManager
graphs: {
  ConfigurationManagementGraph: conf/janusgraph-cql-configurationgraph.properties,
 graph: conf/graph.properties,
 graph1: conf/graph1.properties,
 graph2: conf/graph2.properties
}

authentication: {
  authenticator: org.janusgraph.graphdb.tinkerpop.gremlin.server.auth.SaslAndHMACAuthenticator,
  authenticationHandler: org.janusgraph.graphdb.tinkerpop.gremlin.server.handler.SaslAndHMACAuthenticationHandler,
  config: {
    #defaultUsername: user,
    #defaultPassword: password,
    hmacSecret: secret,
    credentialsDb: conf/janusgraph-credentials-server.properties
  }
}
scriptEngines: {
  gremlin-groovy: {
    plugins: { org.janusgraph.graphdb.tinkerpop.plugin.JanusGraphGremlinPlugin: {},
               org.apache.tinkerpop.gremlin.server.jsr223.GremlinServerGremlinPlugin: {},
               org.apache.tinkerpop.gremlin.tinkergraph.jsr223.TinkerGraphGremlinPlugin: {},
               org.apache.tinkerpop.gremlin.jsr223.ImportGremlinPlugin: {classImports: [java.lang.Math], methodImports: [java.lang.Math#*]},
               org.apache.tinkerpop.gremlin.jsr223.ScriptFileGremlinPlugin: {files: []}}}}
serializers:
  - { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV3d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
  - { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV3d0, config: { serializeResultToString: true }}
  - { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV3d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
 # Older serialization versions for backward compatibility:
  - { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV1d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
  - { className: org.apache.tinkerpop.gremlin.driver.ser.GryoLiteMessageSerializerV1d0, config: {ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
  - { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV1d0, config: { serializeResultToString: true }}
  - { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerGremlinV2d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
  - { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerGremlinV1d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistryV1d0] }}
  - { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV1d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistryV1d0] }}
processors:
  - { className: org.apache.tinkerpop.gremlin.server.op.session.SessionOpProcessor, config: { sessionTimeout: 28800000 }}
  - { className: org.apache.tinkerpop.gremlin.server.op.traversal.TraversalOpProcessor, config: { cacheExpirationTime: 600000, cacheMaxSize: 1000 }}
metrics: {
  consoleReporter: {enabled: true, interval: 180000},
  csvReporter: {enabled: true, interval: 180000, fileName: /tmp/gremlin-server-metrics.csv},
  jmxReporter: {enabled: true},
  slf4jReporter: {enabled: true, interval: 180000},
  gangliaReporter: {enabled: false, interval: 180000, addressingMode: MULTICAST},
  graphiteReporter: {enabled: false, interval: 180000}}
maxInitialLineLength: 4096
maxHeaderSize: 8192
maxChunkSize: 8192
maxContentLength: 65536
maxAccumulationBufferComponents: 1024
resultIterationBatchSize: 64
writeBufferLowWaterMark: 32768
writeBufferHighWaterMark: 65536

janusgraph-cql-configurationgraph.properties

# Copyright 2019 JanusGraph Authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#      http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# JanusGraph configuration sample: Cassandra over a socket
#
# This file connects to a Cassandra daemon running on localhost via
# CQL.  Cassandra must already be started before starting JanusGraph
# with this file.

gremlin.graph=org.janusgraph.core.ConfiguredGraphFactory

# The primary persistence provider used by JanusGraph.  This is required.
# It should be set one of JanusGraph's built-in shorthand names for its
# standard storage backends (shorthands: berkeleyje, cql, cassandrathrift,
# cassandra, astyanax, embeddedcassandra, hbase, inmemory) or to the full
# package and classname of a custom/third-party StoreManager
# implementation.
#
# Default:    (no default value)
# Data Type:  String
# Mutability: LOCAL
storage.backend=cql

# The configuration management graph reserved for use with JanusGraphManager
# and ConfiguredGraphFactory
graph.graphname=ConfigurationManagementGraph
# The hostname or comma-separated list of hostnames of storage backend
# servers.  This is only applicable to some storage backends, such as
# cassandra and hbase.
#
# Default:    127.0.0.1
# Data Type:  class java.lang.String[]
# Mutability: LOCAL
storage.hostname=127.0.0.1

# Whether to enable JanusGraph's database-level cache, which is shared
# across all transactions. Enabling this option speeds up traversals by
# holding hot graph elements in memory, but also increases the likelihood
# of reading stale data.  Disabling it forces each transaction to
# holding hot graph elements in memory, but also increases the likelihood
# of reading stale data.  Disabling it forces each transaction to
# independently fetch graph elements from storage before reading/writing
# them.
#
# Default:    false
# Data Type:  Boolean
# Mutability: MASKABLE
cache.db-cache = true

# How long, in milliseconds, database-level cache will keep entries after
# flushing them.  This option is only useful on distributed storage
# backends that are capable of acknowledging writes without necessarily
# making them immediately visible.
#
# Default:    50
# Data Type:  Integer
# Mutability: GLOBAL_OFFLINE
#
# Settings with mutability GLOBAL_OFFLINE are centrally managed in
# JanusGraph's storage backend.  After starting the database for the first
# time, this file's copy of this setting is ignored.  Use JanusGraph's
# Management System to read or modify this value after bootstrapping.
cache.db-cache-clean-wait = 20

# Default expiration time, in milliseconds, for entries in the
# database-level cache. Entries are evicted when they reach this age even
# if the cache has room to spare. Set to 0 to disable expiration (cache
# entries live forever or until memory pressure triggers eviction when set
# to 0).
#
# Default:    10000
# Data Type:  Long
# Mutability: GLOBAL_OFFLINE
#
# Settings with mutability GLOBAL_OFFLINE are centrally managed in
# JanusGraph's storage backend.  After starting the database for the first
# time, this file's copy of this setting is ignored.  Use JanusGraph's
# Management System to read or modify this value after bootstrapping.
cache.db-cache-time = 180000

# Size of JanusGraph's database level cache.  Values between 0 and 1 are
# interpreted as a percentage of VM heap, while larger values are
# interpreted as an absolute size in bytes.
#
# Default:    0.3
# Data Type:  Double
# Mutability: MASKABLE
cache.db-cache-size = 0.5

Error:

gremlin> :remote connect tinkerpop.server conf/remote.yaml session
==>Configured localhost/127.0.0.1:8182-[8671c17b-e724-484a-aa9f-6b0d255b5ded]
gremlin> :remote console
==>All scripts will now be sent to Gremlin Server - [localhost/127.0.0.1:8182]-[8671c17b-e724-484a-aa9f-6b0d255b5ded] - type ':remote console' to return to local mode
gremlin> ConfiguredGraphFactory.getGraphNames()
Please add a key named "ConfigurationManagementGraph" to the "graphs" property in your YAML file and restart the server to be able to use the functionality of the ConfigurationManagementGraph class.

Request you to check and help me to solve the issue.

Thanks & Regards,
Vinayak


Re: JanusGraphIndex how to retrieve constraint (indexOnly) specified for the global index?

cmilowka
 

Thank you, Index class is having tx() transaction description, but I have not found it there as well,

I have added these comments to 1163 as suggested.

Regards, CM


Re: JanusGraphIndex how to retrieve constraint (indexOnly) specified for the global index?

hadoopmarc@...
 

Confirmed, could not find it either. Could you please add it as a comment to:

https://github.com/JanusGraph/janusgraph/issues/1163

Unless someone comes up with a solution, it seems you will have to keep track of label constraints yourself.

Best wishes,   Marc


Re: Gremlin Query to return count for nodes and edges

hadoopmarc@...
 

Hi Vinayak,

Speeding up your query depends on your setup. 15.000 vertices/second is already fast. Is this the janusgraph inmemory backend? Or ScyllaDB?

In a perfect world, not there yet, your query would profit from parallelization (OLAP). JanusGraph supports both the withComputer() and withComputer(SparkGraphComputer) start steps, but the former is undocumented and the performance gains of the latter are often disappointing.

Best wishes,    Marc