Help needed! Configure janusgraph-1.0.0-docker (either rebuild from base or docker compose with env variable setting) to use ConfigurationManagementGraph, and then initialize it with own graph & schema


Yingjie Li
 

Hello,

Sorry if this post is kind of a continuation of my last  post. As I have not received any suggestion, so i would like to start a new thread to surface up the problem. 

I would like to know the exactly code for me to be able to configure janusgraph-1.0.0-docker (either rebuild a new image from this base) or docker-compose it with env variable settings.  So that I can use ConfigurationManagementGraph and graph traversal for query.  I will need it to be initialized with my own schema groovy (where I define the graph name,  set up other configurations and created composite/mixed index into Cassandra/ElasticSearch( I used OpenSearch instead and it kind of worked).  A sample schema (init_graph_sample1.groovy) is attached, This is similar to the schema I have been using in Januagraph prepcakged versions.

From the previous response I got, it seems all these should have been supported already, but unfortunately it does not work for me with my various tries. I feel I am on the right track, but not exactly there yet. 

I attached a Docker file  (please ignore the jar removal part for now)  that I would like your help to adding the necessary code to use ConfigurationManagementGraph. Only after I get the configuration right, I can initize it with my own schema as attached, 

 

If you have a docker compose that shows the env settings for the above, I am glad to try that as well. I have been trying getting these settings in a env file of docker compose, but it did not work except for the last line. 

gremlinserver.graphs.ConfigurationManagementGraph=/etc/opt/janusgraph/janusgraph.properties
janusgraph.gremlin.graph=org.janusgraph.core.ConfiguredGraphFactory
janusgraph.graph.graphname=ConfigurationManagementGraph
gremlinserver%d.graphs.graph=
gremlinserver.evaluationTimeout=300000

 

I have been using jansguraph prepackage versions, where I know exactly what yaml file to replace/change  to enable the above. But it becomes kind of mystery of how to do it in the unprepackage version.

I also tried having a local install of janusgraph-1.0.0-rc2, and invoke server by providing a configuration file which i think should be used: 

./bin/janusgraph-server.sh start conf/gremlin-server/gremlin-server-configuration-cql-es.yaml

And after starting gremlin.sh, I can create my own graph graph1 using ConfiguredGraphFactory, and can create mixed index, and I can see the graph1 is created in the backend cassandra and my indexes (not exactly the same name)  are created in the backend OpenSearch (ElasticSearch), pictures are attached, but I am not able to use graph1_traversal to query the graph. 

 

My request is kind of urgent as Janusgraph has been the core of our project that has been going on for two/three years, and we really need your help to bring it to the next level  for potential production. We have been used prepackaged version starting from 0.5.3, and recently we need to replace the all in one (janusgraph1.0.0-rc1-ful) with the lastest (janusgraph1.0.0-rc2) and use Cassandra and OpenSearch.  

Due to a tight deadline, would appreciate somebody can help me as soon as possible.  


Yingjie Li
 

To report back, If I directly use janusgraph-1.0.0-rc local install,
after starting the server with  ./bin/janusgraph-server.sh start conf/gremlin-server/gremlin-server-configuration.yaml
i can create graph use ConfigureGraphFactory, and initize it with my own schema with composite and mxied index. 

To be able to traveral, I need to restart the server. then I can do the followings sucessfully: 

gremlin> :remote connect tinkerpop.server conf/remote.yaml session

08:41:02 INFO  com.jcabi.log.Logger.infoForced - 0 attributes loaded from 328 stream(s) in 56ms, 104 saved, 5165 ignored: []

08:41:03 INFO  org.apache.tinkerpop.gremlin.driver.Connection.<init> - Created new connection for ws://localhost:8182/gremlin

08:41:03 INFO  org.apache.tinkerpop.gremlin.driver.ConnectionPool.<init> - Opening connection pool on Host{address=localhost/127.0.0.1:8182, hostUri=ws://localhost:8182/gremlin} with core size of 1

==>Configured localhost/127.0.0.1:8182-[e732c251-4d85-4a3e-89b6-f5c8db565b47]

gremlin> :remote console

==>All scripts will now be sent to Gremlin Server - [localhost/127.0.0.1:8182]-[e732c251-4d85-4a3e-89b6-f5c8db565b47] - type ':remote console' to return to local mode

gremlin> ConfiguredGraphFactory.getGraphNames()

==>graph2

==>graph1

gremlin> graph2_traversal.V().count()

==>0

gremlin> graph1_traversal.V().count()

==>0

 

But I still will need to know how to use the docker unless there is no choice but to build the docker from source code.

 

Thanks,
Yingjei


Florian Hockmann
 

You can simply set all required config via environment variables with the Docker image. If you are already building your own image based on the official one, then you can just add env vars there. Just add them like this:

 

ENV key=value

 

e.g. to configure JanusGraph for CQL & ES:

 

ENV JANUS_PROPS_TEMPLATE=cql-es

 

You can then check whether the resulting config (both the JanusGraph Server YAML file as well as the JanusGraph .properties file) looks like you expect, by using the `show-config` command:

 

rocker run –rm -it my-custom-docker-image janusgraph show-config

 

as described here: https://github.com/JanusGraph/janusgraph-docker#generate-config

 

That should enable you to compare the config used by the Docker image with the config that you apparently already got working without Docker.

 

Von: janusgraph-users@... <janusgraph-users@...> Im Auftrag von Yingjie Li
Gesendet: Dienstag, 21. März 2023 14:05
An: janusgraph-users@...
Betreff: Re: [janusgraph-users] Help needed! Configure janusgraph-1.0.0-docker (either rebuild from base or docker compose with env variable setting) to use ConfigurationManagementGraph, and then initialize it with own graph & schema

 

To report back, If I directly use janusgraph-1.0.0-rc local install,
after starting the server with  ./bin/janusgraph-server.sh start conf/gremlin-server/gremlin-server-configuration.yaml
i can create graph use ConfigureGraphFactory, and initize it with my own schema with composite and mxied index. 

To be able to traveral, I need to restart the server. then I can do the followings sucessfully: 

gremlin> :remote connect tinkerpop.server conf/remote.yaml session

08:41:02 INFO  com.jcabi.log.Logger.infoForced - 0 attributes loaded from 328 stream(s) in 56ms, 104 saved, 5165 ignored: []

08:41:03 INFO  org.apache.tinkerpop.gremlin.driver.Connection.<init> - Created new connection for ws://localhost:8182/gremlin

08:41:03 INFO  org.apache.tinkerpop.gremlin.driver.ConnectionPool.<init> - Opening connection pool on Host{address=localhost/127.0.0.1:8182, hostUri=ws://localhost:8182/gremlin} with core size of 1

==>Configured localhost/127.0.0.1:8182-[e732c251-4d85-4a3e-89b6-f5c8db565b47]

gremlin> :remote console

==>All scripts will now be sent to Gremlin Server - [localhost/127.0.0.1:8182]-[e732c251-4d85-4a3e-89b6-f5c8db565b47] - type ':remote console' to return to local mode

gremlin> ConfiguredGraphFactory.getGraphNames()

==>graph2

==>graph1

gremlin> graph2_traversal.V().count()

==>0

gremlin> graph1_traversal.V().count()

==>0

 

But I still will need to know how to use the docker unless there is no choice but to build the docker from source code.

 

Thanks,
Yingjei


Yingjie Li
 

Thanks Florian. Can you please help me with 

1. the exact ENV code  that I can add to my Docker to enable ConfigurationManagementGraph so that I can apply my own schema (using ConfiguredGraphFactory to create graph and create index) when docker start. 

2. the exact janusgprah env of the above if I use docker-compose to compose 3 containers janusgraph1.0.0-rc2, cassandra, and OpenSearch(ElasticSearch). I put the below lines in the env as you suggested in other email chain, and unfortunately they are not working except the last one. So what am missing here?

gremlinserver.graphs.ConfigurationManagementGraph=/etc/opt/janusgraph/janusgraph.properties

janusgraph.gremlin.graph=org.janusgraph.core.ConfiguredGraphFactory

janusgraph.graph.graphname=ConfigurationManagementGraph

gremlinserver%d.graphs.graph=

gremlinserver.evaluationTimeout=300000

I know the general concept,and I can set other env like gremlinserver.evaluationTimeout. I just can not get ConfiguredGraphFactory work me using docker or docker-compose. 

For now, yes, I get the local installation of janugraph1.0.0-rc2 work (e.g., use ConfiguredGraphFactory  to create graph etc) with the other two containers  by simply invoking janusgprah server with config: 
./bin/janusgraph-server.sh start conf/gremlin-server/gremlin-server-configuration.yaml and initialize it with my own schema,

i was able to inject near million nodes/edges into the graph (3-5 faster than the jansugraph-full version), but I would perfer to not creating my image from source code if there is docker image we can leverage.

Thanks 

Thank you

Yingjie