Date   

Re: Python output to mgmt queries

hadoopmarc@...
 

I am not sure what you are up to and the API changes in remote connections may have confused you.

If you want to see the labels of all vertices in the graph (for janusgraph-0.6.x with gremlin_python-3.5.1):
from gremlin_python.process.anonymous_traversal import traversal
g = traversal().withRemote(DriverRemoteConnection('ws://localhost:8182/gremlin','g'))
g.V().label().toList
If you want to see the vertex labels that you defined in the JanusgGraph schema:
from gremlin_python.driver.client import Client
client = Client('ws://localhost:8182/gremlin', 'graph')
for line in client.submit("graph.openManagement().printSchema()").next():
    print(line)
Best wishes,    Marc




Re: Using a user-supplied string as vertex ID

Boxuan Li
 

Hi Scott,

Currently, JanusGraph does not support user-specified string identifiers. You could check out https://github.com/JanusGraph/janusgraph/issues/1221 to see discussions on this topic.

Best,
Boxuan


Re: Important | Queries for edge label connections

Boxuan Li
 
Edited

Hi Pawan,

Regarding your first question, try this in Java:

mgmt.getEdgeLabel("belongsTo").mappedConnections()

which should give you a list of Java objects that contain the outgoing and incoming labels for each connection.

In the Gremlin console, you could do this:

mgmt.getEdgeLabel("knows").mappedConnections()[0].incomingVertexLabel
mgmt.getEdgeLabel("knows").mappedConnections()[0].outgoingVertexLabel
which is a bit hacky but hopefully shall work. Feel free to create a feature request on GitHub and link to this thread.

Regarding your second question, IIRC there is no such API available.

Best regards,
Boxuan


Using a user-supplied string as vertex ID

Scott Friedman
 

Greetings,

I'd like to specify unique string IDs for newly-added vertices in JanusGraph.  I've verified that I can set graph.set-vertex-id to True and then add integer IDs via my (python) client as expected.

Does JanusGraph support user-specified string identifiers in any fashion?  If not, is there a recommended way to map into integers (e.g., a potentially lengthy MD5 hash?) or will such a long number damage JanusGraph's ID indexing?

Thanks much for your time!

Scott


JG as a 3store, rdf support

Matthew Nguyen <nguyenm9@...>
 

Hey folks, been playing with JG the last couple weeks and am able to import a few million triples using rdf2g (cassandra/solr backend).  I'm processing around 1000 triples/sec currently after turning on batch-loading and disabling a few pre-conditions :-). While this may be suitable for loading a few million triples, it will take far too long to load a billion+.  I've also gotten sparql-gremlin working but haven't yet run it through its paces though I'm disheartened to see that the project appears to have been abandoned. 

I'm looking to communicate with others interested in trying to use JG as a 3store given the lack of available enterprise capable 3store opensource projects currently available.  After some searching on here, there appears to have been some bits & pieces of conversations from various people through the years re: RDF processing.  

Has anyone on here made any significant strides with rdf & JG and can share their experiences?  And if there's a better place to discuss this topic, please advise.

thx, matt


Important | Queries for edge label connections

Pawan Shriwas
 

Hi All,

I need a solution for these two things, but I tried but was not able to find the solution.

1. I want to list the edgeLabel connection created in janusgraph 
       mgmt.addConnection(“belongsTo”, vertexLabel1, vertexLabel2);

       mgmt.addConnection("belongsTo", vertexLabel1, vertexLabel3);

       mgmt.addConnection("belongsTo", vertexLabel3, vertexLabel4);

       mgmt.addConnection("belongsTo", vertexLabel5, vertexLabel6);


    Can see only edge labels in printSchema but not how many time it used between Vertex labels. which is created after above steps.


2.  I want to update the direction of one connection of edgeLabel.


        current ->     mgmt.addConnection(“belongsTo”, vertexLabel1vertexLabel2);   //outdirection towards the vertexLabel2

       

       I want to update the direction of this created connection. like below 

       Expected Direction -->       mgmt.addConnection(“belongsTo”, vertexLabel2 ,vertexLabel1);     // I want only one kind of direction to exist between these 2 nodes types. If this option creates another connection then I want the previous direction to be removed.



Please review and let me know how I can achieve this.  Thanks in advance.



Thanks,

Pawan 

    

 

     




Python output to mgmt queries

dimi
 
Edited

Hi! I am trying to parse some basic info from the schema via Python but I am probably doing something wrong. 

I can request the management info with the Client object in python:

from gremlin_python.driver.client import Client
client = Client('ws://localhost:8182/gremlin', 'mygraph')
mgmt = "mygraph.openManagement()"
get_v_labels = mgmt + ".getVertexLabels()"
tt = client.submit(get_v_labels).all().result()

I have 16 labels, and if I run it in the gremlin console I obtain a list of labels. In python, instead, 
I get
[v[74253], v[74765], v[75277], v[75789], v[76301], v[76813], v[77325], v[77837], v[78349], v[78861], v[79373], v[79885], v[80397], v[80909], v[81421], v[81933]]

If I do
for t in tt:
    for p in g.V(t.id).properties():
         print("key:",p.label, "| value: " ,p.value)
I do not get any output. How can I get the list of labels from the schema?


Re: high-scale-lib dependency

sergeymetallic@...
 

Hm, in Janusgraph version 0.6.0 there is a different library used https://github.com/datastax/java-driver , is there any point to have the dependency on apache cassandra?


Re: high-scale-lib dependency

Clement de Groc
 

Hey! Just wanted to report that we had a similar issue with high-scale-lib.
Replacing high-scale-lib with JCTools sounds like a good option, but I'm not sure it will work for all modules: if I'm not mistaken, Cassandra relies on `high-scale-lib` too.
Another solution could be to exclude all classes under `java/util` from JanusGraph uber-jars.


high-scale-lib dependency

sergeymetallic@...
 
Edited

There is a java library dependency in Janusgraph "high-scale-lib" which is old and unsupported (last update happened 8 years back, see https://github.com/boundary/high-scale-lib). When Janusgraph is included as library into another project it causes IDE issues when loaded into Eclipse or VScode and using Java 11, just because it contains packages like "java.*"

As a solution I would suggest to migrate to a successor project which is actively developed https://github.com/JCTools/JCTools and does not have such issues


Re: High HBase backend 'configuration' row contention

hadoopmarc@...
 

Hi Tendai,

Just one thing came to my mind: did you apply the JanusGraphFactory inside a singleton object so that all tasks from all cores in a spark executor use the same JanusGraph instance? If not, this is an easy change to lower overhead due to connection setups.

Best wishes,   Marc


Re: High HBase backend 'configuration' row contention

Tendai Munetsi
 

Hi Marc,

Thanks for responding back. The configuration row in question, which is created by Janusgraph when the HBase table is first initialized, is having slow read performance due to the simultaneous access by the Spark executors (400+). Again, each executor creates an embedded Janusgraph instance, and we found that the Janusgraph instance accesses the config row every time the JanusGraphFactory’s open() method is called (numerous times per executor). This leads to the executors trying to access this row at the same time and is causing the row to respond back slow. The rest of the other graph data rows do NOT have this latency while reading/writing the graph. I hope that provides some clarification on the issue.

Regards,
Tendai


Re: Bindings for graphs created using ConfiguredGraphFactory not working as expected

anya.sharma@...
 

Hello Marc,

Thank you for your reply. In response to your suggestions and the questions you posed:
Are you sure you did start Cassandra ("cassandra/bin/cassandra") before starting JanusGraph? - Yes, the Cassandra server is running. I am not using the Cassandra bundled with JanusGraph, but my own separate installation which was running at the time when I faced this issue.
Also check whether you did not mix up the graph1 and graph1_config graph.graphname values - I double-checked, and the values for the graph name that I am using are correct.
I guess you found out to do (before running bin/janusgraph-server.sh):
export JANUSGRAPH_YAML=conf/gremlin-server/gremlin-server-configuration.yaml - I am running the server using the command - gremlin-server.bat conf/gremlin-server/gremlin-server-configuration.yaml instead of storing the yaml file in an environment variable. 

Apart from the above, I also ran the same commands as mentioned in my question on a 0.3.1 JanusGraph server and it worked:

Creating the graph:


Accessing the graph through the implicit variables:



The same steps give the issue posed in the question when run using JanusGraph 0.6.0:


Thanks
Anya


JanusGraph and Apache Log4J CVE-2021-44228 (aka "log4shell")

Florian Hockmann
 

A critical security vulnerability was reported in Apache Log4j 2 on December 9, 2021: CVE-2021-44228 [1]. JanusGraph itself is not directly affected by this vulnerability as it still uses log4j 1.2 and the CVE only affects log4j2.

 

However, the two index backends Elasticsearch and Apache Solr are affected by the CVE. Both projects have already published reports on how they are affected and how the vulnerability can be mitigated:

Apache HBase, Apache TinkerPop, Apache Spark, and Apache Hadoop are all also still on log4j 1 and are therefore also not affected.

 

We recommend all JanusGraph users who use JanusGraph together with Elasticsearch or Apache Solr to follow the recommendations listed in the reports linked above. This mostly comes down to updating the backends once an update with a fix is available and to set the JVM option -Dlog4j2.formatMsgNoLookups=true as a mitigation until then.

 

We also plan to release patch versions for 0.5 and 0.6 shortly for the Elasticsearch version that is distributed as part of the pre-packed (“full”) distribution of JanusGraph to adopt these recommendations by default for users who start JanusGraph via the `bin/janusgraph.sh` script which also starts Elasticsearch.

 

Given that Log4j 1 already reached EOL (end of life) and is also affected by other CVEs, we also plan to move away from that version in a future release of JanusGraph.

 

Best regards,

The JanusGraph Technical Steering Committee

 


Re: Drop graph not working

hadoopmarc@...
 

Thanks for your report. Can you please provide the following details:
  1. Which version of JanusGraph do you use?
  2. Are you sure you did not set the storage.drop-on-clear property to false?
  3. Do you have permissions to drop the table manually with hbase shell?


Re: High HBase backend 'configuration' row contention

hadoopmarc@...
 

Hi Tendai,

I do not understand the concept of row contention. Is not this config row just the row that is retrieved most often on the region servers that contain it and are not other rows on these servers served equally slow?

HBase tends to compact tables to a limited number of large size regions (typically 20GB). So, if you have an hdfs replication factor of 3 and your graph has a size of just two regions, at best 6 region servers of your HBase cluster can serve your 500 spark executors.

So, maybe this gives you some hint on what is happening. Or maybe you have more details on how you came to the conclusion that there is such a thing as row contention?

Best wishes,    Marc


High HBase backend 'configuration' row contention

Tendai Munetsi
 

Hi,

 

We are running embedded Janusgraph (0.5.3) with an HBase backend (2.1.6) in our Spark jobs. Each Spark executor creates an instance of Janusgraph. At times there can be over 500 executors running simultaneously. Under those conditions, we observe heavy row contention for the ‘configuration’ row that Janusgraph creates as part of the initialization of the HBase table. Is there any recommendation on how to prevent/reduce this HBase row contention? As the row is only created during HBase initialization and is never updated subsequently, can the data held by the configuration row be moved out of HBase and into a static file?

 

Thanks,

Tendai


This e-mail may contain information that is privileged or confidential. If you are not the intended recipient, please delete the e-mail and any attachments and notify us immediately.



Re: Problem adding edges with the same label

Miroslav Smiljanic
 

The same problem does not exist when using InMemory Storage Backend.


On Tue, Dec 7, 2021 at 1:51 PM Miroslav Smiljanic <miroslav@...> wrote:
Hi All,

I have the setup using Cassandra backend (Azure Cosmos DB fully managed).

Seems it is not possible to add two edges with the same label.

gremlin> g.addV()
==>v[4288]
gremlin> g.addV()
==>v[4200]
gremlin> g.addV()
==>v[4344]
gremlin> g.addE('child').from(__.V(4288)).to(__.V(4200))
==>e[t4-3b4-t1-38o][4288-child->4200]
gremlin> g.addE('child').from(__.V(4200)).to(__.V(4344))
java.lang.NullPointerException
Type ':help' or ':h' for help.
Display stack trace? [yN]


This is the error in server log

388895 [gremlin-server-exec-7] WARN  org.apache.tinkerpop.gremlin.server.op.AbstractEvalOpProcessor  - Exception processing a script on request [RequestMessage{, requestId=a04edabb-920e-4526-ab3a-1d1d22a8ed82, op='eval', processor='', args={gremlin=g.addE('child').from(__.V(4200)).to(__.V(4344)), userAgent=Gremlin Console/1.21.0, batchSize=64}}].
java.lang.NullPointerException
at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.getUniquenessLock(StandardJanusGraphTx.java:720)
at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.addEdge(StandardJanusGraphTx.java:784)
at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.addEdge(StandardJanusGraphTx.java:774)
at org.janusgraph.graphdb.vertices.AbstractVertex.addEdge(AbstractVertex.java:188)
at org.janusgraph.graphdb.vertices.AbstractVertex.addEdge(AbstractVertex.java:45)
at org.apache.tinkerpop.gremlin.process.traversal.step.map.AddEdgeStartStep.processNextStart(AddEdgeStartStep.java:137)
at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:150)
at org.apache.tinkerpop.gremlin.process.traversal.util.DefaultTraversal.hasNext(DefaultTraversal.java:222)
at org.apache.tinkerpop.gremlin.server.op.AbstractOpProcessor.handleIterator(AbstractOpProcessor.java:97)
at org.apache.tinkerpop.gremlin.server.op.AbstractEvalOpProcessor.lambda$evalOpInternal$5(AbstractEvalOpProcessor.java:263)
at org.apache.tinkerpop.gremlin.groovy.engine.GremlinExecutor.lambda$eval$0(GremlinExecutor.java:283)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)

Does anyone know what could be the problem?

Regards,
Miroslav 


Problem adding edges with the same label

Miroslav Smiljanic
 

Hi All,

I have the setup using Cassandra backend (Azure Cosmos DB fully managed).

Seems it is not possible to add two edges with the same label.

gremlin> g.addV()
==>v[4288]
gremlin> g.addV()
==>v[4200]
gremlin> g.addV()
==>v[4344]
gremlin> g.addE('child').from(__.V(4288)).to(__.V(4200))
==>e[t4-3b4-t1-38o][4288-child->4200]
gremlin> g.addE('child').from(__.V(4200)).to(__.V(4344))
java.lang.NullPointerException
Type ':help' or ':h' for help.
Display stack trace? [yN]


This is the error in server log

388895 [gremlin-server-exec-7] WARN  org.apache.tinkerpop.gremlin.server.op.AbstractEvalOpProcessor  - Exception processing a script on request [RequestMessage{, requestId=a04edabb-920e-4526-ab3a-1d1d22a8ed82, op='eval', processor='', args={gremlin=g.addE('child').from(__.V(4200)).to(__.V(4344)), userAgent=Gremlin Console/1.21.0, batchSize=64}}].
java.lang.NullPointerException
at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.getUniquenessLock(StandardJanusGraphTx.java:720)
at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.addEdge(StandardJanusGraphTx.java:784)
at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.addEdge(StandardJanusGraphTx.java:774)
at org.janusgraph.graphdb.vertices.AbstractVertex.addEdge(AbstractVertex.java:188)
at org.janusgraph.graphdb.vertices.AbstractVertex.addEdge(AbstractVertex.java:45)
at org.apache.tinkerpop.gremlin.process.traversal.step.map.AddEdgeStartStep.processNextStart(AddEdgeStartStep.java:137)
at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:150)
at org.apache.tinkerpop.gremlin.process.traversal.util.DefaultTraversal.hasNext(DefaultTraversal.java:222)
at org.apache.tinkerpop.gremlin.server.op.AbstractOpProcessor.handleIterator(AbstractOpProcessor.java:97)
at org.apache.tinkerpop.gremlin.server.op.AbstractEvalOpProcessor.lambda$evalOpInternal$5(AbstractEvalOpProcessor.java:263)
at org.apache.tinkerpop.gremlin.groovy.engine.GremlinExecutor.lambda$eval$0(GremlinExecutor.java:283)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)

Does anyone know what could be the problem?

Regards,
Miroslav 


Drop graph not working

51kumarakhil@...
 

Hi all. 
I'm creating dynamic graphs using configuredGraphFactory in bigTable. I'm able to create them but when I try to delete them then it seems that they only get deleted from janusgraph-server and not from bigTable.
When I try to access the graph it says graph doesnt exists.. but I can still see it in the bigTable. When I try to manually delete the graph from BigTable server throw error like graph not found.
I used below cmd to delete the graph 
> ConfiguredGraphFactory.drop('graph_name') 

:_(

381 - 400 of 6663