Date   

Re: New Property keys in existing index getting stuck in registered state

Boxuan Li
 

Can you post the stacktrace (or the place where NPE is thrown)?

On Wed, Jan 12, 2022 at 2:50 AM Harshit Sharma <harshit.sharma1080@...> wrote:
That is not working. According to Documentation https://docs.janusgraph.org/schema/index-management/index-performance/
After build index i'm calling ManagementSystem.awaitGraphIndexStatus(graph, INDEX_NAME).call()
but this call is throwing nullPointerException.

On Tue, Jan 11, 2022 at 8:18 PM Boxuan Li <liboxuan@...> wrote:
Hi Harshit,


On Tue, Jan 11, 2022 at 10:43 PM Harshit Sharma <harshit.sharma1080@...> wrote:
Sorry because of type posting again -

If I'm adding new keys to an existing index, many keys are getting stuck in the registered or installed state
for example, I indexed a key "graphId" for the vertex in an index "graphVertexIndex"
and I indexed the same key for edges in index names "graphEdgeIndex".
Now it is getting enabled in "graphVertexIndex" but getting stuck in the Registered state in "graphEdgeIndex"

On Tue, Jan 11, 2022 at 8:10 PM Harshit Sharma via lists.lfaidata.foundation <harshit.sharma1080=gmail.com@...> wrote:
If I'm adding new keys to an existing index, many keys are getting stuck in the registered or installed state
for example, I indexed a key "graphId" for the vertex in an index graphVertexIndex
while I indexed the same key for an edge in graphEdgeIndex
it is getting enabled in graphVertexIndex but stuck in the Registered state

--
Regards,

Harshit Sharma
+91-9901459920



--
Regards,

Harshit Sharma
+91-9901459920



--
Regards,

Harshit Sharma
+91-9901459920


Re: New Property keys in existing index getting stuck in registered state

Harshit Sharma
 

That is not working. According to Documentation https://docs.janusgraph.org/schema/index-management/index-performance/
After build index i'm calling ManagementSystem.awaitGraphIndexStatus(graph, INDEX_NAME).call()
but this call is throwing nullPointerException.

On Tue, Jan 11, 2022 at 8:18 PM Boxuan Li <liboxuan@...> wrote:
Hi Harshit,


On Tue, Jan 11, 2022 at 10:43 PM Harshit Sharma <harshit.sharma1080@...> wrote:
Sorry because of type posting again -

If I'm adding new keys to an existing index, many keys are getting stuck in the registered or installed state
for example, I indexed a key "graphId" for the vertex in an index "graphVertexIndex"
and I indexed the same key for edges in index names "graphEdgeIndex".
Now it is getting enabled in "graphVertexIndex" but getting stuck in the Registered state in "graphEdgeIndex"

On Tue, Jan 11, 2022 at 8:10 PM Harshit Sharma via lists.lfaidata.foundation <harshit.sharma1080=gmail.com@...> wrote:
If I'm adding new keys to an existing index, many keys are getting stuck in the registered or installed state
for example, I indexed a key "graphId" for the vertex in an index graphVertexIndex
while I indexed the same key for an edge in graphEdgeIndex
it is getting enabled in graphVertexIndex but stuck in the Registered state

--
Regards,

Harshit Sharma
+91-9901459920



--
Regards,

Harshit Sharma
+91-9901459920



--
Regards,

Harshit Sharma
+91-9901459920


Re: Concurrent Transactions on JG Edges

Aman <amandeep.srivastava1996@...>
 

Hi,

Can someone please advise if previously mentioned way is incorrect for using lock consistencies on edge property keys (version in my case)?

Regards,
Aman

On Mon, 10 Jan, 2022, 10:26 pm Aman via lists.lfaidata.foundation, <amandeep.srivastava1996=gmail.com@...> wrote:
Hi Marc,

Thank you for your response. This is how I'm using it:

1. I created a property key called version, which is incremented at the service layer every time an update is made on an element (vertex or edge). 

mgmt = graph.openManagement()
version = mgmt.makePropertyKey('version').dataType(Integer.class).make()
mgmt.setConsistency(name, ConsistencyModifier.LOCK)
mgmt.buildIndex("vertexIndex", Vertex.class).addKey(version, Mapping.DEFAULT.asParameter()).buildMixedIndex("esIndex")
mgmt.buildIndex("edgeIndex", Edge.class).addKey(version, Mapping.DEFAULT.asParameter()).buildMixedIndex("esIndex")
mgmt.commit()

In case mixed index already exists, I simply add the property in both vertex and edge mixed indices:
index = mgmt.getGraphIndex("edgeIndex or vertexIndex")
mgmt.addIndexKey(index, version)
In case

2. For every new edge label I create, I'm attaching this property to the edge.

mgmt = graph.openManagement() relatedTo = mgmt.makeEdgeLabel('relatedTo').make() mgmt.addProperties(relatedTo, version) mgmt.commit()

How I tested: When I'm making multiple concurrent calls via Jmeter to update an existing vertex, only one request passes, and rest throw a permanent locking exception. However, when I run same test for edges, multiple updates are successful. Few that fail are because of version check logic at service layer which kicks in a few ms after first few updates are successful.

I'm using Janus with Cassandra and Elastic Search.

Regards,

Aman


Re: New Property keys in existing index getting stuck in registered state

Boxuan Li
 

Hi Harshit,


On Tue, Jan 11, 2022 at 10:43 PM Harshit Sharma <harshit.sharma1080@...> wrote:
Sorry because of type posting again -

If I'm adding new keys to an existing index, many keys are getting stuck in the registered or installed state
for example, I indexed a key "graphId" for the vertex in an index "graphVertexIndex"
and I indexed the same key for edges in index names "graphEdgeIndex".
Now it is getting enabled in "graphVertexIndex" but getting stuck in the Registered state in "graphEdgeIndex"

On Tue, Jan 11, 2022 at 8:10 PM Harshit Sharma via lists.lfaidata.foundation <harshit.sharma1080=gmail.com@...> wrote:
If I'm adding new keys to an existing index, many keys are getting stuck in the registered or installed state
for example, I indexed a key "graphId" for the vertex in an index graphVertexIndex
while I indexed the same key for an edge in graphEdgeIndex
it is getting enabled in graphVertexIndex but stuck in the Registered state

--
Regards,

Harshit Sharma
+91-9901459920



--
Regards,

Harshit Sharma
+91-9901459920


Re: New Property keys in existing index getting stuck in registered state

Harshit Sharma
 

Sorry because of type posting again -

If I'm adding new keys to an existing index, many keys are getting stuck in the registered or installed state
for example, I indexed a key "graphId" for the vertex in an index "graphVertexIndex"
and I indexed the same key for edges in index names "graphEdgeIndex".
Now it is getting enabled in "graphVertexIndex" but getting stuck in the Registered state in "graphEdgeIndex"


On Tue, Jan 11, 2022 at 8:10 PM Harshit Sharma via lists.lfaidata.foundation <harshit.sharma1080=gmail.com@...> wrote:
If I'm adding new keys to an existing index, many keys are getting stuck in the registered or installed state
for example, I indexed a key "graphId" for the vertex in an index graphVertexIndex
while I indexed the same key for an edge in graphEdgeIndex
it is getting enabled in graphVertexIndex but stuck in the Registered state

--
Regards,

Harshit Sharma
+91-9901459920



--
Regards,

Harshit Sharma
+91-9901459920


New Property keys in existing index getting stuck in registered state

Harshit Sharma
 

If I'm adding new keys to an existing index, many keys are getting stuck in the registered or installed state
for example, I indexed a key "graphId" for the vertex in an index graphVertexIndex
while I indexed the same key for an edge in graphEdgeIndex
it is getting enabled in graphVertexIndex but stuck in the Registered state

--
Regards,

Harshit Sharma
+91-9901459920


JG Schema - addConnection seem to create duplicate connections

Peter Molnar
 

Hi All,

I have a strange behaviour while using the addConnection method for creating JG schema constraints. It seems it creates duplicated connections in some cases. Please see below how to reproduce this with JG v0.6 and Cassandra v3.11.11 backend. I just used the below snippets in Gremlin Console 3.5.1 to connect to JG remotely.

The graph is supposed model transactions and parties involved in the transaction. It has transaction, person and entity nodes. A transaction is supposed to have one "from party" (either person or entity) connected with FROM edge and one "to party" (either person or entity) connected with TO edge. For example (Person A) --FROM--> (Transaction #1) --TO--> (Entity A) for expressing that "Transaction #1" was performed between "Person A" and "Entity A" and the source of the transaction was "Person A".

Creating dynamic graph and schema
-------
map = new HashMap();
map.put("storage.backend","cql");
map.put("storage.hostname","cassandra");
map.put("query.force-index", "false");
map.put("schema.default", "default");
map.put("schema.constraints", "false");
map.put("graph.graphname", "transactionGraph")
ConfiguredGraphFactory.createConfiguration(new MapConfiguration(map));
 
graph = ConfiguredGraphFactory.open("transactionGraph");
mgmt = graph.openManagement();
 
transaction = mgmt.makeVertexLabel('transaction').make();
person = mgmt.makeVertexLabel('person').make();
entity = mgmt.makeVertexLabel('entity').make();
 
fromEdge = mgmt.makeEdgeLabel('FROM').multiplicity(ONE2ONE).make()
mgmt.addConnection(fromEdge, person, transaction)
mgmt.addConnection(fromEdge, entity, transaction)
 
toEdge = mgmt.makeEdgeLabel('TO').multiplicity(ONE2ONE).make()
mgmt.addConnection(toEdge, transaction, person)
mgmt.addConnection(toEdge, transaction, entity)
 
mgmt.commit()

Checking connections of the schema
----------------
mgmt = graph.openManagement()
edges = mgmt.getRelationTypes(EdgeLabel.class)
fromEdge = edges[0]
toEdge = edges[1]
 
fromEdge.mappedConnections().size() // as I would expect, it has two connections
toEdge.mappedConnections().size() // why 4 connections are here? I would expect only two connections similarly to the FROM edge
 
mgmt.close()

--------------

Could you please have a look and let me know if this is a feature or a bug?

Thanks, Peter


Re: Janusgraph not able to find suitable index for a index enabled property key

Harshit Sharma
 

I think I found the issue, I was using default mapping for all properties which are created in separate transactions.

I have a few questions -

1. Does JG support TEXTSTRING mapping SET cardinality?
2. Just want to be sure, if Default Mapping is equal to TEXT Mapping?

On Mon, Jan 10, 2022 at 10:08 PM Harshit Sharma via lists.lfaidata.foundation <harshit.sharma1080=gmail.com@...> wrote:
It looks something like this ->


graph_search_graph-index-edge: {

mappings: {

- properties: {
   - all: { 
        type: "text"
      }, 
- graphId: {
     type: "text",
     -copy_to: [ "all"]
   }
- graphId__STRING: { type: "keyword" }

- svc_edge_key_s: {
      type: "text", 
      - copy_to: [ "all"]
}
}
}

On Mon, Jan 10, 2022 at 9:46 PM <sergeymetallic@...> wrote:
Could you please provide a ES mapping that was generated for this property?



--
Regards,

Harshit Sharma
+91-9901459920



--
Regards,

Harshit Sharma
+91-9901459920


Re: Concurrent Transactions on JG Edges

Aman <amandeep.srivastava1996@...>
 

Hi Marc,

Thank you for your response. This is how I'm using it:

1. I created a property key called version, which is incremented at the service layer every time an update is made on an element (vertex or edge). 

mgmt = graph.openManagement()
version = mgmt.makePropertyKey('version').dataType(Integer.class).make()
mgmt.setConsistency(name, ConsistencyModifier.LOCK)
mgmt.buildIndex("vertexIndex", Vertex.class).addKey(version, Mapping.DEFAULT.asParameter()).buildMixedIndex("esIndex")
mgmt.buildIndex("edgeIndex", Edge.class).addKey(version, Mapping.DEFAULT.asParameter()).buildMixedIndex("esIndex")
mgmt.commit()

In case mixed index already exists, I simply add the property in both vertex and edge mixed indices:
index = mgmt.getGraphIndex("edgeIndex or vertexIndex")
mgmt.addIndexKey(index, version)
In case

2. For every new edge label I create, I'm attaching this property to the edge.

mgmt = graph.openManagement() relatedTo = mgmt.makeEdgeLabel('relatedTo').make() mgmt.addProperties(relatedTo, version) mgmt.commit()

How I tested: When I'm making multiple concurrent calls via Jmeter to update an existing vertex, only one request passes, and rest throw a permanent locking exception. However, when I run same test for edges, multiple updates are successful. Few that fail are because of version check logic at service layer which kicks in a few ms after first few updates are successful.

I'm using Janus with Cassandra and Elastic Search.

Regards,

Aman


Re: Janusgraph not able to find suitable index for a index enabled property key

Harshit Sharma
 

It looks something like this ->


graph_search_graph-index-edge: {

mappings: {

- properties: {
   - all: { 
        type: "text"
      }, 
- graphId: {
     type: "text",
     -copy_to: [ "all"]
   }
- graphId__STRING: { type: "keyword" }

- svc_edge_key_s: {
      type: "text", 
      - copy_to: [ "all"]
}
}
}

On Mon, Jan 10, 2022 at 9:46 PM <sergeymetallic@...> wrote:
Could you please provide a ES mapping that was generated for this property?



--
Regards,

Harshit Sharma
+91-9901459920


Re: Concurrent Transactions on JG Edges

hadoopmarc@...
 

Hi Aman,

The JanusGraph docs on consistency, cqlLocking do not limit its use to vertices. Can you please show how you translated the example from https://docs.janusgraph.org/advanced-topics/eventual-consistency/#data-consistency for vertex properties to the locking of edge properties?

Best wishes,    Marc


Re: Janusgraph not able to find suitable index for a index enabled property key

sergeymetallic@...
 

Could you please provide a ES mapping that was generated for this property?


Re: Janusgraph not able to find suitable index for a index enabled property key

Harshit Sharma
 

Yes it is TEXTSTRING


On Mon, Jan 10, 2022 at 5:33 PM Boxuan Li <liboxuan@...> wrote:
Sorry, I meant to ask

Are you sure "key1" has TEXTSTRING mapping, rather than TEXT mapping?

On Mon, Jan 10, 2022 at 8:01 PM Harshit Sharma <harshit.sharma1080@...> wrote:
Yes, key1 is of TEXT String mapping because I'm creating only TEXT or TEXTSTRING type of mapping.

I can try to update but looks like an upgrade from 0.5.2 to 0.6.0 involve some major changes and things are breaking in my local

On Mon, Jan 10, 2022 at 5:23 PM Boxuan Li <liboxuan@...> wrote:
You mentioned that 
 
g.V().has("key1", textContains("val") ) is working while 
g.V().has("key1","val13") is not working
 

Are you sure "key1" has TEXTSTRING mapping, rather than STRING mapping?

I am unable to reproduce it on 0.6.0. Can you try the 0.6.0 version?



--
Regards,

Harshit Sharma
+91-9901459920



--
Regards,

Harshit Sharma
+91-9901459920


Concurrent Transactions on JG Edges

Amandeep Srivastava <amandeep.srivastava1996@...>
 

Hi, 

Is there a way by which I can prevent concurrent updates to edges? For vertices, I'm simply maintaining a version field with Lock consistency, so a concurrent update results in Permanent Locking Exception being thrown, but the same doesn't seem to work for edges.

--
Regards,
Aman


Concurrent Transactions on JG Edges

Aman <amandeep.srivastava1996@...>
 

Hi, 

Is there a way by which I can prevent concurrent updates to edges? For vertices, I'm simply maintaining a version field with Lock consistency, so a concurrent update results in Permanent Locking Exception being thrown, but the same doesn't seem to work for edges.

--
Regards,
Aman


Re: Janusgraph not able to find suitable index for a index enabled property key

Boxuan Li
 

Sorry, I meant to ask

Are you sure "key1" has TEXTSTRING mapping, rather than TEXT mapping?

On Mon, Jan 10, 2022 at 8:01 PM Harshit Sharma <harshit.sharma1080@...> wrote:
Yes, key1 is of TEXT String mapping because I'm creating only TEXT or TEXTSTRING type of mapping.

I can try to update but looks like an upgrade from 0.5.2 to 0.6.0 involve some major changes and things are breaking in my local

On Mon, Jan 10, 2022 at 5:23 PM Boxuan Li <liboxuan@...> wrote:
You mentioned that 
 
g.V().has("key1", textContains("val") ) is working while 
g.V().has("key1","val13") is not working
 

Are you sure "key1" has TEXTSTRING mapping, rather than STRING mapping?

I am unable to reproduce it on 0.6.0. Can you try the 0.6.0 version?



--
Regards,

Harshit Sharma
+91-9901459920


Re: Janusgraph not able to find suitable index for a index enabled property key

Harshit Sharma
 

Yes, key1 is of TEXT String mapping because I'm creating only TEXT or TEXTSTRING type of mapping.

I can try to update but looks like an upgrade from 0.5.2 to 0.6.0 involve some major changes and things are breaking in my local

On Mon, Jan 10, 2022 at 5:23 PM Boxuan Li <liboxuan@...> wrote:
You mentioned that 
 
g.V().has("key1", textContains("val") ) is working while 
g.V().has("key1","val13") is not working
 
Are you sure "key1" has TEXTSTRING mapping, rather than STRING mapping?

I am unable to reproduce it on 0.6.0. Can you try the 0.6.0 version?



--
Regards,

Harshit Sharma
+91-9901459920


Re: Janusgraph not able to find suitable index for a index enabled property key

Boxuan Li
 

You mentioned that 
 
g.V().has("key1", textContains("val") ) is working while 
g.V().has("key1","val13") is not working
 
Are you sure "key1" has TEXTSTRING mapping, rather than STRING mapping?

I am unable to reproduce it on 0.6.0. Can you try the 0.6.0 version?


Re: Janusgraph not able to find suitable index for a index enabled property key

Harshit Sharma
 

But this has query is working for other properties which are created in the same transaction as that of the index. They also have mapping as TEXTSTRING


On Mon, Jan 10, 2022 at 3:55 PM <sergeymetallic@...> wrote:
For these kind of queries you need TEXTSTRING mapping. As an alternative you can create composite index to "equals" queries



--
Regards,

Harshit Sharma
+91-9901459920


Re: Janusgraph not able to find suitable index for a index enabled property key

sergeymetallic@...
 

For these kind of queries you need TEXTSTRING mapping. As an alternative you can create composite index to "equals" queries