Date   

Re: Janusgraph embedded multi instance(JVM) data sync issue

Pawan Shriwas
 

Hi Marc,

I have removed cache properties from instances and we already have new transactions for each api operation but still facing stale data issues in other instances for some time.

Below is the code which is used for the new transaction for each operation.

In my embedded janusgraph service, We always create new translations for each api operation using below code and do commit or rollback at the end of api operation.  but sometimes it works and sometimes not. Is it a sync kind of issue which varies between graph instances in multiple services(JVM).

// Create graph instance code(once service start) 
  String filePath = ConfigUtils.getString(GraphConstants.GRAPH_FILE_PATH);
  JanusGraph graphinstance = embeddedConnection.open(filePath);

// create transaction code for each api operation
  JanusgraphTransaction threadedTransaction=  graphinstance.getGraphInstance().newTransaction();

// we do commit or rollback at end of each api operation
        threadedTransaction.commit();
                 //or 
        threadedTransaction.rollback();

Let me know if anything related to configuration or any code needs to tried for the same.

Thanks,
Pawan

On Fri, Jan 7, 2022 at 1:45 PM <hadoopmarc@...> wrote:
Hi Pawan,

Your requirement for instant synchronization cannot work with JanusGraph caches enabled, because JanusGraph will get data from the cache if available, instead of getting the latest data from the backend. So,

  • cache.db-cache = false
  • be sure to start a new transaction before querying for the latest data (e.g. by executing a g.tx().commit())
Best wishes,    Marc



--
Thanks & Regard

PAWAN SHRIWAS


Fastest way to check if a property key is mixed indexed or not

Harshit Sharma
 

Is there a way I can check if a particular property is indexed or not?

I know the following method but there I will have to traverse all indexes

List<JanusgraphIndex> indexList = mgmt.getIndexes(Vertex.class)
For(index : indexList){
  propertyKeys = index.getFieldKeys()
  if (propertyKeys.contains("KEY1")
      return true;
}
return false;

is there a better way to do the same?
--
Regards,

Harshit Sharma
+91-9901459920


Re: New Property keys in existing index getting stuck in registered state

Harshit Sharma
 

Is it allowed to index the same property key in two different indexes.
For example I created a property key graphId and created two indexes vertexIndex, edgeIndex.
Index graphId in both indexes.

The problem I'm facing is this graphId index is getting enabled in vertexIndex because I'm creating it first but it is getting stuck in REGISTERED state for edgeIndex

On Wed, Jan 12, 2022 at 6:38 AM Boxuan Li <liboxuan@...> wrote:
Can you post the stacktrace (or the place where NPE is thrown)?

On Wed, Jan 12, 2022 at 2:50 AM Harshit Sharma <harshit.sharma1080@...> wrote:
That is not working. According to Documentation https://docs.janusgraph.org/schema/index-management/index-performance/
After build index i'm calling ManagementSystem.awaitGraphIndexStatus(graph, INDEX_NAME).call()
but this call is throwing nullPointerException.

On Tue, Jan 11, 2022 at 8:18 PM Boxuan Li <liboxuan@...> wrote:
Hi Harshit,


On Tue, Jan 11, 2022 at 10:43 PM Harshit Sharma <harshit.sharma1080@...> wrote:
Sorry because of type posting again -

If I'm adding new keys to an existing index, many keys are getting stuck in the registered or installed state
for example, I indexed a key "graphId" for the vertex in an index "graphVertexIndex"
and I indexed the same key for edges in index names "graphEdgeIndex".
Now it is getting enabled in "graphVertexIndex" but getting stuck in the Registered state in "graphEdgeIndex"

On Tue, Jan 11, 2022 at 8:10 PM Harshit Sharma via lists.lfaidata.foundation <harshit.sharma1080=gmail.com@...> wrote:
If I'm adding new keys to an existing index, many keys are getting stuck in the registered or installed state
for example, I indexed a key "graphId" for the vertex in an index graphVertexIndex
while I indexed the same key for an edge in graphEdgeIndex
it is getting enabled in graphVertexIndex but stuck in the Registered state

--
Regards,

Harshit Sharma
+91-9901459920



--
Regards,

Harshit Sharma
+91-9901459920



--
Regards,

Harshit Sharma
+91-9901459920



--
Regards,

Harshit Sharma
+91-9901459920


Re: New Property keys in existing index getting stuck in registered state

Boxuan Li
 

Can you post the stacktrace (or the place where NPE is thrown)?

On Wed, Jan 12, 2022 at 2:50 AM Harshit Sharma <harshit.sharma1080@...> wrote:
That is not working. According to Documentation https://docs.janusgraph.org/schema/index-management/index-performance/
After build index i'm calling ManagementSystem.awaitGraphIndexStatus(graph, INDEX_NAME).call()
but this call is throwing nullPointerException.

On Tue, Jan 11, 2022 at 8:18 PM Boxuan Li <liboxuan@...> wrote:
Hi Harshit,


On Tue, Jan 11, 2022 at 10:43 PM Harshit Sharma <harshit.sharma1080@...> wrote:
Sorry because of type posting again -

If I'm adding new keys to an existing index, many keys are getting stuck in the registered or installed state
for example, I indexed a key "graphId" for the vertex in an index "graphVertexIndex"
and I indexed the same key for edges in index names "graphEdgeIndex".
Now it is getting enabled in "graphVertexIndex" but getting stuck in the Registered state in "graphEdgeIndex"

On Tue, Jan 11, 2022 at 8:10 PM Harshit Sharma via lists.lfaidata.foundation <harshit.sharma1080=gmail.com@...> wrote:
If I'm adding new keys to an existing index, many keys are getting stuck in the registered or installed state
for example, I indexed a key "graphId" for the vertex in an index graphVertexIndex
while I indexed the same key for an edge in graphEdgeIndex
it is getting enabled in graphVertexIndex but stuck in the Registered state

--
Regards,

Harshit Sharma
+91-9901459920



--
Regards,

Harshit Sharma
+91-9901459920



--
Regards,

Harshit Sharma
+91-9901459920


Re: New Property keys in existing index getting stuck in registered state

Harshit Sharma
 

That is not working. According to Documentation https://docs.janusgraph.org/schema/index-management/index-performance/
After build index i'm calling ManagementSystem.awaitGraphIndexStatus(graph, INDEX_NAME).call()
but this call is throwing nullPointerException.

On Tue, Jan 11, 2022 at 8:18 PM Boxuan Li <liboxuan@...> wrote:
Hi Harshit,


On Tue, Jan 11, 2022 at 10:43 PM Harshit Sharma <harshit.sharma1080@...> wrote:
Sorry because of type posting again -

If I'm adding new keys to an existing index, many keys are getting stuck in the registered or installed state
for example, I indexed a key "graphId" for the vertex in an index "graphVertexIndex"
and I indexed the same key for edges in index names "graphEdgeIndex".
Now it is getting enabled in "graphVertexIndex" but getting stuck in the Registered state in "graphEdgeIndex"

On Tue, Jan 11, 2022 at 8:10 PM Harshit Sharma via lists.lfaidata.foundation <harshit.sharma1080=gmail.com@...> wrote:
If I'm adding new keys to an existing index, many keys are getting stuck in the registered or installed state
for example, I indexed a key "graphId" for the vertex in an index graphVertexIndex
while I indexed the same key for an edge in graphEdgeIndex
it is getting enabled in graphVertexIndex but stuck in the Registered state

--
Regards,

Harshit Sharma
+91-9901459920



--
Regards,

Harshit Sharma
+91-9901459920



--
Regards,

Harshit Sharma
+91-9901459920


Re: Concurrent Transactions on JG Edges

Aman <amandeep.srivastava1996@...>
 

Hi,

Can someone please advise if previously mentioned way is incorrect for using lock consistencies on edge property keys (version in my case)?

Regards,
Aman

On Mon, 10 Jan, 2022, 10:26 pm Aman via lists.lfaidata.foundation, <amandeep.srivastava1996=gmail.com@...> wrote:
Hi Marc,

Thank you for your response. This is how I'm using it:

1. I created a property key called version, which is incremented at the service layer every time an update is made on an element (vertex or edge). 

mgmt = graph.openManagement()
version = mgmt.makePropertyKey('version').dataType(Integer.class).make()
mgmt.setConsistency(name, ConsistencyModifier.LOCK)
mgmt.buildIndex("vertexIndex", Vertex.class).addKey(version, Mapping.DEFAULT.asParameter()).buildMixedIndex("esIndex")
mgmt.buildIndex("edgeIndex", Edge.class).addKey(version, Mapping.DEFAULT.asParameter()).buildMixedIndex("esIndex")
mgmt.commit()

In case mixed index already exists, I simply add the property in both vertex and edge mixed indices:
index = mgmt.getGraphIndex("edgeIndex or vertexIndex")
mgmt.addIndexKey(index, version)
In case

2. For every new edge label I create, I'm attaching this property to the edge.

mgmt = graph.openManagement() relatedTo = mgmt.makeEdgeLabel('relatedTo').make() mgmt.addProperties(relatedTo, version) mgmt.commit()

How I tested: When I'm making multiple concurrent calls via Jmeter to update an existing vertex, only one request passes, and rest throw a permanent locking exception. However, when I run same test for edges, multiple updates are successful. Few that fail are because of version check logic at service layer which kicks in a few ms after first few updates are successful.

I'm using Janus with Cassandra and Elastic Search.

Regards,

Aman


Re: New Property keys in existing index getting stuck in registered state

Boxuan Li
 

Hi Harshit,


On Tue, Jan 11, 2022 at 10:43 PM Harshit Sharma <harshit.sharma1080@...> wrote:
Sorry because of type posting again -

If I'm adding new keys to an existing index, many keys are getting stuck in the registered or installed state
for example, I indexed a key "graphId" for the vertex in an index "graphVertexIndex"
and I indexed the same key for edges in index names "graphEdgeIndex".
Now it is getting enabled in "graphVertexIndex" but getting stuck in the Registered state in "graphEdgeIndex"

On Tue, Jan 11, 2022 at 8:10 PM Harshit Sharma via lists.lfaidata.foundation <harshit.sharma1080=gmail.com@...> wrote:
If I'm adding new keys to an existing index, many keys are getting stuck in the registered or installed state
for example, I indexed a key "graphId" for the vertex in an index graphVertexIndex
while I indexed the same key for an edge in graphEdgeIndex
it is getting enabled in graphVertexIndex but stuck in the Registered state

--
Regards,

Harshit Sharma
+91-9901459920



--
Regards,

Harshit Sharma
+91-9901459920


Re: New Property keys in existing index getting stuck in registered state

Harshit Sharma
 

Sorry because of type posting again -

If I'm adding new keys to an existing index, many keys are getting stuck in the registered or installed state
for example, I indexed a key "graphId" for the vertex in an index "graphVertexIndex"
and I indexed the same key for edges in index names "graphEdgeIndex".
Now it is getting enabled in "graphVertexIndex" but getting stuck in the Registered state in "graphEdgeIndex"


On Tue, Jan 11, 2022 at 8:10 PM Harshit Sharma via lists.lfaidata.foundation <harshit.sharma1080=gmail.com@...> wrote:
If I'm adding new keys to an existing index, many keys are getting stuck in the registered or installed state
for example, I indexed a key "graphId" for the vertex in an index graphVertexIndex
while I indexed the same key for an edge in graphEdgeIndex
it is getting enabled in graphVertexIndex but stuck in the Registered state

--
Regards,

Harshit Sharma
+91-9901459920



--
Regards,

Harshit Sharma
+91-9901459920


New Property keys in existing index getting stuck in registered state

Harshit Sharma
 

If I'm adding new keys to an existing index, many keys are getting stuck in the registered or installed state
for example, I indexed a key "graphId" for the vertex in an index graphVertexIndex
while I indexed the same key for an edge in graphEdgeIndex
it is getting enabled in graphVertexIndex but stuck in the Registered state

--
Regards,

Harshit Sharma
+91-9901459920


JG Schema - addConnection seem to create duplicate connections

Peter Molnar
 

Hi All,

I have a strange behaviour while using the addConnection method for creating JG schema constraints. It seems it creates duplicated connections in some cases. Please see below how to reproduce this with JG v0.6 and Cassandra v3.11.11 backend. I just used the below snippets in Gremlin Console 3.5.1 to connect to JG remotely.

The graph is supposed model transactions and parties involved in the transaction. It has transaction, person and entity nodes. A transaction is supposed to have one "from party" (either person or entity) connected with FROM edge and one "to party" (either person or entity) connected with TO edge. For example (Person A) --FROM--> (Transaction #1) --TO--> (Entity A) for expressing that "Transaction #1" was performed between "Person A" and "Entity A" and the source of the transaction was "Person A".

Creating dynamic graph and schema
-------
map = new HashMap();
map.put("storage.backend","cql");
map.put("storage.hostname","cassandra");
map.put("query.force-index", "false");
map.put("schema.default", "default");
map.put("schema.constraints", "false");
map.put("graph.graphname", "transactionGraph")
ConfiguredGraphFactory.createConfiguration(new MapConfiguration(map));
 
graph = ConfiguredGraphFactory.open("transactionGraph");
mgmt = graph.openManagement();
 
transaction = mgmt.makeVertexLabel('transaction').make();
person = mgmt.makeVertexLabel('person').make();
entity = mgmt.makeVertexLabel('entity').make();
 
fromEdge = mgmt.makeEdgeLabel('FROM').multiplicity(ONE2ONE).make()
mgmt.addConnection(fromEdge, person, transaction)
mgmt.addConnection(fromEdge, entity, transaction)
 
toEdge = mgmt.makeEdgeLabel('TO').multiplicity(ONE2ONE).make()
mgmt.addConnection(toEdge, transaction, person)
mgmt.addConnection(toEdge, transaction, entity)
 
mgmt.commit()

Checking connections of the schema
----------------
mgmt = graph.openManagement()
edges = mgmt.getRelationTypes(EdgeLabel.class)
fromEdge = edges[0]
toEdge = edges[1]
 
fromEdge.mappedConnections().size() // as I would expect, it has two connections
toEdge.mappedConnections().size() // why 4 connections are here? I would expect only two connections similarly to the FROM edge
 
mgmt.close()

--------------

Could you please have a look and let me know if this is a feature or a bug?

Thanks, Peter


Re: Janusgraph not able to find suitable index for a index enabled property key

Harshit Sharma
 

I think I found the issue, I was using default mapping for all properties which are created in separate transactions.

I have a few questions -

1. Does JG support TEXTSTRING mapping SET cardinality?
2. Just want to be sure, if Default Mapping is equal to TEXT Mapping?

On Mon, Jan 10, 2022 at 10:08 PM Harshit Sharma via lists.lfaidata.foundation <harshit.sharma1080=gmail.com@...> wrote:
It looks something like this ->


graph_search_graph-index-edge: {

mappings: {

- properties: {
   - all: { 
        type: "text"
      }, 
- graphId: {
     type: "text",
     -copy_to: [ "all"]
   }
- graphId__STRING: { type: "keyword" }

- svc_edge_key_s: {
      type: "text", 
      - copy_to: [ "all"]
}
}
}

On Mon, Jan 10, 2022 at 9:46 PM <sergeymetallic@...> wrote:
Could you please provide a ES mapping that was generated for this property?



--
Regards,

Harshit Sharma
+91-9901459920



--
Regards,

Harshit Sharma
+91-9901459920


Re: Concurrent Transactions on JG Edges

Aman <amandeep.srivastava1996@...>
 

Hi Marc,

Thank you for your response. This is how I'm using it:

1. I created a property key called version, which is incremented at the service layer every time an update is made on an element (vertex or edge). 

mgmt = graph.openManagement()
version = mgmt.makePropertyKey('version').dataType(Integer.class).make()
mgmt.setConsistency(name, ConsistencyModifier.LOCK)
mgmt.buildIndex("vertexIndex", Vertex.class).addKey(version, Mapping.DEFAULT.asParameter()).buildMixedIndex("esIndex")
mgmt.buildIndex("edgeIndex", Edge.class).addKey(version, Mapping.DEFAULT.asParameter()).buildMixedIndex("esIndex")
mgmt.commit()

In case mixed index already exists, I simply add the property in both vertex and edge mixed indices:
index = mgmt.getGraphIndex("edgeIndex or vertexIndex")
mgmt.addIndexKey(index, version)
In case

2. For every new edge label I create, I'm attaching this property to the edge.

mgmt = graph.openManagement() relatedTo = mgmt.makeEdgeLabel('relatedTo').make() mgmt.addProperties(relatedTo, version) mgmt.commit()

How I tested: When I'm making multiple concurrent calls via Jmeter to update an existing vertex, only one request passes, and rest throw a permanent locking exception. However, when I run same test for edges, multiple updates are successful. Few that fail are because of version check logic at service layer which kicks in a few ms after first few updates are successful.

I'm using Janus with Cassandra and Elastic Search.

Regards,

Aman


Re: Janusgraph not able to find suitable index for a index enabled property key

Harshit Sharma
 

It looks something like this ->


graph_search_graph-index-edge: {

mappings: {

- properties: {
   - all: { 
        type: "text"
      }, 
- graphId: {
     type: "text",
     -copy_to: [ "all"]
   }
- graphId__STRING: { type: "keyword" }

- svc_edge_key_s: {
      type: "text", 
      - copy_to: [ "all"]
}
}
}

On Mon, Jan 10, 2022 at 9:46 PM <sergeymetallic@...> wrote:
Could you please provide a ES mapping that was generated for this property?



--
Regards,

Harshit Sharma
+91-9901459920


Re: Concurrent Transactions on JG Edges

hadoopmarc@...
 

Hi Aman,

The JanusGraph docs on consistency, cqlLocking do not limit its use to vertices. Can you please show how you translated the example from https://docs.janusgraph.org/advanced-topics/eventual-consistency/#data-consistency for vertex properties to the locking of edge properties?

Best wishes,    Marc


Re: Janusgraph not able to find suitable index for a index enabled property key

sergeymetallic@...
 

Could you please provide a ES mapping that was generated for this property?


Re: Janusgraph not able to find suitable index for a index enabled property key

Harshit Sharma
 

Yes it is TEXTSTRING


On Mon, Jan 10, 2022 at 5:33 PM Boxuan Li <liboxuan@...> wrote:
Sorry, I meant to ask

Are you sure "key1" has TEXTSTRING mapping, rather than TEXT mapping?

On Mon, Jan 10, 2022 at 8:01 PM Harshit Sharma <harshit.sharma1080@...> wrote:
Yes, key1 is of TEXT String mapping because I'm creating only TEXT or TEXTSTRING type of mapping.

I can try to update but looks like an upgrade from 0.5.2 to 0.6.0 involve some major changes and things are breaking in my local

On Mon, Jan 10, 2022 at 5:23 PM Boxuan Li <liboxuan@...> wrote:
You mentioned that 
 
g.V().has("key1", textContains("val") ) is working while 
g.V().has("key1","val13") is not working
 

Are you sure "key1" has TEXTSTRING mapping, rather than STRING mapping?

I am unable to reproduce it on 0.6.0. Can you try the 0.6.0 version?



--
Regards,

Harshit Sharma
+91-9901459920



--
Regards,

Harshit Sharma
+91-9901459920


Concurrent Transactions on JG Edges

Amandeep Srivastava <amandeep.srivastava1996@...>
 

Hi, 

Is there a way by which I can prevent concurrent updates to edges? For vertices, I'm simply maintaining a version field with Lock consistency, so a concurrent update results in Permanent Locking Exception being thrown, but the same doesn't seem to work for edges.

--
Regards,
Aman


Concurrent Transactions on JG Edges

Aman <amandeep.srivastava1996@...>
 

Hi, 

Is there a way by which I can prevent concurrent updates to edges? For vertices, I'm simply maintaining a version field with Lock consistency, so a concurrent update results in Permanent Locking Exception being thrown, but the same doesn't seem to work for edges.

--
Regards,
Aman


Re: Janusgraph not able to find suitable index for a index enabled property key

Boxuan Li
 

Sorry, I meant to ask

Are you sure "key1" has TEXTSTRING mapping, rather than TEXT mapping?

On Mon, Jan 10, 2022 at 8:01 PM Harshit Sharma <harshit.sharma1080@...> wrote:
Yes, key1 is of TEXT String mapping because I'm creating only TEXT or TEXTSTRING type of mapping.

I can try to update but looks like an upgrade from 0.5.2 to 0.6.0 involve some major changes and things are breaking in my local

On Mon, Jan 10, 2022 at 5:23 PM Boxuan Li <liboxuan@...> wrote:
You mentioned that 
 
g.V().has("key1", textContains("val") ) is working while 
g.V().has("key1","val13") is not working
 

Are you sure "key1" has TEXTSTRING mapping, rather than STRING mapping?

I am unable to reproduce it on 0.6.0. Can you try the 0.6.0 version?



--
Regards,

Harshit Sharma
+91-9901459920


Re: Janusgraph not able to find suitable index for a index enabled property key

Harshit Sharma
 

Yes, key1 is of TEXT String mapping because I'm creating only TEXT or TEXTSTRING type of mapping.

I can try to update but looks like an upgrade from 0.5.2 to 0.6.0 involve some major changes and things are breaking in my local

On Mon, Jan 10, 2022 at 5:23 PM Boxuan Li <liboxuan@...> wrote:
You mentioned that 
 
g.V().has("key1", textContains("val") ) is working while 
g.V().has("key1","val13") is not working
 
Are you sure "key1" has TEXTSTRING mapping, rather than STRING mapping?

I am unable to reproduce it on 0.6.0. Can you try the 0.6.0 version?



--
Regards,

Harshit Sharma
+91-9901459920

321 - 340 of 6661