|
Re: JG Schema - addConnection seem to create duplicate connections
Peter Molnar
Hi Marc,
Thanks a lot for looking into this. As requested, I filled an issue about this on Github: https://github.com/JanusGraph/janusgraph/issues/2950 Thanks, Peter
|
|
|
|
Re: Janusgraph embedded multi instance(JVM) data sync issue
Pawan Shriwas
Hi Marc, I don't think data cache was created due to elastic search/mixed index only. I have seen this on basic property/node without index as well. I am thinking let's work on basic node/property then we can plan for mixed index cases. Any suggestions for basic case without an index backend? Thanks, Pawan
On Sat, Jan 15, 2022 at 5:16 PM <hadoopmarc@...> wrote: Hi Pawan, --
Thanks & Regard PAWAN SHRIWAS
|
|
|
|
Re: JG Schema - addConnection seem to create duplicate connections
hadoopmarc@...
Hi Peter,
toggle quoted messageShow quoted text
Thanks for reporting. I think it is a bug. I checked with the standalone gremlin REPL of janusgraph-0.6.0, using: graph = JanusGraphFactory.open('conf/janusgraph-inmemory.properties') This gives the same results and if you add the from toEdge connections first, the FromEdge gets 4 connections. You can check that two of the four connections are redundant, that is, they refer to the same edge in the schema: gremlin> edges[1].mappedConnections() ==>org.janusgraph.core.Connection@1fecfaea ==>org.janusgraph.core.Connection@4872669f ==>org.janusgraph.core.Connection@483f286e ==>org.janusgraph.core.Connection@4bb147ec gremlin> edges[1].mappedConnections()[0].getConnectionEdge() ==>e[hs0-el-1th-st][525-~T$SchemaRelated->1037] gremlin> edges[1].mappedConnections()[1].getConnectionEdge() ==>e[ikg-el-1th-171][525-~T$SchemaRelated->1549] gremlin> edges[1].mappedConnections()[2].getConnectionEdge() ==>e[hs0-el-1th-st][525-~T$SchemaRelated->1037] gremlin> edges[1].mappedConnections()[3].getConnectionEdge() ==>e[ikg-el-1th-171][525-~T$SchemaRelated->1549] Finally, I checked that the schema results remain the same if you add the following config properties to the graph (as suggested by the ref docs): schema.default=none schema.constraints=true Can you please report this as an issue on: https://github.com/JanusGraph/janusgraph/issues Best wishes, Marc
On Tue, Jan 11, 2022 at 01:06 PM, Peter Molnar wrote: mgmt = graph.openManagement();
|
|
|
|
Re: Fastest way to check if a property key is mixed indexed or not
hadoopmarc@...
Hi Harshit,
The performance impact for JanusGraph when including a property key in multiple mixed indices, is negligable (the selection of the index for a specific query will be a tat slower). Additional mixed indices imply a heavier load on the indexing backend (in particular memory and storage, CPU during inserts) but with little impact on response times if the cluster is dimensioned properly. Marc
|
|
|
|
Re: Janusgraph embedded multi instance(JVM) data sync issue
hadoopmarc@...
Hi Pawan,
OK, let's investigate further. You say that the issue occurs for both vertex creation and modification. Let's take the clearest case first: vertex creation with an indexed property. So, in your system setup, if you have added a new vertex with embedded intance1, sometimes it takes a minute or more before a query for this vertex (based on its property value) on instance2 returns the vertex. This can only mean that the elasticserch index sometimes does not return the new property value. This on its turn means that an elasticsearch replica has not yet been synced with the data about the new vertex. Indeed, the janusgraph-elastic configs have a key index.[X].elasticsearch.bulk-refresh (default: false) which can be set to any of the values in: https://www.elastic.co/guide/en/elasticsearch/reference/7.16/docs-refresh.html One can check the correspondence between this janusgraph config item and the elasticsearch API parameter in: https://github.com/JanusGraph/janusgraph/blob/v0.6.0/janusgraph-es/src/main/java/org/janusgraph/diskstorage/es/rest/RestElasticSearchClient.java So, can you see what happens with the other possible values for index.[X].elasticsearch.bulk-refresh? Best wishes, Marc
|
|
|
|
Re: Fastest way to check if a property key is mixed indexed or not
Harshit Sharma
Will there be any performance impact if i will index a property key in multiple indices (mixed index)?
On Sat, 15 Jan, 2022, 3:55 pm , <hadoopmarc@...> wrote: Hi Harshit,
|
|
|
|
Re: Fastest way to check if a property key is mixed indexed or not
hadoopmarc@...
Hi Harshit,
The concept "property is indexed or not" is ambiguous because an index can have multiple property keys. If you want to know if there is an index with a specific property key as the only key, indeed you would have to do something like in your example code (but modified). Best wishes, Marc
|
|
|
|
Re: Janusgraph embedded multi instance(JVM) data sync issue
Pawan Shriwas
Hi Marc, I have removed cache properties from instances and we already have new transactions for each api operation but still facing stale data issues in other instances for some time. Below is the code which is used for the new transaction for each operation. In my embedded janusgraph service, We always create new translations for each api operation using below code and do commit or rollback at the end of api operation. but sometimes it works and sometimes not. Is it a sync kind of issue which varies between graph instances in multiple services(JVM). // Create graph instance code(once service start) String filePath = ConfigUtils.getString(GraphConstants.GRAPH_FILE_PATH); JanusGraph graphinstance = embeddedConnection.open(filePath); // create transaction code for each api operation JanusgraphTransaction threadedTransaction= graphinstance.getGraphInstance().newTransaction(); // we do commit or rollback at end of each api operation threadedTransaction.commit(); //or threadedTransaction.rollback(); Let me know if anything related to configuration or any code needs to tried for the same. Thanks, Pawan
On Fri, Jan 7, 2022 at 1:45 PM <hadoopmarc@...> wrote: Hi Pawan, --
Thanks & Regard PAWAN SHRIWAS
|
|
|
|
Fastest way to check if a property key is mixed indexed or not
Harshit Sharma
Is there a way I can check if a particular property is indexed or not? I know the following method but there I will have to traverse all indexes List<JanusgraphIndex> indexList = mgmt.getIndexes(Vertex.class) For(index : indexList){ propertyKeys = index.getFieldKeys() if (propertyKeys.contains("KEY1") return true; } return false; is there a better way to do the same? -- Regards, Harshit Sharma +91-9901459920
|
|
|
|
Re: New Property keys in existing index getting stuck in registered state
Harshit Sharma
Is it allowed to index the same property key in two different indexes. For example I created a property key graphId and created two indexes vertexIndex, edgeIndex. Index graphId in both indexes. The problem I'm facing is this graphId index is getting enabled in vertexIndex because I'm creating it first but it is getting stuck in REGISTERED state for edgeIndex
On Wed, Jan 12, 2022 at 6:38 AM Boxuan Li <liboxuan@...> wrote:
--
Regards, Harshit Sharma +91-9901459920
|
|
|
|
Re: New Property keys in existing index getting stuck in registered state
Boxuan Li
Can you post the stacktrace (or the place where NPE is thrown)?
On Wed, Jan 12, 2022 at 2:50 AM Harshit Sharma <harshit.sharma1080@...> wrote:
|
|
|
|
Re: New Property keys in existing index getting stuck in registered state
Harshit Sharma
That is not working. According to Documentation https://docs.janusgraph.org/schema/index-management/index-performance/ After build index i'm calling ManagementSystem.awaitGraphIndexStatus(graph, INDEX_NAME).call() but this call is throwing nullPointerException.
On Tue, Jan 11, 2022 at 8:18 PM Boxuan Li <liboxuan@...> wrote:
--
Regards, Harshit Sharma +91-9901459920
|
|
|
|
Re: Concurrent Transactions on JG Edges
Aman <amandeep.srivastava1996@...>
Hi, Regards, Aman
On Mon, 10 Jan, 2022, 10:26 pm Aman via lists.lfaidata.foundation, <amandeep.srivastava1996=gmail.com@...> wrote: Hi Marc,
|
|
|
|
Re: New Property keys in existing index getting stuck in registered state
Boxuan Li
Hi Harshit,
On Tue, Jan 11, 2022 at 10:43 PM Harshit Sharma <harshit.sharma1080@...> wrote:
|
|
|
|
Re: New Property keys in existing index getting stuck in registered state
Harshit Sharma
Sorry because of type posting again - If I'm adding new keys to an existing index, many keys are getting stuck in the registered or installed state for example, I indexed a key "graphId" for the vertex in an index "graphVertexIndex" and I indexed the same key for edges in index names "graphEdgeIndex". Now it is getting enabled in "graphVertexIndex" but getting stuck in the Registered state in "graphEdgeIndex"
On Tue, Jan 11, 2022 at 8:10 PM Harshit Sharma via lists.lfaidata.foundation <harshit.sharma1080=gmail.com@...> wrote:
--
Regards, Harshit Sharma +91-9901459920
|
|
|
|
New Property keys in existing index getting stuck in registered state
Harshit Sharma
If I'm adding new keys to an existing index, many keys are getting stuck in the registered or installed state for example, I indexed a key "graphId" for the vertex in an index graphVertexIndex while I indexed the same key for an edge in graphEdgeIndex it is getting enabled in graphVertexIndex but stuck in the Registered state Regards, Harshit Sharma +91-9901459920
|
|
|
|
JG Schema - addConnection seem to create duplicate connections
Peter Molnar
Hi All,
I have a strange behaviour while using the addConnection method for creating JG schema constraints. It seems it creates duplicated connections in some cases. Please see below how to reproduce this with JG v0.6 and Cassandra v3.11.11 backend. I just used the below snippets in Gremlin Console 3.5.1 to connect to JG remotely. The graph is supposed model transactions and parties involved in the transaction. It has transaction, person and entity nodes. A transaction is supposed to have one "from party" (either person or entity) connected with FROM edge and one "to party" (either person or entity) connected with TO edge. For example (Person A) --FROM--> (Transaction #1) --TO--> (Entity A) for expressing that "Transaction #1" was performed between "Person A" and "Entity A" and the source of the transaction was "Person A". Creating dynamic graph and schema ------- map = new HashMap();
map.put("storage.backend","cql");
map.put("storage.hostname","cassandra");
map.put("query.force-index", "false");
map.put("schema.default", "default");
map.put("schema.constraints", "false");
map.put("graph.graphname", "transactionGraph")
ConfiguredGraphFactory.createConfiguration(new MapConfiguration(map));
graph = ConfiguredGraphFactory.open("transactionGraph");
mgmt = graph.openManagement();
transaction = mgmt.makeVertexLabel('transaction').make();
person = mgmt.makeVertexLabel('person').make();
entity = mgmt.makeVertexLabel('entity').make();
fromEdge = mgmt.makeEdgeLabel('FROM').multiplicity(ONE2ONE).make()
mgmt.addConnection(fromEdge, person, transaction)
mgmt.addConnection(fromEdge, entity, transaction)
toEdge = mgmt.makeEdgeLabel('TO').multiplicity(ONE2ONE).make()
mgmt.addConnection(toEdge, transaction, person)
mgmt.addConnection(toEdge, transaction, entity)
mgmt.commit()
Checking connections of the schema ---------------- mgmt = graph.openManagement()
edges = mgmt.getRelationTypes(EdgeLabel.class)
fromEdge = edges[0]
toEdge = edges[1]
fromEdge.mappedConnections().size() // as I would expect, it has two connections
toEdge.mappedConnections().size() // why 4 connections are here? I would expect only two connections similarly to the FROM edge
mgmt.close() -------------- Could you please have a look and let me know if this is a feature or a bug? Thanks, Peter
|
|
|
|
Re: Janusgraph not able to find suitable index for a index enabled property key
Harshit Sharma
I think I found the issue, I was using default mapping for all properties which are created in separate transactions. I have a few questions - 1. Does JG support TEXTSTRING mapping SET cardinality? 2. Just want to be sure, if Default Mapping is equal to TEXT Mapping?
On Mon, Jan 10, 2022 at 10:08 PM Harshit Sharma via lists.lfaidata.foundation <harshit.sharma1080=gmail.com@...> wrote:
--
Regards, Harshit Sharma +91-9901459920
|
|
|
|
Re: Concurrent Transactions on JG Edges
Aman <amandeep.srivastava1996@...>
Hi Marc,
Thank you for your response. This is how I'm using it: 1. I created a property key called version, which is incremented at the service layer every time an update is made on an element (vertex or edge). mgmt = graph.openManagement() version = mgmt.makePropertyKey('version').dataType(Integer.class).make() mgmt.setConsistency(name, ConsistencyModifier.LOCK) mgmt.buildIndex("vertexIndex", Vertex.class).addKey(version, Mapping.DEFAULT.asParameter()).buildMixedIndex("esIndex") mgmt.buildIndex("edgeIndex", Edge.class).addKey(version, Mapping.DEFAULT.asParameter()).buildMixedIndex("esIndex") mgmt.commit() In case mixed index already exists, I simply add the property in both vertex and edge mixed indices: index = mgmt.getGraphIndex("edgeIndex or vertexIndex") mgmt.addIndexKey(index, version) In case 2. For every new edge label I create, I'm attaching this property to the edge. mgmt = graph.openManagement() relatedTo = mgmt.makeEdgeLabel('relatedTo').make() mgmt.addProperties(relatedTo, version) mgmt.commit() How I tested: When I'm making multiple concurrent calls via Jmeter to update an existing vertex, only one request passes, and rest throw a permanent locking exception. However, when I run same test for edges, multiple updates are successful. Few that fail are because of version check logic at service layer which kicks in a few ms after first few updates are successful. I'm using Janus with Cassandra and Elastic Search. Regards, Aman
|
|
|
|
Re: Janusgraph not able to find suitable index for a index enabled property key
Harshit Sharma
It looks something like this -> mappings: { - properties: { - all: { type: "text" }, - graphId: { type: "text", -copy_to: [ "all"] } - graphId__STRING: { type: "keyword" } - svc_edge_key_s: { type: "text", - copy_to: [ "all"] } } }
On Mon, Jan 10, 2022 at 9:46 PM <sergeymetallic@...> wrote: Could you please provide a ES mapping that was generated for this property? --
Regards, Harshit Sharma +91-9901459920
|
|
|