Hi Harshit,

Can you please describe the steps you have taken in more detail:

• creation of property keys and indices in the schema + commit,
• creation of a vertex with a property in one committed transaction,
• query of a vertex based on a property value in another transaction.

Marc

Hi Marc,

1. Creation of property keys and indices in the schema + commit

      graph = (Janusgraph) GraphDBConfigurator.getGraph()

      mgmt = graph.openManagement()

      

     For Vertex ->

           propertyKey = mgmt.makePropertyKey(propertyKeyName).dataType (String.class).cardinality(Cardinality.SET).make()

          if(existingIndex) 

                janusgraphIndex = mgmt.getGraphIndex(VERTEX_INDEX_NAME)

                mgmt.addIndex(janusgraphIndex, propertyKey)

          else

               indexBuilder = mgmt.buildIndex(VERTEX_INDEX_NAME,  Vertex.class)

               if(cardinality is SET)

                        indexBuilder.addKey(propertyKey, String.class.equals(propertyKey.dataType) ? Mapping.TEXT.asParameter() : Mapping.DEFAULT.asParameter())

               else

                        indexBuilder.addKey(propertyKey, String.class.equals(propertyKey.dataType) ? Mapping.TEXTSTRING.asParameter() : Mapping.DEFAULT.asParameter())

  

    For Edge ->

            propertyKey = mgmt.makePropertyKey(propertyKeyName).dataType (String.class).cardinality(Cardinality.SINGLE).make()

             propertyKey = mgmt.makePropertyKey(propertyKeyName).dataType (String.class).cardinality(Cardinality.SINGLE).make()

          if(existingIndex) 

                janusgraphIndex = mgmt.getGraphIndex(EDGE_INDEX_NAME)

                mgmt.addIndex(janusgraphIndex, propertyKey)

          else

               indexBuilder = mgmt.buildIndex(EDGE_INDEX_NAME,  Vertex.class)

               if(cardinality is SET)

                        indexBuilder.addKey(propertyKey, String.class.equals(propertyKey.dataType) ? Mapping.TEXT.asParameter() : Mapping.DEFAULT.asParameter())

               else

                        indexBuilder.addKey(propertyKey, String.class.equals(propertyKey.dataType) ? Mapping.TEXTSTRING.asParameter() : Mapping.DEFAULT.asParameter())

         

            mgmt.commit();

    

       3.   Query of a vertex based on a property value in another transaction

             Let say i created a property key "domain_vertex_key1_s" for vertex

             Query = g.V().has("domain_vertex_key1_s", "val1);

             

             Let say I created a property key "domain_edge_key1_s" for edge

            

             Query = g.V().has("domain_vertex_key1_s","val2");

          

On Mon, 10 Jan, 2022, 12:46 am , <hadoopmarc@...> wrote:

Hi Harshit,

Can you please describe the steps you have taken in more detail:

• creation of property keys and indices in the schema + commit,
• creation of a vertex with a property in one committed transaction,
• query of a vertex based on a property value in another transaction.

Marc

Hi Harshit,

You should use STRING mapping if you want exact string match. TEXT mapping is for full text search. See 

https://docs.janusgraph.org/index-backend/text-search/#full-text-and-string-search

Best,

Boxuan

On Mon, Jan 10, 2022 at 1:15 PM Harshit Sharma <harshit.sharma1080@...> wrote:

Hi Marc,

1. Creation of property keys and indices in the schema + commit

      graph = (Janusgraph) GraphDBConfigurator.getGraph()

      mgmt = graph.openManagement()

      

     For Vertex ->

           propertyKey = mgmt.makePropertyKey(propertyKeyName).dataType (String.class).cardinality(Cardinality.SET).make()

          if(existingIndex) 

                janusgraphIndex = mgmt.getGraphIndex(VERTEX_INDEX_NAME)

                mgmt.addIndex(janusgraphIndex, propertyKey)

          else

               indexBuilder = mgmt.buildIndex(VERTEX_INDEX_NAME,  Vertex.class)

               if(cardinality is SET)

                        indexBuilder.addKey(propertyKey, String.class.equals(propertyKey.dataType) ? Mapping.TEXT.asParameter() : Mapping.DEFAULT.asParameter())

               else

                        indexBuilder.addKey(propertyKey, String.class.equals(propertyKey.dataType) ? Mapping.TEXTSTRING.asParameter() : Mapping.DEFAULT.asParameter())

  

    For Edge ->

            propertyKey = mgmt.makePropertyKey(propertyKeyName).dataType (String.class).cardinality(Cardinality.SINGLE).make()

             propertyKey = mgmt.makePropertyKey(propertyKeyName).dataType (String.class).cardinality(Cardinality.SINGLE).make()

          if(existingIndex) 

                janusgraphIndex = mgmt.getGraphIndex(EDGE_INDEX_NAME)

                mgmt.addIndex(janusgraphIndex, propertyKey)

          else

               indexBuilder = mgmt.buildIndex(EDGE_INDEX_NAME,  Vertex.class)

               if(cardinality is SET)

                        indexBuilder.addKey(propertyKey, String.class.equals(propertyKey.dataType) ? Mapping.TEXT.asParameter() : Mapping.DEFAULT.asParameter())

               else

                        indexBuilder.addKey(propertyKey, String.class.equals(propertyKey.dataType) ? Mapping.TEXTSTRING.asParameter() : Mapping.DEFAULT.asParameter())

         

            mgmt.commit();

    

       3.   Query of a vertex based on a property value in another transaction

             Let say i created a property key "domain_vertex_key1_s" for vertex

             Query = g.V().has("domain_vertex_key1_s", "val1);

             

             Let say I created a property key "domain_edge_key1_s" for edge

            

             Query = g.V().has("domain_vertex_key1_s","val2");

          

On Mon, 10 Jan, 2022, 12:46 am , <hadoopmarc@...> wrote:

Hi Harshit,

Can you please describe the steps you have taken in more detail:

• creation of property keys and indices in the schema + commit,
• creation of a vertex with a property in one committed transaction,
• query of a vertex based on a property value in another transaction.

Marc

but is that the reason I'm not able to use property keys created in a new transaction?

Also, can we mix index property keys of cardinality SET?

On Mon, Jan 10, 2022 at 10:56 AM Boxuan Li <liboxuan@...> wrote:

Hi Harshit,

You should use STRING mapping if you want exact string match. TEXT mapping is for full text search. See 

https://docs.janusgraph.org/index-backend/text-search/#full-text-and-string-search

Best,

Boxuan

On Mon, Jan 10, 2022 at 1:15 PM Harshit Sharma <harshit.sharma1080@...> wrote:

Hi Marc,

1. Creation of property keys and indices in the schema + commit

      graph = (Janusgraph) GraphDBConfigurator.getGraph()

      mgmt = graph.openManagement()

      

     For Vertex ->

           propertyKey = mgmt.makePropertyKey(propertyKeyName).dataType (String.class).cardinality(Cardinality.SET).make()

          if(existingIndex) 

                janusgraphIndex = mgmt.getGraphIndex(VERTEX_INDEX_NAME)

                mgmt.addIndex(janusgraphIndex, propertyKey)

          else

               indexBuilder = mgmt.buildIndex(VERTEX_INDEX_NAME,  Vertex.class)

               if(cardinality is SET)

                        indexBuilder.addKey(propertyKey, String.class.equals(propertyKey.dataType) ? Mapping.TEXT.asParameter() : Mapping.DEFAULT.asParameter())

               else

                        indexBuilder.addKey(propertyKey, String.class.equals(propertyKey.dataType) ? Mapping.TEXTSTRING.asParameter() : Mapping.DEFAULT.asParameter())

  

    For Edge ->

            propertyKey = mgmt.makePropertyKey(propertyKeyName).dataType (String.class).cardinality(Cardinality.SINGLE).make()

             propertyKey = mgmt.makePropertyKey(propertyKeyName).dataType (String.class).cardinality(Cardinality.SINGLE).make()

          if(existingIndex) 

                janusgraphIndex = mgmt.getGraphIndex(EDGE_INDEX_NAME)

                mgmt.addIndex(janusgraphIndex, propertyKey)

          else

               indexBuilder = mgmt.buildIndex(EDGE_INDEX_NAME,  Vertex.class)

               if(cardinality is SET)

                        indexBuilder.addKey(propertyKey, String.class.equals(propertyKey.dataType) ? Mapping.TEXT.asParameter() : Mapping.DEFAULT.asParameter())

               else

                        indexBuilder.addKey(propertyKey, String.class.equals(propertyKey.dataType) ? Mapping.TEXTSTRING.asParameter() : Mapping.DEFAULT.asParameter())

         

            mgmt.commit();

    

       3.   Query of a vertex based on a property value in another transaction

             Let say i created a property key "domain_vertex_key1_s" for vertex

             Query = g.V().has("domain_vertex_key1_s", "val1);

             

             Let say I created a property key "domain_edge_key1_s" for edge

            

             Query = g.V().has("domain_vertex_key1_s","val2");

          

On Mon, 10 Jan, 2022, 12:46 am , <hadoopmarc@...> wrote:

Hi Harshit,

Can you please describe the steps you have taken in more detail:

• creation of property keys and indices in the schema + commit,
• creation of a vertex with a property in one committed transaction,
• query of a vertex based on a property value in another transaction.

Marc

--

Regards,

Harshit Sharma

+91-9901459920

Only For SET cardinality, I'm using TEXT mapping because looks like SET does not work with TEXTSTRING.

On Mon, Jan 10, 2022 at 11:07 AM Harshit Sharma via lists.lfaidata.foundation <harshit.sharma1080=gmail.com@...> wrote:

but is that the reason I'm not able to use property keys created in a new transaction?

Also, can we mix index property keys of cardinality SET?

On Mon, Jan 10, 2022 at 10:56 AM Boxuan Li <liboxuan@...> wrote:

Hi Harshit,

You should use STRING mapping if you want exact string match. TEXT mapping is for full text search. See 

https://docs.janusgraph.org/index-backend/text-search/#full-text-and-string-search

Best,

Boxuan

On Mon, Jan 10, 2022 at 1:15 PM Harshit Sharma <harshit.sharma1080@...> wrote:

Hi Marc,

1. Creation of property keys and indices in the schema + commit

      graph = (Janusgraph) GraphDBConfigurator.getGraph()

      mgmt = graph.openManagement()

      

     For Vertex ->

           propertyKey = mgmt.makePropertyKey(propertyKeyName).dataType (String.class).cardinality(Cardinality.SET).make()

          if(existingIndex) 

                janusgraphIndex = mgmt.getGraphIndex(VERTEX_INDEX_NAME)

                mgmt.addIndex(janusgraphIndex, propertyKey)

          else

               indexBuilder = mgmt.buildIndex(VERTEX_INDEX_NAME,  Vertex.class)

               if(cardinality is SET)

                        indexBuilder.addKey(propertyKey, String.class.equals(propertyKey.dataType) ? Mapping.TEXT.asParameter() : Mapping.DEFAULT.asParameter())

               else

                        indexBuilder.addKey(propertyKey, String.class.equals(propertyKey.dataType) ? Mapping.TEXTSTRING.asParameter() : Mapping.DEFAULT.asParameter())

  

    For Edge ->

            propertyKey = mgmt.makePropertyKey(propertyKeyName).dataType (String.class).cardinality(Cardinality.SINGLE).make()

             propertyKey = mgmt.makePropertyKey(propertyKeyName).dataType (String.class).cardinality(Cardinality.SINGLE).make()

          if(existingIndex) 

                janusgraphIndex = mgmt.getGraphIndex(EDGE_INDEX_NAME)

                mgmt.addIndex(janusgraphIndex, propertyKey)

          else

               indexBuilder = mgmt.buildIndex(EDGE_INDEX_NAME,  Vertex.class)

               if(cardinality is SET)

                        indexBuilder.addKey(propertyKey, String.class.equals(propertyKey.dataType) ? Mapping.TEXT.asParameter() : Mapping.DEFAULT.asParameter())

               else

                        indexBuilder.addKey(propertyKey, String.class.equals(propertyKey.dataType) ? Mapping.TEXTSTRING.asParameter() : Mapping.DEFAULT.asParameter())

         

            mgmt.commit();

    

       3.   Query of a vertex based on a property value in another transaction

             Let say i created a property key "domain_vertex_key1_s" for vertex

             Query = g.V().has("domain_vertex_key1_s", "val1);

             

             Let say I created a property key "domain_edge_key1_s" for edge

            

             Query = g.V().has("domain_vertex_key1_s","val2");

          

On Mon, 10 Jan, 2022, 12:46 am , <hadoopmarc@...> wrote:

Hi Harshit,

Can you please describe the steps you have taken in more detail:

• creation of property keys and indices in the schema + commit,
• creation of a vertex with a property in one committed transaction,
• query of a vertex based on a property value in another transaction.

Marc

--

Regards,

Harshit Sharma

+91-9901459920

--

Regards,

Harshit Sharma

+91-9901459920

Sorry I misread your email. I thought you were using TEXT mapping only.

You said you were able to create index, add vertices and edges, and query them utilizing index in the same transaction, but not in a new transaction. Can you give complete code examples that demonstrate how you do so in the same transaction AND in a new transaction, as Marc asked? Specifically, do include how you open and commit/rollback transactions.

In addition, I wonder if this always happens no matter how long you wait, or only happens in a short period of time. Schema propagation takes some time, and you may see a sync delay across JanusGraph instances. If you create the index on one JanusGraph instance, and immediately insert & query data in another JanusGraph instance, it might be the case that schema propagation is not done yet and your other JanusGraph instance is not aware of the recently created index.

Another thing you could check is the Elasticsearch. After creating the index, do you see the index on your ES cluster? After inserting vertices/edges, do you see documents created at ES side?

Best,

Boxuan

On Mon, Jan 10, 2022 at 1:41 PM Harshit Sharma <harshit.sharma1080@...> wrote:

Only For SET cardinality, I'm using TEXT mapping because looks like SET does not work with TEXTSTRING.

On Mon, Jan 10, 2022 at 11:07 AM Harshit Sharma via lists.lfaidata.foundation <harshit.sharma1080=gmail.com@...> wrote:

but is that the reason I'm not able to use property keys created in a new transaction?

Also, can we mix index property keys of cardinality SET?

On Mon, Jan 10, 2022 at 10:56 AM Boxuan Li <liboxuan@...> wrote:

Hi Harshit,

You should use STRING mapping if you want exact string match. TEXT mapping is for full text search. See 

https://docs.janusgraph.org/index-backend/text-search/#full-text-and-string-search

Best,

Boxuan

On Mon, Jan 10, 2022 at 1:15 PM Harshit Sharma <harshit.sharma1080@...> wrote:

Hi Marc,

1. Creation of property keys and indices in the schema + commit

      graph = (Janusgraph) GraphDBConfigurator.getGraph()

      mgmt = graph.openManagement()

      

     For Vertex ->

           propertyKey = mgmt.makePropertyKey(propertyKeyName).dataType (String.class).cardinality(Cardinality.SET).make()

          if(existingIndex) 

                janusgraphIndex = mgmt.getGraphIndex(VERTEX_INDEX_NAME)

                mgmt.addIndex(janusgraphIndex, propertyKey)

          else

               indexBuilder = mgmt.buildIndex(VERTEX_INDEX_NAME,  Vertex.class)

               if(cardinality is SET)

                        indexBuilder.addKey(propertyKey, String.class.equals(propertyKey.dataType) ? Mapping.TEXT.asParameter() : Mapping.DEFAULT.asParameter())

               else

                        indexBuilder.addKey(propertyKey, String.class.equals(propertyKey.dataType) ? Mapping.TEXTSTRING.asParameter() : Mapping.DEFAULT.asParameter())

  

    For Edge ->

            propertyKey = mgmt.makePropertyKey(propertyKeyName).dataType (String.class).cardinality(Cardinality.SINGLE).make()

             propertyKey = mgmt.makePropertyKey(propertyKeyName).dataType (String.class).cardinality(Cardinality.SINGLE).make()

          if(existingIndex) 

                janusgraphIndex = mgmt.getGraphIndex(EDGE_INDEX_NAME)

                mgmt.addIndex(janusgraphIndex, propertyKey)

          else

               indexBuilder = mgmt.buildIndex(EDGE_INDEX_NAME,  Vertex.class)

               if(cardinality is SET)

                        indexBuilder.addKey(propertyKey, String.class.equals(propertyKey.dataType) ? Mapping.TEXT.asParameter() : Mapping.DEFAULT.asParameter())

               else

                        indexBuilder.addKey(propertyKey, String.class.equals(propertyKey.dataType) ? Mapping.TEXTSTRING.asParameter() : Mapping.DEFAULT.asParameter())

         

            mgmt.commit();

    

       3.   Query of a vertex based on a property value in another transaction

             Let say i created a property key "domain_vertex_key1_s" for vertex

             Query = g.V().has("domain_vertex_key1_s", "val1);

             

             Let say I created a property key "domain_edge_key1_s" for edge

            

             Query = g.V().has("domain_vertex_key1_s","val2");

          

On Mon, 10 Jan, 2022, 12:46 am , <hadoopmarc@...> wrote:

Hi Harshit,

Can you please describe the steps you have taken in more detail:

• creation of property keys and indices in the schema + commit,
• creation of a vertex with a property in one committed transaction,
• query of a vertex based on a property value in another transaction.

Marc

--

Regards,

Harshit Sharma

+91-9901459920

--

Regards,

Harshit Sharma

+91-9901459920

Hi Boxuan,

Thanks for the quick response. Actually, I'm following a strict schema in my service. 

i.e we first register a schema and then use it for vertex/edge creation.

So I'm not creating schema and vertex/edge in the same transaction. Only property key and index creation are happening in one transaction that is also at starting of the service only. From next time is index already exists then my service adds new properties to it.

These new properties are created after the creation of the index in a separate transaction are visible in management API printIndexes and printSchema. Also, they are visible in elastic mapping.

On Mon, Jan 10, 2022 at 12:26 PM Boxuan Li <liboxuan@...> wrote:

Sorry I misread your email. I thought you were using TEXT mapping only.

You said you were able to create index, add vertices and edges, and query them utilizing index in the same transaction, but not in a new transaction. Can you give complete code examples that demonstrate how you do so in the same transaction AND in a new transaction, as Marc asked? Specifically, do include how you open and commit/rollback transactions.

In addition, I wonder if this always happens no matter how long you wait, or only happens in a short period of time. Schema propagation takes some time, and you may see a sync delay across JanusGraph instances. If you create the index on one JanusGraph instance, and immediately insert & query data in another JanusGraph instance, it might be the case that schema propagation is not done yet and your other JanusGraph instance is not aware of the recently created index.

Another thing you could check is the Elasticsearch. After creating the index, do you see the index on your ES cluster? After inserting vertices/edges, do you see documents created at ES side?

Best,

Boxuan

On Mon, Jan 10, 2022 at 1:41 PM Harshit Sharma <harshit.sharma1080@...> wrote:

Only For SET cardinality, I'm using TEXT mapping because looks like SET does not work with TEXTSTRING.

On Mon, Jan 10, 2022 at 11:07 AM Harshit Sharma via lists.lfaidata.foundation <harshit.sharma1080=gmail.com@...> wrote:

but is that the reason I'm not able to use property keys created in a new transaction?

Also, can we mix index property keys of cardinality SET?

On Mon, Jan 10, 2022 at 10:56 AM Boxuan Li <liboxuan@...> wrote:

Hi Harshit,

You should use STRING mapping if you want exact string match. TEXT mapping is for full text search. See 

https://docs.janusgraph.org/index-backend/text-search/#full-text-and-string-search

Best,

Boxuan

On Mon, Jan 10, 2022 at 1:15 PM Harshit Sharma <harshit.sharma1080@...> wrote:

Hi Marc,

1. Creation of property keys and indices in the schema + commit

      graph = (Janusgraph) GraphDBConfigurator.getGraph()

      mgmt = graph.openManagement()

      

     For Vertex ->

           propertyKey = mgmt.makePropertyKey(propertyKeyName).dataType (String.class).cardinality(Cardinality.SET).make()

          if(existingIndex) 

                janusgraphIndex = mgmt.getGraphIndex(VERTEX_INDEX_NAME)

                mgmt.addIndex(janusgraphIndex, propertyKey)

          else

               indexBuilder = mgmt.buildIndex(VERTEX_INDEX_NAME,  Vertex.class)

               if(cardinality is SET)

                        indexBuilder.addKey(propertyKey, String.class.equals(propertyKey.dataType) ? Mapping.TEXT.asParameter() : Mapping.DEFAULT.asParameter())

               else

                        indexBuilder.addKey(propertyKey, String.class.equals(propertyKey.dataType) ? Mapping.TEXTSTRING.asParameter() : Mapping.DEFAULT.asParameter())

  

    For Edge ->

            propertyKey = mgmt.makePropertyKey(propertyKeyName).dataType (String.class).cardinality(Cardinality.SINGLE).make()

             propertyKey = mgmt.makePropertyKey(propertyKeyName).dataType (String.class).cardinality(Cardinality.SINGLE).make()

          if(existingIndex) 

                janusgraphIndex = mgmt.getGraphIndex(EDGE_INDEX_NAME)

                mgmt.addIndex(janusgraphIndex, propertyKey)

          else

               indexBuilder = mgmt.buildIndex(EDGE_INDEX_NAME,  Vertex.class)

               if(cardinality is SET)

                        indexBuilder.addKey(propertyKey, String.class.equals(propertyKey.dataType) ? Mapping.TEXT.asParameter() : Mapping.DEFAULT.asParameter())

               else

                        indexBuilder.addKey(propertyKey, String.class.equals(propertyKey.dataType) ? Mapping.TEXTSTRING.asParameter() : Mapping.DEFAULT.asParameter())

         

            mgmt.commit();

    

       3.   Query of a vertex based on a property value in another transaction

             Let say i created a property key "domain_vertex_key1_s" for vertex

             Query = g.V().has("domain_vertex_key1_s", "val1);

             

             Let say I created a property key "domain_edge_key1_s" for edge

            

             Query = g.V().has("domain_vertex_key1_s","val2");

          

On Mon, 10 Jan, 2022, 12:46 am , <hadoopmarc@...> wrote:

Hi Harshit,

Can you please describe the steps you have taken in more detail:

• creation of property keys and indices in the schema + commit,
• creation of a vertex with a property in one committed transaction,
• query of a vertex based on a property value in another transaction.

Marc

--

Regards,

Harshit Sharma

+91-9901459920

--

Regards,

Harshit Sharma

+91-9901459920

--

Regards,

Harshit Sharma

+91-9901459920

Hi Harshit,

Thanks for the explanation. I noticed your example code contains the following snippet:

if(existingIndex) 

                janusgraphIndex = mgmt.getGraphIndex(VERTEX_INDEX_NAME)

                mgmt.addIndex(janusgraphIndex, propertyKey)

I don’t have a computer with me right now, but IIRC there is no “addIndex” API. Do you mean by “addIndexKey”?

Another question: let’s say you create a new mixed index with property A. Later, you add a new property B to it using “mgmt.addIndexKey” API. Then you create a vertex with both property A and property B. Now, are you only able to leverage the mixed index to query property A only, but not property B? Also, what new document(s) do you observe on ES? A concrete example would be greatly appreciated if possible.

Best,

Boxuan

On Mon, Jan 10, 2022 at 3:04 PM Harshit Sharma <harshit.sharma1080@...> wrote:

Hi Boxuan,

Thanks for the quick response. Actually, I'm following a strict schema in my service. 

i.e we first register a schema and then use it for vertex/edge creation.

So I'm not creating schema and vertex/edge in the same transaction. Only property key and index creation are happening in one transaction that is also at starting of the service only. From next time is index already exists then my service adds new properties to it.

These new properties are created after the creation of the index in a separate transaction are visible in management API printIndexes and printSchema. Also, they are visible in elastic mapping.

On Mon, Jan 10, 2022 at 12:26 PM Boxuan Li <liboxuan@...> wrote:

Sorry I misread your email. I thought you were using TEXT mapping only.

You said you were able to create index, add vertices and edges, and query them utilizing index in the same transaction, but not in a new transaction. Can you give complete code examples that demonstrate how you do so in the same transaction AND in a new transaction, as Marc asked? Specifically, do include how you open and commit/rollback transactions.

In addition, I wonder if this always happens no matter how long you wait, or only happens in a short period of time. Schema propagation takes some time, and you may see a sync delay across JanusGraph instances. If you create the index on one JanusGraph instance, and immediately insert & query data in another JanusGraph instance, it might be the case that schema propagation is not done yet and your other JanusGraph instance is not aware of the recently created index.

Another thing you could check is the Elasticsearch. After creating the index, do you see the index on your ES cluster? After inserting vertices/edges, do you see documents created at ES side?

Best,

Boxuan

On Mon, Jan 10, 2022 at 1:41 PM Harshit Sharma <harshit.sharma1080@...> wrote:

Only For SET cardinality, I'm using TEXT mapping because looks like SET does not work with TEXTSTRING.

On Mon, Jan 10, 2022 at 11:07 AM Harshit Sharma via lists.lfaidata.foundation <harshit.sharma1080=gmail.com@...> wrote:

but is that the reason I'm not able to use property keys created in a new transaction?

Also, can we mix index property keys of cardinality SET?

On Mon, Jan 10, 2022 at 10:56 AM Boxuan Li <liboxuan@...> wrote:

Hi Harshit,

You should use STRING mapping if you want exact string match. TEXT mapping is for full text search. See 

https://docs.janusgraph.org/index-backend/text-search/#full-text-and-string-search

Best,

Boxuan

On Mon, Jan 10, 2022 at 1:15 PM Harshit Sharma <harshit.sharma1080@...> wrote:

Hi Marc,

1. Creation of property keys and indices in the schema + commit

      graph = (Janusgraph) GraphDBConfigurator.getGraph()

      mgmt = graph.openManagement()

      

     For Vertex ->

           propertyKey = mgmt.makePropertyKey(propertyKeyName).dataType (String.class).cardinality(Cardinality.SET).make()

          if(existingIndex) 

                janusgraphIndex = mgmt.getGraphIndex(VERTEX_INDEX_NAME)

                mgmt.addIndex(janusgraphIndex, propertyKey)

          else

               indexBuilder = mgmt.buildIndex(VERTEX_INDEX_NAME,  Vertex.class)

               if(cardinality is SET)

                        indexBuilder.addKey(propertyKey, String.class.equals(propertyKey.dataType) ? Mapping.TEXT.asParameter() : Mapping.DEFAULT.asParameter())

               else

                        indexBuilder.addKey(propertyKey, String.class.equals(propertyKey.dataType) ? Mapping.TEXTSTRING.asParameter() : Mapping.DEFAULT.asParameter())

  

    For Edge ->

            propertyKey = mgmt.makePropertyKey(propertyKeyName).dataType (String.class).cardinality(Cardinality.SINGLE).make()

             propertyKey = mgmt.makePropertyKey(propertyKeyName).dataType (String.class).cardinality(Cardinality.SINGLE).make()

          if(existingIndex) 

                janusgraphIndex = mgmt.getGraphIndex(EDGE_INDEX_NAME)

                mgmt.addIndex(janusgraphIndex, propertyKey)

          else

               indexBuilder = mgmt.buildIndex(EDGE_INDEX_NAME,  Vertex.class)

               if(cardinality is SET)

                        indexBuilder.addKey(propertyKey, String.class.equals(propertyKey.dataType) ? Mapping.TEXT.asParameter() : Mapping.DEFAULT.asParameter())

               else

                        indexBuilder.addKey(propertyKey, String.class.equals(propertyKey.dataType) ? Mapping.TEXTSTRING.asParameter() : Mapping.DEFAULT.asParameter())

         

            mgmt.commit();

    

       3.   Query of a vertex based on a property value in another transaction

             Let say i created a property key "domain_vertex_key1_s" for vertex

             Query = g.V().has("domain_vertex_key1_s", "val1);

             

             Let say I created a property key "domain_edge_key1_s" for edge

            

             Query = g.V().has("domain_vertex_key1_s","val2");

          

On Mon, 10 Jan, 2022, 12:46 am , <hadoopmarc@...> wrote:

Hi Harshit,

Can you please describe the steps you have taken in more detail:

• creation of property keys and indices in the schema + commit,
• creation of a vertex with a property in one committed transaction,
• query of a vertex based on a property value in another transaction.

Marc

--

Regards,

Harshit Sharma

+91-9901459920

--

Regards,

Harshit Sharma

+91-9901459920

--

Regards,

Harshit Sharma

+91-9901459920

sorry for the typo,  it is addIndexKey() only what I'm using. 

Example-

Created a vertex label -> vertexA

created a property key -> vertexA_key_s

create mixedIndex if not exist -> vertex_index 

add property key "vertexA_key_s" in vertex_index

Created a vertex label -> vertexB

created a proproperty key -> vertexB_key_s

create mixedIndex if not exist -> vertex_index 

add property key "vertexB_key_s" in vertex_index

Created a edge label -> edge1

created a proproperty key -> edge1_key_s

create mixedIndex if not exist -> edge_index 

*Note -> We are crating index (vertex_index, edge_index) on the start of the service with some system properties like graphID, name, etc)

PrintIndexes in janusgraph -

Vertex Index Name. |    Type.  | Unique.  | Backing.   | key:   Status

--------------------------------------------------------------------------------------------

vertex_index. |    Mixed | false.  | search.   | vertexA_key_s: ENABLED

| vertexB_key_s: ENABLED

Edge Index (VCI) Name. |    Type.  | Unique.  | Backing.   | key  | Status

--------------------------------------------------------------------------------------------

edge_index. |    Mixed | false.  | search.   | vedge1_key_s: ENABLED

In elastic i'm able to search on that property using endpoint /_search?q=oltp_vertexA_key_s

On Mon, Jan 10, 2022 at 12:51 PM Boxuan Li <liboxuan@...> wrote:

Hi Harshit,

Thanks for the explanation. I noticed your example code contains the following snippet:

if(existingIndex) 

                janusgraphIndex = mgmt.getGraphIndex(VERTEX_INDEX_NAME)

                mgmt.addIndex(janusgraphIndex, propertyKey)

I don’t have a computer with me right now, but IIRC there is no “addIndex” API. Do you mean by “addIndexKey”?

Another question: let’s say you create a new mixed index with property A. Later, you add a new property B to it using “mgmt.addIndexKey” API. Then you create a vertex with both property A and property B. Now, are you only able to leverage the mixed index to query property A only, but not property B? Also, what new document(s) do you observe on ES? A concrete example would be greatly appreciated if possible.

Best,

Boxuan

On Mon, Jan 10, 2022 at 3:04 PM Harshit Sharma <harshit.sharma1080@...> wrote:

Hi Boxuan,

Thanks for the quick response. Actually, I'm following a strict schema in my service. 

i.e we first register a schema and then use it for vertex/edge creation.

So I'm not creating schema and vertex/edge in the same transaction. Only property key and index creation are happening in one transaction that is also at starting of the service only. From next time is index already exists then my service adds new properties to it.

These new properties are created after the creation of the index in a separate transaction are visible in management API printIndexes and printSchema. Also, they are visible in elastic mapping.

On Mon, Jan 10, 2022 at 12:26 PM Boxuan Li <liboxuan@...> wrote:

Sorry I misread your email. I thought you were using TEXT mapping only.

You said you were able to create index, add vertices and edges, and query them utilizing index in the same transaction, but not in a new transaction. Can you give complete code examples that demonstrate how you do so in the same transaction AND in a new transaction, as Marc asked? Specifically, do include how you open and commit/rollback transactions.

In addition, I wonder if this always happens no matter how long you wait, or only happens in a short period of time. Schema propagation takes some time, and you may see a sync delay across JanusGraph instances. If you create the index on one JanusGraph instance, and immediately insert & query data in another JanusGraph instance, it might be the case that schema propagation is not done yet and your other JanusGraph instance is not aware of the recently created index.

Another thing you could check is the Elasticsearch. After creating the index, do you see the index on your ES cluster? After inserting vertices/edges, do you see documents created at ES side?

Best,

Boxuan

On Mon, Jan 10, 2022 at 1:41 PM Harshit Sharma <harshit.sharma1080@...> wrote:

Only For SET cardinality, I'm using TEXT mapping because looks like SET does not work with TEXTSTRING.

On Mon, Jan 10, 2022 at 11:07 AM Harshit Sharma via lists.lfaidata.foundation <harshit.sharma1080=gmail.com@...> wrote:

but is that the reason I'm not able to use property keys created in a new transaction?

Also, can we mix index property keys of cardinality SET?

On Mon, Jan 10, 2022 at 10:56 AM Boxuan Li <liboxuan@...> wrote:

Hi Harshit,

You should use STRING mapping if you want exact string match. TEXT mapping is for full text search. See 

https://docs.janusgraph.org/index-backend/text-search/#full-text-and-string-search

Best,

Boxuan

On Mon, Jan 10, 2022 at 1:15 PM Harshit Sharma <harshit.sharma1080@...> wrote:

Hi Marc,

1. Creation of property keys and indices in the schema + commit

      graph = (Janusgraph) GraphDBConfigurator.getGraph()

      mgmt = graph.openManagement()

      

     For Vertex ->

           propertyKey = mgmt.makePropertyKey(propertyKeyName).dataType (String.class).cardinality(Cardinality.SET).make()

          if(existingIndex) 

                janusgraphIndex = mgmt.getGraphIndex(VERTEX_INDEX_NAME)

                mgmt.addIndex(janusgraphIndex, propertyKey)

          else

               indexBuilder = mgmt.buildIndex(VERTEX_INDEX_NAME,  Vertex.class)

               if(cardinality is SET)

                        indexBuilder.addKey(propertyKey, String.class.equals(propertyKey.dataType) ? Mapping.TEXT.asParameter() : Mapping.DEFAULT.asParameter())

               else

                        indexBuilder.addKey(propertyKey, String.class.equals(propertyKey.dataType) ? Mapping.TEXTSTRING.asParameter() : Mapping.DEFAULT.asParameter())

  

    For Edge ->

            propertyKey = mgmt.makePropertyKey(propertyKeyName).dataType (String.class).cardinality(Cardinality.SINGLE).make()

             propertyKey = mgmt.makePropertyKey(propertyKeyName).dataType (String.class).cardinality(Cardinality.SINGLE).make()

          if(existingIndex) 

                janusgraphIndex = mgmt.getGraphIndex(EDGE_INDEX_NAME)

                mgmt.addIndex(janusgraphIndex, propertyKey)

          else

               indexBuilder = mgmt.buildIndex(EDGE_INDEX_NAME,  Vertex.class)

               if(cardinality is SET)

                        indexBuilder.addKey(propertyKey, String.class.equals(propertyKey.dataType) ? Mapping.TEXT.asParameter() : Mapping.DEFAULT.asParameter())

               else

                        indexBuilder.addKey(propertyKey, String.class.equals(propertyKey.dataType) ? Mapping.TEXTSTRING.asParameter() : Mapping.DEFAULT.asParameter())

         

            mgmt.commit();

    

       3.   Query of a vertex based on a property value in another transaction

             Let say i created a property key "domain_vertex_key1_s" for vertex

             Query = g.V().has("domain_vertex_key1_s", "val1);

             

             Let say I created a property key "domain_edge_key1_s" for edge

            

             Query = g.V().has("domain_vertex_key1_s","val2");

          

On Mon, 10 Jan, 2022, 12:46 am , <hadoopmarc@...> wrote:

Hi Harshit,

Can you please describe the steps you have taken in more detail:

• creation of property keys and indices in the schema + commit,
• creation of a vertex with a property in one committed transaction,
• query of a vertex based on a property value in another transaction.

Marc

--

Regards,

Harshit Sharma

+91-9901459920

--

Regards,

Harshit Sharma

+91-9901459920

--

Regards,

Harshit Sharma

+91-9901459920

--

Regards,

Harshit Sharma

+91-9901459920

Do we need reindexing even for properties created after index?

On Mon, Jan 10, 2022 at 1:39 PM Harshit Sharma via lists.lfaidata.foundation <harshit.sharma1080=gmail.com@...> wrote:

sorry for the typo,  it is addIndexKey() only what I'm using. 

Example-

Created a vertex label -> vertexA

created a property key -> vertexA_key_s

create mixedIndex if not exist -> vertex_index 

add property key "vertexA_key_s" in vertex_index

Created a vertex label -> vertexB

created a proproperty key -> vertexB_key_s

create mixedIndex if not exist -> vertex_index 

add property key "vertexB_key_s" in vertex_index

Created a edge label -> edge1

created a proproperty key -> edge1_key_s

create mixedIndex if not exist -> edge_index 

*Note -> We are crating index (vertex_index, edge_index) on the start of the service with some system properties like graphID, name, etc)

PrintIndexes in janusgraph -

Vertex Index Name. |    Type.  | Unique.  | Backing.   | key:   Status

--------------------------------------------------------------------------------------------

vertex_index. |    Mixed | false.  | search.   | vertexA_key_s: ENABLED

| vertexB_key_s: ENABLED

Edge Index (VCI) Name. |    Type.  | Unique.  | Backing.   | key  | Status

--------------------------------------------------------------------------------------------

edge_index. |    Mixed | false.  | search.   | vedge1_key_s: ENABLED

In elastic i'm able to search on that property using endpoint /_search?q=oltp_vertexA_key_s

On Mon, Jan 10, 2022 at 12:51 PM Boxuan Li <liboxuan@...> wrote:

Hi Harshit,

Thanks for the explanation. I noticed your example code contains the following snippet:

if(existingIndex) 

                janusgraphIndex = mgmt.getGraphIndex(VERTEX_INDEX_NAME)

                mgmt.addIndex(janusgraphIndex, propertyKey)

I don’t have a computer with me right now, but IIRC there is no “addIndex” API. Do you mean by “addIndexKey”?

Another question: let’s say you create a new mixed index with property A. Later, you add a new property B to it using “mgmt.addIndexKey” API. Then you create a vertex with both property A and property B. Now, are you only able to leverage the mixed index to query property A only, but not property B? Also, what new document(s) do you observe on ES? A concrete example would be greatly appreciated if possible.

Best,

Boxuan

On Mon, Jan 10, 2022 at 3:04 PM Harshit Sharma <harshit.sharma1080@...> wrote:

Hi Boxuan,

Thanks for the quick response. Actually, I'm following a strict schema in my service. 

i.e we first register a schema and then use it for vertex/edge creation.

So I'm not creating schema and vertex/edge in the same transaction. Only property key and index creation are happening in one transaction that is also at starting of the service only. From next time is index already exists then my service adds new properties to it.

These new properties are created after the creation of the index in a separate transaction are visible in management API printIndexes and printSchema. Also, they are visible in elastic mapping.

On Mon, Jan 10, 2022 at 12:26 PM Boxuan Li <liboxuan@...> wrote:

Sorry I misread your email. I thought you were using TEXT mapping only.

You said you were able to create index, add vertices and edges, and query them utilizing index in the same transaction, but not in a new transaction. Can you give complete code examples that demonstrate how you do so in the same transaction AND in a new transaction, as Marc asked? Specifically, do include how you open and commit/rollback transactions.

In addition, I wonder if this always happens no matter how long you wait, or only happens in a short period of time. Schema propagation takes some time, and you may see a sync delay across JanusGraph instances. If you create the index on one JanusGraph instance, and immediately insert & query data in another JanusGraph instance, it might be the case that schema propagation is not done yet and your other JanusGraph instance is not aware of the recently created index.

Another thing you could check is the Elasticsearch. After creating the index, do you see the index on your ES cluster? After inserting vertices/edges, do you see documents created at ES side?

Best,

Boxuan

On Mon, Jan 10, 2022 at 1:41 PM Harshit Sharma <harshit.sharma1080@...> wrote:

Only For SET cardinality, I'm using TEXT mapping because looks like SET does not work with TEXTSTRING.

On Mon, Jan 10, 2022 at 11:07 AM Harshit Sharma via lists.lfaidata.foundation <harshit.sharma1080=gmail.com@...> wrote:

but is that the reason I'm not able to use property keys created in a new transaction?

Also, can we mix index property keys of cardinality SET?

On Mon, Jan 10, 2022 at 10:56 AM Boxuan Li <liboxuan@...> wrote:

Hi Harshit,

You should use STRING mapping if you want exact string match. TEXT mapping is for full text search. See 

https://docs.janusgraph.org/index-backend/text-search/#full-text-and-string-search

Best,

Boxuan

On Mon, Jan 10, 2022 at 1:15 PM Harshit Sharma <harshit.sharma1080@...> wrote:

Hi Marc,

1. Creation of property keys and indices in the schema + commit

      graph = (Janusgraph) GraphDBConfigurator.getGraph()

      mgmt = graph.openManagement()

      

     For Vertex ->

           propertyKey = mgmt.makePropertyKey(propertyKeyName).dataType (String.class).cardinality(Cardinality.SET).make()

          if(existingIndex) 

                janusgraphIndex = mgmt.getGraphIndex(VERTEX_INDEX_NAME)

                mgmt.addIndex(janusgraphIndex, propertyKey)

          else

               indexBuilder = mgmt.buildIndex(VERTEX_INDEX_NAME,  Vertex.class)

               if(cardinality is SET)

                        indexBuilder.addKey(propertyKey, String.class.equals(propertyKey.dataType) ? Mapping.TEXT.asParameter() : Mapping.DEFAULT.asParameter())

               else

                        indexBuilder.addKey(propertyKey, String.class.equals(propertyKey.dataType) ? Mapping.TEXTSTRING.asParameter() : Mapping.DEFAULT.asParameter())

  

    For Edge ->

            propertyKey = mgmt.makePropertyKey(propertyKeyName).dataType (String.class).cardinality(Cardinality.SINGLE).make()

             propertyKey = mgmt.makePropertyKey(propertyKeyName).dataType (String.class).cardinality(Cardinality.SINGLE).make()

          if(existingIndex) 

                janusgraphIndex = mgmt.getGraphIndex(EDGE_INDEX_NAME)

                mgmt.addIndex(janusgraphIndex, propertyKey)

          else

               indexBuilder = mgmt.buildIndex(EDGE_INDEX_NAME,  Vertex.class)

               if(cardinality is SET)

                        indexBuilder.addKey(propertyKey, String.class.equals(propertyKey.dataType) ? Mapping.TEXT.asParameter() : Mapping.DEFAULT.asParameter())

               else

                        indexBuilder.addKey(propertyKey, String.class.equals(propertyKey.dataType) ? Mapping.TEXTSTRING.asParameter() : Mapping.DEFAULT.asParameter())

         

            mgmt.commit();

    

       3.   Query of a vertex based on a property value in another transaction

             Let say i created a property key "domain_vertex_key1_s" for vertex

             Query = g.V().has("domain_vertex_key1_s", "val1);

             

             Let say I created a property key "domain_edge_key1_s" for edge

            

             Query = g.V().has("domain_vertex_key1_s","val2");

          

On Mon, 10 Jan, 2022, 12:46 am , <hadoopmarc@...> wrote:

Hi Harshit,

Can you please describe the steps you have taken in more detail:

• creation of property keys and indices in the schema + commit,
• creation of a vertex with a property in one committed transaction,
• query of a vertex based on a property value in another transaction.

Marc

--

Regards,

Harshit Sharma

+91-9901459920

--

Regards,

Harshit Sharma

+91-9901459920

--

Regards,

Harshit Sharma

+91-9901459920

--

Regards,

Harshit Sharma

+91-9901459920

--

Regards,

Harshit Sharma

+91-9901459920

For these kind of queries you need TEXTSTRING mapping. As an alternative you can create composite index to "equals" queries

You mentioned that 

 

g.V().has("key1", textContains("val") ) is working while 

g.V().has("key1","val13") is not working

 

Are you sure "key1" has TEXTSTRING mapping, rather than STRING mapping?

I am unable to reproduce it on 0.6.0. Can you try the 0.6.0 version?

Yes, key1 is of TEXT String mapping because I'm creating only TEXT or TEXTSTRING type of mapping.

I can try to update but looks like an upgrade from 0.5.2 to 0.6.0 involve some major changes and things are breaking in my local

On Mon, Jan 10, 2022 at 5:23 PM Boxuan Li <liboxuan@...> wrote:

You mentioned that 

 

g.V().has("key1", textContains("val") ) is working while 

g.V().has("key1","val13") is not working

 

Are you sure "key1" has TEXTSTRING mapping, rather than STRING mapping?

I am unable to reproduce it on 0.6.0. Can you try the 0.6.0 version?

--

Regards,

Harshit Sharma

+91-9901459920

Sorry, I meant to ask

Are you sure "key1" has TEXTSTRING mapping, rather than TEXT mapping?

On Mon, Jan 10, 2022 at 8:01 PM Harshit Sharma <harshit.sharma1080@...> wrote:

Yes, key1 is of TEXT String mapping because I'm creating only TEXT or TEXTSTRING type of mapping.

I can try to update but looks like an upgrade from 0.5.2 to 0.6.0 involve some major changes and things are breaking in my local

On Mon, Jan 10, 2022 at 5:23 PM Boxuan Li <liboxuan@...> wrote:

You mentioned that 

 

g.V().has("key1", textContains("val") ) is working while 

g.V().has("key1","val13") is not working

 

Are you sure "key1" has TEXTSTRING mapping, rather than STRING mapping?

I am unable to reproduce it on 0.6.0. Can you try the 0.6.0 version?

--

Regards,

Harshit Sharma

+91-9901459920

Could you please provide a ES mapping that was generated for this property?