Date   

Janusgraph connect with MySQL Storage Backend

liu molong <1854...@...>
 

Hello,

I request your help regardding the janusgraph connect with Mysql or other  Relational DBMS. Because I'm trying to code a module which can connects to Mysql.
And current this module can connect with Mysql Server and do some simple operations like add/delete/query and so on.
But There‘’s some different when do query, I see that sliceStart/sliceEnd when do getSlice。

question 1:
Is it necessary to use sliceStart/sliceEnd when query?Beacuse I just try to  replace the Storage with MySQL,I am not sure whether the slice is necessary for JanusGraph.

question 2:
why don't you plan to support Mysql? Is it for some special reason? 

I will be very appreciate that if you can give some suggestion for the Mysql plan.

Thanks


Re: Aggregating edges based on the source & target vertex attributes

HadoopMarc <bi...@...>
 

Hi Vishnu,

The processing time does not really surprise me, JanusGraph has to do everything in java. For the typical JanusGraph use case, the storage backend is the limiting factor and the java processing does not really matter. If you want to do this query fast in memory with multiple cores, you are better off with python dask or the like (and do the aggregation on a single dataframe with the edge id, inV label and outV label). I would not be surprised if pandas, using a single core, already does this within a second.

For the queries given above I believe only a single core is used when run as OLTP query. Because this N x N query is not easy to parallelize for TinkerPop, you have to take care how to run it as OLAP query. I would guess that with(SparkGraphComputer) with a single spark executor with 8 cores will work best because then the spark cores share the memory. This is automatically true for spark.master=local[*] .

Best wishes,    Marc

PS Thanks for introducing me into the Indian numbering system. Happily, you do not have 1.5 crore vertices!


Op maandag 21 december 2020 om 09:16:08 UTC+1 schreef vishnu gajendran:


Thank you Kevin and Marc for quick response. I tried both the queries and they are working as expected. My use case demands to run such query for a bigger dataset. I ran the query for 1 lakh vertices and 5 million edges in my desktop using the in-memory backend (assuming that in-memory will be faster compared to other external data stores) and it took roughly 2 minutes to execute. My desktop contains 8 logical cores and 64 GB RAM. Few questions regarding the same:

1. Is this the expected performance for such aggregation queries in JanusGraph?
2. Will increasing the number of cores (i.e. processing power) improve the performance of the query?

The dataset I am dealing with can be as big as 1.5 lakh vertices and 20 million edges and I would like to support the above aggregation query in real time (i.e. in few seconds and not in minutes). Can we achieve the same using JanusGraph?
On Thursday, December 17, 2020 at 7:51:56 PM UTC+5:30 kt...@... wrote:
Thanks for improving it!  Always good to learn more.

On Thu, Dec 17, 2020 at 6:11 AM HadoopMarc <b...@...> wrote:
And here a small variation without the keys and with some code formatting:

g.V().as('a').outE().as('e').inV().as('b').
    group().by(
        union(select('a').values('organization'), select('b').values('organization')).fold()
    ).by(
        select('e').by('collaborationHours').sum()
    ).unfold()
==>[marketing, engineering]=2
==>[sales, marketing]=2
==>[engineering, sales]=3
==>[engineering, marketing]=2
Marc



Op donderdag 17 december 2020 om 14:50:11 UTC+1 schreef kt...@...:
Vishu,

This may not be optimal, but seems to work:

g.E().hasLabel('collaboration').as('e').outV().values('organization').as('1').select('e').inV().values('organization').as('2').select('e').group().by(select('1', '2')).by(values('collaborationHours').sum()).unfold();

==>{1=engineering, 2=marketing}=2
==>{1=marketing, 2=engineering}=2
==>{1=engineering, 2=sales}=3
==>{1=sales, 2=marketing}=2

Note, you have some leading spaces in your Gremlin on 'collaborationHours' I had to remove, and with the data you provided the engineering/sales total is 3, not 4.

Kevin 

On Wed, Dec 16, 2020 at 11:57 PM vishnu gajendran <gg...@...> wrote:
Hello,

I request your help regarding the janus graph query which I am trying to construct. Let's consider the following graph where each vertex denotes a person and the edge between any two vertex denotes collaboration between them.

Vertices:
p1 = graph.addVertex('person')
p1.property('personId', 1)
p1.property('organization', "engineering")

p2 = graph.addVertex('person')
p2.property('personId', 2)
p2.property('organization', "sales")

p3 = graph.addVertex('person')
p3.property('personId', 3)
p3.property('organization', "marketing")

p4 = graph.addVertex('person')
p4.property('personId', 4)
p4.property('organization', "engineering")

Edges:
p1.addEdge('collaboration', p2, 'collaborationHours', 1)
p1.addEdge('collaboration', p3, 'collaborationHours', 2)

p2.addEdge('collaboration', p3, 'collaborationHours', 2)

p3.addEdge('collaboration', p4, ' collaborationHours', 2)

p4.addEdge('collaboration', p2, ' collaborationHours', 2)

Expected Result is the following table:

Organization1  Organization2 Total Collaboration Hours
Engineering      Sales                 4
Engineering      Marketing         2
Sales                 Marketing          2
Marketing         Engineering       2

Here, I am trying to aggregate the "person to person" graph into "organization to organization" graph. Does JanusGraph support such aggregation queries? If yes, can you please help me with the query for the same?

Thanks

--
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgr...@....
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/5fb20cf1-0aeb-4128-91da-857ec6295587n%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgr...@....


Re: JanusGraph 0.5.2 and BigTable

BO XUAN LI <libo...@...>
 

> About the locking, what do you consider a JVM instance? An instance of the Gremlin server? JanusGraph iteslf? If I try and use Janus as a cluster (multiple dockers instead of one), will that translate to having more than 1 JVM?

Sorry I wasn’t very clear about this. I meant by the JVM instance where JanusGraph itself runs. To be accurate, you could see local lock contentions when multiple threads, under the same process, contend for the same lock. This is due to JanusGraph’s locking mechanism:

Step 1: Local lock resolution (inter-thread synchronization), utilizing in-memory data structures (concurrent hashmap). If conflict detected, you typically see error message like "Local lock contention”.
Step 2: Inter-process synchronization, utilizing data backend (e.g. HBase). If conflict detected, you typically see other error messages like “Lock write retry count exceeded”.


If you have multiple transactions contending for the same lock, then it’s better to have them running on the same JVM instance because local lock synchronization is faster and can let conflicting transactions fail early.

Glad to hear you don’t have the problem anymore. To be honest I don’t know why switching to composite indexes helps you resolve the locking exception issues.

Cheers,
Boxuan

On Dec 19, 2020, at 3:41 PM, Assaf Schwartz <schw...@...> wrote:

Thanks a lot Boxuan!

For some reason I missed being notified on your response.
The indexes were indeed the issue (as I began to suspect), switching them to being composite indexes (there was no real need for them being mixed) solved the issue :)

About the locking, what do you consider a JVM instance? An instance of the Gremlin server? JanusGraph iteslf? If I try and use Janus as a cluster (multiple dockers instead of one), will that translate to having more than 1 JVM?

Again thanks,
Assaf

On Thursday, December 17, 2020 at 12:39:48 PM UTC+2 libo...@connect.hku.hk wrote:
Hi Assaf,

I am not familiar with GKE but I can try to answer some of your questions:

> how does a traversal behave when looking up based on an index key when the key is not yet indexed

Assuming the index has been enabled. If a particular key is still in the indexing process (e.g. you are in the middle of a committing process) in one thread, then another thread will not be able to find any data because it finds nothing in the index key lookup. Note that when you are using mixed index, the data is written to your primary backend (e.g. hbase) first, and then mixed index backend (e.g. Elasticsearch). If the data has already been written into hbase but not into Elasticsearch yet, the querying thread cannot find any data (if JanusGraph decides your query can be satisfied by a mixed index).

> org.janusgraph.diskstorage.locking.PermanentLockingException: Local lock contention at org.janusgraph.diskstorage.locking.AbstractLocker.writeLock(AbstractLocker.java:327) 

This usually happens when you have multiple local threads (running on the same JVM instance) contending for the same lock. You might want to check your application logic.

Best regards,
Boxuan

On Dec 17, 2020, at 6:16 PM, Assaf Schwartz <sc...@...> wrote:

Could this be related to delays in indexing? I don't know how to figure out of such exists, but assuming this happens - 
how does a traversal behave when looking up based on an index key when the key is not yet indexed?

On Thursday, December 17, 2020 at 10:54:32 AM UTC+2 Assaf Schwartz wrote:
Hi All,

I'm experiencing an issues with running JanusGraph (on top of GKE) against BigTable.
This is the general setup description:
  • We are using a single node BigTable cluster (for development / integration purposes) with the vanilla 0.5.2 docker.
  • Indexing is configured to be done with ES (also running on GKE)
  • JanusGraph is configured through environment variables:
  • Interaction with JanusGraph are done only through a single gRPC server that is running gremlin-python, let's call it DB-SERVER.
  • Last time we've done testing against BT was with version 0.4.1 of JanusGraph, precompiled to support HBase1.
  • All of our components communicate via gRPC.
Description of the problem:
  1. The DB-SERVER creates a Vertex i, generate some XML to represent work to be done, and sends it to another service for processing, let's call in ORCHESTRATOR.
  2. The ORCHESTRATOR generates two properties, w and r (local identifiers) and sends them back to the DB-SERVER, so they will be set as properties on Vertex i. These two properties are also mixed String indexes.
  3. After setting the properties, DB-SERVER will ack ORCHESTRATOR, which will start processing. As part of the processing, ORCHESTRATOR will send updates back to the DB-SERVER using w and r.
  4. On getting these updates DB-SERVER, it will try looking up Vertex i based on w and r, like so:
    g.V().has("r", <some_r>).has("w", <some_w>).next()
  5. At that point, a null / None is returned as the traversal fails to find Vertex i.
  6. Trying the same traversal in a separate console (python and gremlin) does fetch the vertex. Since it's a single instance cluster, I ruled out any eventual consistency issues.
I'm not sure if it's a regression introduced after 0.4.1.
I've also validated that db-caching is turned off.

Help! :)
Many thanks in advance,
Assaf




-- 
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgr...@....
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/d4373c5a-ab97-4aa4-a143-f26c3ce50677n%40googlegroups.com.


--
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgra...@....
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/1afb5dd4-2802-4be5-b62d-c7d13ce9b7d2n%40googlegroups.com.


Re: JanusGraph 0.5.2 and BigTable

Assaf Schwartz <schw...@...>
 

Thanks a lot Boxuan!

For some reason I missed being notified on your response.
The indexes were indeed the issue (as I began to suspect), switching them to being composite indexes (there was no real need for them being mixed) solved the issue :)

About the locking, what do you consider a JVM instance? An instance of the Gremlin server? JanusGraph iteslf? If I try and use Janus as a cluster (multiple dockers instead of one), will that translate to having more than 1 JVM?

Again thanks,
Assaf

On Thursday, December 17, 2020 at 12:39:48 PM UTC+2 li...@... wrote:
Hi Assaf,

I am not familiar with GKE but I can try to answer some of your questions:

> how does a traversal behave when looking up based on an index key when the key is not yet indexed

Assuming the index has been enabled. If a particular key is still in the indexing process (e.g. you are in the middle of a committing process) in one thread, then another thread will not be able to find any data because it finds nothing in the index key lookup. Note that when you are using mixed index, the data is written to your primary backend (e.g. hbase) first, and then mixed index backend (e.g. Elasticsearch). If the data has already been written into hbase but not into Elasticsearch yet, the querying thread cannot find any data (if JanusGraph decides your query can be satisfied by a mixed index).

> org.janusgraph.diskstorage.locking.PermanentLockingException: Local lock contention at org.janusgraph.diskstorage.locking.AbstractLocker.writeLock(AbstractLocker.java:327) 

This usually happens when you have multiple local threads (running on the same JVM instance) contending for the same lock. You might want to check your application logic.

Best regards,
Boxuan

On Dec 17, 2020, at 6:16 PM, Assaf Schwartz <sc...@...> wrote:

Could this be related to delays in indexing? I don't know how to figure out of such exists, but assuming this happens - 
how does a traversal behave when looking up based on an index key when the key is not yet indexed?

On Thursday, December 17, 2020 at 10:54:32 AM UTC+2 Assaf Schwartz wrote:
Hi All,

I'm experiencing an issues with running JanusGraph (on top of GKE) against BigTable.
This is the general setup description:
  • We are using a single node BigTable cluster (for development / integration purposes) with the vanilla 0.5.2 docker.
  • Indexing is configured to be done with ES (also running on GKE)
  • JanusGraph is configured through environment variables:
  • Interaction with JanusGraph are done only through a single gRPC server that is running gremlin-python, let's call it DB-SERVER.
  • Last time we've done testing against BT was with version 0.4.1 of JanusGraph, precompiled to support HBase1.
  • All of our components communicate via gRPC.
Description of the problem:
  1. The DB-SERVER creates a Vertex i, generate some XML to represent work to be done, and sends it to another service for processing, let's call in ORCHESTRATOR.
  2. The ORCHESTRATOR generates two properties, w and r (local identifiers) and sends them back to the DB-SERVER, so they will be set as properties on Vertex i. These two properties are also mixed String indexes.
  3. After setting the properties, DB-SERVER will ack ORCHESTRATOR, which will start processing. As part of the processing, ORCHESTRATOR will send updates back to the DB-SERVER using w and r.
  4. On getting these updates DB-SERVER, it will try looking up Vertex i based on w and r, like so:
    g.V().has("r", <some_r>).has("w", <some_w>).next()
  5. At that point, a null / None is returned as the traversal fails to find Vertex i.
  6. Trying the same traversal in a separate console (python and gremlin) does fetch the vertex. Since it's a single instance cluster, I ruled out any eventual consistency issues.
I'm not sure if it's a regression introduced after 0.4.1.
I've also validated that db-caching is turned off.

Help! :)
Many thanks in advance,
Assaf




-- 
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgr...@....
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/d4373c5a-ab97-4aa4-a143-f26c3ce50677n%40googlegroups.com.


Re: Aggregating edges based on the source & target vertex attributes

vishnu gajendran <ggvis...@...>
 


Thank you Kevin and Marc for quick response. I tried both the queries and they are working as expected. My use case demands to run such query for a bigger dataset. I ran the query for 1 lakh vertices and 5 million edges in my desktop using the in-memory backend (assuming that in-memory will be faster compared to other external data stores) and it took roughly 2 minutes to execute. My desktop contains 8 logical cores and 64 GB RAM. Few questions regarding the same:

1. Is this the expected performance for such aggregation queries in JanusGraph?
2. Will increasing the number of cores (i.e. processing power) improve the performance of the query?

The dataset I am dealing with can be as big as 1.5 lakh vertices and 20 million edges and I would like to support the above aggregation query in real time (i.e. in few seconds and not in minutes). Can we achieve the same using JanusGraph?

On Thursday, December 17, 2020 at 7:51:56 PM UTC+5:30 kt...@... wrote:
Thanks for improving it!  Always good to learn more.

On Thu, Dec 17, 2020 at 6:11 AM HadoopMarc <b...@...> wrote:
And here a small variation without the keys and with some code formatting:

g.V().as('a').outE().as('e').inV().as('b').
    group().by(
        union(select('a').values('organization'), select('b').values('organization')).fold()
    ).by(
        select('e').by('collaborationHours').sum()
    ).unfold()
==>[marketing, engineering]=2
==>[sales, marketing]=2
==>[engineering, sales]=3
==>[engineering, marketing]=2
Marc



Op donderdag 17 december 2020 om 14:50:11 UTC+1 schreef kt...@...:
Vishu,

This may not be optimal, but seems to work:

g.E().hasLabel('collaboration').as('e').outV().values('organization').as('1').select('e').inV().values('organization').as('2').select('e').group().by(select('1', '2')).by(values('collaborationHours').sum()).unfold();

==>{1=engineering, 2=marketing}=2
==>{1=marketing, 2=engineering}=2
==>{1=engineering, 2=sales}=3
==>{1=sales, 2=marketing}=2

Note, you have some leading spaces in your Gremlin on 'collaborationHours' I had to remove, and with the data you provided the engineering/sales total is 3, not 4.

Kevin 

On Wed, Dec 16, 2020 at 11:57 PM vishnu gajendran <gg...@...> wrote:
Hello,

I request your help regarding the janus graph query which I am trying to construct. Let's consider the following graph where each vertex denotes a person and the edge between any two vertex denotes collaboration between them.

Vertices:
p1 = graph.addVertex('person')
p1.property('personId', 1)
p1.property('organization', "engineering")

p2 = graph.addVertex('person')
p2.property('personId', 2)
p2.property('organization', "sales")

p3 = graph.addVertex('person')
p3.property('personId', 3)
p3.property('organization', "marketing")

p4 = graph.addVertex('person')
p4.property('personId', 4)
p4.property('organization', "engineering")

Edges:
p1.addEdge('collaboration', p2, 'collaborationHours', 1)
p1.addEdge('collaboration', p3, 'collaborationHours', 2)

p2.addEdge('collaboration', p3, 'collaborationHours', 2)

p3.addEdge('collaboration', p4, ' collaborationHours', 2)

p4.addEdge('collaboration', p2, ' collaborationHours', 2)

Expected Result is the following table:

Organization1  Organization2 Total Collaboration Hours
Engineering      Sales                 4
Engineering      Marketing         2
Sales                 Marketing          2
Marketing         Engineering       2

Here, I am trying to aggregate the "person to person" graph into "organization to organization" graph. Does JanusGraph support such aggregation queries? If yes, can you please help me with the query for the same?

Thanks

--
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgr...@....
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/5fb20cf1-0aeb-4128-91da-857ec6295587n%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgr...@....


Re: using In-Memory in JanusGraph Verison 0.5.2 Throws Exception

Boxuan Li <libo...@...>
 

Make sure you have included janusgraph-inmemory in your dependency.

Cheers,
Boxuan


On Friday, December 18, 2020 at 4:07:10 PM UTC+8 kri...@... wrote:
Hi folks


i have updated janusgraph version from 0.4.1 to 0.5.2. all my previous unit testcases with inmemory config throws expection below. should i want to add any more particular dependency to my project for running inmemory graph?
can you please help me regarding this issue.

using below config for inmemory janusgraph instance.
graph = JanusGraphFactory.build().set("storage.backend", "inmemory")
.set("graph.timestamps",TimestampProviders.NANO) .open();

java.lang.IllegalArgumentException: Could not find implementation class: org.janusgraph.diskstorage.inmemory.InMemoryStoreManager
at org.janusgraph.util.system.ConfigurationUtil.instantiate(ConfigurationUtil.java:60)
at org.janusgraph.diskstorage.Backend.getImplementationClass(Backend.java:440)
at org.janusgraph.diskstorage.Backend.getStorageManager(Backend.java:411)
at org.janusgraph.graphdb.configuration.builder.GraphDatabaseConfigurationBuilder.build(GraphDatabaseConfigurationBuilder.java:50)
at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:161)
at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:132)
at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:122)
at org.janusgraph.core.JanusGraphFactory$Builder.open(JanusGraphFactory.java:261)


Thanks & Regards
Krishna Sailesh


Re: using In-Memory in JanusGraph Verison 0.5.2 Throws Exception

Nicolas Trangosi <nicolas...@...>
 

Hi,
Have you added the janusgraph-inmemory dependency in your pom.xml file? 

   <dependency>
     <groupId>org.janusgraph</groupId>
     <artifactId>janusgraph-inmemory</artifactId>
     <version>${janusgraph.version}</version>
     <scope>test</scope>
</dependency>


Regards,
Nicolas


Le ven. 18 déc. 2020 à 09:07, kri...@... <krishna...@...> a écrit :
Hi folks


i have updated janusgraph version from 0.4.1 to 0.5.2. all my previous unit testcases with inmemory config throws expection below. should i want to add any more particular dependency to my project for running inmemory graph?
can you please help me regarding this issue.

using below config for inmemory janusgraph instance.
graph = JanusGraphFactory.build().set("storage.backend", "inmemory")
.set("graph.timestamps",TimestampProviders.NANO) .open();

java.lang.IllegalArgumentException: Could not find implementation class: org.janusgraph.diskstorage.inmemory.InMemoryStoreManager
at org.janusgraph.util.system.ConfigurationUtil.instantiate(ConfigurationUtil.java:60)
at org.janusgraph.diskstorage.Backend.getImplementationClass(Backend.java:440)
at org.janusgraph.diskstorage.Backend.getStorageManager(Backend.java:411)
at org.janusgraph.graphdb.configuration.builder.GraphDatabaseConfigurationBuilder.build(GraphDatabaseConfigurationBuilder.java:50)
at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:161)
at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:132)
at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:122)
at org.janusgraph.core.JanusGraphFactory$Builder.open(JanusGraphFactory.java:261)


Thanks & Regards
Krishna Sailesh

--
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgra...@....
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/d0a46a98-59d5-4895-bf22-a25b0250fec1n%40googlegroups.com.


--

  

Nicolas Trangosi

Lead back

+33 (0)6 77 86 66 44      

   



Ce message et ses pièces jointes peuvent contenir des informations confidentielles ou privilégiées et ne doivent donc pas être diffusés, exploités ou copiés sans autorisation. 
Si vous avez reçu ce message par erreur, veuillez le signaler a l'expéditeur et le détruire ainsi que les pièces jointes. 
Les messages électroniques étant susceptibles d'altération, DCbrain décline toute responsabilité si ce message a été altéré, déformé ou falsifié. Merci. 

This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, DCbrain is not liable for messages that have been modified, changed or falsified. Thank you.


using In-Memory in JanusGraph Verison 0.5.2 Throws Exception

"kri...@gmail.com" <krishna...@...>
 

Hi folks


i have updated janusgraph version from 0.4.1 to 0.5.2. all my previous unit testcases with inmemory config throws expection below. should i want to add any more particular dependency to my project for running inmemory graph?
can you please help me regarding this issue.

using below config for inmemory janusgraph instance.
graph = JanusGraphFactory.build().set("storage.backend", "inmemory")
.set("graph.timestamps",TimestampProviders.NANO) .open();

java.lang.IllegalArgumentException: Could not find implementation class: org.janusgraph.diskstorage.inmemory.InMemoryStoreManager
at org.janusgraph.util.system.ConfigurationUtil.instantiate(ConfigurationUtil.java:60)
at org.janusgraph.diskstorage.Backend.getImplementationClass(Backend.java:440)
at org.janusgraph.diskstorage.Backend.getStorageManager(Backend.java:411)
at org.janusgraph.graphdb.configuration.builder.GraphDatabaseConfigurationBuilder.build(GraphDatabaseConfigurationBuilder.java:50)
at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:161)
at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:132)
at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:122)
at org.janusgraph.core.JanusGraphFactory$Builder.open(JanusGraphFactory.java:261)


Thanks & Regards
Krishna Sailesh


Re: Aggregating edges based on the source & target vertex attributes

Kevin Schmidt <ktsc...@...>
 

Thanks for improving it!  Always good to learn more.


On Thu, Dec 17, 2020 at 6:11 AM HadoopMarc <bi...@...> wrote:
And here a small variation without the keys and with some code formatting:

g.V().as('a').outE().as('e').inV().as('b').
    group().by(
        union(select('a').values('organization'), select('b').values('organization')).fold()
    ).by(
        select('e').by('collaborationHours').sum()
    ).unfold()
==>[marketing, engineering]=2
==>[sales, marketing]=2
==>[engineering, sales]=3
==>[engineering, marketing]=2
Marc



Op donderdag 17 december 2020 om 14:50:11 UTC+1 schreef kt...@...:
Vishu,

This may not be optimal, but seems to work:

g.E().hasLabel('collaboration').as('e').outV().values('organization').as('1').select('e').inV().values('organization').as('2').select('e').group().by(select('1', '2')).by(values('collaborationHours').sum()).unfold();

==>{1=engineering, 2=marketing}=2
==>{1=marketing, 2=engineering}=2
==>{1=engineering, 2=sales}=3
==>{1=sales, 2=marketing}=2

Note, you have some leading spaces in your Gremlin on 'collaborationHours' I had to remove, and with the data you provided the engineering/sales total is 3, not 4.

Kevin 

On Wed, Dec 16, 2020 at 11:57 PM vishnu gajendran <gg...@...> wrote:
Hello,

I request your help regarding the janus graph query which I am trying to construct. Let's consider the following graph where each vertex denotes a person and the edge between any two vertex denotes collaboration between them.

Vertices:
p1 = graph.addVertex('person')
p1.property('personId', 1)
p1.property('organization', "engineering")

p2 = graph.addVertex('person')
p2.property('personId', 2)
p2.property('organization', "sales")

p3 = graph.addVertex('person')
p3.property('personId', 3)
p3.property('organization', "marketing")

p4 = graph.addVertex('person')
p4.property('personId', 4)
p4.property('organization', "engineering")

Edges:
p1.addEdge('collaboration', p2, 'collaborationHours', 1)
p1.addEdge('collaboration', p3, 'collaborationHours', 2)

p2.addEdge('collaboration', p3, 'collaborationHours', 2)

p3.addEdge('collaboration', p4, ' collaborationHours', 2)

p4.addEdge('collaboration', p2, ' collaborationHours', 2)

Expected Result is the following table:

Organization1  Organization2 Total Collaboration Hours
Engineering      Sales                 4
Engineering      Marketing         2
Sales                 Marketing          2
Marketing         Engineering       2

Here, I am trying to aggregate the "person to person" graph into "organization to organization" graph. Does JanusGraph support such aggregation queries? If yes, can you please help me with the query for the same?

Thanks

--
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgr...@....
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/5fb20cf1-0aeb-4128-91da-857ec6295587n%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgra...@....
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/16a43300-59ae-4727-920c-b5d4cdb96820n%40googlegroups.com.


Re: Aggregating edges based on the source & target vertex attributes

HadoopMarc <bi...@...>
 

And here a small variation without the keys and with some code formatting:

g.V().as('a').outE().as('e').inV().as('b').
    group().by(
        union(select('a').values('organization'), select('b').values('organization')).fold()
    ).by(
        select('e').by('collaborationHours').sum()
    ).unfold()
==>[marketing, engineering]=2
==>[sales, marketing]=2
==>[engineering, sales]=3
==>[engineering, marketing]=2
Marc



Op donderdag 17 december 2020 om 14:50:11 UTC+1 schreef kt...@...:

Vishu,

This may not be optimal, but seems to work:

g.E().hasLabel('collaboration').as('e').outV().values('organization').as('1').select('e').inV().values('organization').as('2').select('e').group().by(select('1', '2')).by(values('collaborationHours').sum()).unfold();

==>{1=engineering, 2=marketing}=2
==>{1=marketing, 2=engineering}=2
==>{1=engineering, 2=sales}=3
==>{1=sales, 2=marketing}=2

Note, you have some leading spaces in your Gremlin on 'collaborationHours' I had to remove, and with the data you provided the engineering/sales total is 3, not 4.

Kevin 

On Wed, Dec 16, 2020 at 11:57 PM vishnu gajendran <gg...@...> wrote:
Hello,

I request your help regarding the janus graph query which I am trying to construct. Let's consider the following graph where each vertex denotes a person and the edge between any two vertex denotes collaboration between them.

Vertices:
p1 = graph.addVertex('person')
p1.property('personId', 1)
p1.property('organization', "engineering")

p2 = graph.addVertex('person')
p2.property('personId', 2)
p2.property('organization', "sales")

p3 = graph.addVertex('person')
p3.property('personId', 3)
p3.property('organization', "marketing")

p4 = graph.addVertex('person')
p4.property('personId', 4)
p4.property('organization', "engineering")

Edges:
p1.addEdge('collaboration', p2, 'collaborationHours', 1)
p1.addEdge('collaboration', p3, 'collaborationHours', 2)

p2.addEdge('collaboration', p3, 'collaborationHours', 2)

p3.addEdge('collaboration', p4, ' collaborationHours', 2)

p4.addEdge('collaboration', p2, ' collaborationHours', 2)

Expected Result is the following table:

Organization1  Organization2 Total Collaboration Hours
Engineering      Sales                 4
Engineering      Marketing         2
Sales                 Marketing          2
Marketing         Engineering       2

Here, I am trying to aggregate the "person to person" graph into "organization to organization" graph. Does JanusGraph support such aggregation queries? If yes, can you please help me with the query for the same?

Thanks

--
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgr...@....
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/5fb20cf1-0aeb-4128-91da-857ec6295587n%40googlegroups.com.


Re: Aggregating edges based on the source & target vertex attributes

Kevin Schmidt <ktsc...@...>
 

Vishu,

This may not be optimal, but seems to work:

g.E().hasLabel('collaboration').as('e').outV().values('organization').as('1').select('e').inV().values('organization').as('2').select('e').group().by(select('1', '2')).by(values('collaborationHours').sum()).unfold();

==>{1=engineering, 2=marketing}=2
==>{1=marketing, 2=engineering}=2
==>{1=engineering, 2=sales}=3
==>{1=sales, 2=marketing}=2

Note, you have some leading spaces in your Gremlin on 'collaborationHours' I had to remove, and with the data you provided the engineering/sales total is 3, not 4.

Kevin 

On Wed, Dec 16, 2020 at 11:57 PM vishnu gajendran <ggvis...@...> wrote:
Hello,

I request your help regarding the janus graph query which I am trying to construct. Let's consider the following graph where each vertex denotes a person and the edge between any two vertex denotes collaboration between them.

Vertices:
p1 = graph.addVertex('person')
p1.property('personId', 1)
p1.property('organization', "engineering")

p2 = graph.addVertex('person')
p2.property('personId', 2)
p2.property('organization', "sales")

p3 = graph.addVertex('person')
p3.property('personId', 3)
p3.property('organization', "marketing")

p4 = graph.addVertex('person')
p4.property('personId', 4)
p4.property('organization', "engineering")

Edges:
p1.addEdge('collaboration', p2, 'collaborationHours', 1)
p1.addEdge('collaboration', p3, 'collaborationHours', 2)

p2.addEdge('collaboration', p3, 'collaborationHours', 2)

p3.addEdge('collaboration', p4, ' collaborationHours', 2)

p4.addEdge('collaboration', p2, ' collaborationHours', 2)

Expected Result is the following table:

Organization1  Organization2 Total Collaboration Hours
Engineering      Sales                 4
Engineering      Marketing         2
Sales                 Marketing          2
Marketing         Engineering       2

Here, I am trying to aggregate the "person to person" graph into "organization to organization" graph. Does JanusGraph support such aggregation queries? If yes, can you please help me with the query for the same?

Thanks

--
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgra...@....
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/5fb20cf1-0aeb-4128-91da-857ec6295587n%40googlegroups.com.


Re: JanusGraph 0.5.2 and BigTable

BO XUAN LI <libo...@...>
 

Hi Assaf,

I am not familiar with GKE but I can try to answer some of your questions:

> how does a traversal behave when looking up based on an index key when the key is not yet indexed

Assuming the index has been enabled. If a particular key is still in the indexing process (e.g. you are in the middle of a committing process) in one thread, then another thread will not be able to find any data because it finds nothing in the index key lookup. Note that when you are using mixed index, the data is written to your primary backend (e.g. hbase) first, and then mixed index backend (e.g. Elasticsearch). If the data has already been written into hbase but not into Elasticsearch yet, the querying thread cannot find any data (if JanusGraph decides your query can be satisfied by a mixed index).

> org.janusgraph.diskstorage.locking.PermanentLockingException: Local lock contention at org.janusgraph.diskstorage.locking.AbstractLocker.writeLock(AbstractLocker.java:327) 

This usually happens when you have multiple local threads (running on the same JVM instance) contending for the same lock. You might want to check your application logic.

Best regards,
Boxuan

On Dec 17, 2020, at 6:16 PM, Assaf Schwartz <schw...@...> wrote:

Could this be related to delays in indexing? I don't know how to figure out of such exists, but assuming this happens - 
how does a traversal behave when looking up based on an index key when the key is not yet indexed?

On Thursday, December 17, 2020 at 10:54:32 AM UTC+2 Assaf Schwartz wrote:
Hi All,

I'm experiencing an issues with running JanusGraph (on top of GKE) against BigTable.
This is the general setup description:
  • We are using a single node BigTable cluster (for development / integration purposes) with the vanilla 0.5.2 docker.
  • Indexing is configured to be done with ES (also running on GKE)
  • JanusGraph is configured through environment variables:
  • Interaction with JanusGraph are done only through a single gRPC server that is running gremlin-python, let's call it DB-SERVER.
  • Last time we've done testing against BT was with version 0.4.1 of JanusGraph, precompiled to support HBase1.
  • All of our components communicate via gRPC.
Description of the problem:
  1. The DB-SERVER creates a Vertex i, generate some XML to represent work to be done, and sends it to another service for processing, let's call in ORCHESTRATOR.
  2. The ORCHESTRATOR generates two properties, w and r (local identifiers) and sends them back to the DB-SERVER, so they will be set as properties on Vertex i. These two properties are also mixed String indexes.
  3. After setting the properties, DB-SERVER will ack ORCHESTRATOR, which will start processing. As part of the processing, ORCHESTRATOR will send updates back to the DB-SERVER using w and r.
  4. On getting these updates DB-SERVER, it will try looking up Vertex i based on w and r, like so:
    g.V().has("r", <some_r>).has("w", <some_w>).next()
  5. At that point, a null / None is returned as the traversal fails to find Vertex i.
  6. Trying the same traversal in a separate console (python and gremlin) does fetch the vertex. Since it's a single instance cluster, I ruled out any eventual consistency issues.
I'm not sure if it's a regression introduced after 0.4.1.
I've also validated that db-caching is turned off.

Help! :)
Many thanks in advance,
Assaf




-- 
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgra...@....
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/d4373c5a-ab97-4aa4-a143-f26c3ce50677n%40googlegroups.com.


Re: JanusGraph 0.5.2 and BigTable

"ow...@gmail.com" <owner...@...>
 

You can try to use composite index for checking mixedindex delay
четверг, 17 декабря 2020 г. в 13:16:43 UTC+3, Assaf Schwartz:

Could this be related to delays in indexing? I don't know how to figure out of such exists, but assuming this happens - 
how does a traversal behave when looking up based on an index key when the key is not yet indexed?

On Thursday, December 17, 2020 at 10:54:32 AM UTC+2 Assaf Schwartz wrote:
Hi All,

I'm experiencing an issues with running JanusGraph (on top of GKE) against BigTable.
This is the general setup description:
  • We are using a single node BigTable cluster (for development / integration purposes) with the vanilla 0.5.2 docker.
  • Indexing is configured to be done with ES (also running on GKE)
  • JanusGraph is configured through environment variables:
  • Interaction with JanusGraph are done only through a single gRPC server that is running gremlin-python, let's call it DB-SERVER.
  • Last time we've done testing against BT was with version 0.4.1 of JanusGraph, precompiled to support HBase1.
  • All of our components communicate via gRPC.
Description of the problem:
  1. The DB-SERVER creates a Vertex i, generate some XML to represent work to be done, and sends it to another service for processing, let's call in ORCHESTRATOR.
  2. The ORCHESTRATOR generates two properties, w and r (local identifiers) and sends them back to the DB-SERVER, so they will be set as properties on Vertex i. These two properties are also mixed String indexes.
  3. After setting the properties, DB-SERVER will ack ORCHESTRATOR, which will start processing. As part of the processing, ORCHESTRATOR will send updates back to the DB-SERVER using w and r.
  4. On getting these updates DB-SERVER, it will try looking up Vertex i based on w and r, like so:
    g.V().has("r", <some_r>).has("w", <some_w>).next()
  5. At that point, a null / None is returned as the traversal fails to find Vertex i.
  6. Trying the same traversal in a separate console (python and gremlin) does fetch the vertex. Since it's a single instance cluster, I ruled out any eventual consistency issues.
I'm not sure if it's a regression introduced after 0.4.1.
I've also validated that db-caching is turned off.

Help! :)
Many thanks in advance,
Assaf




Re: JanusGraph 0.5.2 and BigTable

Assaf Schwartz <schw...@...>
 

Could this be related to delays in indexing? I don't know how to figure out of such exists, but assuming this happens - 
how does a traversal behave when looking up based on an index key when the key is not yet indexed?

On Thursday, December 17, 2020 at 10:54:32 AM UTC+2 Assaf Schwartz wrote:
Hi All,

I'm experiencing an issues with running JanusGraph (on top of GKE) against BigTable.
This is the general setup description:
  • We are using a single node BigTable cluster (for development / integration purposes) with the vanilla 0.5.2 docker.
  • Indexing is configured to be done with ES (also running on GKE)
  • JanusGraph is configured through environment variables:
  • Interaction with JanusGraph are done only through a single gRPC server that is running gremlin-python, let's call it DB-SERVER.
  • Last time we've done testing against BT was with version 0.4.1 of JanusGraph, precompiled to support HBase1.
  • All of our components communicate via gRPC.
Description of the problem:
  1. The DB-SERVER creates a Vertex i, generate some XML to represent work to be done, and sends it to another service for processing, let's call in ORCHESTRATOR.
  2. The ORCHESTRATOR generates two properties, w and r (local identifiers) and sends them back to the DB-SERVER, so they will be set as properties on Vertex i. These two properties are also mixed String indexes.
  3. After setting the properties, DB-SERVER will ack ORCHESTRATOR, which will start processing. As part of the processing, ORCHESTRATOR will send updates back to the DB-SERVER using w and r.
  4. On getting these updates DB-SERVER, it will try looking up Vertex i based on w and r, like so:
    g.V().has("r", <some_r>).has("w", <some_w>).next()
  5. At that point, a null / None is returned as the traversal fails to find Vertex i.
  6. Trying the same traversal in a separate console (python and gremlin) does fetch the vertex. Since it's a single instance cluster, I ruled out any eventual consistency issues.
I'm not sure if it's a regression introduced after 0.4.1.
I've also validated that db-caching is turned off.

Help! :)
Many thanks in advance,
Assaf




Re: JanusGraph 0.5.2 and BigTable

Assaf Schwartz <schw...@...>
 

From time to time, usually after setting up a fresh copy of the BT and Janus, I'll encounter errors relating to locking. However this doesn't happen every time.
Sorry, I can't see to be able copy the logs nicely from the GCP Cloud logging.


org.janusgraph.diskstorage.locking.PermanentLockingException: Local lock contention at org.janusgraph.diskstorage.locking.AbstractLocker.writeLock(AbstractLocker.java:327) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStore.acquireLock(ExpectedValueCheckingStore.java:103) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.acquireLock(KCVSProxy.java:51) at org.janusgraph.diskstorage.BackendTransaction.acquireIndexLock(BackendTransaction.java:255) at org.janusgraph.graphdb.database.StandardJanusGraph.prepareCommit(StandardJanusGraph.java:591) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:712) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1425) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:297) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.apache.tinkerpop.gremlin.server.op.traversal.TraversalOpProcessor.onTraversalSuccess(TraversalOpProcessor.java:471) at org.apache.tinkerpop.gremlin.server.op.traversal.TraversalOpProcessor.handleIterator(TraversalOpProcessor.java:603) at org.apache.tinkerpop.gremlin.server.op.traversal.TraversalOpProcessor.lambda$iterateBytecodeTraversal$4(TraversalOpProcessor.java:411) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)


On Thursday, December 17, 2020 at 10:54:32 AM UTC+2 Assaf Schwartz wrote:
Hi All,

I'm experiencing an issues with running JanusGraph (on top of GKE) against BigTable.
This is the general setup description:
  • We are using a single node BigTable cluster (for development / integration purposes) with the vanilla 0.5.2 docker.
  • Indexing is configured to be done with ES (also running on GKE)
  • JanusGraph is configured through environment variables:
  • Interaction with JanusGraph are done only through a single gRPC server that is running gremlin-python, let's call it DB-SERVER.
  • Last time we've done testing against BT was with version 0.4.1 of JanusGraph, precompiled to support HBase1.
  • All of our components communicate via gRPC.
Description of the problem:
  1. The DB-SERVER creates a Vertex i, generate some XML to represent work to be done, and sends it to another service for processing, let's call in ORCHESTRATOR.
  2. The ORCHESTRATOR generates two properties, w and r (local identifiers) and sends them back to the DB-SERVER, so they will be set as properties on Vertex i. These two properties are also mixed String indexes.
  3. After setting the properties, DB-SERVER will ack ORCHESTRATOR, which will start processing. As part of the processing, ORCHESTRATOR will send updates back to the DB-SERVER using w and r.
  4. On getting these updates DB-SERVER, it will try looking up Vertex i based on w and r, like so:
    g.V().has("r", <some_r>).has("w", <some_w>).next()
  5. At that point, a null / None is returned as the traversal fails to find Vertex i.
  6. Trying the same traversal in a separate console (python and gremlin) does fetch the vertex. Since it's a single instance cluster, I ruled out any eventual consistency issues.
I'm not sure if it's a regression introduced after 0.4.1.
I've also validated that db-caching is turned off.

Help! :)
Many thanks in advance,
Assaf




JanusGraph 0.5.2 and BigTable

Assaf Schwartz <schw...@...>
 

Hi All,

I'm experiencing an issues with running JanusGraph (on top of GKE) against BigTable.
This is the general setup description:
  • We are using a single node BigTable cluster (for development / integration purposes) with the vanilla 0.5.2 docker.
  • Indexing is configured to be done with ES (also running on GKE)
  • JanusGraph is configured through environment variables:
    • janusgraph.storage.backend=hbase
    • janusgraph.storage.hbase.ext.hbase.client.connection.impl=com.google.cloud.bigtable.hbase2_x.BigtableConnection
    • janusgraph.storage.hbase.ext.google.bigtable.project.id=<project_id>
    • janusgraph.storage.hbase.ext.google.bigtable.instance.id=<bt>
  • Interaction with JanusGraph are done only through a single gRPC server that is running gremlin-python, let's call it DB-SERVER.
  • Last time we've done testing against BT was with version 0.4.1 of JanusGraph, precompiled to support HBase1.
  • All of our components communicate via gRPC.
Description of the problem:
  1. The DB-SERVER creates a Vertex i, generate some XML to represent work to be done, and sends it to another service for processing, let's call in ORCHESTRATOR.
  2. The ORCHESTRATOR generates two properties, w and r (local identifiers) and sends them back to the DB-SERVER, so they will be set as properties on Vertex i. These two properties are also mixed String indexes.
  3. After setting the properties, DB-SERVER will ack ORCHESTRATOR, which will start processing. As part of the processing, ORCHESTRATOR will send updates back to the DB-SERVER using w and r.
  4. On getting these updates DB-SERVER, it will try looking up Vertex i based on w and r, like so:
    g.V().has("r", <some_r>).has("w", <some_w>).next()
  5. At that point, a null / None is returned as the traversal fails to find Vertex i.
  6. Trying the same traversal in a separate console (python and gremlin) does fetch the vertex. Since it's a single instance cluster, I ruled out any eventual consistency issues.
I'm not sure if it's a regression introduced after 0.4.1.
I've also validated that db-caching is turned off.

Help! :)
Many thanks in advance,
Assaf




Re: Use index for sorting

toom <to...@...>
 

Thank you very much, I'll try that.

Toom.

On Thursday, December 17, 2020 at 8:28:52 AM UTC+1 ow...@... wrote:
We use special strategy that rewrite order comparator for supporting missing values https://gist.github.com/mad/6bef960d51d7a0d89a8c310851023303

пятница, 4 декабря 2020 г. в 09:59:13 UTC+3, t...@...:

The problem of using custom value for null is that we need to choose a value for each data type, and hope that nobody will try to use this particular value. I suppose it is feasible for data type like string, date or double but not for boolean.

Toom.

On Friday, December 4, 2020 at 2:22:02 AM UTC+1 li...@... wrote:
No, null support is an optional feature for graph providers. JanusGraph does not allow null value and I don’t think it will be supported (in near future).

Apart from the solution suggested by Marc, is it possible for you to come up with some custom value to represent null?

Best regards,
Boxuan

「toom <t...@...>」在 2020年12月4日 週五,上午3:17 寫道:

Hi Marc,

Thank you for your response.
If I understand correctly, with TinkerPop 3.5 I will be able to sort on property with missing values. It is a good news.
Do you know it JanusGraph 0.6.0 will be release with that version ?

Regarding the impact of the step order on index use, I wrote a strategy [1] that put HasStep and OrderStep before FilterStep if they follow a GraphStep.

Best regards,

Toom.

On Thursday, December 3, 2020 at 8:17:27 AM UTC+1 HadoopMarc wrote:
Hi Toom,

No solution, but the exception that you mention comes from TinkerPop:

Apparently, you want all selected vertices, including the ones with null properties, so I would wait for TinkerPop 3.5 and in the mean time use your own workaround for a single filter criterion and do the ordering outside gremlin for more complex sets of filtering criteria.

Best wishes,      Marc

Op woensdag 2 december 2020 om 08:13:05 UTC+1 schreef t...@...:
Hello,

I'm using JanusGraph with Cassandra (0.5.2) and ElasticSearch.

I try to optimize my queries and use the mixed indexes as much as possible, in particular for sortings, but I have some difficulties:

It is not possible to sort by properties that can have missing values (or I get a "The property does not exist as the key has no associated value for the provided element"). Therefore I used ".order().by(coalesce(values('closingDate'), new Date()))" but in this case, the index is not used.

If there is only one sorting criterion, I probably can do something like:

g.inject(1).union(
  g.V().hasLabel('Case').has('closingDate').order().by('closingDate'),
  g.V().hasLabel('Case').hasNot('closingDate'))

But what is my best option if I want to use several criteria?


I also note that the FilterRankingStrategy strategy can have negative effect on performance when there are filters that don't use  index. For example, the following query is faster without step reordering.

g.V().hasLabel('Case').has('closingDate').order().by('closingDate').filter(out('attachment').has('file'))

FilterRanking swaps order() and filter() steps and then index is not used for sorting.

Any help will be much appreciated.

Toom.

--
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgr...@....
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/c6bcb878-bfc2-4d8f-a3ee-38cc214382bcn%40googlegroups.com.


Re: Use index for sorting

"ow...@gmail.com" <owner...@...>
 

We use special strategy that rewrite order comparator for supporting missing values https://gist.github.com/mad/6bef960d51d7a0d89a8c310851023303

пятница, 4 декабря 2020 г. в 09:59:13 UTC+3, t...@...:

The problem of using custom value for null is that we need to choose a value for each data type, and hope that nobody will try to use this particular value. I suppose it is feasible for data type like string, date or double but not for boolean.

Toom.

On Friday, December 4, 2020 at 2:22:02 AM UTC+1 li...@... wrote:
No, null support is an optional feature for graph providers. JanusGraph does not allow null value and I don’t think it will be supported (in near future).

Apart from the solution suggested by Marc, is it possible for you to come up with some custom value to represent null?

Best regards,
Boxuan

「toom <t...@...>」在 2020年12月4日 週五,上午3:17 寫道:

Hi Marc,

Thank you for your response.
If I understand correctly, with TinkerPop 3.5 I will be able to sort on property with missing values. It is a good news.
Do you know it JanusGraph 0.6.0 will be release with that version ?

Regarding the impact of the step order on index use, I wrote a strategy [1] that put HasStep and OrderStep before FilterStep if they follow a GraphStep.

Best regards,

Toom.

On Thursday, December 3, 2020 at 8:17:27 AM UTC+1 HadoopMarc wrote:
Hi Toom,

No solution, but the exception that you mention comes from TinkerPop:

Apparently, you want all selected vertices, including the ones with null properties, so I would wait for TinkerPop 3.5 and in the mean time use your own workaround for a single filter criterion and do the ordering outside gremlin for more complex sets of filtering criteria.

Best wishes,      Marc

Op woensdag 2 december 2020 om 08:13:05 UTC+1 schreef t...@...:
Hello,

I'm using JanusGraph with Cassandra (0.5.2) and ElasticSearch.

I try to optimize my queries and use the mixed indexes as much as possible, in particular for sortings, but I have some difficulties:

It is not possible to sort by properties that can have missing values (or I get a "The property does not exist as the key has no associated value for the provided element"). Therefore I used ".order().by(coalesce(values('closingDate'), new Date()))" but in this case, the index is not used.

If there is only one sorting criterion, I probably can do something like:

g.inject(1).union(
  g.V().hasLabel('Case').has('closingDate').order().by('closingDate'),
  g.V().hasLabel('Case').hasNot('closingDate'))

But what is my best option if I want to use several criteria?


I also note that the FilterRankingStrategy strategy can have negative effect on performance when there are filters that don't use  index. For example, the following query is faster without step reordering.

g.V().hasLabel('Case').has('closingDate').order().by('closingDate').filter(out('attachment').has('file'))

FilterRanking swaps order() and filter() steps and then index is not used for sorting.

Any help will be much appreciated.

Toom.

--
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgr...@....
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/c6bcb878-bfc2-4d8f-a3ee-38cc214382bcn%40googlegroups.com.


Aggregating edges based on the source & target vertex attributes

vishnu gajendran <ggvis...@...>
 

Hello,

I request your help regarding the janus graph query which I am trying to construct. Let's consider the following graph where each vertex denotes a person and the edge between any two vertex denotes collaboration between them.

Vertices:
p1 = graph.addVertex('person')
p1.property('personId', 1)
p1.property('organization', "engineering")

p2 = graph.addVertex('person')
p2.property('personId', 2)
p2.property('organization', "sales")

p3 = graph.addVertex('person')
p3.property('personId', 3)
p3.property('organization', "marketing")

p4 = graph.addVertex('person')
p4.property('personId', 4)
p4.property('organization', "engineering")

Edges:
p1.addEdge('collaboration', p2, 'collaborationHours', 1)
p1.addEdge('collaboration', p3, 'collaborationHours', 2)

p2.addEdge('collaboration', p3, 'collaborationHours', 2)

p3.addEdge('collaboration', p4, ' collaborationHours', 2)

p4.addEdge('collaboration', p2, ' collaborationHours', 2)

Expected Result is the following table:

Organization1  Organization2 Total Collaboration Hours
Engineering      Sales                 4
Engineering      Marketing         2
Sales                 Marketing          2
Marketing         Engineering       2

Here, I am trying to aggregate the "person to person" graph into "organization to organization" graph. Does JanusGraph support such aggregation queries? If yes, can you please help me with the query for the same?

Thanks


Re: Condition where-inV-is does not work

HadoopMarc <bi...@...>
 

Hi Anton,

To compare objects you need the eq() predicate, not the is() step (see https://tinkerpop.apache.org/docs/current/reference/#a-note-on-predicates). Also note that JanusGraph uses RelationIdentifier objects as id's.
So, give the following a try (not tested):
g.V(110792).as('a').V(41099392).outE('LINK').where(inV().eq('a'))

Best wishes,    Marc

Op woensdag 16 december 2020 om 10:03:33 UTC+1 schreef Anton Eroshenko:

Thanks for reply.
In real query I don't have an ID, that's why I use traversal in where clause. Tried this one, but it does not work either:
g.V(41099392).outE('LINK').where(inV().hasId(V(110792).id()))

On Wednesday, December 16, 2020 at 1:45:50 AM UTC+7 Amiya wrote:

Try  g.V(41099392).outE('LINK').where(inV().hasId(110792))

On Tuesday, 15 December 2020 at 20:45:40 UTC+5:30 Anton Eroshenko wrote:

Hi, JanusGraph users.
I try a simple query from TinkerPop docs with my JanusGraph installation, but it does not work as expected. How is it possible:
gremlin> g.V(41099392).outE('LINK').inV()
==>v[110792]
==>v[81993864]
gremlin> g.V(41099392).outE('LINK').where(inV().is(V(110792)))
gremlin>

The last query returns nothing...
How to filter out vertices by traversal? Appreciate any help

1221 - 1240 of 6663