janusgraph-users@lists.lfaidata.foundation

Re: Multiple writers cause inconsistent vertices

Robert Dale <rob...@...>

Do you `mgt.commit()`? Do you `mgt.awaitGraphIndexStatus(graph, 'UniqueURI').call()`?

You can use this script in the console to help see the state of the index - https://gist.github.com/robertdale/ad4c63910009dd1118abe67b33ce41e1

Robert Dale

Show quoted text

On Sat, Sep 23, 2017 at 6:36 AM, Ajay Srivastava <Ajay.Sr...@...> wrote:

Hi,

I am using janusgraph-0.1.1 with HBase.

The data is being loaded in graph using three clients connecting to same gremlin server. The clients are executing same code that checks if vertex is not already present in the graph then it inserts the vertex.

I was verifying the data and found following problem -

scala> graph.V().hasLabel("Root").toList

15:27:22,361 WARN StandardJanusGraphTx:1273 - Query requires iterating over all vertices [(~label = Root)]. For better performance, use indexes

res11: List[gremlin.scala.Vertex] = List(v[737304], v[4136], v[442432])

Results is three vertices.

scala> graph.V().hasLabel("Root").properties("URI").toList

15:27:52,275 WARN StandardJanusGraphTx:1273 - Query requires iterating over all vertices [(~label = Root)]. For better performance, use indexes

res13: List[gremlin.scala.Property[Any]] = List(vp[URI->Root], vp[URI->Root], vp[URI->Root])

Result is three vertices having same URI.

scala> val uri = Key[String]("URI")

scala> graph.V().has(uri, "Root").toList

res12: List[gremlin.scala.Vertex] = List(v[442432])

Since vertices are uniquely indexed on URI, this result is correct. Janusgraph should not have allowed to insert vertices having same URI but it did as displayed in above two outputs.

I am new to janusgraph and have many questions -

1) What am I doing wrong here ?

2) Multiple clients writing to same gremlin server may create problem ?

3) How to read back the schema created by me ?

4) Below is the code for creating schema. Is this correct ?

/* Creating three types of vertices having same properties and indexed on same property URI */

def createVertexSchema : Boolean = {

val vertexLabels = Array("Root", "Lang", "Cocpt")

val GUID = mgt.makePropertyKey("GUID").dataType(classOf[String]).make

val Name = mgt.makePropertyKey("Name").dataType(classOf[String]).make

val URI = mgt.makePropertyKey("URI").dataType(classOf[String]).make

vertexLabels.foreach {

vertexLabel =>

val vLabel = mgt.makeVertexLabel(vertexLabel).make

}

mgt.buildIndex("UniqueURI", classOf[Vertex]).addKey(URI).unique().buildCompositeIndex()

true

}

Regards,

Ajay

--
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/D8F1F502-3482-4A8E-AB9A-5021273DC697%40guavus.com.
For more options, visit https://groups.google.com/d/optout.

Can I use JanusGraph with C#.net and if so how?

biniam...@...

#853

Hello,

I am new to JanusGraph and I want to use it in a C# project. I plan to use AWS DynamoDB as a storage engine.

Does JanusGraph support writing the implementation in C# programming language?

If it does, what library can I use?

Can you please provide me with a sample project (may be in Github) or a blog post?

Thanks.

Multiple writers cause inconsistent vertices

Ajay Srivastava <Ajay.Sr...@...>

#856

Hi,

I am using janusgraph-0.1.1 with HBase.

The data is being loaded in graph using three clients connecting to same gremlin server. The clients are executing same code that checks if vertex is not already present in the graph then it inserts the vertex.

I was verifying the data and found following problem -

scala> graph.V().hasLabel("Root").toList

15:27:22,361  WARN StandardJanusGraphTx:1273 - Query requires iterating over all vertices [(~label = Root)]. For better performance, use indexes

res11: List[gremlin.scala.Vertex] = List(v[737304], v[4136], v[442432])

Results is three vertices.

scala> graph.V().hasLabel("Root").properties("URI").toList

15:27:52,275  WARN StandardJanusGraphTx:1273 - Query requires iterating over all vertices [(~label = Root)]. For better performance, use indexes

res13: List[gremlin.scala.Property[Any]] = List(vp[URI->Root], vp[URI->Root], vp[URI->Root])

Result is three vertices having same URI.

scala> val uri = Key[String]("URI")

scala> graph.V().has(uri, "Root").toList

res12: List[gremlin.scala.Vertex] = List(v[442432])

Since vertices are uniquely indexed on URI, this result is correct. Janusgraph should not have allowed to insert vertices having same URI but it did as displayed in above two outputs.

I am new to janusgraph and have many questions -

1) What am I doing wrong here ?

2) Multiple clients writing to same gremlin server may create problem ?

3) How to read back the schema created by me ?

4) Below is the code for creating schema. Is this correct ?

/* Creating three types of vertices having same properties and indexed on same property URI */

    def createVertexSchema : Boolean = {

        val vertexLabels = Array("Root", "Lang", "Cocpt")

        val GUID     = mgt.makePropertyKey("GUID").dataType(classOf[String]).make

        val Name = mgt.makePropertyKey("Name").dataType(classOf[String]).make

        val URI      = mgt.makePropertyKey("URI").dataType(classOf[String]).make

        vertexLabels.foreach {

            vertexLabel =>

                val vLabel   = mgt.makeVertexLabel(vertexLabel).make

}

        mgt.buildIndex("UniqueURI", classOf[Vertex]).addKey(URI).unique().buildCompositeIndex()

        true

}

Regards,

Ajay

Re: PageRank on Large Graph

HadoopMarc <bi...@...>

#850

Hi Joe,

This question reminds me to an earlier discussion we had on the performance of OLAP traversals for janusgraph-hbase. My conclusion there that janusgraph-hbase needs a better HbaseInputFormat that delivers more partitions than one partition per HBase region. I guess Pagerank suffers from that in the same way. Do you maybe have the option to use Cassandra, which has a configurable cassandra.inpit.split.size ? I did not try this myself.

HTH, Marc

Op vrijdag 22 september 2017 15:41:12 UTC+2 schreef Joseph Obernberger:

Show quoted text

Hi All - I've been experimenting with SparkGraphComputer, and have it
working, but I'm having performance issues. What is the best way to run
PageRank against a very large graph stored inside of JanusGraph?

Thank you!

-Joe

Re: ES with JG

Suny <sahithiy...@...>

#848

I have Cassandra and ES hosted on AWS.

JG and ES on one node and Cassandra running on different node.

Do you think that is causing issues ?

I am trying to setup JS,ES and Cassandra on same node and try it out. Any other suggestions ?

Show quoted text

On Friday, September 22, 2017 at 11:29:04 AM UTC-4, Robert Dale wrote:

First, I'm not sure if an index is being used here. Second, my full table scans with 10k vertices is faster. (I'm also using 0.2.0-SNAPSHOT) So you probably have some other issues going on. Is your Cassandra and ES local or over internet?

No index, no cache:

gremlin> g.V().has('type',textContains('car')).valueMap().profile()
11:15:01 WARN org.janusgraph.graphdb.transaction.StandardJanusGraphTx - Query requires iterating over all vertices [(type textContains car)]. For better performance, use indexes
==>Traversal Metrics
Step Count Traversers Time (ms) % Dur
=============================================================================================================
JanusGraphStep([],[type.textContains(car)]) 10000 10000 1640.758 95.02
\_condition=(type textContains car)
\_isFitted=false
\_query=[]
\_orders=[]
\_isOrdered=true
optimization 3.493
scan 0.000
\_condition=VERTEX
\_query=[]
\_fullscan=true
PropertyMapStep(value) 10000 10000 85.950 4.98
>TOTAL - - 1726.709 -

With Index, No Cache:

gremlin> g.V().has('type',textContains('car')).valueMap().profile()
==>Traversal Metrics
Step Count Traversers Time (ms) % Dur
=============================================================================================================
JanusGraphStep([],[type.textContains(car)]) 10000 10000 910.588 35.06
\_condition=(type textContains car)
\_isFitted=true
\_query=[(type textContains car)]:typeIndex
\_index=typeIndex
\_orders=[]
\_isOrdered=true
\_index_impl=search
optimization 6.669
backend-query 0.000
\_query=typeIndex:[(type textContains car)]:typeIndex
backend-query 0.000
\_query=typeIndex:[(type textContains car)]:typeIndex
backend-query 0.000
\_query=typeIndex:[(type textContains car)]:typeIndex
backend-query 10000 700.633
\_query=typeIndex:[(type textContains car)]:typeIndex
PropertyMapStep(value) 10000 10000 1686.415 64.94
>TOTAL - - 2597.003 -

On Friday, September 22, 2017 at 10:26:46 AM UTC-4, Suny wrote:
The index is enabled in ES. I verified the same query g.V().has('type',textContains('site')) by turning on force-index property to true and it works.

The initial hit tool 46000ms and later ones tool 600-750ms.

For the second query - g.V().has('type',textContains('site')).valueMap()

The initial hit tool 129826ms and later ones took 550 - 700ms

gremlin> :> g.V().has('type',textContains('site')).valueMap().profile()

==>Traversal Metrics

Step Count Traversers Time (ms) % Dur

=============================================================================================================

JanusGraphStep([],[type.textContains(site)]) 1186 1186 17.913 0.03

optimization 0.203

backend-query 1186 5.714

PropertyMapStep(value) 1186 1186 64199.498 99.97

>TOTAL - - 64217.412 -

On Friday, September 22, 2017 at 10:17:23 AM UTC-4, Robert Dale wrote:
How much time is 'slow'?. Are your indexes actually enabled? What is the byte size of the results? Have you ruled out resource issues - cpu, ram, disk, network?

What does `g.V().has('type',textContains('site')).valueMap().profile()` show?

Robert Dale

On Fri, Sep 22, 2017 at 10:10 AM, Suny <sahi...@...> wrote:
values are of small size. like strings of length 20.

On Friday, September 22, 2017 at 10:03:34 AM UTC-4, Suny wrote:
Each vertex has 3-5 attributes on it.

On Friday, September 22, 2017 at 9:02:15 AM UTC-4, Jason Plurad wrote:
1500 vertices is a fair amount, but how much data is getting returned in the valueMap? Is there a large number of properties in the map or perhaps are the values very large? Very easily could be the cost of serializing that map. In another post you were asking about string size restrictions...

On Thursday, September 21, 2017 at 2:40:06 PM UTC-4, Suny wrote:
I implemented a mixed index on 'type' attribute on vertex.

Whenever i query for :> g.V().has('type',textContains('car'))

the result comes out really really fast.

When I do :> g.V().has('type',textContains('site')).valueMap() it is taking so much time to retrieve data. I have 1500 vertices of type car.

Is it because for 1st query it gets the result using ES for the second query it gets vertex id's from ES but for getting attributes it needs to hit JG again ?

Is there a way to fasten up the second query ?

--
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/8a3c629b-10c4-4d55-8f8f-8e5397f6ff87%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Re: ES with JG

Robert Dale <rob...@...>

#847

First, I'm not sure if an index is being used here. Second, my full table scans with 10k vertices is faster. (I'm also using 0.2.0-SNAPSHOT) So you probably have some other issues going on. Is your Cassandra and ES local or over internet?

No index, no cache:

gremlin> g.V().has('type',textContains('car')).valueMap().profile()

11:15:01 WARN org.janusgraph.graphdb.transaction.StandardJanusGraphTx - Query requires iterating over all vertices [(type textContains car)]. For better performance, use indexes

==>Traversal Metrics

Step Count Traversers Time (ms) % Dur

=============================================================================================================

JanusGraphStep([],[type.textContains(car)]) 10000 10000 1640.758 95.02

\_condition=(type textContains car)

\_isFitted=false

\_query=[]

\_orders=[]

\_isOrdered=true

optimization 3.493

scan 0.000

\_condition=VERTEX

\_query=[]

\_fullscan=true

PropertyMapStep(value) 10000 10000 85.950 4.98

>TOTAL - - 1726.709 -

With Index, No Cache:

gremlin> g.V().has('type',textContains('car')).valueMap().profile()

==>Traversal Metrics

Step Count Traversers Time (ms) % Dur

=============================================================================================================

JanusGraphStep([],[type.textContains(car)]) 10000 10000 910.588 35.06

\_condition=(type textContains car)

\_isFitted=true

\_query=[(type textContains car)]:typeIndex

\_index=typeIndex

\_orders=[]

\_isOrdered=true

\_index_impl=search

optimization 6.669

backend-query 0.000

\_query=typeIndex:[(type textContains car)]:typeIndex

backend-query 0.000

\_query=typeIndex:[(type textContains car)]:typeIndex

backend-query 0.000

\_query=typeIndex:[(type textContains car)]:typeIndex

backend-query 10000 700.633

\_query=typeIndex:[(type textContains car)]:typeIndex

PropertyMapStep(value) 10000 10000 1686.415 64.94

>TOTAL - - 2597.003 -

Show quoted text

On Friday, September 22, 2017 at 10:26:46 AM UTC-4, Suny wrote:

The index is enabled in ES. I verified the same query g.V().has('type',textContains('site')) by turning on force-index property to true and it works.

The initial hit tool 46000ms and later ones tool 600-750ms.

For the second query - g.V().has('type',textContains('site')).valueMap()

The initial hit tool 129826ms and later ones took 550 - 700ms

gremlin> :> g.V().has('type',textContains('site')).valueMap().profile()

==>Traversal Metrics

Step Count Traversers Time (ms) % Dur

=============================================================================================================

JanusGraphStep([],[type.textContains(site)]) 1186 1186 17.913 0.03

optimization 0.203

backend-query 1186 5.714

PropertyMapStep(value) 1186 1186 64199.498 99.97

>TOTAL - - 64217.412 -

On Friday, September 22, 2017 at 10:17:23 AM UTC-4, Robert Dale wrote:
How much time is 'slow'?. Are your indexes actually enabled? What is the byte size of the results? Have you ruled out resource issues - cpu, ram, disk, network?

What does `g.V().has('type',textContains('site')).valueMap().profile()` show?

Robert Dale

On Fri, Sep 22, 2017 at 10:10 AM, Suny <sahi...@...> wrote:
values are of small size. like strings of length 20.

On Friday, September 22, 2017 at 10:03:34 AM UTC-4, Suny wrote:
Each vertex has 3-5 attributes on it.

On Friday, September 22, 2017 at 9:02:15 AM UTC-4, Jason Plurad wrote:
1500 vertices is a fair amount, but how much data is getting returned in the valueMap? Is there a large number of properties in the map or perhaps are the values very large? Very easily could be the cost of serializing that map. In another post you were asking about string size restrictions...

On Thursday, September 21, 2017 at 2:40:06 PM UTC-4, Suny wrote:
I implemented a mixed index on 'type' attribute on vertex.

Whenever i query for :> g.V().has('type',textContains('car'))

the result comes out really really fast.

When I do :> g.V().has('type',textContains('site')).valueMap() it is taking so much time to retrieve data. I have 1500 vertices of type car.

Is it because for 1st query it gets the result using ES for the second query it gets vertex id's from ES but for getting attributes it needs to hit JG again ?

Is there a way to fasten up the second query ?

--
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/8a3c629b-10c4-4d55-8f8f-8e5397f6ff87%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Re: ES with JG

Suny <sahithiy...@...>

#846

The index is enabled in ES. I verified the same query g.V().has('type',textContains('site')) by turning on force-index property to true and it works.

The initial hit tool 46000ms and later ones tool 600-750ms.

For the second query - g.V().has('type',textContains('site')).valueMap()

The initial hit tool 129826ms and later ones took 550 - 700ms

gremlin> :> g.V().has('type',textContains('site')).valueMap().profile()

==>Traversal Metrics

Step Count Traversers Time (ms) % Dur

=============================================================================================================

JanusGraphStep([],[type.textContains(site)]) 1186 1186 17.913 0.03

optimization 0.203

backend-query 1186 5.714

PropertyMapStep(value) 1186 1186 64199.498 99.97

>TOTAL - - 64217.412 -

Show quoted text

On Friday, September 22, 2017 at 10:17:23 AM UTC-4, Robert Dale wrote:

How much time is 'slow'?. Are your indexes actually enabled? What is the byte size of the results? Have you ruled out resource issues - cpu, ram, disk, network?

What does `g.V().has('type',textContains('site')).valueMap().profile()` show?

Robert Dale

On Fri, Sep 22, 2017 at 10:10 AM, Suny <sahi...@...> wrote:
values are of small size. like strings of length 20.

On Friday, September 22, 2017 at 10:03:34 AM UTC-4, Suny wrote:
Each vertex has 3-5 attributes on it.

On Friday, September 22, 2017 at 9:02:15 AM UTC-4, Jason Plurad wrote:
1500 vertices is a fair amount, but how much data is getting returned in the valueMap? Is there a large number of properties in the map or perhaps are the values very large? Very easily could be the cost of serializing that map. In another post you were asking about string size restrictions...

On Thursday, September 21, 2017 at 2:40:06 PM UTC-4, Suny wrote:
I implemented a mixed index on 'type' attribute on vertex.

Whenever i query for :> g.V().has('type',textContains('car'))

the result comes out really really fast.

When I do :> g.V().has('type',textContains('site')).valueMap() it is taking so much time to retrieve data. I have 1500 vertices of type car.

Is it because for 1st query it gets the result using ES for the second query it gets vertex id's from ES but for getting attributes it needs to hit JG again ?

Is there a way to fasten up the second query ?

--
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/8a3c629b-10c4-4d55-8f8f-8e5397f6ff87%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Re: Using ES for traversal queries ?

Suny <sahithiy...@...>

#845

I defined timestamp as a string.

Here is how i created those

final PropertyKey myTimestamp = mgmt.makePropertyKey("timestamp").dataType(String.class).make();

EdgeLabel connectedTo = mgmt.getOrCreateEdgeLabel("connected_to");

mgmt.buildEdgeIndex(connectedTo, "forAllEdgesOnTimestamp", Direction.BOTH, myTimestamp);

Show quoted text

On Friday, September 22, 2017 at 10:18:46 AM UTC-4, Jason Plurad wrote:

How did you define the 'timestamp' property and the vertex-centric index that uses it?
Seems strange to me that you're looking for has('timestamp, '') rather than comparing Long values.

On Friday, September 22, 2017 at 10:12:44 AM UTC-4, Suny wrote:
yes. the first time i query it took 129826 ms, and then 600-700 ms later on (which i believe is because of caching). Without that it is taking around 128000ms

On Friday, September 22, 2017 at 10:10:57 AM UTC-4, Jason Plurad wrote:
is this slow?

g.V().has('type',textContains('car')).inE().has('timestamp',eq('')).inV().id()

On Friday, September 22, 2017 at 10:07:26 AM UTC-4, Suny wrote:
Thanks. I am dealing with count here. I need the attributes list on those vertices.

g.V().has('type',textContains('car')) - This is coming out very fast, which i assume is because of index in ES.

g.V().has('type',textContains('car')).inE().has('timestamp',eq('')).inV().valueMap() - This is slow. I have 1500 vertices and each vertex has about 3-5 attributes on it. I implemented vertex-centric index, on timestamp property but not sure if it is being used.

On Friday, September 22, 2017 at 8:54:54 AM UTC-4, Jason Plurad wrote:
No, explain() doesn't give any clear indication currently whether an index will be utilized. I opened up an issue for that.

What counts are you dealing with here?

g.V().has('type',textContains('car')).count()
g.V().has('type',textContains('car')).inE().count()
g.V().has('type',textContains('car')).inE().has('timestamp',eq('')).count()

On Thursday, September 21, 2017 at 1:59:37 PM UTC-4, Suny wrote:
I implemented vertex-centric index on edge label.

Here is the query i am doing

g.V().has('type',textContains('car')).inE().has('timestamp',eq('')).inV().valueMap()

The documentation says - JanusGraph is intelligent enough to use vertex-centric indices when available.

So can i assume that Janusgraph uses the vertex-centric index for this query ?

On Thursday, September 21, 2017 at 11:05:37 AM UTC-4, Daniel Kuppitz wrote:
Only in the underlying storage backend.

See: http://docs.janusgraph.org/0.2.0-SNAPSHOT/data-model.html

Cheers,
Daniel

On Thu, Sep 21, 2017 at 7:57 AM, Suny <sahi...@...> wrote:
Thanks. Is the vertex-centric index not stored in Elastic search. Is it only in JG ?

On Thursday, September 21, 2017 at 10:28:47 AM UTC-4, Daniel Kuppitz wrote:
Thanks, All the in edges will have same label. Does it still traverse through all edges or directly find the edges with empty timestamp ?

I assume that Janus can only leverage a vertex centric index, if you specify the label, since that was the case in Titan and I don't think I heard about any changes in this area.

So in cases where i need to do lot of traversal, ES is not helpful ?

Again, ES is helpful for the initial vertex lookup. Once you start to traverse through the graph, you can only rely on vertex centric indices (or a good model that doesn't require much filtering).

Also can you explain a bit more on OLAP query

You can read up on OLAP here: http://docs.janusgraph.org/0.2.0-SNAPSHOT/hadoop-tp3.html
...and here: http://tinkerpop.apache.org/docs/current/reference/#_olap_hadoop_gremlin

Cheers,
Daniel

On Thu, Sep 21, 2017 at 7:07 AM, Suny <sahi...@...> wrote:
Thanks, All the in edges will have same label. Does it still traverse through all edges or directly find the edges with empty timestamp ?

So in cases where i need to do lot of traversal, ES is not helpful ?

Also can you explain a bit more on OLAP query

On Wednesday, September 20, 2017 at 2:35:27 PM UTC-4, Daniel Kuppitz wrote:
You should
provide an edge label for inE()
have a vertex centric index on timestamp
use a simple filter, instead of traversing back to the previous vertex (unless you rely on duplicates in your result)
Something like this:

g.V().has('type', textContains('car')).
filter(inE('edge-label').has('timestamp','')).
valueMap()

If you expect very large results, you'd be better off using an OLAP query.

Can I make JG to use ES for traversal too ?

ES is only useful for initial / global vertex lookups.

Cheers,
Daniel

On Wed, Sep 20, 2017 at 7:41 AM, Suny <sahi...@...> wrote:
Hi,

I am using JG with Cassandra and ES.

I have a type attribute on all vertices based on which i can differentiate group of vertices.

The query i want to do is

:> g.V().has('type',textContains('car')).inE().has('timestamp','').inV().valueMap()

I created an index on type attribute. If I just query for :> g.V().has('type',textContains('car')) it is coming back very fast with result. If I add the traversal part it is slowing down.

So, JG is using ES to retrieve the vertices of type 'car' and then for traversal it is using just JG. Can I make JG to use ES for traversal too ?

If i add index on edge attribute 'timestamp', does this fasten the query ?

Thanks

--
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/5aeb2fdc-6199-45f4-8953-866118c2d81a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/94e4f5f4-8670-403d-9038-fd64382b61a5%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/e24480c8-52eb-4dd7-b511-b2722ad1bc2b%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Re: janusgraph solr cassandra GraphOfTheGodsFactory

Jason Plurad <plu...@...>

#844

Solr 4.3 is not a supported indexing backend.
http://docs.janusgraph.org/latest/version-compat.html

Show quoted text

On Friday, September 22, 2017 at 10:17:48 AM UTC-4, Ankur Goel wrote:

Yes i am able to see core using Solr UI. i have created core by adding directory in core folder and updating solr.xml.

Just to update i am using solr 4.3.

~AnkurG

On Fri, Sep 22, 2017 at 7:39 PM, Jason Plurad wrote:
That error indicates the core isn't available. Did you create it with

bin/solr create_core -c janusSolr -d $JANUSGRAPH_HOME/conf/solr -p 8983

Are you able to see the core in the Solr UI? http://localhost:8983/solr/#/~cores/janusSolr

Were you able to get Graph of the Gods example working with mixed indexes?

On Friday, September 22, 2017 at 9:26:35 AM UTC-4, Ankur Goel wrote:
Hi I am using the same configuration and have created core with name janusSolr for mixed index, when adding a vertex, getting below error:

org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://localhost:8983/solr: Expected mime type application/octet-stream but got text/html. <html>

<head>

<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"/>

<title>Error 404 Not Found</title>

</head>

<body><h2>HTTP ERROR 404</h2>

<p>Problem accessing /solr/janusSolr/update. Reason:

<pre> Not Found</pre></p><hr /><i><small>Powered by Jetty://</small></i><br/>

i checked following URL are working:
http://localhost:8983/solr/janusSolr/select
http://localhost:8983/solr/janusSolr/query

but http://localhost:8983/solr/janusSolr/update is throwing 404.

I am no solr expert, any advise?

~AnkurG

On Saturday, July 8, 2017 at 8:41:04 AM UTC+5:30, mahendiran chandrasekar wrote:
Error trying to load GraphOfTheGodsFactory into janusgraph-cassandra-solr

Steps:
1) download cassandra, copy cassandra.yaml from janusgraph/cassandra/cassandra.yaml -> cassandra_installationdir/conf/cassandra.yaml
2) download solr 5.2.1, copy all conf files from janusgraph/conf/solr/* -> solr_installation/server/solr/config_sets/basic_configs/*
3) Started cassandra and solr
4)janusgraph/bin/gremlin.sh

graph = JanusGraphFactory.open('conf/janusgraph-cassandra-solr.properties')
==>standardjanusgraph[cassandrathrift:[127.0.0.1]]

gremlin> GraphOfTheGodsFactory.load(graph)

causes this stack trace

15:07:39 ERROR org.janusgraph.diskstorage.solr.SolrIndex - Unable to save documents to Solr as one of the shape objects stored were not compatible with Solr.
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://localhost:8983/solr: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
<title>Error 404 Not Found</title>
</head>
<body><h2>HTTP ERROR 404</h2>
<p>Problem accessing /solr/edges/update. Reason:
<pre> Not Found</pre></p><hr><i><small>Powered by Jetty://</small></i><hr/>

</body>
</html>

at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:529)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:235)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:227)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:526)
at org.janusgraph.diskstorage.solr.SolrIndex.commitDocumentChanges(SolrIndex.java:407)
at org.janusgraph.diskstorage.solr.SolrIndex.mutate(SolrIndex.java:318)
at org.janusgraph.diskstorage.indexing.IndexTransaction$1.call(IndexTransaction.java:137)
at org.janusgraph.diskstorage.indexing.IndexTransaction$1.call(IndexTransaction.java:134)
at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:69)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:55)
at org.janusgraph.diskstorage.indexing.IndexTransaction.flushInternal(IndexTransaction.java:134)
at org.janusgraph.diskstorage.indexing.IndexTransaction.commit(IndexTransaction.java:115)
at org.janusgraph.diskstorage.BackendTransaction.commitIndexes(BackendTransaction.java:140)
at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:743)
at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1363)
at org.janusgraph.example.GraphOfTheGodsFactory.load(GraphOfTheGodsFactory.java:144)
at org.janusgraph.example.GraphOfTheGodsFactory.load(GraphOfTheGodsFactory.java:63)
at org.janusgraph.example.GraphOfTheGodsFactory$load.call(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
at groovysh_evaluate.run(groovysh_evaluate:3)
at org.codehaus.groovy.vmplugin.v7.IndyInterface.selectMethod(IndyInterface.java:232)
at org.codehaus.groovy.tools.shell.Interpreter.evaluate(Interpreter.groovy:70)
at org.codehaus.groovy.tools.shell.Groovysh.execute(Groovysh.groovy:190)
at org.apache.tinkerpop.gremlin.console.GremlinGroovysh.super$3$execute(GremlinGroovysh.groovy)
at sun.reflect.GeneratedMethodAccessor14.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1215)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuperN(ScriptBytecodeAdapter.java:132)
at org.apache.tinkerpop.gremlin.console.GremlinGroovysh.execute(GremlinGroovysh.groovy:72)
at org.codehaus.groovy.tools.shell.Shell.leftShift(Shell.groovy:122)
at org.codehaus.groovy.tools.shell.ShellRunner.work(ShellRunner.groovy:95)
at org.codehaus.groovy.tools.shell.InteractiveShellRunner.super$2$work(InteractiveShellRunner.groovy)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1215)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuperN(ScriptBytecodeAdapter.java:132)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuper0(ScriptBytecodeAdapter.java:152)
at org.codehaus.groovy.tools.shell.InteractiveShellRunner.work(InteractiveShellRunner.groovy:124)
at org.codehaus.groovy.tools.shell.ShellRunner.run(ShellRunner.groovy:59)
at org.codehaus.groovy.tools.shell.InteractiveShellRunner.super$2$run(InteractiveShellRunner.groovy)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1215)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuperN(ScriptBytecodeAdapter.java:132)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuper0(ScriptBytecodeAdapter.java:152)
at org.codehaus.groovy.tools.shell.InteractiveShellRunner.run(InteractiveShellRunner.groovy:83)
at org.codehaus.groovy.vmplugin.v7.IndyInterface.selectMethod(IndyInterface.java:232)
at org.apache.tinkerpop.gremlin.console.Console.<init>(Console.groovy:152)
at org.codehaus.groovy.vmplugin.v7.IndyInterface.selectMethod(IndyInterface.java:232)
at org.apache.tinkerpop.gremlin.console.Console.main(Console.groovy:455)
15:07:39 ERROR org.janusgraph.diskstorage.solr.SolrIndex - Details in failed document batch:
15:07:39 ERROR org.janusgraph.diskstorage.solr.SolrIndex - id:5y1-6h4-9hx-38w
15:07:39 ERROR org.janusgraph.diskstorage.solr.SolrIndex - place_g:POINT(23.7 38.1)
15:07:39 ERROR org.janusgraph.diskstorage.solr.SolrIndex - id:2s2-39s-b2t-cqw
15:07:39 ERROR org.janusgraph.diskstorage.solr.SolrIndex - reason_t:no fear of death
15:07:39 ERROR org.janusgraph.diskstorage.solr.SolrIndex - id:5xt-39k-b2t-6fs
15:07:39 ERROR org.janusgraph.diskstorage.solr.SolrIndex - reason_t:loves waves
15:07:39 ERROR org.janusgraph.diskstorage.solr.SolrIndex - id:6qh-6h4-9hx-6eo
15:07:39 ERROR org.janusgraph.diskstorage.solr.SolrIndex - place_g:POINT(22.0 39.0)
15:07:39 ERROR org.janusgraph.diskstorage.solr.SolrIndex - id:3yx-3bc-b2t-3cg
15:07:39 ERROR org.janusgraph.diskstorage.solr.SolrIndex - reason_t:loves fresh breezes
15:07:39 ERROR org.janusgraph.diskstorage.solr.SolrIndex - id:6c9-6h4-9hx-9l4
15:07:39 ERROR org.janusgraph.diskstorage.solr.SolrIndex - place_g:POINT(23.9 37.7)
15:07:52 ERROR org.janusgraph.diskstorage.solr.SolrIndex - Unable to save documents to Solr as one of the shape objects stored were not compatible with Solr.
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://localhost:8983/solr: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
<title>Error 404 Not Found</title>
</head>
<body><h2>HTTP ERROR 404</h2>
<p>Problem accessing /solr/edges/update. Reason:
<pre> Not Found</pre></p><hr><i><small>Powered by Jetty://</small></i><hr/>

</body>
</html>

at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:529)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:235)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:227)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:526)
at org.janusgraph.diskstorage.solr.SolrIndex.commitDocumentChanges(SolrIndex.java:407)
at org.janusgraph.diskstorage.solr.SolrIndex.mutate(SolrIndex.java:318)
at org.janusgraph.diskstorage.indexing.IndexTransaction$1.call(IndexTransaction.java:137)
at org.janusgraph.diskstorage.indexing.IndexTransaction$1.call(IndexTransaction.java:134)
at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:69)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:55)
at org.janusgraph.diskstorage.indexing.IndexTransaction.flushInternal(IndexTransaction.java:134)
at org.janusgraph.diskstorage.indexing.IndexTransaction.commit(IndexTransaction.java:115)
at org.janusgraph.diskstorage.BackendTransaction.commitIndexes(BackendTransaction.java:140)
at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:743)
at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1363)
at org.janusgraph.example.GraphOfTheGodsFactory.load(GraphOfTheGodsFactory.java:144)
at org.janusgraph.example.GraphOfTheGodsFactory.load(GraphOfTheGodsFactory.java:63)
at org.janusgraph.example.GraphOfTheGodsFactory$load.call(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
at groovysh_evaluate.run(groovysh_evaluate:3)
at org.codehaus.groovy.vmplugin.v7.IndyInterface.selectMethod(IndyInterface.java:232)
at org.codehaus.groovy.tools.shell.Interpreter.evaluate(Interpreter.groovy:70)
at org.codehaus.groovy.tools.shell.Groovysh.execute(Groovysh.groovy:190)
at org.apache.tinkerpop.gremlin.console.GremlinGroovysh.super$3$execute(GremlinGroovysh.groovy)
at sun.reflect.GeneratedMethodAccessor14.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1215)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuperN(ScriptBytecodeAdapter.java:132)
at org.apache.tinkerpop.gremlin.console.GremlinGroovysh.execute(GremlinGroovysh.groovy:72)
at org.codehaus.groovy.tools.shell.Shell.leftShift(Shell.groovy:122)
at org.codehaus.groovy.tools.shell.ShellRunner.work(ShellRunner.groovy:95)
at org.codehaus.groovy.tools.shell.InteractiveShellRunner.super$2$work(InteractiveShellRunner.groovy)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1215)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuperN(ScriptBytecodeAdapter.java:132)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuper0(ScriptBytecodeAdapter.java:152)
at org.codehaus.groovy.tools.shell.InteractiveShellRunner.work(InteractiveShellRunner.groovy:124)
at org.codehaus.groovy.tools.shell.ShellRunner.run(ShellRunner.groovy:59)
at org.codehaus.groovy.tools.shell.InteractiveShellRunner.super$2$run(InteractiveShellRunner.groovy)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1215)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuperN(ScriptBytecodeAdapter.java:132)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuper0(ScriptBytecodeAdapter.java:152)
at org.codehaus.groovy.tools.shell.InteractiveShellRunner.run(InteractiveShellRunner.groovy:83)
at org.codehaus.groovy.vmplugin.v7.IndyInterface.selectMethod(IndyInterface.java:232)
at org.apache.tinkerpop.gremlin.console.Console.<init>(Console.groovy:152)
at org.codehaus.groovy.vmplugin.v7.IndyInterface.selectMethod(IndyInterface.java:232)
at org.apache.tinkerpop.gremlin.console.Console.main(Console.groovy:455)
15:07:52 ERROR org.janusgraph.diskstorage.solr.SolrIndex - Details in failed document batch:
15:07:52 ERROR org.janusgraph.diskstorage.solr.SolrIndex - id:5y1-6h4-9hx-38w
15:07:52 ERROR org.janusgraph.diskstorage.solr.SolrIndex - place_g:POINT(23.7 38.1)
15:07:52 ERROR org.janusgraph.diskstorage.solr.SolrIndex - id:2s2-39s-b2t-cqw
15:07:52 ERROR org.janusgraph.diskstorage.solr.SolrIndex - reason_t:no fear of death
15:07:52 ERROR org.janusgraph.diskstorage.solr.SolrIndex - id:5xt-39k-b2t-6fs
15:07:52 ERROR org.janusgraph.diskstorage.solr.SolrIndex - reason_t:loves waves
15:07:52 ERROR org.janusgraph.diskstorage.solr.SolrIndex - id:6qh-6h4-9hx-6eo
15:07:52 ERROR org.janusgraph.diskstorage.solr.SolrIndex - place_g:POINT(22.0 39.0)
15:07:52 ERROR org.janusgraph.diskstorage.solr.SolrIndex - id:3yx-3bc-b2t-3cg
15:07:52 ERROR org.janusgraph.diskstorage.solr.SolrIndex - reason_t:loves fresh breezes
15:07:52 ERROR org.janusgraph.diskstorage.solr.SolrIndex - id:6c9-6h4-9hx-9l4
15:07:52 ERROR org.janusgraph.diskstorage.solr.SolrIndex - place_g:POINT(23.9 37.7)

--
You received this message because you are subscribed to a topic in the Google Groups "JanusGraph users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/janusgraph-users/wWUu6Y6CKAY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to janusgraph-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/6a7164fe-46da-49d6-ac38-bdde3b38df56%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Re: Using ES for traversal queries ?

Jason Plurad <plu...@...>

#843

How did you define the 'timestamp' property and the vertex-centric index that uses it?
Seems strange to me that you're looking for has('timestamp, '') rather than comparing Long values.

Show quoted text

On Friday, September 22, 2017 at 10:12:44 AM UTC-4, Suny wrote:

yes. the first time i query it took 129826 ms, and then 600-700 ms later on (which i believe is because of caching). Without that it is taking around 128000ms

On Friday, September 22, 2017 at 10:10:57 AM UTC-4, Jason Plurad wrote:
is this slow?

g.V().has('type',textContains('car')).inE().has('timestamp',eq('')).inV().id()

On Friday, September 22, 2017 at 10:07:26 AM UTC-4, Suny wrote:
Thanks. I am dealing with count here. I need the attributes list on those vertices.

g.V().has('type',textContains('car')) - This is coming out very fast, which i assume is because of index in ES.

g.V().has('type',textContains('car')).inE().has('timestamp',eq('')).inV().valueMap() - This is slow. I have 1500 vertices and each vertex has about 3-5 attributes on it. I implemented vertex-centric index, on timestamp property but not sure if it is being used.

On Friday, September 22, 2017 at 8:54:54 AM UTC-4, Jason Plurad wrote:
No, explain() doesn't give any clear indication currently whether an index will be utilized. I opened up an issue for that.

What counts are you dealing with here?

g.V().has('type',textContains('car')).count()
g.V().has('type',textContains('car')).inE().count()
g.V().has('type',textContains('car')).inE().has('timestamp',eq('')).count()

On Thursday, September 21, 2017 at 1:59:37 PM UTC-4, Suny wrote:
I implemented vertex-centric index on edge label.

Here is the query i am doing

g.V().has('type',textContains('car')).inE().has('timestamp',eq('')).inV().valueMap()

The documentation says - JanusGraph is intelligent enough to use vertex-centric indices when available.

So can i assume that Janusgraph uses the vertex-centric index for this query ?

On Thursday, September 21, 2017 at 11:05:37 AM UTC-4, Daniel Kuppitz wrote:
Only in the underlying storage backend.

See: http://docs.janusgraph.org/0.2.0-SNAPSHOT/data-model.html

Cheers,
Daniel

On Thu, Sep 21, 2017 at 7:57 AM, Suny <sahi...@...> wrote:
Thanks. Is the vertex-centric index not stored in Elastic search. Is it only in JG ?

On Thursday, September 21, 2017 at 10:28:47 AM UTC-4, Daniel Kuppitz wrote:
Thanks, All the in edges will have same label. Does it still traverse through all edges or directly find the edges with empty timestamp ?

I assume that Janus can only leverage a vertex centric index, if you specify the label, since that was the case in Titan and I don't think I heard about any changes in this area.

So in cases where i need to do lot of traversal, ES is not helpful ?

Again, ES is helpful for the initial vertex lookup. Once you start to traverse through the graph, you can only rely on vertex centric indices (or a good model that doesn't require much filtering).

Also can you explain a bit more on OLAP query

You can read up on OLAP here: http://docs.janusgraph.org/0.2.0-SNAPSHOT/hadoop-tp3.html
...and here: http://tinkerpop.apache.org/docs/current/reference/#_olap_hadoop_gremlin

Cheers,
Daniel

On Thu, Sep 21, 2017 at 7:07 AM, Suny <sahi...@...> wrote:
Thanks, All the in edges will have same label. Does it still traverse through all edges or directly find the edges with empty timestamp ?

So in cases where i need to do lot of traversal, ES is not helpful ?

Also can you explain a bit more on OLAP query

On Wednesday, September 20, 2017 at 2:35:27 PM UTC-4, Daniel Kuppitz wrote:
You should
provide an edge label for inE()
have a vertex centric index on timestamp
use a simple filter, instead of traversing back to the previous vertex (unless you rely on duplicates in your result)
Something like this:

g.V().has('type', textContains('car')).
filter(inE('edge-label').has('timestamp','')).
valueMap()

If you expect very large results, you'd be better off using an OLAP query.

Can I make JG to use ES for traversal too ?

ES is only useful for initial / global vertex lookups.

Cheers,
Daniel

On Wed, Sep 20, 2017 at 7:41 AM, Suny <sahi...@...> wrote:
Hi,

I am using JG with Cassandra and ES.

I have a type attribute on all vertices based on which i can differentiate group of vertices.

The query i want to do is

:> g.V().has('type',textContains('car')).inE().has('timestamp','').inV().valueMap()

I created an index on type attribute. If I just query for :> g.V().has('type',textContains('car')) it is coming back very fast with result. If I add the traversal part it is slowing down.

So, JG is using ES to retrieve the vertices of type 'car' and then for traversal it is using just JG. Can I make JG to use ES for traversal too ?

If i add index on edge attribute 'timestamp', does this fasten the query ?

Thanks

--
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/5aeb2fdc-6199-45f4-8953-866118c2d81a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/94e4f5f4-8670-403d-9038-fd64382b61a5%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/e24480c8-52eb-4dd7-b511-b2722ad1bc2b%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Re: janusgraph solr cassandra GraphOfTheGodsFactory

Ankur Goel <ankur...@...>

#852

Yes i am able to see core using Solr UI. i have created core by adding directory in core folder and updating solr.xml.

Just to update i am using solr 4.3.

~AnkurG

Show quoted text

On Fri, Sep 22, 2017 at 7:39 PM, Jason Plurad <plu...@...> wrote:

That error indicates the core isn't available. Did you create it with

bin/solr create_core -c janusSolr -d $JANUSGRAPH_HOME/conf/solr -p 8983

Are you able to see the core in the Solr UI? http://localhost:8983/solr/#/~cores/janusSolr

Were you able to get Graph of the Gods example working with mixed indexes?

On Friday, September 22, 2017 at 9:26:35 AM UTC-4, Ankur Goel wrote:
Hi I am using the same configuration and have created core with name janusSolr for mixed index, when adding a vertex, getting below error:

org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://localhost:8983/solr: Expected mime type application/octet-stream but got text/html. <html>

<head>

<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"/>

<title>Error 404 Not Found</title>

</head>

<body><h2>HTTP ERROR 404</h2>

<p>Problem accessing /solr/janusSolr/update. Reason:

<pre> Not Found</pre></p><hr /><i><small>Powered by Jetty://</small></i><br/>

i checked following URL are working:
http://localhost:8983/solr/janusSolr/select
http://localhost:8983/solr/janusSolr/query

but http://localhost:8983/solr/janusSolr/update is throwing 404.

I am no solr expert, any advise?

~AnkurG

On Saturday, July 8, 2017 at 8:41:04 AM UTC+5:30, mahendiran chandrasekar wrote:
Error trying to load GraphOfTheGodsFactory into janusgraph-cassandra-solr

Steps:
1) download cassandra, copy cassandra.yaml from janusgraph/cassandra/cassandra.yaml -> cassandra_installationdir/conf/cassandra.yaml
2) download solr 5.2.1, copy all conf files from janusgraph/conf/solr/* -> solr_installation/server/solr/config_sets/basic_configs/*
3) Started cassandra and solr
4)janusgraph/bin/gremlin.sh

graph = JanusGraphFactory.open('conf/janusgraph-cassandra-solr.properties')
==>standardjanusgraph[cassandrathrift:[127.0.0.1]]

gremlin> GraphOfTheGodsFactory.load(graph)

causes this stack trace

15:07:39 ERROR org.janusgraph.diskstorage.solr.SolrIndex - Unable to save documents to Solr as one of the shape objects stored were not compatible with Solr.
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://localhost:8983/solr: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
<title>Error 404 Not Found</title>
</head>
<body><h2>HTTP ERROR 404</h2>
<p>Problem accessing /solr/edges/update. Reason:
<pre> Not Found</pre></p><hr><i><small>Powered by Jetty://</small></i><hr/>

</body>
</html>

at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:529)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:235)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:227)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:526)
at org.janusgraph.diskstorage.solr.SolrIndex.commitDocumentChanges(SolrIndex.java:407)
at org.janusgraph.diskstorage.solr.SolrIndex.mutate(SolrIndex.java:318)
at org.janusgraph.diskstorage.indexing.IndexTransaction$1.call(IndexTransaction.java:137)
at org.janusgraph.diskstorage.indexing.IndexTransaction$1.call(IndexTransaction.java:134)
at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:69)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:55)
at org.janusgraph.diskstorage.indexing.IndexTransaction.flushInternal(IndexTransaction.java:134)
at org.janusgraph.diskstorage.indexing.IndexTransaction.commit(IndexTransaction.java:115)
at org.janusgraph.diskstorage.BackendTransaction.commitIndexes(BackendTransaction.java:140)
at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:743)
at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1363)
at org.janusgraph.example.GraphOfTheGodsFactory.load(GraphOfTheGodsFactory.java:144)
at org.janusgraph.example.GraphOfTheGodsFactory.load(GraphOfTheGodsFactory.java:63)
at org.janusgraph.example.GraphOfTheGodsFactory$load.call(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
at groovysh_evaluate.run(groovysh_evaluate:3)
at org.codehaus.groovy.vmplugin.v7.IndyInterface.selectMethod(IndyInterface.java:232)
at org.codehaus.groovy.tools.shell.Interpreter.evaluate(Interpreter.groovy:70)
at org.codehaus.groovy.tools.shell.Groovysh.execute(Groovysh.groovy:190)
at org.apache.tinkerpop.gremlin.console.GremlinGroovysh.super$3$execute(GremlinGroovysh.groovy)
at sun.reflect.GeneratedMethodAccessor14.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1215)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuperN(ScriptBytecodeAdapter.java:132)
at org.apache.tinkerpop.gremlin.console.GremlinGroovysh.execute(GremlinGroovysh.groovy:72)
at org.codehaus.groovy.tools.shell.Shell.leftShift(Shell.groovy:122)
at org.codehaus.groovy.tools.shell.ShellRunner.work(ShellRunner.groovy:95)
at org.codehaus.groovy.tools.shell.InteractiveShellRunner.super$2$work(InteractiveShellRunner.groovy)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1215)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuperN(ScriptBytecodeAdapter.java:132)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuper0(ScriptBytecodeAdapter.java:152)
at org.codehaus.groovy.tools.shell.InteractiveShellRunner.work(InteractiveShellRunner.groovy:124)
at org.codehaus.groovy.tools.shell.ShellRunner.run(ShellRunner.groovy:59)
at org.codehaus.groovy.tools.shell.InteractiveShellRunner.super$2$run(InteractiveShellRunner.groovy)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1215)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuperN(ScriptBytecodeAdapter.java:132)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuper0(ScriptBytecodeAdapter.java:152)
at org.codehaus.groovy.tools.shell.InteractiveShellRunner.run(InteractiveShellRunner.groovy:83)
at org.codehaus.groovy.vmplugin.v7.IndyInterface.selectMethod(IndyInterface.java:232)
at org.apache.tinkerpop.gremlin.console.Console.<init>(Console.groovy:152)
at org.codehaus.groovy.vmplugin.v7.IndyInterface.selectMethod(IndyInterface.java:232)
at org.apache.tinkerpop.gremlin.console.Console.main(Console.groovy:455)
15:07:39 ERROR org.janusgraph.diskstorage.solr.SolrIndex - Details in failed document batch:
15:07:39 ERROR org.janusgraph.diskstorage.solr.SolrIndex - id:5y1-6h4-9hx-38w
15:07:39 ERROR org.janusgraph.diskstorage.solr.SolrIndex - place_g:POINT(23.7 38.1)
15:07:39 ERROR org.janusgraph.diskstorage.solr.SolrIndex - id:2s2-39s-b2t-cqw
15:07:39 ERROR org.janusgraph.diskstorage.solr.SolrIndex - reason_t:no fear of death
15:07:39 ERROR org.janusgraph.diskstorage.solr.SolrIndex - id:5xt-39k-b2t-6fs
15:07:39 ERROR org.janusgraph.diskstorage.solr.SolrIndex - reason_t:loves waves
15:07:39 ERROR org.janusgraph.diskstorage.solr.SolrIndex - id:6qh-6h4-9hx-6eo
15:07:39 ERROR org.janusgraph.diskstorage.solr.SolrIndex - place_g:POINT(22.0 39.0)
15:07:39 ERROR org.janusgraph.diskstorage.solr.SolrIndex - id:3yx-3bc-b2t-3cg
15:07:39 ERROR org.janusgraph.diskstorage.solr.SolrIndex - reason_t:loves fresh breezes
15:07:39 ERROR org.janusgraph.diskstorage.solr.SolrIndex - id:6c9-6h4-9hx-9l4
15:07:39 ERROR org.janusgraph.diskstorage.solr.SolrIndex - place_g:POINT(23.9 37.7)
15:07:52 ERROR org.janusgraph.diskstorage.solr.SolrIndex - Unable to save documents to Solr as one of the shape objects stored were not compatible with Solr.
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://localhost:8983/solr: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
<title>Error 404 Not Found</title>
</head>
<body><h2>HTTP ERROR 404</h2>
<p>Problem accessing /solr/edges/update. Reason:
<pre> Not Found</pre></p><hr><i><small>Powered by Jetty://</small></i><hr/>

</body>
</html>

at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:529)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:235)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:227)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:526)
at org.janusgraph.diskstorage.solr.SolrIndex.commitDocumentChanges(SolrIndex.java:407)
at org.janusgraph.diskstorage.solr.SolrIndex.mutate(SolrIndex.java:318)
at org.janusgraph.diskstorage.indexing.IndexTransaction$1.call(IndexTransaction.java:137)
at org.janusgraph.diskstorage.indexing.IndexTransaction$1.call(IndexTransaction.java:134)
at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:69)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:55)
at org.janusgraph.diskstorage.indexing.IndexTransaction.flushInternal(IndexTransaction.java:134)
at org.janusgraph.diskstorage.indexing.IndexTransaction.commit(IndexTransaction.java:115)
at org.janusgraph.diskstorage.BackendTransaction.commitIndexes(BackendTransaction.java:140)
at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:743)
at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1363)
at org.janusgraph.example.GraphOfTheGodsFactory.load(GraphOfTheGodsFactory.java:144)
at org.janusgraph.example.GraphOfTheGodsFactory.load(GraphOfTheGodsFactory.java:63)
at org.janusgraph.example.GraphOfTheGodsFactory$load.call(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
at groovysh_evaluate.run(groovysh_evaluate:3)
at org.codehaus.groovy.vmplugin.v7.IndyInterface.selectMethod(IndyInterface.java:232)
at org.codehaus.groovy.tools.shell.Interpreter.evaluate(Interpreter.groovy:70)
at org.codehaus.groovy.tools.shell.Groovysh.execute(Groovysh.groovy:190)
at org.apache.tinkerpop.gremlin.console.GremlinGroovysh.super$3$execute(GremlinGroovysh.groovy)
at sun.reflect.GeneratedMethodAccessor14.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1215)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuperN(ScriptBytecodeAdapter.java:132)
at org.apache.tinkerpop.gremlin.console.GremlinGroovysh.execute(GremlinGroovysh.groovy:72)
at org.codehaus.groovy.tools.shell.Shell.leftShift(Shell.groovy:122)
at org.codehaus.groovy.tools.shell.ShellRunner.work(ShellRunner.groovy:95)
at org.codehaus.groovy.tools.shell.InteractiveShellRunner.super$2$work(InteractiveShellRunner.groovy)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1215)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuperN(ScriptBytecodeAdapter.java:132)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuper0(ScriptBytecodeAdapter.java:152)
at org.codehaus.groovy.tools.shell.InteractiveShellRunner.work(InteractiveShellRunner.groovy:124)
at org.codehaus.groovy.tools.shell.ShellRunner.run(ShellRunner.groovy:59)
at org.codehaus.groovy.tools.shell.InteractiveShellRunner.super$2$run(InteractiveShellRunner.groovy)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1215)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuperN(ScriptBytecodeAdapter.java:132)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuper0(ScriptBytecodeAdapter.java:152)
at org.codehaus.groovy.tools.shell.InteractiveShellRunner.run(InteractiveShellRunner.groovy:83)
at org.codehaus.groovy.vmplugin.v7.IndyInterface.selectMethod(IndyInterface.java:232)
at org.apache.tinkerpop.gremlin.console.Console.<init>(Console.groovy:152)
at org.codehaus.groovy.vmplugin.v7.IndyInterface.selectMethod(IndyInterface.java:232)
at org.apache.tinkerpop.gremlin.console.Console.main(Console.groovy:455)
15:07:52 ERROR org.janusgraph.diskstorage.solr.SolrIndex - Details in failed document batch:
15:07:52 ERROR org.janusgraph.diskstorage.solr.SolrIndex - id:5y1-6h4-9hx-38w
15:07:52 ERROR org.janusgraph.diskstorage.solr.SolrIndex - place_g:POINT(23.7 38.1)
15:07:52 ERROR org.janusgraph.diskstorage.solr.SolrIndex - id:2s2-39s-b2t-cqw
15:07:52 ERROR org.janusgraph.diskstorage.solr.SolrIndex - reason_t:no fear of death
15:07:52 ERROR org.janusgraph.diskstorage.solr.SolrIndex - id:5xt-39k-b2t-6fs
15:07:52 ERROR org.janusgraph.diskstorage.solr.SolrIndex - reason_t:loves waves
15:07:52 ERROR org.janusgraph.diskstorage.solr.SolrIndex - id:6qh-6h4-9hx-6eo
15:07:52 ERROR org.janusgraph.diskstorage.solr.SolrIndex - place_g:POINT(22.0 39.0)
15:07:52 ERROR org.janusgraph.diskstorage.solr.SolrIndex - id:3yx-3bc-b2t-3cg
15:07:52 ERROR org.janusgraph.diskstorage.solr.SolrIndex - reason_t:loves fresh breezes
15:07:52 ERROR org.janusgraph.diskstorage.solr.SolrIndex - id:6c9-6h4-9hx-9l4
15:07:52 ERROR org.janusgraph.diskstorage.solr.SolrIndex - place_g:POINT(23.9 37.7)

--
You received this message because you are subscribed to a topic in the Google Groups "JanusGraph users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/janusgraph-users/wWUu6Y6CKAY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to janusgraph-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/6a7164fe-46da-49d6-ac38-bdde3b38df56%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Re: ES with JG

Robert Dale <rob...@...>

#851

How much time is 'slow'?. Are your indexes actually enabled? What is the byte size of the results? Have you ruled out resource issues - cpu, ram, disk, network?

What does `g.V().has('type',textContains('site')).valueMap().profile()` show?

Robert Dale

Show quoted text

On Fri, Sep 22, 2017 at 10:10 AM, Suny <sahithiy...@...> wrote:

values are of small size. like strings of length 20.

On Friday, September 22, 2017 at 10:03:34 AM UTC-4, Suny wrote:
Each vertex has 3-5 attributes on it.

On Friday, September 22, 2017 at 9:02:15 AM UTC-4, Jason Plurad wrote:
1500 vertices is a fair amount, but how much data is getting returned in the valueMap? Is there a large number of properties in the map or perhaps are the values very large? Very easily could be the cost of serializing that map. In another post you were asking about string size restrictions...

On Thursday, September 21, 2017 at 2:40:06 PM UTC-4, Suny wrote:
I implemented a mixed index on 'type' attribute on vertex.

Whenever i query for :> g.V().has('type',textContains('car'))

the result comes out really really fast.

When I do :> g.V().has('type',textContains('site')).valueMap() it is taking so much time to retrieve data. I have 1500 vertices of type car.

Is it because for 1st query it gets the result using ES for the second query it gets vertex id's from ES but for getting attributes it needs to hit JG again ?

Is there a way to fasten up the second query ?

--
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/8a3c629b-10c4-4d55-8f8f-8e5397f6ff87%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Re: Using ES for traversal queries ?

Suny <sahithiy...@...>

#842

yes. the first time i query it took 129826 ms, and then 600-700 ms later on (which i believe is because of caching). Without that it is taking around 128000ms

Show quoted text

On Friday, September 22, 2017 at 10:10:57 AM UTC-4, Jason Plurad wrote:

is this slow?

g.V().has('type',textContains('car')).inE().has('timestamp',eq('')).inV().id()

On Friday, September 22, 2017 at 10:07:26 AM UTC-4, Suny wrote:
Thanks. I am dealing with count here. I need the attributes list on those vertices.

g.V().has('type',textContains('car')) - This is coming out very fast, which i assume is because of index in ES.

g.V().has('type',textContains('car')).inE().has('timestamp',eq('')).inV().valueMap() - This is slow. I have 1500 vertices and each vertex has about 3-5 attributes on it. I implemented vertex-centric index, on timestamp property but not sure if it is being used.

On Friday, September 22, 2017 at 8:54:54 AM UTC-4, Jason Plurad wrote:
No, explain() doesn't give any clear indication currently whether an index will be utilized. I opened up an issue for that.

What counts are you dealing with here?

g.V().has('type',textContains('car')).count()
g.V().has('type',textContains('car')).inE().count()
g.V().has('type',textContains('car')).inE().has('timestamp',eq('')).count()

On Thursday, September 21, 2017 at 1:59:37 PM UTC-4, Suny wrote:
I implemented vertex-centric index on edge label.

Here is the query i am doing

g.V().has('type',textContains('car')).inE().has('timestamp',eq('')).inV().valueMap()

The documentation says - JanusGraph is intelligent enough to use vertex-centric indices when available.

So can i assume that Janusgraph uses the vertex-centric index for this query ?

On Thursday, September 21, 2017 at 11:05:37 AM UTC-4, Daniel Kuppitz wrote:
Only in the underlying storage backend.

See: http://docs.janusgraph.org/0.2.0-SNAPSHOT/data-model.html

Cheers,
Daniel

On Thu, Sep 21, 2017 at 7:57 AM, Suny <sahi...@...> wrote:
Thanks. Is the vertex-centric index not stored in Elastic search. Is it only in JG ?

On Thursday, September 21, 2017 at 10:28:47 AM UTC-4, Daniel Kuppitz wrote:
Thanks, All the in edges will have same label. Does it still traverse through all edges or directly find the edges with empty timestamp ?

I assume that Janus can only leverage a vertex centric index, if you specify the label, since that was the case in Titan and I don't think I heard about any changes in this area.

So in cases where i need to do lot of traversal, ES is not helpful ?

Again, ES is helpful for the initial vertex lookup. Once you start to traverse through the graph, you can only rely on vertex centric indices (or a good model that doesn't require much filtering).

Also can you explain a bit more on OLAP query

You can read up on OLAP here: http://docs.janusgraph.org/0.2.0-SNAPSHOT/hadoop-tp3.html
...and here: http://tinkerpop.apache.org/docs/current/reference/#_olap_hadoop_gremlin

Cheers,
Daniel

On Thu, Sep 21, 2017 at 7:07 AM, Suny <sahi...@...> wrote:
Thanks, All the in edges will have same label. Does it still traverse through all edges or directly find the edges with empty timestamp ?

So in cases where i need to do lot of traversal, ES is not helpful ?

Also can you explain a bit more on OLAP query

On Wednesday, September 20, 2017 at 2:35:27 PM UTC-4, Daniel Kuppitz wrote:
You should
provide an edge label for inE()
have a vertex centric index on timestamp
use a simple filter, instead of traversing back to the previous vertex (unless you rely on duplicates in your result)
Something like this:

g.V().has('type', textContains('car')).
filter(inE('edge-label').has('timestamp','')).
valueMap()

If you expect very large results, you'd be better off using an OLAP query.

Can I make JG to use ES for traversal too ?

ES is only useful for initial / global vertex lookups.

Cheers,
Daniel

On Wed, Sep 20, 2017 at 7:41 AM, Suny <sahi...@...> wrote:
Hi,

I am using JG with Cassandra and ES.

I have a type attribute on all vertices based on which i can differentiate group of vertices.

The query i want to do is

:> g.V().has('type',textContains('car')).inE().has('timestamp','').inV().valueMap()

I created an index on type attribute. If I just query for :> g.V().has('type',textContains('car')) it is coming back very fast with result. If I add the traversal part it is slowing down.

So, JG is using ES to retrieve the vertices of type 'car' and then for traversal it is using just JG. Can I make JG to use ES for traversal too ?

If i add index on edge attribute 'timestamp', does this fasten the query ?

Thanks

--
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/5aeb2fdc-6199-45f4-8953-866118c2d81a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/94e4f5f4-8670-403d-9038-fd64382b61a5%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/e24480c8-52eb-4dd7-b511-b2722ad1bc2b%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Re: Using ES for traversal queries ?

Jason Plurad <plu...@...>

#841

is this slow?

g.V().has('type',textContains('car')).inE().has('timestamp',eq('')).inV().id()

Show quoted text

On Friday, September 22, 2017 at 10:07:26 AM UTC-4, Suny wrote:

Thanks. I am dealing with count here. I need the attributes list on those vertices.

g.V().has('type',textContains('car')) - This is coming out very fast, which i assume is because of index in ES.

g.V().has('type',textContains('car')).inE().has('timestamp',eq('')).inV().valueMap() - This is slow. I have 1500 vertices and each vertex has about 3-5 attributes on it. I implemented vertex-centric index, on timestamp property but not sure if it is being used.

On Friday, September 22, 2017 at 8:54:54 AM UTC-4, Jason Plurad wrote:
No, explain() doesn't give any clear indication currently whether an index will be utilized. I opened up an issue for that.

What counts are you dealing with here?

g.V().has('type',textContains('car')).count()
g.V().has('type',textContains('car')).inE().count()
g.V().has('type',textContains('car')).inE().has('timestamp',eq('')).count()

On Thursday, September 21, 2017 at 1:59:37 PM UTC-4, Suny wrote:
I implemented vertex-centric index on edge label.

Here is the query i am doing

g.V().has('type',textContains('car')).inE().has('timestamp',eq('')).inV().valueMap()

The documentation says - JanusGraph is intelligent enough to use vertex-centric indices when available.

So can i assume that Janusgraph uses the vertex-centric index for this query ?

On Thursday, September 21, 2017 at 11:05:37 AM UTC-4, Daniel Kuppitz wrote:
Only in the underlying storage backend.

See: http://docs.janusgraph.org/0.2.0-SNAPSHOT/data-model.html

Cheers,
Daniel

On Thu, Sep 21, 2017 at 7:57 AM, Suny <sahi...@...> wrote:
Thanks. Is the vertex-centric index not stored in Elastic search. Is it only in JG ?

On Thursday, September 21, 2017 at 10:28:47 AM UTC-4, Daniel Kuppitz wrote:
Thanks, All the in edges will have same label. Does it still traverse through all edges or directly find the edges with empty timestamp ?

I assume that Janus can only leverage a vertex centric index, if you specify the label, since that was the case in Titan and I don't think I heard about any changes in this area.

So in cases where i need to do lot of traversal, ES is not helpful ?

Again, ES is helpful for the initial vertex lookup. Once you start to traverse through the graph, you can only rely on vertex centric indices (or a good model that doesn't require much filtering).

Also can you explain a bit more on OLAP query

You can read up on OLAP here: http://docs.janusgraph.org/0.2.0-SNAPSHOT/hadoop-tp3.html
...and here: http://tinkerpop.apache.org/docs/current/reference/#_olap_hadoop_gremlin

Cheers,
Daniel

On Thu, Sep 21, 2017 at 7:07 AM, Suny <sahi...@...> wrote:
Thanks, All the in edges will have same label. Does it still traverse through all edges or directly find the edges with empty timestamp ?

So in cases where i need to do lot of traversal, ES is not helpful ?

Also can you explain a bit more on OLAP query

On Wednesday, September 20, 2017 at 2:35:27 PM UTC-4, Daniel Kuppitz wrote:
You should
provide an edge label for inE()
have a vertex centric index on timestamp
use a simple filter, instead of traversing back to the previous vertex (unless you rely on duplicates in your result)
Something like this:

g.V().has('type', textContains('car')).
filter(inE('edge-label').has('timestamp','')).
valueMap()

If you expect very large results, you'd be better off using an OLAP query.

Can I make JG to use ES for traversal too ?

ES is only useful for initial / global vertex lookups.

Cheers,
Daniel

On Wed, Sep 20, 2017 at 7:41 AM, Suny <sahi...@...> wrote:
Hi,

I am using JG with Cassandra and ES.

I have a type attribute on all vertices based on which i can differentiate group of vertices.

The query i want to do is

:> g.V().has('type',textContains('car')).inE().has('timestamp','').inV().valueMap()

I created an index on type attribute. If I just query for :> g.V().has('type',textContains('car')) it is coming back very fast with result. If I add the traversal part it is slowing down.

So, JG is using ES to retrieve the vertices of type 'car' and then for traversal it is using just JG. Can I make JG to use ES for traversal too ?

If i add index on edge attribute 'timestamp', does this fasten the query ?

Thanks

--
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/5aeb2fdc-6199-45f4-8953-866118c2d81a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/94e4f5f4-8670-403d-9038-fd64382b61a5%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/e24480c8-52eb-4dd7-b511-b2722ad1bc2b%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Re: ES with JG

Suny <sahithiy...@...>

#840

values are of small size. like strings of length 20.

Show quoted text

On Friday, September 22, 2017 at 10:03:34 AM UTC-4, Suny wrote:

Each vertex has 3-5 attributes on it.

On Friday, September 22, 2017 at 9:02:15 AM UTC-4, Jason Plurad wrote:
1500 vertices is a fair amount, but how much data is getting returned in the valueMap? Is there a large number of properties in the map or perhaps are the values very large? Very easily could be the cost of serializing that map. In another post you were asking about string size restrictions...

On Thursday, September 21, 2017 at 2:40:06 PM UTC-4, Suny wrote:
I implemented a mixed index on 'type' attribute on vertex.

Whenever i query for :> g.V().has('type',textContains('car'))

the result comes out really really fast.

When I do :> g.V().has('type',textContains('site')).valueMap() it is taking so much time to retrieve data. I have 1500 vertices of type car.

Is it because for 1st query it gets the result using ES for the second query it gets vertex id's from ES but for getting attributes it needs to hit JG again ?

Is there a way to fasten up the second query ?

Re: janusgraph solr cassandra GraphOfTheGodsFactory

Jason Plurad <plu...@...>

#839

That error indicates the core isn't available. Did you create it with

bin/solr create_core -c janusSolr -d $JANUSGRAPH_HOME/conf/solr -p 8983

Are you able to see the core in the Solr UI? http://localhost:8983/solr/#/~cores/janusSolr

Were you able to get Graph of the Gods example working with mixed indexes?

Show quoted text

On Friday, September 22, 2017 at 9:26:35 AM UTC-4, Ankur Goel wrote:

Hi I am using the same configuration and have created core with name janusSolr for mixed index, when adding a vertex, getting below error:

org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://localhost:8983/solr: Expected mime type application/octet-stream but got text/html. <html>

<head>

<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"/>

<title>Error 404 Not Found</title>

</head>

<body><h2>HTTP ERROR 404</h2>

<p>Problem accessing /solr/janusSolr/update. Reason:

<pre> Not Found</pre></p><hr /><i><small>Powered by Jetty://</small></i><br/>

i checked following URL are working:
http://localhost:8983/solr/janusSolr/select
http://localhost:8983/solr/janusSolr/query

but http://localhost:8983/solr/janusSolr/update is throwing 404.

I am no solr expert, any advise?

~AnkurG

On Saturday, July 8, 2017 at 8:41:04 AM UTC+5:30, mahendiran chandrasekar wrote:
Error trying to load GraphOfTheGodsFactory into janusgraph-cassandra-solr

Steps:
1) download cassandra, copy cassandra.yaml from janusgraph/cassandra/cassandra.yaml -> cassandra_installationdir/conf/cassandra.yaml
2) download solr 5.2.1, copy all conf files from janusgraph/conf/solr/* -> solr_installation/server/solr/config_sets/basic_configs/*
3) Started cassandra and solr
4)janusgraph/bin/gremlin.sh

graph = JanusGraphFactory.open('conf/janusgraph-cassandra-solr.properties')
==>standardjanusgraph[cassandrathrift:[127.0.0.1]]

gremlin> GraphOfTheGodsFactory.load(graph)

causes this stack trace

15:07:39 ERROR org.janusgraph.diskstorage.solr.SolrIndex - Unable to save documents to Solr as one of the shape objects stored were not compatible with Solr.
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://localhost:8983/solr: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
<title>Error 404 Not Found</title>
</head>
<body><h2>HTTP ERROR 404</h2>
<p>Problem accessing /solr/edges/update. Reason:
<pre> Not Found</pre></p><hr><i><small>Powered by Jetty://</small></i><hr/>

</body>
</html>

at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:529)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:235)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:227)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:526)
at org.janusgraph.diskstorage.solr.SolrIndex.commitDocumentChanges(SolrIndex.java:407)
at org.janusgraph.diskstorage.solr.SolrIndex.mutate(SolrIndex.java:318)
at org.janusgraph.diskstorage.indexing.IndexTransaction$1.call(IndexTransaction.java:137)
at org.janusgraph.diskstorage.indexing.IndexTransaction$1.call(IndexTransaction.java:134)
at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:69)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:55)
at org.janusgraph.diskstorage.indexing.IndexTransaction.flushInternal(IndexTransaction.java:134)
at org.janusgraph.diskstorage.indexing.IndexTransaction.commit(IndexTransaction.java:115)
at org.janusgraph.diskstorage.BackendTransaction.commitIndexes(BackendTransaction.java:140)
at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:743)
at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1363)
at org.janusgraph.example.GraphOfTheGodsFactory.load(GraphOfTheGodsFactory.java:144)
at org.janusgraph.example.GraphOfTheGodsFactory.load(GraphOfTheGodsFactory.java:63)
at org.janusgraph.example.GraphOfTheGodsFactory$load.call(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
at groovysh_evaluate.run(groovysh_evaluate:3)
at org.codehaus.groovy.vmplugin.v7.IndyInterface.selectMethod(IndyInterface.java:232)
at org.codehaus.groovy.tools.shell.Interpreter.evaluate(Interpreter.groovy:70)
at org.codehaus.groovy.tools.shell.Groovysh.execute(Groovysh.groovy:190)
at org.apache.tinkerpop.gremlin.console.GremlinGroovysh.super$3$execute(GremlinGroovysh.groovy)
at sun.reflect.GeneratedMethodAccessor14.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1215)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuperN(ScriptBytecodeAdapter.java:132)
at org.apache.tinkerpop.gremlin.console.GremlinGroovysh.execute(GremlinGroovysh.groovy:72)
at org.codehaus.groovy.tools.shell.Shell.leftShift(Shell.groovy:122)
at org.codehaus.groovy.tools.shell.ShellRunner.work(ShellRunner.groovy:95)
at org.codehaus.groovy.tools.shell.InteractiveShellRunner.super$2$work(InteractiveShellRunner.groovy)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1215)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuperN(ScriptBytecodeAdapter.java:132)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuper0(ScriptBytecodeAdapter.java:152)
at org.codehaus.groovy.tools.shell.InteractiveShellRunner.work(InteractiveShellRunner.groovy:124)
at org.codehaus.groovy.tools.shell.ShellRunner.run(ShellRunner.groovy:59)
at org.codehaus.groovy.tools.shell.InteractiveShellRunner.super$2$run(InteractiveShellRunner.groovy)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1215)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuperN(ScriptBytecodeAdapter.java:132)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuper0(ScriptBytecodeAdapter.java:152)
at org.codehaus.groovy.tools.shell.InteractiveShellRunner.run(InteractiveShellRunner.groovy:83)
at org.codehaus.groovy.vmplugin.v7.IndyInterface.selectMethod(IndyInterface.java:232)
at org.apache.tinkerpop.gremlin.console.Console.<init>(Console.groovy:152)
at org.codehaus.groovy.vmplugin.v7.IndyInterface.selectMethod(IndyInterface.java:232)
at org.apache.tinkerpop.gremlin.console.Console.main(Console.groovy:455)
15:07:39 ERROR org.janusgraph.diskstorage.solr.SolrIndex - Details in failed document batch:
15:07:39 ERROR org.janusgraph.diskstorage.solr.SolrIndex - id:5y1-6h4-9hx-38w
15:07:39 ERROR org.janusgraph.diskstorage.solr.SolrIndex - place_g:POINT(23.7 38.1)
15:07:39 ERROR org.janusgraph.diskstorage.solr.SolrIndex - id:2s2-39s-b2t-cqw
15:07:39 ERROR org.janusgraph.diskstorage.solr.SolrIndex - reason_t:no fear of death
15:07:39 ERROR org.janusgraph.diskstorage.solr.SolrIndex - id:5xt-39k-b2t-6fs
15:07:39 ERROR org.janusgraph.diskstorage.solr.SolrIndex - reason_t:loves waves
15:07:39 ERROR org.janusgraph.diskstorage.solr.SolrIndex - id:6qh-6h4-9hx-6eo
15:07:39 ERROR org.janusgraph.diskstorage.solr.SolrIndex - place_g:POINT(22.0 39.0)
15:07:39 ERROR org.janusgraph.diskstorage.solr.SolrIndex - id:3yx-3bc-b2t-3cg
15:07:39 ERROR org.janusgraph.diskstorage.solr.SolrIndex - reason_t:loves fresh breezes
15:07:39 ERROR org.janusgraph.diskstorage.solr.SolrIndex - id:6c9-6h4-9hx-9l4
15:07:39 ERROR org.janusgraph.diskstorage.solr.SolrIndex - place_g:POINT(23.9 37.7)
15:07:52 ERROR org.janusgraph.diskstorage.solr.SolrIndex - Unable to save documents to Solr as one of the shape objects stored were not compatible with Solr.
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://localhost:8983/solr: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
<title>Error 404 Not Found</title>
</head>
<body><h2>HTTP ERROR 404</h2>
<p>Problem accessing /solr/edges/update. Reason:
<pre> Not Found</pre></p><hr><i><small>Powered by Jetty://</small></i><hr/>

</body>
</html>

at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:529)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:235)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:227)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:526)
at org.janusgraph.diskstorage.solr.SolrIndex.commitDocumentChanges(SolrIndex.java:407)
at org.janusgraph.diskstorage.solr.SolrIndex.mutate(SolrIndex.java:318)
at org.janusgraph.diskstorage.indexing.IndexTransaction$1.call(IndexTransaction.java:137)
at org.janusgraph.diskstorage.indexing.IndexTransaction$1.call(IndexTransaction.java:134)
at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:69)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:55)
at org.janusgraph.diskstorage.indexing.IndexTransaction.flushInternal(IndexTransaction.java:134)
at org.janusgraph.diskstorage.indexing.IndexTransaction.commit(IndexTransaction.java:115)
at org.janusgraph.diskstorage.BackendTransaction.commitIndexes(BackendTransaction.java:140)
at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:743)
at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1363)
at org.janusgraph.example.GraphOfTheGodsFactory.load(GraphOfTheGodsFactory.java:144)
at org.janusgraph.example.GraphOfTheGodsFactory.load(GraphOfTheGodsFactory.java:63)
at org.janusgraph.example.GraphOfTheGodsFactory$load.call(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
at groovysh_evaluate.run(groovysh_evaluate:3)
at org.codehaus.groovy.vmplugin.v7.IndyInterface.selectMethod(IndyInterface.java:232)
at org.codehaus.groovy.tools.shell.Interpreter.evaluate(Interpreter.groovy:70)
at org.codehaus.groovy.tools.shell.Groovysh.execute(Groovysh.groovy:190)
at org.apache.tinkerpop.gremlin.console.GremlinGroovysh.super$3$execute(GremlinGroovysh.groovy)
at sun.reflect.GeneratedMethodAccessor14.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1215)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuperN(ScriptBytecodeAdapter.java:132)
at org.apache.tinkerpop.gremlin.console.GremlinGroovysh.execute(GremlinGroovysh.groovy:72)
at org.codehaus.groovy.tools.shell.Shell.leftShift(Shell.groovy:122)
at org.codehaus.groovy.tools.shell.ShellRunner.work(ShellRunner.groovy:95)
at org.codehaus.groovy.tools.shell.InteractiveShellRunner.super$2$work(InteractiveShellRunner.groovy)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1215)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuperN(ScriptBytecodeAdapter.java:132)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuper0(ScriptBytecodeAdapter.java:152)
at org.codehaus.groovy.tools.shell.InteractiveShellRunner.work(InteractiveShellRunner.groovy:124)
at org.codehaus.groovy.tools.shell.ShellRunner.run(ShellRunner.groovy:59)
at org.codehaus.groovy.tools.shell.InteractiveShellRunner.super$2$run(InteractiveShellRunner.groovy)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1215)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuperN(ScriptBytecodeAdapter.java:132)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuper0(ScriptBytecodeAdapter.java:152)
at org.codehaus.groovy.tools.shell.InteractiveShellRunner.run(InteractiveShellRunner.groovy:83)
at org.codehaus.groovy.vmplugin.v7.IndyInterface.selectMethod(IndyInterface.java:232)
at org.apache.tinkerpop.gremlin.console.Console.<init>(Console.groovy:152)
at org.codehaus.groovy.vmplugin.v7.IndyInterface.selectMethod(IndyInterface.java:232)
at org.apache.tinkerpop.gremlin.console.Console.main(Console.groovy:455)
15:07:52 ERROR org.janusgraph.diskstorage.solr.SolrIndex - Details in failed document batch:
15:07:52 ERROR org.janusgraph.diskstorage.solr.SolrIndex - id:5y1-6h4-9hx-38w
15:07:52 ERROR org.janusgraph.diskstorage.solr.SolrIndex - place_g:POINT(23.7 38.1)
15:07:52 ERROR org.janusgraph.diskstorage.solr.SolrIndex - id:2s2-39s-b2t-cqw
15:07:52 ERROR org.janusgraph.diskstorage.solr.SolrIndex - reason_t:no fear of death
15:07:52 ERROR org.janusgraph.diskstorage.solr.SolrIndex - id:5xt-39k-b2t-6fs
15:07:52 ERROR org.janusgraph.diskstorage.solr.SolrIndex - reason_t:loves waves
15:07:52 ERROR org.janusgraph.diskstorage.solr.SolrIndex - id:6qh-6h4-9hx-6eo
15:07:52 ERROR org.janusgraph.diskstorage.solr.SolrIndex - place_g:POINT(22.0 39.0)
15:07:52 ERROR org.janusgraph.diskstorage.solr.SolrIndex - id:3yx-3bc-b2t-3cg
15:07:52 ERROR org.janusgraph.diskstorage.solr.SolrIndex - reason_t:loves fresh breezes
15:07:52 ERROR org.janusgraph.diskstorage.solr.SolrIndex - id:6c9-6h4-9hx-9l4
15:07:52 ERROR org.janusgraph.diskstorage.solr.SolrIndex - place_g:POINT(23.9 37.7)

Re: Using ES for traversal queries ?

Suny <sahithiy...@...>

#838

Thanks. I am dealing with count here. I need the attributes list on those vertices.

g.V().has('type',textContains('car')) - This is coming out very fast, which i assume is because of index in ES.

g.V().has('type',textContains('car')).inE().has('timestamp',eq('')).inV().valueMap() - This is slow. I have 1500 vertices and each vertex has about 3-5 attributes on it. I implemented vertex-centric index, on timestamp property but not sure if it is being used.

Show quoted text

On Friday, September 22, 2017 at 8:54:54 AM UTC-4, Jason Plurad wrote:

No, explain() doesn't give any clear indication currently whether an index will be utilized. I opened up an issue for that.

What counts are you dealing with here?

g.V().has('type',textContains('car')).count()
g.V().has('type',textContains('car')).inE().count()
g.V().has('type',textContains('car')).inE().has('timestamp',eq('')).count()

On Thursday, September 21, 2017 at 1:59:37 PM UTC-4, Suny wrote:
I implemented vertex-centric index on edge label.

Here is the query i am doing

g.V().has('type',textContains('car')).inE().has('timestamp',eq('')).inV().valueMap()

The documentation says - JanusGraph is intelligent enough to use vertex-centric indices when available.

So can i assume that Janusgraph uses the vertex-centric index for this query ?

On Thursday, September 21, 2017 at 11:05:37 AM UTC-4, Daniel Kuppitz wrote:
Only in the underlying storage backend.

See: http://docs.janusgraph.org/0.2.0-SNAPSHOT/data-model.html

Cheers,
Daniel

On Thu, Sep 21, 2017 at 7:57 AM, Suny <sahi...@...> wrote:
Thanks. Is the vertex-centric index not stored in Elastic search. Is it only in JG ?

On Thursday, September 21, 2017 at 10:28:47 AM UTC-4, Daniel Kuppitz wrote:
Thanks, All the in edges will have same label. Does it still traverse through all edges or directly find the edges with empty timestamp ?

I assume that Janus can only leverage a vertex centric index, if you specify the label, since that was the case in Titan and I don't think I heard about any changes in this area.

So in cases where i need to do lot of traversal, ES is not helpful ?

Again, ES is helpful for the initial vertex lookup. Once you start to traverse through the graph, you can only rely on vertex centric indices (or a good model that doesn't require much filtering).

Also can you explain a bit more on OLAP query

You can read up on OLAP here: http://docs.janusgraph.org/0.2.0-SNAPSHOT/hadoop-tp3.html
...and here: http://tinkerpop.apache.org/docs/current/reference/#_olap_hadoop_gremlin

Cheers,
Daniel

On Thu, Sep 21, 2017 at 7:07 AM, Suny <sahi...@...> wrote:
Thanks, All the in edges will have same label. Does it still traverse through all edges or directly find the edges with empty timestamp ?

So in cases where i need to do lot of traversal, ES is not helpful ?

Also can you explain a bit more on OLAP query

On Wednesday, September 20, 2017 at 2:35:27 PM UTC-4, Daniel Kuppitz wrote:
You should
provide an edge label for inE()
have a vertex centric index on timestamp
use a simple filter, instead of traversing back to the previous vertex (unless you rely on duplicates in your result)
Something like this:

g.V().has('type', textContains('car')).
filter(inE('edge-label').has('timestamp','')).
valueMap()

If you expect very large results, you'd be better off using an OLAP query.

Can I make JG to use ES for traversal too ?

ES is only useful for initial / global vertex lookups.

Cheers,
Daniel

On Wed, Sep 20, 2017 at 7:41 AM, Suny <sahi...@...> wrote:
Hi,

I am using JG with Cassandra and ES.

I have a type attribute on all vertices based on which i can differentiate group of vertices.

The query i want to do is

:> g.V().has('type',textContains('car')).inE().has('timestamp','').inV().valueMap()

I created an index on type attribute. If I just query for :> g.V().has('type',textContains('car')) it is coming back very fast with result. If I add the traversal part it is slowing down.

So, JG is using ES to retrieve the vertices of type 'car' and then for traversal it is using just JG. Can I make JG to use ES for traversal too ?

If i add index on edge attribute 'timestamp', does this fasten the query ?

Thanks

--
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/5aeb2fdc-6199-45f4-8953-866118c2d81a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/94e4f5f4-8670-403d-9038-fd64382b61a5%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/e24480c8-52eb-4dd7-b511-b2722ad1bc2b%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Re: ES with JG

Suny <sahithiy...@...>

#837

Each vertex has 3-5 attributes on it.

Show quoted text

On Friday, September 22, 2017 at 9:02:15 AM UTC-4, Jason Plurad wrote:

1500 vertices is a fair amount, but how much data is getting returned in the valueMap? Is there a large number of properties in the map or perhaps are the values very large? Very easily could be the cost of serializing that map. In another post you were asking about string size restrictions...

On Thursday, September 21, 2017 at 2:40:06 PM UTC-4, Suny wrote:
I implemented a mixed index on 'type' attribute on vertex.

Whenever i query for :> g.V().has('type',textContains('car'))

the result comes out really really fast.

When I do :> g.V().has('type',textContains('site')).valueMap() it is taking so much time to retrieve data. I have 1500 vertices of type car.

Is it because for 1st query it gets the result using ES for the second query it gets vertex id's from ES but for getting attributes it needs to hit JG again ?

Is there a way to fasten up the second query ?

Custom type Kryo serializers

simone...@...

#836

I'm running a JanusGraph 0.1 and I need to set a custom class as attribute type.

package com.contentwise.janus.test;

public class MyClass implements Comparable<MyClass> {

 private String title;
 private Integer rank;

 public MyClass(){
 }

 public MyClass(String title, Integer rank){
 this.title = title;
 this.rank = rank;
 }

 public String getTitle() {
 return title;
 }

 public void setTitle(String title) {
 this.title = title;
 }

 public Integer getRank() {
 return rank;
 }

 public void setRank(Integer rank) {
 this.rank = rank;
 }

 @Override
 public int compareTo(MyClass o) {
 return rank.compareTo(o.rank);
 }

 @Override
 public String toString(){
 return "Compless[" + title + ", " + rank + "]";
 }
}

I defined the following Janus serializer

package com.contentwise.janus.test;

import org.janusgraph.core.attribute.AttributeSerializer;
import org.janusgraph.diskstorage.ScanBuffer;
import org.janusgraph.diskstorage.WriteBuffer;
import org.janusgraph.graphdb.database.serialize.attribute.IntegerSerializer;
import org.janusgraph.graphdb.database.serialize.attribute.StringSerializer;

public class MyClassSerializer implements AttributeSerializer<MyClass> {

 private final StringSerializer titleSerializer = new StringSerializer();
 private final IntegerSerializer rankSerializer = new IntegerSerializer();

 @Override
 public MyClass read(ScanBuffer buffer) {
 String title = titleSerializer.read(buffer);
 Integer rank = rankSerializer.read(buffer);
 return new MyClass(title, rank);
 }

 @Override
 public void write(WriteBuffer buffer, MyClass attribute) {
 titleSerializer.write(buffer, attribute.getTitle());
 rankSerializer.write(buffer,attribute.getRank());
 }
}

and then added the references in the janus-cassandra-es.properties file (I already defined the serializers for arrays and linkedmaps)

attributes.custom.attribute10.attribute-class=java.util.ArrayList
attributes.custom.attribute10.serializer-class=com.contentwise.janus.serializer.ArrayListSerializer
attributes.custom.attribute11.attribute-class=java.util.LinkedHashMap
attributes.custom.attribute11.serializer-class=com.contentwise.janus.serializer.LinkedHashMapSerializer
attributes.custom.attribute12.attribute-class=com.contentwise.janus.test.MyClass
attributes.custom.attribute12.serializer-class=com.contentwise.janus.test.MyClassSerializer

Restarting the graph I was able to create new attributes of type MyClass using the console and I was able to perform some queries, too (ex. order by and filter)

:remote connect tinkerpop.server conf/remote.yaml
:> g.addV().property("uri", "tst://test")
:> g.addV().property("uri", "tst://test/1").property("attr1", "AAA").property("attr2", 3).property("attr3", 2.23f).property("attr4", new com.contentwise.janus.test.MyClass('AAA', 3)).as("source").V().has("uri", "tst://test").addE("instance_of").from("source").outV()
:> g.addV().property("uri", "tst://test/2").property("attr1", "BBB").property("attr2", 1).property("attr3", 2.2f).property("attr4", new com.contentwise.janus.test.MyClass('BBB', 1)).as("source").V().has("uri", "tst://test").addE("instance_of").from("source").outV()
:> g.V().has("uri", "tst://test").in("instance_of").order().by("attr4", incr).values("attr4")

Problems came out performing query from my app (trying both gremlin driver and java GVL). I wrote the Kyro serializer, too

package com.contentwise.janus.test;

import org.apache.tinkerpop.shaded.kryo.Kryo;
import org.apache.tinkerpop.shaded.kryo.Serializer;
import org.apache.tinkerpop.shaded.kryo.io.Input;
import org.apache.tinkerpop.shaded.kryo.io.Output;

public class MyClassKryoSerializer extends Serializer<MyClass> {

 @Override
 public void write(Kryo kryo, Output output, MyClass myClass) {
 output.writeString(myClass.getTitle());
 output.writeInt(myClass.getRank());
 }

 @Override
 public MyClass read(Kryo kryo, Input input, Class<MyClass> aClass) {
 String title = input.readString();
 Integer rank = input.readInt();
 return new MyClass(title, rank);
 }

}

then I specified it in my gremlin-server.yaml

serializers:
 - { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV1d0, config: { serializeResultToString: true }}
 - { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV1d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
 - { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV1d0, config: { custom: [com.contentwise.janus.test.MyClass;com.contentwise.janus.test.MyClassKryoSerializer] }}

Running the same query (the one sorting the nodes by attr4) using the driver, the gremlin-server log reported that the new class is not registered.

janus_1                  | 2017-09-22 13:56:10,744 [gremlin-server-exec-6] WARN  org.apache.tinkerpop.gremlin.driver.MessageSerializer  - Response [ResponseMessage{requestId=d6817560-f747-4c04-bbfe-d9b7ec961cdc, status=ResponseStatus{code=SUCCESS, message='', attributes={}}, result=ResponseResult{data=[{attr2=[3], uri=[tst://test/1], attr1=[AAA], attr4=[Compless[AAA, 3]], attr3=[2.23]}, {attr2=[1], uri=[tst://test/2], attr1=[BBB], attr4=[Compless[BBB, 1]], attr3=[2.2]}], meta={}}}] could not be serialized by org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV1d0.
janus_1                  | 2017-09-22 13:56:10,838 [gremlin-server-exec-6] WARN  org.apache.tinkerpop.gremlin.server.op.AbstractEvalOpProcessor  - The result [[{attr2=[3], uri=[tst://test/1], attr1=[AAA], attr4=[Compless[AAA, 3]], attr3=[2.23]}, {attr2=[1], uri=[tst://test/2], attr1=[BBB], attr4=[Compless[BBB, 1]], attr3=[2.2]}]] in the request d6817560-f747-4c04-bbfe-d9b7ec961cdc could not be serialized and returned.
janus_1                  | org.apache.tinkerpop.gremlin.driver.ser.SerializationException: java.lang.IllegalArgumentException: Class is not registered: com.contentwise.janus.test.MyClass
janus_1                  | Note: To register this class use: kryo.register(com.contentwise.janus.test.MyClass.class);
janus_1                  | at org.apache.tinkerpop.gremlin.driver.ser.AbstractGryoMessageSerializerV1d0.serializeResponseAsBinary(AbstractGryoMessageSerializerV1d0.java:234)
janus_1                  | at org.apache.tinkerpop.gremlin.server.op.AbstractOpProcessor.makeFrame(AbstractOpProcessor.java:260)
janus_1                  | at org.apache.tinkerpop.gremlin.server.op.AbstractOpProcessor.handleIterator(AbstractOpProcessor.java:145)
janus_1                  | at org.apache.tinkerpop.gremlin.server.op.AbstractEvalOpProcessor.lambda$evalOpInternal$64(AbstractEvalOpProcessor.java:248)
janus_1                  | at org.apache.tinkerpop.gremlin.groovy.engine.GremlinExecutor.lambda$eval$2(GremlinExecutor.java:298)
janus_1                  | at java.util.concurrent.FutureTask.run(FutureTask.java:266)
janus_1                  | at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
janus_1                  | at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
janus_1                  | at java.lang.Thread.run(Thread.java:745)
janus_1                  | Caused by: java.lang.IllegalArgumentException: Class is not registered: com.contentwise.janus.test.MyClass
janus_1                  | Note: To register this class use: kryo.register(com.contentwise.janus.test.MyClass.class);
janus_1                  | at org.apache.tinkerpop.shaded.kryo.Kryo.getRegistration(Kryo.java:484)
janus_1                  | at org.apache.tinkerpop.gremlin.structure.io.gryo.GryoClassResolver.writeClass(GryoClassResolver.java:136)
janus_1                  | at org.apache.tinkerpop.shaded.kryo.Kryo.writeClass(Kryo.java:514)
janus_1                  | at org.apache.tinkerpop.shaded.kryo.Kryo.writeClassAndObject(Kryo.java:619)
janus_1                  | at org.apache.tinkerpop.shaded.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:100)
janus_1                  | at org.apache.tinkerpop.shaded.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:40)
janus_1                  | at org.apache.tinkerpop.shaded.kryo.Kryo.writeClassAndObject(Kryo.java:625)
janus_1                  | at org.apache.tinkerpop.shaded.kryo.serializers.MapSerializer.write(MapSerializer.java:113)
janus_1                  | at org.apache.tinkerpop.shaded.kryo.serializers.MapSerializer.write(MapSerializer.java:39)
janus_1                  | at org.apache.tinkerpop.shaded.kryo.Kryo.writeClassAndObject(Kryo.java:625)
janus_1                  | at org.apache.tinkerpop.shaded.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:100)
janus_1                  | at org.apache.tinkerpop.shaded.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:40)
janus_1                  | at org.apache.tinkerpop.shaded.kryo.Kryo.writeClassAndObject(Kryo.java:625)
janus_1                  | at org.apache.tinkerpop.gremlin.driver.ser.AbstractGryoMessageSerializerV1d0.serializeResponseAsBinary(AbstractGryoMessageSerializerV1d0.java:217)
janus_1                  | ... 8 more

Am I missing some configuration?

PageRank on Large Graph

Joe Obernberger <joseph.o...@...>

#849

Hi All - I've been experimenting with SparkGraphComputer, and have it working, but I'm having performance issues.  What is the best way to run PageRank against a very large graph stored inside of JanusGraph?

Thank you!

-Joe