Date   

JanusGraph System Requirements

csconnor257@...
 

Hello, 

What would the system requirements for JanusGraph be with 100 GB of data?

Thanks


Re: How to filter out step 3 vertex list based on step 1 vertex

Boxuan Li
 

That’s a great question! To be honest I am not sure about the reason. My assumption is the __.out("edgeCB").is("B") is an anonymous child traversal within where() step, and thus it has no access to the label “B” which is defined in the outer traversal.

I am not sure if my understanding is correct. It might be a better idea to ask in the gremlin-users group.

Cheers,
Boxuan

「Ronnie via lists.lfaidata.foundation <rputhukkeril=qualys.com@...>」在 2021年7月9日 週五,上午3:07 寫道:

Thanks Boxuan! I tried the single query and that worked accurately!

On the other hand, I am still trying to figure out why the where traversal e.g. where(__.out("edgeCB").is("B")) didnt work. Is it because "B" is considered as literal instead of step label?

Also thanks for pointing me to the gremlin-users group.

Thanks!
Ronnie


Re: How to filter out step 3 vertex list based on step 1 vertex

Ronnie
 

Thanks Boxuan! I tried the single query and that worked accurately!

On the other hand, I am still trying to figure out why the where traversal e.g. where(__.out("edgeCB").is("B")) didnt work. Is it because "B" is considered as literal instead of step label?

Also thanks for pointing me to the gremlin-users group.

Thanks!
Ronnie


Re: How to filter out step 3 vertex list based on step 1 vertex

Boxuan Li
 

Hi Ronnie,

Not sure if it's optimal but this should work:

g.V().hasLabel("VertexB").as("B").in("edgeAB").out("edgeAC").as("C").out("edgeCB").as("B2").where("B", eq("B2")).select("C")

You can also do this in two steps:

b = g.V().hasLabel("VertexB").as("B").next()
g.V(b).in("edgeAB").out("edgeAC").where(out("edgeCB").is(b))

FYI, for general gremlin query questions, you can also ask in the gremlin-users mailing list: https://groups.google.com/g/gremlin-users

Best,
Boxuan


Re: How to filter out step 3 vertex list based on step 1 vertex

Ronnie
 

Sorry the gremlin queries should be as below

g.V().hasLabel("VertexB").as("B").in("edgeAB").out("edgeAC").where(__.out("edgeCB").is("B"))
g.V().hasLabel("VertexB").as("B").in("edgeAB").out("edgeAC").where(__.out("edgeCB").hasId(__.select("B").id()))


How to filter out step 3 vertex list based on step 1 vertex

Ronnie
 

Hi,
Assuming following schema
VertexA--edgeAB-->VertexB
VertexA--edgeAC-->VertexC
VertexC--edgeCB-->VertexB

Traversal
step 1: start with VertexB,
step 2: traverse edgeAB to find connected VertexA,
step 3: traverse edgeAC to find connected VertexC
step 4: how to filter out VertexC which are not connect to VertexB from step 1 ?

Gremlin queries that i tried:
g.V().hasLabel("VertexB").as("B").in("edgeAB").out("edgeAC").where(__.out("edgeCB").is("cert"))
g.V().hasLabel("VertexB").as("B").in("edgeAB").out("edgeAC").where(__.out("edgeCB").hasId(__.select("cert").id()))

Expected result: list of VertexC vertices which are connect back to the same VertexB vertex from step 1
Actual result: empty

Any pointers to why these gremlin queries don't work as expected?

Thanks,
Ronnie


Re: Multiple or-steps are conflated when using the textRegex() predicate

Mladen Marović
 

Hi,

Thanks for the quick answer. That seems to be it. Hopefully the fix will be available soon.

Best regards,

Mladen Marović


Re: Multiple or-steps are conflated when using the textRegex() predicate

Boxuan Li
 

Hi,


Best,
Boxuan

On Jul 7, 2021, at 8:29 PM, Mladen Marović <mladen.marovic@...> wrote:

Hello!

I came upon some unexpected behavior when running queries with multiple or() steps and string searches on mixed indexes and would like some clarification if this is intended or not.

I have a graph with a vertex label of type person and an edge label of type sent-message-to. All edges have the properties sender and receiver, and some others. A mixed index backed by elasticsearch is created for the edge label. Both sender and receiver are indexed (as STRING types), as well as some others.

The query that's causing me problems is:

g.E() \
    .hasLabel('sent-message-to') \
    .or( \
        has('sender', textRegex('.*alice.*')), \
        has('receiver', textRegex('.*alice.*')) \
    ).or( \
        has('sender', textRegex('.*bob.*')), \
        has('receiver', textRegex('.*bob.*')) \
    ).toList()

The query should return (roughly) messages between alices and bobs (and some edge cases where an alice bobowitz talks to an eve, but that's not important here). However, I'm getting some unexpected results where, for example, neither the sender nor the recipient contain the substring bob.

The explain plan for the query is as follows:

gremlin> g.E() \
......1>     .hasLabel('sent-message-to') \
......2>     .or( \
......3>         has('sender', textRegex('.*alice.*')), \
......4>         has('receiver', textRegex('.*alice.*')) \
......5>     ).or( \
......6>         has('sender', textRegex('.*bob.*')), \
......7>         has('receiver', textRegex('.*bob.*')) \
......8>     ).explain()
==>Traversal Explanation
=======================================================================================================================================================================================================================================================================================================
Original Traversal                          [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.textRegex(.*alice.*)])], [HasStep([receiver.textRegex(.*alice.*)])]]), OrStep([[HasStep([sender.textRegex(.*bob.*)])], [HasStep([receiver.textRegex(.*bob.*)])]])]

ConnectiveStrategy                    [D]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.textRegex(.*alice.*)])], [HasStep([receiver.textRegex(.*alice.*)])]]), OrStep([[HasStep([sender.textRegex(.*bob.*)])], [HasStep([receiver.textRegex(.*bob.*)])]])]
IncidentToAdjacentStrategy            [O]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.textRegex(.*alice.*)])], [HasStep([receiver.textRegex(.*alice.*)])]]), OrStep([[HasStep([sender.textRegex(.*bob.*)])], [HasStep([receiver.textRegex(.*bob.*)])]])]
RepeatUnrollStrategy                  [O]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.textRegex(.*alice.*)])], [HasStep([receiver.textRegex(.*alice.*)])]]), OrStep([[HasStep([sender.textRegex(.*bob.*)])], [HasStep([receiver.textRegex(.*bob.*)])]])]
MatchPredicateStrategy                [O]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.textRegex(.*alice.*)])], [HasStep([receiver.textRegex(.*alice.*)])]]), OrStep([[HasStep([sender.textRegex(.*bob.*)])], [HasStep([receiver.textRegex(.*bob.*)])]])]
PathRetractionStrategy                [O]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.textRegex(.*alice.*)])], [HasStep([receiver.textRegex(.*alice.*)])]]), OrStep([[HasStep([sender.textRegex(.*bob.*)])], [HasStep([receiver.textRegex(.*bob.*)])]])]
EarlyLimitStrategy                    [O]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.textRegex(.*alice.*)])], [HasStep([receiver.textRegex(.*alice.*)])]]), OrStep([[HasStep([sender.textRegex(.*bob.*)])], [HasStep([receiver.textRegex(.*bob.*)])]])]
FilterRankingStrategy                 [O]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.textRegex(.*alice.*)])], [HasStep([receiver.textRegex(.*alice.*)])]]), OrStep([[HasStep([sender.textRegex(.*bob.*)])], [HasStep([receiver.textRegex(.*bob.*)])]])]
InlineFilterStrategy                  [O]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.textRegex(.*alice.*)])], [HasStep([receiver.textRegex(.*alice.*)])]]), OrStep([[HasStep([sender.textRegex(.*bob.*)])], [HasStep([receiver.textRegex(.*bob.*)])]])]
AdjacentToIncidentStrategy            [O]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.textRegex(.*alice.*)])], [HasStep([receiver.textRegex(.*alice.*)])]]), OrStep([[HasStep([sender.textRegex(.*bob.*)])], [HasStep([receiver.textRegex(.*bob.*)])]])]
CountStrategy                         [O]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.textRegex(.*alice.*)])], [HasStep([receiver.textRegex(.*alice.*)])]]), OrStep([[HasStep([sender.textRegex(.*bob.*)])], [HasStep([receiver.textRegex(.*bob.*)])]])]
LazyBarrierStrategy                   [O]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.textRegex(.*alice.*)])], [HasStep([receiver.textRegex(.*alice.*)])]]), OrStep([[HasStep([sender.textRegex(.*bob.*)])], [HasStep([receiver.textRegex(.*bob.*)])]])]
AdjacentVertexFilterOptimizerStrategy [P]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.textRegex(.*alice.*)])], [HasStep([receiver.textRegex(.*alice.*)])]]), OrStep([[HasStep([sender.textRegex(.*bob.*)])], [HasStep([receiver.textRegex(.*bob.*)])]])]
AdjacentVertexHasIdOptimizerStrategy  [P]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.textRegex(.*alice.*)])], [HasStep([receiver.textRegex(.*alice.*)])]]), OrStep([[HasStep([sender.textRegex(.*bob.*)])], [HasStep([receiver.textRegex(.*bob.*)])]])]
AdjacentVertexIsOptimizerStrategy     [P]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.textRegex(.*alice.*)])], [HasStep([receiver.textRegex(.*alice.*)])]]), OrStep([[HasStep([sender.textRegex(.*bob.*)])], [HasStep([receiver.textRegex(.*bob.*)])]])]
JanusGraphLocalQueryOptimizerStrategy [P]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.textRegex(.*alice.*)])], [HasStep([receiver.textRegex(.*alice.*)])]]), OrStep([[HasStep([sender.textRegex(.*bob.*)])], [HasStep([receiver.textRegex(.*bob.*)])]])]
JanusGraphStepStrategy                [P]   [JanusGraphStep([],[~label.eq(sent-message-to)]).Or(JanusGraphStep([],[sender.textRegex(.*alice.*)]),JanusGraphStep([],[receiver.textRegex(.*alice.*)]),JanusGraphStep([],[sender.textRegex(.*bob.*)]),JanusGraphStep([],[receiver.textRegex(.*bob.*)]))]
JanusGraphIoRegistrationStrategy      [P]   [JanusGraphStep([],[~label.eq(sent-message-to)]).Or(JanusGraphStep([],[sender.textRegex(.*alice.*)]),JanusGraphStep([],[receiver.textRegex(.*alice.*)]),JanusGraphStep([],[sender.textRegex(.*bob.*)]),JanusGraphStep([],[receiver.textRegex(.*bob.*)]))]
ProfileStrategy                       [F]   [JanusGraphStep([],[~label.eq(sent-message-to)]).Or(JanusGraphStep([],[sender.textRegex(.*alice.*)]),JanusGraphStep([],[receiver.textRegex(.*alice.*)]),JanusGraphStep([],[sender.textRegex(.*bob.*)]),JanusGraphStep([],[receiver.textRegex(.*bob.*)]))]
StandardVerificationStrategy          [V]   [JanusGraphStep([],[~label.eq(sent-message-to)]).Or(JanusGraphStep([],[sender.textRegex(.*alice.*)]),JanusGraphStep([],[receiver.textRegex(.*alice.*)]),JanusGraphStep([],[sender.textRegex(.*bob.*)]),JanusGraphStep([],[receiver.textRegex(.*bob.*)]))]

Final Traversal                             [JanusGraphStep([],[~label.eq(sent-message-to)]).Or(JanusGraphStep([],[sender.textRegex(.*alice.*)]),JanusGraphStep([],[receiver.textRegex(.*alice.*)]),JanusGraphStep([],[sender.textRegex(.*bob.*)]),JanusGraphStep([],[receiver.textRegex(.*bob.*)]))]

The final (seemingly incorrect) traversal explains the results I'm getting. However, what strikes me as odd is that the JanusGraphLocalQueryOptimizerStrategy seems to return a correct traversal with two separate or steps:

[
	GraphStep(edge,[]),
	HasStep([~label.eq(sent-message-to)]),
	OrStep([
		[HasStep([sender.textRegex(.*alice.*)])],
		[HasStep([receiver.textRegex(.*alice.*)])]
	]),
	OrStep([
		[HasStep([sender.textRegex(.*bob.*)])],
		[HasStep([receiver.textRegex(.*bob.*)])]
	])
]

but the following JanusGraphStepStrategy conflates the two or steps into a single one:

[
	JanusGraphStep(
		[],[~label.eq(sent-message-to)]
	)
	.Or(
		JanusGraphStep([],[sender.eq(alice)]),
		JanusGraphStep([],[receiver.eq(alice)]),
		JanusGraphStep([],[sender.eq(bob)]),
		JanusGraphStep([],[receiver.eq(bob)])
	)
]

, which should not be correct because (A or B) and (C or D) is not equal to (A or B or C or D).

What's more confusing is that if I replace the textRegex() predicate with the tinkerpop predicate containing(), I get the proper results, because the explain plan is different:

gremlin> g.E() \
......1>     .hasLabel('sent-message-to') \
......2>     .or( \
......3>         has('sender', containing('alice')), \
......4>         has('receiver', containing('alice')) \
......5>     ).or( \
......6>         has('sender', containing('bob')), \
......7>         has('receiver', containing('bob')) \
......8>     ).explain()
==>Traversal Explanation
=======================================================================================================================================================================================================================================================================================
Original Traversal                          [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.containing(alice)])], [HasStep([receiver.containing(alice)])]]), OrStep([[HasStep([sender.containing(bob)])], [HasStep([receiver.containing(bob)])]])]

ConnectiveStrategy                    [D]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.containing(alice)])], [HasStep([receiver.containing(alice)])]]), OrStep([[HasStep([sender.containing(bob)])], [HasStep([receiver.containing(bob)])]])]
IncidentToAdjacentStrategy            [O]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.containing(alice)])], [HasStep([receiver.containing(alice)])]]), OrStep([[HasStep([sender.containing(bob)])], [HasStep([receiver.containing(bob)])]])]
RepeatUnrollStrategy                  [O]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.containing(alice)])], [HasStep([receiver.containing(alice)])]]), OrStep([[HasStep([sender.containing(bob)])], [HasStep([receiver.containing(bob)])]])]
MatchPredicateStrategy                [O]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.containing(alice)])], [HasStep([receiver.containing(alice)])]]), OrStep([[HasStep([sender.containing(bob)])], [HasStep([receiver.containing(bob)])]])]
PathRetractionStrategy                [O]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.containing(alice)])], [HasStep([receiver.containing(alice)])]]), OrStep([[HasStep([sender.containing(bob)])], [HasStep([receiver.containing(bob)])]])]
EarlyLimitStrategy                    [O]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.containing(alice)])], [HasStep([receiver.containing(alice)])]]), OrStep([[HasStep([sender.containing(bob)])], [HasStep([receiver.containing(bob)])]])]
FilterRankingStrategy                 [O]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.containing(alice)])], [HasStep([receiver.containing(alice)])]]), OrStep([[HasStep([sender.containing(bob)])], [HasStep([receiver.containing(bob)])]])]
InlineFilterStrategy                  [O]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.containing(alice)])], [HasStep([receiver.containing(alice)])]]), OrStep([[HasStep([sender.containing(bob)])], [HasStep([receiver.containing(bob)])]])]
AdjacentToIncidentStrategy            [O]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.containing(alice)])], [HasStep([receiver.containing(alice)])]]), OrStep([[HasStep([sender.containing(bob)])], [HasStep([receiver.containing(bob)])]])]
CountStrategy                         [O]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.containing(alice)])], [HasStep([receiver.containing(alice)])]]), OrStep([[HasStep([sender.containing(bob)])], [HasStep([receiver.containing(bob)])]])]
LazyBarrierStrategy                   [O]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.containing(alice)])], [HasStep([receiver.containing(alice)])]]), OrStep([[HasStep([sender.containing(bob)])], [HasStep([receiver.containing(bob)])]])]
AdjacentVertexFilterOptimizerStrategy [P]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.containing(alice)])], [HasStep([receiver.containing(alice)])]]), OrStep([[HasStep([sender.containing(bob)])], [HasStep([receiver.containing(bob)])]])]
AdjacentVertexHasIdOptimizerStrategy  [P]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.containing(alice)])], [HasStep([receiver.containing(alice)])]]), OrStep([[HasStep([sender.containing(bob)])], [HasStep([receiver.containing(bob)])]])]
AdjacentVertexIsOptimizerStrategy     [P]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.containing(alice)])], [HasStep([receiver.containing(alice)])]]), OrStep([[HasStep([sender.containing(bob)])], [HasStep([receiver.containing(bob)])]])]
JanusGraphLocalQueryOptimizerStrategy [P]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.containing(alice)])], [HasStep([receiver.containing(alice)])]]), OrStep([[HasStep([sender.containing(bob)])], [HasStep([receiver.containing(bob)])]])]
JanusGraphStepStrategy                [P]   [JanusGraphStep([],[~label.eq(sent-message-to)]), OrStep([[HasStep([sender.containing(alice)])], [HasStep([receiver.containing(alice)])]]), OrStep([[HasStep([sender.containing(bob)])], [HasStep([receiver.containing(bob)])]])]
JanusGraphIoRegistrationStrategy      [P]   [JanusGraphStep([],[~label.eq(sent-message-to)]), OrStep([[HasStep([sender.containing(alice)])], [HasStep([receiver.containing(alice)])]]), OrStep([[HasStep([sender.containing(bob)])], [HasStep([receiver.containing(bob)])]])]
ProfileStrategy                       [F]   [JanusGraphStep([],[~label.eq(sent-message-to)]), OrStep([[HasStep([sender.containing(alice)])], [HasStep([receiver.containing(alice)])]]), OrStep([[HasStep([sender.containing(bob)])], [HasStep([receiver.containing(bob)])]])]
StandardVerificationStrategy          [V]   [JanusGraphStep([],[~label.eq(sent-message-to)]), OrStep([[HasStep([sender.containing(alice)])], [HasStep([receiver.containing(alice)])]]), OrStep([[HasStep([sender.containing(bob)])], [HasStep([receiver.containing(bob)])]])]

Final Traversal                             [JanusGraphStep([],[~label.eq(sent-message-to)]), OrStep([[HasStep([sender.containing(alice)])], [HasStep([receiver.containing(alice)])]]), OrStep([[HasStep([sender.containing(bob)])], [HasStep([receiver.containing(bob)])]])]

and the final traversal contains two or steps:

[
	JanusGraphStep([],[~label.eq(sent-message-to)]),
	OrStep([
		[HasStep([sender.containing(alice)])],
		[HasStep([receiver.containing(alice)])]
	]),
	OrStep([
		[HasStep([sender.containing(bob)])],
		[HasStep([receiver.containing(bob)])]
	])]

I'd like to use the underlying mixed index to fetch the results, and preferably only one index query should be performed under the hood.

Is there a way to force this query to properly use the mixed index? Why is the explain plan in these two cases different?

Kind regards,

Mladen Marović



Multiple or-steps are conflated when using the textRegex() predicate

Mladen Marović
 

Hello!

I came upon some unexpected behavior when running queries with multiple or() steps and string searches on mixed indexes and would like some clarification if this is intended or not.

I have a graph with a vertex label of type person and an edge label of type sent-message-to. All edges have the properties sender and receiver, and some others. A mixed index backed by elasticsearch is created for the edge label. Both sender and receiver are indexed (as STRING types), as well as some others.

The query that's causing me problems is:

g.E() \
    .hasLabel('sent-message-to') \
    .or( \
        has('sender', textRegex('.*alice.*')), \
        has('receiver', textRegex('.*alice.*')) \
    ).or( \
        has('sender', textRegex('.*bob.*')), \
        has('receiver', textRegex('.*bob.*')) \
    ).toList()

The query should return (roughly) messages between alices and bobs (and some edge cases where an alice bobowitz talks to an eve, but that's not important here). However, I'm getting some unexpected results where, for example, neither the sender nor the recipient contain the substring bob.

The explain plan for the query is as follows:

gremlin> g.E() \
......1>     .hasLabel('sent-message-to') \
......2>     .or( \
......3>         has('sender', textRegex('.*alice.*')), \
......4>         has('receiver', textRegex('.*alice.*')) \
......5>     ).or( \
......6>         has('sender', textRegex('.*bob.*')), \
......7>         has('receiver', textRegex('.*bob.*')) \
......8>     ).explain()
==>Traversal Explanation
=======================================================================================================================================================================================================================================================================================================
Original Traversal                          [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.textRegex(.*alice.*)])], [HasStep([receiver.textRegex(.*alice.*)])]]), OrStep([[HasStep([sender.textRegex(.*bob.*)])], [HasStep([receiver.textRegex(.*bob.*)])]])]

ConnectiveStrategy                    [D]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.textRegex(.*alice.*)])], [HasStep([receiver.textRegex(.*alice.*)])]]), OrStep([[HasStep([sender.textRegex(.*bob.*)])], [HasStep([receiver.textRegex(.*bob.*)])]])]
IncidentToAdjacentStrategy            [O]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.textRegex(.*alice.*)])], [HasStep([receiver.textRegex(.*alice.*)])]]), OrStep([[HasStep([sender.textRegex(.*bob.*)])], [HasStep([receiver.textRegex(.*bob.*)])]])]
RepeatUnrollStrategy                  [O]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.textRegex(.*alice.*)])], [HasStep([receiver.textRegex(.*alice.*)])]]), OrStep([[HasStep([sender.textRegex(.*bob.*)])], [HasStep([receiver.textRegex(.*bob.*)])]])]
MatchPredicateStrategy                [O]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.textRegex(.*alice.*)])], [HasStep([receiver.textRegex(.*alice.*)])]]), OrStep([[HasStep([sender.textRegex(.*bob.*)])], [HasStep([receiver.textRegex(.*bob.*)])]])]
PathRetractionStrategy                [O]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.textRegex(.*alice.*)])], [HasStep([receiver.textRegex(.*alice.*)])]]), OrStep([[HasStep([sender.textRegex(.*bob.*)])], [HasStep([receiver.textRegex(.*bob.*)])]])]
EarlyLimitStrategy                    [O]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.textRegex(.*alice.*)])], [HasStep([receiver.textRegex(.*alice.*)])]]), OrStep([[HasStep([sender.textRegex(.*bob.*)])], [HasStep([receiver.textRegex(.*bob.*)])]])]
FilterRankingStrategy                 [O]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.textRegex(.*alice.*)])], [HasStep([receiver.textRegex(.*alice.*)])]]), OrStep([[HasStep([sender.textRegex(.*bob.*)])], [HasStep([receiver.textRegex(.*bob.*)])]])]
InlineFilterStrategy                  [O]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.textRegex(.*alice.*)])], [HasStep([receiver.textRegex(.*alice.*)])]]), OrStep([[HasStep([sender.textRegex(.*bob.*)])], [HasStep([receiver.textRegex(.*bob.*)])]])]
AdjacentToIncidentStrategy            [O]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.textRegex(.*alice.*)])], [HasStep([receiver.textRegex(.*alice.*)])]]), OrStep([[HasStep([sender.textRegex(.*bob.*)])], [HasStep([receiver.textRegex(.*bob.*)])]])]
CountStrategy                         [O]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.textRegex(.*alice.*)])], [HasStep([receiver.textRegex(.*alice.*)])]]), OrStep([[HasStep([sender.textRegex(.*bob.*)])], [HasStep([receiver.textRegex(.*bob.*)])]])]
LazyBarrierStrategy                   [O]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.textRegex(.*alice.*)])], [HasStep([receiver.textRegex(.*alice.*)])]]), OrStep([[HasStep([sender.textRegex(.*bob.*)])], [HasStep([receiver.textRegex(.*bob.*)])]])]
AdjacentVertexFilterOptimizerStrategy [P]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.textRegex(.*alice.*)])], [HasStep([receiver.textRegex(.*alice.*)])]]), OrStep([[HasStep([sender.textRegex(.*bob.*)])], [HasStep([receiver.textRegex(.*bob.*)])]])]
AdjacentVertexHasIdOptimizerStrategy  [P]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.textRegex(.*alice.*)])], [HasStep([receiver.textRegex(.*alice.*)])]]), OrStep([[HasStep([sender.textRegex(.*bob.*)])], [HasStep([receiver.textRegex(.*bob.*)])]])]
AdjacentVertexIsOptimizerStrategy     [P]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.textRegex(.*alice.*)])], [HasStep([receiver.textRegex(.*alice.*)])]]), OrStep([[HasStep([sender.textRegex(.*bob.*)])], [HasStep([receiver.textRegex(.*bob.*)])]])]
JanusGraphLocalQueryOptimizerStrategy [P]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.textRegex(.*alice.*)])], [HasStep([receiver.textRegex(.*alice.*)])]]), OrStep([[HasStep([sender.textRegex(.*bob.*)])], [HasStep([receiver.textRegex(.*bob.*)])]])]
JanusGraphStepStrategy                [P]   [JanusGraphStep([],[~label.eq(sent-message-to)]).Or(JanusGraphStep([],[sender.textRegex(.*alice.*)]),JanusGraphStep([],[receiver.textRegex(.*alice.*)]),JanusGraphStep([],[sender.textRegex(.*bob.*)]),JanusGraphStep([],[receiver.textRegex(.*bob.*)]))]
JanusGraphIoRegistrationStrategy      [P]   [JanusGraphStep([],[~label.eq(sent-message-to)]).Or(JanusGraphStep([],[sender.textRegex(.*alice.*)]),JanusGraphStep([],[receiver.textRegex(.*alice.*)]),JanusGraphStep([],[sender.textRegex(.*bob.*)]),JanusGraphStep([],[receiver.textRegex(.*bob.*)]))]
ProfileStrategy                       [F]   [JanusGraphStep([],[~label.eq(sent-message-to)]).Or(JanusGraphStep([],[sender.textRegex(.*alice.*)]),JanusGraphStep([],[receiver.textRegex(.*alice.*)]),JanusGraphStep([],[sender.textRegex(.*bob.*)]),JanusGraphStep([],[receiver.textRegex(.*bob.*)]))]
StandardVerificationStrategy          [V]   [JanusGraphStep([],[~label.eq(sent-message-to)]).Or(JanusGraphStep([],[sender.textRegex(.*alice.*)]),JanusGraphStep([],[receiver.textRegex(.*alice.*)]),JanusGraphStep([],[sender.textRegex(.*bob.*)]),JanusGraphStep([],[receiver.textRegex(.*bob.*)]))]

Final Traversal                             [JanusGraphStep([],[~label.eq(sent-message-to)]).Or(JanusGraphStep([],[sender.textRegex(.*alice.*)]),JanusGraphStep([],[receiver.textRegex(.*alice.*)]),JanusGraphStep([],[sender.textRegex(.*bob.*)]),JanusGraphStep([],[receiver.textRegex(.*bob.*)]))]

The final (seemingly incorrect) traversal explains the results I'm getting. However, what strikes me as odd is that the JanusGraphLocalQueryOptimizerStrategy seems to return a correct traversal with two separate or steps:

[
	GraphStep(edge,[]),
	HasStep([~label.eq(sent-message-to)]),
	OrStep([
		[HasStep([sender.textRegex(.*alice.*)])],
		[HasStep([receiver.textRegex(.*alice.*)])]
	]),
	OrStep([
		[HasStep([sender.textRegex(.*bob.*)])],
		[HasStep([receiver.textRegex(.*bob.*)])]
	])
]

but the following JanusGraphStepStrategy conflates the two or steps into a single one:

[
	JanusGraphStep(
		[],[~label.eq(sent-message-to)]
	)
	.Or(
		JanusGraphStep([],[sender.eq(alice)]),
		JanusGraphStep([],[receiver.eq(alice)]),
		JanusGraphStep([],[sender.eq(bob)]),
		JanusGraphStep([],[receiver.eq(bob)])
	)
]

, which should not be correct because (A or B) and (C or D) is not equal to (A or B or C or D).

What's more confusing is that if I replace the textRegex() predicate with the tinkerpop predicate containing(), I get the proper results, because the explain plan is different:

gremlin> g.E() \
......1>     .hasLabel('sent-message-to') \
......2>     .or( \
......3>         has('sender', containing('alice')), \
......4>         has('receiver', containing('alice')) \
......5>     ).or( \
......6>         has('sender', containing('bob')), \
......7>         has('receiver', containing('bob')) \
......8>     ).explain()
==>Traversal Explanation
=======================================================================================================================================================================================================================================================================================
Original Traversal                          [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.containing(alice)])], [HasStep([receiver.containing(alice)])]]), OrStep([[HasStep([sender.containing(bob)])], [HasStep([receiver.containing(bob)])]])]

ConnectiveStrategy                    [D]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.containing(alice)])], [HasStep([receiver.containing(alice)])]]), OrStep([[HasStep([sender.containing(bob)])], [HasStep([receiver.containing(bob)])]])]
IncidentToAdjacentStrategy            [O]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.containing(alice)])], [HasStep([receiver.containing(alice)])]]), OrStep([[HasStep([sender.containing(bob)])], [HasStep([receiver.containing(bob)])]])]
RepeatUnrollStrategy                  [O]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.containing(alice)])], [HasStep([receiver.containing(alice)])]]), OrStep([[HasStep([sender.containing(bob)])], [HasStep([receiver.containing(bob)])]])]
MatchPredicateStrategy                [O]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.containing(alice)])], [HasStep([receiver.containing(alice)])]]), OrStep([[HasStep([sender.containing(bob)])], [HasStep([receiver.containing(bob)])]])]
PathRetractionStrategy                [O]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.containing(alice)])], [HasStep([receiver.containing(alice)])]]), OrStep([[HasStep([sender.containing(bob)])], [HasStep([receiver.containing(bob)])]])]
EarlyLimitStrategy                    [O]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.containing(alice)])], [HasStep([receiver.containing(alice)])]]), OrStep([[HasStep([sender.containing(bob)])], [HasStep([receiver.containing(bob)])]])]
FilterRankingStrategy                 [O]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.containing(alice)])], [HasStep([receiver.containing(alice)])]]), OrStep([[HasStep([sender.containing(bob)])], [HasStep([receiver.containing(bob)])]])]
InlineFilterStrategy                  [O]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.containing(alice)])], [HasStep([receiver.containing(alice)])]]), OrStep([[HasStep([sender.containing(bob)])], [HasStep([receiver.containing(bob)])]])]
AdjacentToIncidentStrategy            [O]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.containing(alice)])], [HasStep([receiver.containing(alice)])]]), OrStep([[HasStep([sender.containing(bob)])], [HasStep([receiver.containing(bob)])]])]
CountStrategy                         [O]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.containing(alice)])], [HasStep([receiver.containing(alice)])]]), OrStep([[HasStep([sender.containing(bob)])], [HasStep([receiver.containing(bob)])]])]
LazyBarrierStrategy                   [O]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.containing(alice)])], [HasStep([receiver.containing(alice)])]]), OrStep([[HasStep([sender.containing(bob)])], [HasStep([receiver.containing(bob)])]])]
AdjacentVertexFilterOptimizerStrategy [P]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.containing(alice)])], [HasStep([receiver.containing(alice)])]]), OrStep([[HasStep([sender.containing(bob)])], [HasStep([receiver.containing(bob)])]])]
AdjacentVertexHasIdOptimizerStrategy  [P]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.containing(alice)])], [HasStep([receiver.containing(alice)])]]), OrStep([[HasStep([sender.containing(bob)])], [HasStep([receiver.containing(bob)])]])]
AdjacentVertexIsOptimizerStrategy     [P]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.containing(alice)])], [HasStep([receiver.containing(alice)])]]), OrStep([[HasStep([sender.containing(bob)])], [HasStep([receiver.containing(bob)])]])]
JanusGraphLocalQueryOptimizerStrategy [P]   [GraphStep(edge,[]), HasStep([~label.eq(sent-message-to)]), OrStep([[HasStep([sender.containing(alice)])], [HasStep([receiver.containing(alice)])]]), OrStep([[HasStep([sender.containing(bob)])], [HasStep([receiver.containing(bob)])]])]
JanusGraphStepStrategy                [P]   [JanusGraphStep([],[~label.eq(sent-message-to)]), OrStep([[HasStep([sender.containing(alice)])], [HasStep([receiver.containing(alice)])]]), OrStep([[HasStep([sender.containing(bob)])], [HasStep([receiver.containing(bob)])]])]
JanusGraphIoRegistrationStrategy      [P]   [JanusGraphStep([],[~label.eq(sent-message-to)]), OrStep([[HasStep([sender.containing(alice)])], [HasStep([receiver.containing(alice)])]]), OrStep([[HasStep([sender.containing(bob)])], [HasStep([receiver.containing(bob)])]])]
ProfileStrategy                       [F]   [JanusGraphStep([],[~label.eq(sent-message-to)]), OrStep([[HasStep([sender.containing(alice)])], [HasStep([receiver.containing(alice)])]]), OrStep([[HasStep([sender.containing(bob)])], [HasStep([receiver.containing(bob)])]])]
StandardVerificationStrategy          [V]   [JanusGraphStep([],[~label.eq(sent-message-to)]), OrStep([[HasStep([sender.containing(alice)])], [HasStep([receiver.containing(alice)])]]), OrStep([[HasStep([sender.containing(bob)])], [HasStep([receiver.containing(bob)])]])]

Final Traversal                             [JanusGraphStep([],[~label.eq(sent-message-to)]), OrStep([[HasStep([sender.containing(alice)])], [HasStep([receiver.containing(alice)])]]), OrStep([[HasStep([sender.containing(bob)])], [HasStep([receiver.containing(bob)])]])]

and the final traversal contains two or steps:

[
	JanusGraphStep([],[~label.eq(sent-message-to)]),
	OrStep([
		[HasStep([sender.containing(alice)])],
		[HasStep([receiver.containing(alice)])]
	]),
	OrStep([
		[HasStep([sender.containing(bob)])],
		[HasStep([receiver.containing(bob)])]
	])]

I'd like to use the underlying mixed index to fetch the results, and preferably only one index query should be performed under the hood.

Is there a way to force this query to properly use the mixed index? Why is the explain plan in these two cases different?

Kind regards,

Mladen Marović


Re: Indexing on sub-attribute of custom data type

Ronnie
 

Hi Marc,
Thanks for confirming about "creating an associated vertex which defines this custom data type" approach. In which case i would not be experimenting custom data types for now. Thanks for the details regarding the serializers for custom attributes - i am sure these will come handy for me.

Thanks!
Ronnie


Re: Could not call index

schwartz@...
 

At first I thought it might have something to do with 2 new indices I added yesterday, so I re-indexed them just in case. But the result is still the same.
I'm having a hard time what does the error means - is it an issue with a composite index? a mixed index? which index?

The same traversal steps worked in a development Janus container (whichever storage and index that come out of the box)


Re: Could not call index

schwartz@...
 

This is all I have in Stackdriver. If there's a way to see more details logs from inside the container, please tell how to get them and I'll gladly post them here.


Re: Could not call index

Boxuan Li
 

Hi, is this the full stacktrace? Is there a nested exception?


Could not call index

schwartz@...
 

Hi! Running JanusGraph 0.5.3 against BigTable, and ES is used as the Index backend.
For a particular traversal I'm seeing the error message below. No clue what this means and where to look for a solution.
Any assistance will be greatly appreciated! Thanks, Assaf

org.janusgraph.core.JanusGraphException: Could not call index at org.janusgraph.graphdb.util.SubqueryIterator.<init>(SubqueryIterator.java:68) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1354) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1246) at org.janusgraph.graphdb.query.MetricsQueryExecutor.lambda$execute$3(MetricsQueryExecutor.java:60) at org.janusgraph.graphdb.query.MetricsQueryExecutor.runWithMetrics(MetricsQueryExecutor.java:72) at org.janusgraph.graphdb.query.MetricsQueryExecutor.execute(MetricsQueryExecutor.java:60) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:201) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:69) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:55) at org.janusgraph.graphdb.query.ResultSetIterator.<init>(ResultSetIterator.java:45) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:67) at org.janusgraph.graphdb.query.graph.GraphCentricQueryBuilder$1.iterator(GraphCentricQueryBuilder.java:204) at org.janusgraph.graphdb.query.graph.GraphCentricQueryBuilder$1.iterator(GraphCentricQueryBuilder.java:201) at com.google.common.collect.Iterables.getOnlyElement(Iterables.java:302) at org.janusgraph.graphdb.database.StandardJanusGraph$1.retrieveSchemaByName(StandardJanusGraph.java:383) at org.janusgraph.graphdb.database.cache.MetricInstrumentedSchemaCache$1.retrieveSchemaByName(MetricInstrumentedSchemaCache.java:41) at org.janusgraph.graphdb.database.cache.StandardSchemaCache.getSchemaId(StandardSchemaCache.java:111) at org.janusgraph.graphdb.database.cache.MetricInstrumentedSchemaCache.getSchemaId(MetricInstrumentedSchemaCache.java:59) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.getSchemaVertex(StandardJanusGraphTx.java:950) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.getRelationType(StandardJanusGraphTx.java:970) at org.janusgraph.graphdb.query.QueryUtil.getType(QueryUtil.java:68) at org.janusgraph.graphdb.query.vertex.BasicVertexCentricQueryBuilder.constructQueryWithoutProfile(BasicVertexCentricQueryBuilder.java:483) at org.janusgraph.graphdb.query.vertex.BasicVertexCentricQueryBuilder.constructQuery(BasicVertexCentricQueryBuilder.java:416) at org.janusgraph.graphdb.query.vertex.VertexCentricQueryBuilder.execute(VertexCentricQueryBuilder.java:68) at org.janusgraph.graphdb.query.vertex.VertexCentricQueryBuilder.properties(VertexCentricQueryBuilder.java:100) at org.janusgraph.graphdb.tinkerpop.optimize.JanusGraphPropertiesStep.flatMap(JanusGraphPropertiesStep.java:137) at org.apache.tinkerpop.gremlin.process.traversal.step.map.FlatMapStep.processNextStart(FlatMapStep.java:49) at org.janusgraph.graphdb.tinkerpop.optimize.JanusGraphPropertiesStep.processNextStart(JanusGraphPropertiesStep.java:125) at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:143) at org.apache.tinkerpop.gremlin.process.traversal.util.DefaultTraversal.hasNext(DefaultTraversal.java:197) at org.apache.tinkerpop.gremlin.process.traversal.step.map.CoalesceStep.flatMap(CoalesceStep.java:58) at org.apache.tinkerpop.gremlin.process.traversal.step.map.FlatMapStep.processNextStart(FlatMapStep.java:49) at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:143) at org.apache.tinkerpop.gremlin.process.traversal.step.branch.BranchStep.standardAlgorithm(BranchStep.java:126) at org.apache.tinkerpop.gremlin.process.traversal.step.util.ComputerAwareStep.processNextStart(ComputerAwareStep.java:46) at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:143) at org.apache.tinkerpop.gremlin.process.traversal.step.util.ExpandableStepIterator.hasNext(ExpandableStepIterator.java:42) at org.apache.tinkerpop.gremlin.process.traversal.step.util.ReducingBarrierStep.processAllStarts(ReducingBarrierStep.java:82) at org.apache.tinkerpop.gremlin.process.traversal.step.util.ReducingBarrierStep.processNextStart(ReducingBarrierStep.java:112) at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.next(AbstractStep.java:128) at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.next(AbstractStep.java:38) at org.apache.tinkerpop.gremlin.process.traversal.util.DefaultTraversal.next(DefaultTraversal.java:205) at org.apache.tinkerpop.gremlin.process.traversal.util.TraversalUtil.apply(TraversalUtil.java:44) at org.apache.tinkerpop.gremlin.process.traversal.util.TraversalUtil.applyNullable(TraversalUtil.java:87) at org.apache.tinkerpop.gremlin.process.traversal.step.map.GroupStep.projectTraverser(GroupStep.java:140) at org.apache.tinkerpop.gremlin.process.traversal.step.map.GroupStep.projectTraverser(GroupStep.java:56) at org.apache.tinkerpop.gremlin.process.traversal.step.util.ReducingBarrierStep.processAllStarts(ReducingBarrierStep.java:83) at org.apache.tinkerpop.gremlin.process.traversal.step.util.ReducingBarrierStep.processNextStart(ReducingBarrierStep.java:112) at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:143) at org.apache.tinkerpop.gremlin.process.traversal.step.util.ExpandableStepIterator.next(ExpandableStepIterator.java:50) at org.apache.tinkerpop.gremlin.process.traversal.step.map.MapStep.processNextStart(MapStep.java:36) at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:143) at org.apache.tinkerpop.gremlin.process.traversal.step.util.ExpandableStepIterator.next(ExpandableStepIterator.java:50) at org.apache.tinkerpop.gremlin.process.traversal.step.map.FlatMapStep.processNextStart(FlatMapStep.java:48) at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:143) at org.apache.tinkerpop.gremlin.process.traversal.step.util.ExpandableStepIterator.hasNext(ExpandableStepIterator.java:42) at org.apache.tinkerpop.gremlin.process.traversal.step.util.CollectingBarrierStep.processNextStart(CollectingBarrierStep.java:107) at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:143) at org.apache.tinkerpop.gremlin.process.traversal.step.util.ExpandableStepIterator.hasNext(ExpandableStepIterator.java:42) at org.apache.tinkerpop.gremlin.process.traversal.step.util.ReducingBarrierStep.processAllStarts(ReducingBarrierStep.java:82) at org.apache.tinkerpop.gremlin.process.traversal.step.util.ReducingBarrierStep.processNextStart(ReducingBarrierStep.java:112) at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:143) at org.apache.tinkerpop.gremlin.process.traversal.step.util.ExpandableStepIterator.next(ExpandableStepIterator.java:50) at org.apache.tinkerpop.gremlin.process.traversal.step.map.MapStep.processNextStart(MapStep.java:36) at org.apache.tinkerpop.gremlin.process.traversal.step.map.SelectStep.processNextStart(SelectStep.java:156) at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:143) at org.apache.tinkerpop.gremlin.process.traversal.util.DefaultTraversal.hasNext(DefaultTraversal.java:197) at org.apache.tinkerpop.gremlin.server.util.TraverserIterator.fillBulker(TraverserIterator.java:69) at org.apache.tinkerpop.gremlin.server.util.TraverserIterator.hasNext(TraverserIterator.java:56) at org.apache.tinkerpop.gremlin.server.op.traversal.TraversalOpProcessor.handleIterator(TraversalOpProcessor.java:512) at org.apache.tinkerpop.gremlin.server.op.traversal.TraversalOpProcessor.lambda$iterateBytecodeTraversal$4(TraversalOpProcessor.java:411) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)


Re: Index stuck on INSTALLED (single instance of JanusGraph)

schwartz@...
 

THANK YOU!!! :)


Re: Index stuck on INSTALLED (single instance of JanusGraph)

sergeymetallic@...
 
Edited

I had a similar problem, how I solved it:
mgmt.getGraphIndex('my_index_name').getIndexStatus(mgmt.getPropertyKey("property_i_ised"))
==>INSTALLED
 graph.getOpenTransactions() - > showed me all the open transactions.
 graph.getOpenTransactions().getAt(0).commit() -> for every open transaction
 mgmt = graph.openManagement()
 mgmt.updateIndex(mgmt.getGraphIndex('my_index_name'), SchemaAction.REGISTER_INDEX).get() 
 mgmt.commit()
mgmt = graph.openManagement()
 mgmt.getGraphIndex('my_index_name').getIndexStatus(mgmt.getPropertyKey("property_i_ised"))
==>REGISTERED
 


Re: Index stuck on INSTALLED (single instance of JanusGraph)

schwartz@...
 

It seems that I had lots of instances registered in the cluster, probably due to shutdowns.
I got the list by using mgmt.getOpenInstances().toList()

I closed all instances expect for the current one, and committed, hoping that this would move the index status to REGISTERED.
Yet, nothing happens


Index stuck on INSTALLED (single instance of JanusGraph)

schwartz@...
 

I tried adding a composite index based on 2 existing properties.
As far as I understand, the initial stated is INSTALLED, then after all instances become aware of it, it should be REGISTERED.
Only then, I should re-index to make the index ENABLED.

My index remains INSTALLED. The JanusGraph server has no other instances (GKE deployment, with just 1 replica).
What needs to be done for the index to transition from INSTALLED to REGISTERED?

Many thanks!
Assaf


Re: How to replay a transaction log from the begining

Boxuan Li
 

Hi Ojas,

Ideally, by using Instant.now() to add your log processor, you should be able to see your callback invoked as soon as the transaction completes (if you are using a single in-memory storage backend), or with a minimal delay (depending on the read latency of your storage backend).

The time difference in your log looks a bit weird to me. Can you check if there is a clock drift among your servers?

Best,
Boxuan

「ojas.dubey via lists.lfaidata.foundation <ojas.dubey=amdocs.com@...>」在 2021年6月30日 週三,下午10:12 寫道:

Hi Boxuan,

Thanks. This indeed helped. Initially nothing happened (or at least it appeared that way) so I changed the start time to EPOCH and left the application running for a while and after sometime the callback was executed. 

So I was wondering how the log processor uses the start time value to replay the log and why did it take a long time to replay the logs. Is there a way by which I can reduce the time by setting the correct UTC time to start time (as i dont want to use EPOCH everytime) so that the callback is executed immediately?

Also is there a difference in the values of Instant.now() used by ReadMarker vs the actual local time used by the applicatioon because the ReadMarker initialization logs showed a different time. e.g.

2021-06-30T13:21:32.003+05:30 INFO |InternalEventLogger|||||||o.j.diskstorage.log.kcvs.KCVSLog|Loaded identified ReadMarker start time 2021-06-30T04:00:00Z into org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller@4051e47b
2021-06-30T13:21:32.008+05:30 INFO |InternalEventLogger|||||||o.j.diskstorage.log.kcvs.KCVSLog|Loaded identified ReadMarker start time 2021-06-30T04:00:00Z into org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller@6a332fb7
2021-06-30T13:21:32.013+05:30 INFO |InternalEventLogger|||||||o.j.diskstorage.log.kcvs.KCVSLog|Loaded identified ReadMarker start time 2021-06-30T04:00:00Z into org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller@1f2cf847
2021-06-30T13:21:32.015+05:30 INFO |InternalEventLogger|||||||o.j.diskstorage.log.kcvs.KCVSLog|Loaded identified ReadMarker start time 2021-06-30T04:00:00Z into org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller@3cc2c61b

while the application log shows another time

2021-06-30T13:26:15.794+05:30 INFO |InternalEventLogger||c.a.o.s.b.s.i.Test|Started tx standardjanusgraphtx[0x39c4068c] for requestId 5ba073c8-68c2-4356-8097-2e62ef56299a and batchId 9632dceb-7996-4464-91d8-1b157fc8ca00


Regards,
Ojas 


Re: Indexing on sub-attribute of custom data type

hadoopmarc@...
 

Regarding documentation on custom attributes: Jason Plurad published an example project a few years ago (so, for an older JanusGraph version).

See, https://github.com/pluradj/janusgraph-attribute-serializer

661 - 680 of 6666