Date   

Re: [PROPOSAL] new repository for janusgraph-foundationdb

Anshul Pathak <anshul...@...>
 

 +1 

On Jun 11, 2020, at 7:32 AM, Jason Plurad <plu...@...> wrote:

Several developers from the community have expressed interest in collaborating on a storage backend for FoundationDB. As discussed on this thread, they each have a separate fork (no license changes) from the original experoinc/janusgraph-foundationdb repo that Ted Wilmes created (no updates in ~2 years).

I propose that we create a new project janusgraph-foundationdb under the JanusGraph org so that developers can work together to merge their forks and move forward.

Let's give it 72 hours for lazy consensus.

I'm +1 for this proposal.

-- Jason

--
You received this message because you are subscribed to the Google Groups "JanusGraph developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgr...@....
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-dev/4009696c-98b9-4a04-ad21-eb0572425de7o%40googlegroups.com.


Re: [PROPOSAL] new repository for janusgraph-foundationdb

Christopher Jackson <jackson.ch...@...>
 

+1


On Thursday, June 11, 2020 at 10:32:37 AM UTC-4, Jason Plurad wrote:
Several developers from the community have expressed interest in collaborating on a storage backend for FoundationDB. As discussed on this thread, they each have a separate fork (no license changes) from the original experoinc/janusgraph-foundationdb repo that Ted Wilmes created (no updates in ~2 years).

I propose that we create a new project janusgraph-foundationdb under the JanusGraph org so that developers can work together to merge their forks and move forward.

Let's give it 72 hours for lazy consensus.

I'm +1 for this proposal.

-- Jason


Re: [PROPOSAL] new repository for janusgraph-foundationdb

Misha Brukman <mbru...@...>
 

Vote: +1


On Thu, Jun 11, 2020 at 10:32 AM Jason Plurad <plu...@...> wrote:
Several developers from the community have expressed interest in collaborating on a storage backend for FoundationDB. As discussed on this thread, they each have a separate fork (no license changes) from the original experoinc/janusgraph-foundationdb repo that Ted Wilmes created (no updates in ~2 years).

I propose that we create a new project janusgraph-foundationdb under the JanusGraph org so that developers can work together to merge their forks and move forward.

Let's give it 72 hours for lazy consensus.

I'm +1 for this proposal.

-- Jason

--
You received this message because you are subscribed to the Google Groups "JanusGraph developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgr...@....
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-dev/4009696c-98b9-4a04-ad21-eb0572425de7o%40googlegroups.com.


Re: [PROPOSAL] new repository for janusgraph-foundationdb

Florian Hockmann <f...@...>
 

+1

Great to see progress with the FDB backend.

Am Freitag, 12. Juni 2020 08:44:46 UTC+2 schrieb Jan Jansen:

My vote: +1

Greetings,
Jan

On Thursday, June 11, 2020 at 4:32:37 PM UTC+2, Jason Plurad wrote:
Several developers from the community have expressed interest in collaborating on a storage backend for FoundationDB. As discussed on this thread, they each have a separate fork (no license changes) from the original experoinc/janusgraph-foundationdb repo that Ted Wilmes created (no updates in ~2 years).

I propose that we create a new project janusgraph-foundationdb under the JanusGraph org so that developers can work together to merge their forks and move forward.

Let's give it 72 hours for lazy consensus.

I'm +1 for this proposal.

-- Jason


Re: [PROPOSAL] new repository for janusgraph-foundationdb

Jan Jansen <faro...@...>
 

My vote: +1

Greetings,
Jan

On Thursday, June 11, 2020 at 4:32:37 PM UTC+2, Jason Plurad wrote:
Several developers from the community have expressed interest in collaborating on a storage backend for FoundationDB. As discussed on this thread, they each have a separate fork (no license changes) from the original experoinc/janusgraph-foundationdb repo that Ted Wilmes created (no updates in ~2 years).

I propose that we create a new project janusgraph-foundationdb under the JanusGraph org so that developers can work together to merge their forks and move forward.

Let's give it 72 hours for lazy consensus.

I'm +1 for this proposal.

-- Jason


Violation of UNIQUE INDEX deleted the property from the all the all ready existed vertex

krishna...@...
 

Hi,

i have created a Unique Composite Index for vertices with id, name properties.
  1. create a vertex with an id(-109), name("test").
  2. take the already created vertex which is having different id(-201654672484678624),name('test") which are having few properties like age, createdTime, gender, etc
  3. try to update the already available vertex stated in point 2 with id(-109) of vertex stated in point 1
  4. then the following exception is thrown which is expected
    1. Adding this property for key [id] and value [-109] violates a uniqueness constraint [newUniqueIndex]Type ':help' or ':h' for help.Display stack trace? [yN]yorg.janusgraph.core.SchemaViolationException: Adding this property for key [id] and value [-109] violates a uniqueness constraint [newUniqueIndex]{{ at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.addProperty(StandardJanusGraphTx.java:817)}}{{ at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.addProperty(StandardJanusGraphTx.java:745)}}{{ at org.janusgraph.graphdb.vertices.AbstractVertex.property(AbstractVertex.java:152)}}{{ at org.janusgraph.core.JanusGraphVertex.property(JanusGraphVertex.java:72)}}{{ at org.janusgraph.core.JanusGraphVertex.property(JanusGraphVertex.java:33}}{{}}
  5. But the unexpected thing happening is the id(-201654672484678624) property of the vertex stated in point is removed from it
Note: i am using cassandra as backend and ES for indexing

can you please help me to solve the issue as there is a loss in data as we can't retrive the data which is lost.


attaching the console logs of reproducing the issue through gremlin console


Thanks
Krishna Jalla


Re: [PROPOSAL] new repository for janusgraph-foundationdb

Ted Wilmes <twi...@...>
 

Hey guys, been out of office so catching up on this. Excited to see the FDB interest.

Vote: +1

Thanks,
Ted


Official support for FoundationDB?

Lakshay Rastogi <lakshay1...@...>
 

I would like to offer my assistance and get involved too!


Re: [PROPOSAL] new repository for janusgraph-foundationdb

Rafael Fernandes <luizr...@...>
 

absolutely: +1

rafa


On Thursday, June 11, 2020 at 10:35:57 AM UTC-4, Oleksandr Porunov wrote:
My vote: +1

Best regards,
Oleksandr

On Thursday, June 11, 2020 at 5:32:37 PM UTC+3, Jason Plurad wrote:
Several developers from the community have expressed interest in collaborating on a storage backend for FoundationDB. As discussed on this thread, they each have a separate fork (no license changes) from the original experoinc/janusgraph-foundationdb repo that Ted Wilmes created (no updates in ~2 years).

I propose that we create a new project janusgraph-foundationdb under the JanusGraph org so that developers can work together to merge their forks and move forward.

Let's give it 72 hours for lazy consensus.

I'm +1 for this proposal.

-- Jason


Re: Janusgraph functionality issue

Dmitry Kovalev <dk.g...@...>
 

BTW, the doc suggests that you start with signing the CLA before forking and raising pull request. In case of individual contribution this is easy and fast, but for corporate route (which I am guessing you have to go through) it can be a long and unexciting part of contribution process. 

Likewise, getting your changes to be reviewed can also take time, depending on how big and complex your change is, and how many people are familiar enough with that part of code.

So I suggest that you raise the PR as early in the process as possible, and the CLA bit can catch up (while it's still obviously required before your contribution can be finally accepted)

On Thu, 11 Jun 2020 at 11:17, Dmitry Kovalev <dk.g...@...> wrote:
Hi Shiva,

Thanks for taking time to dive into the issue and deciding to contribute.

Please have a look at this page: https://github.com/JanusGraph/janusgraph/blob/master/CONTRIBUTING.md - it describes the steps to contribute, which are pretty standard.

Thanks,
Dmitry

On Thu, 11 Jun 2020 at 10:40, <shivain...@...> wrote:
Hello Everyone,

I have found the root cause for this issue and also have the fix for the same.

Please guide me if I can contribute for this project as I'm new to this community.

Thanks
Shiva

On Monday, June 8, 2020 at 6:50:04 PM UTC+5:30, Shiva Krishnan wrote:
Hi,

I have noticed a corruption in index after reindexing vertex-centric indices with direction 'IN'.

Issue : A self-link is getting added to every vertex after reindexing.

To demonstrate this, I have taken a very small graph consists of two vertices (A & B) and one edge connecting the two.
Edge Label -> link
Edge has one property key (Integer) -> assocKind

//Creating the Vertex Centrix Index
gremlin > edgeLabel = mgmt.getEdgeLabel("link");
gremlin > assocKind= mgmt.getPropertyKey("assocKind")
gremlin > mgmt.buildEdgeIndex(edgeLabel, "myVertexCentricIndex", Direction.IN, assocKind);

Please note i have given the direction IN for the index.


//Creating the Edge from A to B: (a and b are vertices)
a.addEdge("link",b,"assocKind",1)

Output Before Reindexing :
gremlin> g.V().has('name' , 'A').inE().hasLabel('link').has('assocKind',1)
//no IN edges to vertex A  Correct

gremlin> g.V().has('name' , 'A').outE().hasLabel('link').has('assocKind',1)
==>e[4e1f-b6g-1bit-1pqw][14488-link->80024]   Correct

Now I ran the reindex command
// 'index' is the vertex centric index which is created above
gremlin> m.updateIndex(index, SchemaAction.REINDEX).get()

Output After Reindexing :
gremlin> g.V().has('_objId','A').inE().hasLabel('link').has('assocKind',1)
==>e[4e1f-b6g-1bit-b6g][14488-link->14488] Wrong

gremlin> g.V().has('_objId','A').bothE().hasLabel('link').has('assocKind',1)
==>e[4e1f-b6g-1bit-1pqw][14488-link->80024]
==>e[4e1f-b6g-1bit-b6g][14488-link->14488] Unexpected Link

This issue happens in janusgraph 0.5.2 also.

Thanks
Shiva

--
You received this message because you are subscribed to the Google Groups "JanusGraph developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgr...@....
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-dev/ea55b56a-dc8b-456e-b8c9-071de9b9f35ao%40googlegroups.com.


Re: [PROPOSAL] new repository for janusgraph-foundationdb

Oleksandr Porunov <alexand...@...>
 

My vote: +1

Best regards,
Oleksandr

On Thursday, June 11, 2020 at 5:32:37 PM UTC+3, Jason Plurad wrote:
Several developers from the community have expressed interest in collaborating on a storage backend for FoundationDB. As discussed on this thread, they each have a separate fork (no license changes) from the original experoinc/janusgraph-foundationdb repo that Ted Wilmes created (no updates in ~2 years).

I propose that we create a new project janusgraph-foundationdb under the JanusGraph org so that developers can work together to merge their forks and move forward.

Let's give it 72 hours for lazy consensus.

I'm +1 for this proposal.

-- Jason


[PROPOSAL] new repository for janusgraph-foundationdb

Jason Plurad <plu...@...>
 

Several developers from the community have expressed interest in collaborating on a storage backend for FoundationDB. As discussed on this thread, they each have a separate fork (no license changes) from the original experoinc/janusgraph-foundationdb repo that Ted Wilmes created (no updates in ~2 years).

I propose that we create a new project janusgraph-foundationdb under the JanusGraph org so that developers can work together to merge their forks and move forward.

Let's give it 72 hours for lazy consensus.

I'm +1 for this proposal.

-- Jason


Re: Janusgraph functionality issue

Dmitry Kovalev <dk.g...@...>
 

Hi Shiva,

Thanks for taking time to dive into the issue and deciding to contribute.

Please have a look at this page: https://github.com/JanusGraph/janusgraph/blob/master/CONTRIBUTING.md - it describes the steps to contribute, which are pretty standard.

Thanks,
Dmitry

On Thu, 11 Jun 2020 at 10:40, <shivain...@...> wrote:
Hello Everyone,

I have found the root cause for this issue and also have the fix for the same.

Please guide me if I can contribute for this project as I'm new to this community.

Thanks
Shiva

On Monday, June 8, 2020 at 6:50:04 PM UTC+5:30, Shiva Krishnan wrote:
Hi,

I have noticed a corruption in index after reindexing vertex-centric indices with direction 'IN'.

Issue : A self-link is getting added to every vertex after reindexing.

To demonstrate this, I have taken a very small graph consists of two vertices (A & B) and one edge connecting the two.
Edge Label -> link
Edge has one property key (Integer) -> assocKind

//Creating the Vertex Centrix Index
gremlin > edgeLabel = mgmt.getEdgeLabel("link");
gremlin > assocKind= mgmt.getPropertyKey("assocKind")
gremlin > mgmt.buildEdgeIndex(edgeLabel, "myVertexCentricIndex", Direction.IN, assocKind);

Please note i have given the direction IN for the index.


//Creating the Edge from A to B: (a and b are vertices)
a.addEdge("link",b,"assocKind",1)

Output Before Reindexing :
gremlin> g.V().has('name' , 'A').inE().hasLabel('link').has('assocKind',1)
//no IN edges to vertex A  Correct

gremlin> g.V().has('name' , 'A').outE().hasLabel('link').has('assocKind',1)
==>e[4e1f-b6g-1bit-1pqw][14488-link->80024]   Correct

Now I ran the reindex command
// 'index' is the vertex centric index which is created above
gremlin> m.updateIndex(index, SchemaAction.REINDEX).get()

Output After Reindexing :
gremlin> g.V().has('_objId','A').inE().hasLabel('link').has('assocKind',1)
==>e[4e1f-b6g-1bit-b6g][14488-link->14488] Wrong

gremlin> g.V().has('_objId','A').bothE().hasLabel('link').has('assocKind',1)
==>e[4e1f-b6g-1bit-1pqw][14488-link->80024]
==>e[4e1f-b6g-1bit-b6g][14488-link->14488] Unexpected Link

This issue happens in janusgraph 0.5.2 also.

Thanks
Shiva

--
You received this message because you are subscribed to the Google Groups "JanusGraph developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgr...@....
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-dev/ea55b56a-dc8b-456e-b8c9-071de9b9f35ao%40googlegroups.com.


Re: Janusgraph functionality issue

shivain...@...
 

Hello Everyone,

I have found the root cause for this issue and also have the fix for the same.

Please guide me if I can contribute for this project as I'm new to this community.

Thanks
Shiva

On Monday, June 8, 2020 at 6:50:04 PM UTC+5:30, Shiva Krishnan wrote:
Hi,

I have noticed a corruption in index after reindexing vertex-centric indices with direction 'IN'.

Issue : A self-link is getting added to every vertex after reindexing.

To demonstrate this, I have taken a very small graph consists of two vertices (A & B) and one edge connecting the two.
Edge Label -> link
Edge has one property key (Integer) -> assocKind

//Creating the Vertex Centrix Index
gremlin > edgeLabel = mgmt.getEdgeLabel("link");
gremlin > assocKind= mgmt.getPropertyKey("assocKind")
gremlin > mgmt.buildEdgeIndex(edgeLabel, "myVertexCentricIndex", Direction.IN, assocKind);

Please note i have given the direction IN for the index.


//Creating the Edge from A to B: (a and b are vertices)
a.addEdge("link",b,"assocKind",1)

Output Before Reindexing :
gremlin> g.V().has('name' , 'A').inE().hasLabel('link').has('assocKind',1)
//no IN edges to vertex A  Correct

gremlin> g.V().has('name' , 'A').outE().hasLabel('link').has('assocKind',1)
==>e[4e1f-b6g-1bit-1pqw][14488-link->80024]   Correct

Now I ran the reindex command
// 'index' is the vertex centric index which is created above
gremlin> m.updateIndex(index, SchemaAction.REINDEX).get()

Output After Reindexing :
gremlin> g.V().has('_objId','A').inE().hasLabel('link').has('assocKind',1)
==>e[4e1f-b6g-1bit-b6g][14488-link->14488] Wrong

gremlin> g.V().has('_objId','A').bothE().hasLabel('link').has('assocKind',1)
==>e[4e1f-b6g-1bit-1pqw][14488-link->80024]
==>e[4e1f-b6g-1bit-b6g][14488-link->14488] Unexpected Link

This issue happens in janusgraph 0.5.2 also.

Thanks
Shiva


Re: Official support for FoundationDB?

f.gri...@...
 

I'm of course interested and would like to offer my support for everything that needs to be done.

-- Florian

rngcntr -- Sorry, I didn't see your first name... are you interested as well?


Re: Official support for FoundationDB?

Jason Plurad <plu...@...>
 

Sounds like there is some interest here from Chris (blindenvy) and Yuvaraj (skyrocknroll) on a common fork.

rngcntr -- Sorry, I didn't see your first name... are you interested as well?

I'd think the next steps would be to get a vote approved first, then we'd need a repo to seed to the project. There are a few options there on how to proceed with that, but let's get the above answered first before moving to a vote.

-- Jason

On Friday, June 5, 2020 at 9:51:01 PM UTC-4, Yuvaraj Loganathan wrote:
We would be really interested in a common fork.


Janusgraph functionality issue

shivain...@...
 

Hi,

I have noticed a corruption in index after reindexing vertex-centric indices with direction 'IN'.

Issue : A self-link is getting added to every vertex after reindexing.

To demonstrate this, I have taken a very small graph consists of two vertices (A & B) and one edge connecting the two.
Edge Label -> link
Edge has one property key (Integer) -> assocKind

//Creating the Vertex Centrix Index
gremlin > edgeLabel = mgmt.getEdgeLabel("link");
gremlin > assocKind= mgmt.getPropertyKey("assocKind")
gremlin > mgmt.buildEdgeIndex(edgeLabel, "myVertexCentricIndex", Direction.IN, assocKind);

Please note i have given the direction IN for the index.


//Creating the Edge from A to B: (a and b are vertices)
a.addEdge("link",b,"assocKind",1)

Output Before Reindexing :
gremlin> g.V().has('name' , 'A').inE().hasLabel('link').has('assocKind',1)
//no IN edges to vertex A  Correct

gremlin> g.V().has('name' , 'A').outE().hasLabel('link').has('assocKind',1)
==>e[4e1f-b6g-1bit-1pqw][14488-link->80024]   Correct

Now I ran the reindex command
// 'index' is the vertex centric index which is created above
gremlin> m.updateIndex(index, SchemaAction.REINDEX).get()

Output After Reindexing :
gremlin> g.V().has('_objId','A').inE().hasLabel('link').has('assocKind',1)
==>e[4e1f-b6g-1bit-b6g][14488-link->14488] Wrong

gremlin> g.V().has('_objId','A').bothE().hasLabel('link').has('assocKind',1)
==>e[4e1f-b6g-1bit-1pqw][14488-link->80024]
==>e[4e1f-b6g-1bit-b6g][14488-link->14488] Unexpected Link

This issue happens in janusgraph 0.5.2 also.

Thanks
Shiva


Edge Filter in Janusgraph when working with Spark

rafi ansari <rafi1...@...>
 

Hi All,

I am currently working on using Janusgraph in batch mode using Spark.

I am facing a problem on filtering the edges by label.

Below are the specifications:
Spark = 2.4.5
Janusgraph = 0.5.0

Below is the configuration file for Spark:

conf.setProperty("gremlin.graph", "org.apache.tinkerpop.gremlin.hadoop.structure.HadoopGraph")
conf.setProperty("gremlin.hadoop.graphReader", "org.janusgraph.hadoop.formats.cql.CqlInputFormat")
conf.setProperty("gremlin.hadoop.graphWriter", "org.apache.hadoop.mapreduce.lib.output.NullOutputFormat")
conf.setProperty("spark.cassandra.connection.host", "127.0.0.1")
conf.setProperty("janusgraphmr.ioformat.conf.storage.backend", "cql")
conf.setProperty("janusgraphmr.ioformat.conf.storage.hostname", "127.0.0.1")
conf.setProperty("janusgraphmr.ioformat.conf.storage.port", 9042)
conf.setProperty("janusgraphmr.ioformat.conf.storage.cql.keyspace", "graph_db_1")
conf.setProperty("janusgraphmr.ioformat.conf.index.search.backend", "elasticsearch")
conf.setProperty("janusgraphmr.ioformat.conf.index.search.hostname", "127.0.0.1")
conf.setProperty("janusgraphmr.ioformat.conf.index.search.port", 9200)
conf.setProperty("janusgraphmr.ioformat.conf.index.search.index-name", "graph_1")
conf.setProperty("cassandra.input.partitioner.class","org.apache.cassandra.dht.Murmur3Partitioner")
conf.setProperty("cassandra.input.widerows",true)
conf.setProperty("spark.serializer", "org.apache.spark.serializer.KryoSerializer")
conf.setProperty("spark.kryo.registrator", "org.janusgraph.hadoop.serialize.JanusGraphKryoRegistrator")

Below is the Spark code using newAPIHadoopRDD 


val hadoopConfiguration = ConfUtil.makeHadoopConfiguration(conf)

val rdd: RDD[(NullWritable, VertexWritable)] =spark.sparkContext.newAPIHadoopRDD(hadoopConfiguration,hadoopConfiguration.
getClass(Constants.GREMLIN_HADOOP_GRAPH_READER, classOf[InputFormat[NullWritable, VertexWritable]]).
asInstanceOf[Class[InputFormat[NullWritable, VertexWritable]]],classOf[NullWritable], classOf[VertexWritable])

The above lines give an RDD as output.

rdd: org.apache.spark.rdd.RDD[(org.apache.hadoop.io.NullWritable, org.apache.tinkerpop.gremlin.hadoop.structure.io.VertexWritable)]

rdd.map{case (x,y)=>y.asInstanceOf[VertexWritable]}

res17: Array[String] = Array(v[8344], v[12440], v[4336], v[4320], v[4136], v[8416], v[8192], v[4248], v[4344], v[8432], v[12528], v[4096])

From the res17 above, not sure how to filter the edges by labels


TIA

Regards

Rafi


Re: Official support for FoundationDB?

Yuvaraj Loganathan <uva...@...>
 

We would be really interested in a common fork.


Re: Official support for FoundationDB?

Jason Plurad <plu...@...>
 

+1. I support having a janusgraph-foundationdb repository under the main JanusGraph org on GitHub. There is precedent for this with janusgraph-docker, language-specific client bindings (janusgraph-dotnet and janusgraph-python), and janusgraph-ambari.

It comes down to whether there is a consensus desire to collaborate together instead of having separate forks. There would be some work from those fork authors to agree on how to merge those forks into the one that will serve as the initial repo. There would be more restrictions on getting code committed since the project would be expected to have a code review and approval process in place, like the other JanusGraph repos. Keep in mind that having a repo under the JanusGraph org doesn't mean that it needs to be production worthy from the start.

Thanks to both of you for getting the ball rolling here with FoundationDB. Happy to hear more feedback/discussion from the community.

-- Jason

On Saturday, May 30, 2020 at 5:57:13 PM UTC-4, Christopher Jackson wrote:
I agree we would have to clearly state all limitations the adapter has so everyone is aware of them. 

Would love to hear additional feedback from the community on the topic of making this adapter official. 

On Monday, May 25, 2020 at 3:21:22 AM UTC-4, f....@... wrote:
Hi Christopher,
thanks for offering your support!

I did indeed put a lot of effort into my fork (rngcntr), but I worked alone and rewrote a significant amount of code so it definitely lacks an in-depth review. Other than that, most of what I did was stress testing and finding bugs that need to be fixed. There are stil some open issues but all in all I'm quite happy with the current state of the storage adapter.

What still worries me is the effects of the transaction time limit of FDB. Officially supporting FDB as a storage backend would currently leave the user in charge of handling these problems. Retrying can not solve all problems as some queries will never make it through the 5s limit, even when tried over and over again. This results in the FDB adapter only supporting a subset of all possible queries and this should be stated clearly in case it is officially released.

201 - 220 of 1596