Date   

[REPORT] Open Pull Requests 2018-09-04

Jason Plurad <plu...@...>
 

Open Pull Requests 2018-09-04

This report contains a summary of the open pull requests in the JanusGraph project ecosystem. Everybody interested is encouraged to submit review comments and approvals, but only approvals from committers are considered binding for merge. Please refer to the pull request policy for details.


janusgraph


Release 0.2.2 (2 open)


PR-1118 Issue #1114: fix GremlinServer StackOverflowError

  • Opened 2018-06-14 (3 months ago) by boliza
  • Comments from chupman, sjudeng, pluradj
    • Missing unit test case
  • 0 approvals

PR-1237 Issue #1157: fix REINDEX delay on the newly created index in empty DB

  • Opened 2018-08-29 (6 days ago) by dengziming (First-time contributor)
  • 0 comments
  • 0 approvals

Release 0.3.1 (4 open)


PR-929 Bug Fix (issue #764) & LIST and SET support for Lucene (issue #670)

  • Opened 2018-02-13 (7 months ago) by Lakedaemon
  • 43 comments: analytically, davidclement90, pluradj, sjudeng, twilmes
  • 1 approval: analytically

PR-1152 Add projects built with JanusGraph

  • Opened 2018-07-07 (2 months ago) by mbrukman
  • 5 comments: amcp, hsaputra, jerryjch, pluradj
  • 1 approval: mbrukman

PR-1214 Create a HBase 2 profile #1213

  • Opened 2018-08-08 (26 days ago) by jerryjch
  • 6 comments: analytically, hsaputra
  • 1 approval: jerryjch

PR-1230 Add support for kerberized solr instance (#1138)

  • Opened 2018-08-23 (12 days ago) by blindenvy (First-time contributor)
  • 5 comments: jerryjch
  • 0 approvals

Release 0.4.0 (0 open)


0 pull requests


No Milestone (10 open)


10 pull requests


janusgraph-ambari (1 open)


PR-2 Plugin copied over from chupman/janusgraph-ambari-service

  • Opened 2018-08-27 (8 days ago) by chupman
  • 12 comments: pluradj
  • 1 approval: chupman

janusgraph-docker (1 open)


PR-1 [IN PROGRESS] Creating docker image build for JanusGraph 0.2.0

  • Opened 2018-08-30 (5 days ago) by twilmes
  • 24 comments: chrislbs, FlorianHockmann, sjudeng
  • 0 approvals

janusgraph-dotnet (1 open)


PR-1 Initial version of JanusGraph.Net

  • Opened 2018-08-16 (19 days ago) by FlorianHockmann
  • 33 comments: mbrukman, pluradj
  • 1 approval: FlorianHockmann

janusgraph-python (1 open)


PR-4 Initial version of JanusGraph Python client drivers

  • Opened 2018-09-04 (today) by debasishdebs
  • 0 comments
  • 0 approvals


Re: TinkerPop 3.2 vs TinkerPop 3.3 Serialization difference for Predicate class

Jason Plurad <plu...@...>
 

Apologies for the slow response. Back from a long holiday weekend in the USA.

My opinion is that we need to keep pushing forward on releases and actively encourage users to move up to the latest release. The JanusGraph policy has been for new features to go into the master branch only. If there are products out there in production on JanusGraph 0.2, they aren't using GLVs or are using the plain TinkerPop GLVs and decided that was good enough.

The new dotnet and python drivers are separate from the main repository, so it is possible that they could have a different approach to what JanusGraph releases they want to support. However, I'd suggest that we need to continue sticking with primarily supporting the latest release, especially since GLVs are still a relatively new idea in TinkerPop. Managing the dependency differences between the TinkerPop versions is not worth the extra effort, especially considering that TinkerPop 3.4 is on the horizon.

We haven't been assigning reviewers to specific PRs. As a primarily volunteer community, we rely on committers going to the queue and reviewing what is out there. We discussed the idea of the PR queue email to the mailing list previously, and I will get that started this week. It is especially important now that the PRs are spread across several repositories.

I created issue 1243 to update the docs on the PR workflow.


On Tuesday, September 4, 2018 at 4:52:49 AM UTC-4, Florian Hockmann wrote:
I'm not against merging this into 0.2, but I'm actually not sure whether we want to do that. My understanding is that new features go mainly into master and that the 0.2 branch mostly gets bug fixes and small improvements.
Users should upgrade to 0.3.0 in my opinion when they want to use new features like JanusGraph predicates in languages like Python and C#.

Also I'll need a small walkthrough on what things need to be done for the same.

I'm actually not sure about the workflow myself. Usually, PRs are merged into a feature branch and then from there up into master and not the other way around. One solution is probably to simply cherry-pick the commit. Can someone else shed some light on this?

I was unable to add You, Jason, or Misha to reviewers. Maybe because I'm not committer?

Yes, commit permissions are necessary to add reviewers as far as I know. But I will try to find some time to review this.

Am Sonntag, 2. September 2018 11:41:34 UTC+2 schrieb Debasish Kanhar:
Hi Florian,

Thanks for the link to PR. Didn't know I had stumbled upon the same issue already. Oops :-P

Anyways, quick questions, are there any plan to merge those changes to 0.2 branch? Reason I'm asking is because I feel a lot of users still haven't migrated to 0.3 series of JanusGraph, meaning that the drivers will still remain far from being usable. I did see your code changes in the PR, and they seem like few changes, which can be implemented. But wanted to know what are plans for merging it to 0.2 branch, and if no one else is working, I can take that up. Also I'll need a small walkthrough on what things need to be done for the same.

Also, I've create a PR #2 to push initial version of library to official repo. I was unable to add You, Jason, or Misha to reviewers. Maybe because I'm not committer? Can you check that out. We will need your comments to push that out.

On Monday, 27 August 2018 19:17:26 UTC+5:30, Florian Hockmann wrote:
JanusGraph simply lacks the required GraphSON serializers and deserializers for predicates. I created #1060 to track this issue and fixed it with PR #1061 which was merged into master and not into the 0.2 branch and is therefore not present in version 0.2.1.

That is also why I decided to only support JanusGraph 0.3.0 with JanusGraph.Net.

Am Freitag, 24. August 2018 18:46:25 UTC+2 schrieb Debasish Kanhar:
And, the following are serializers registered in my gremlin-server.yaml for JanusGraph 0.2.1 (I use the default packeged set of serializers):
serializers:
 
- { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV1d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
 
- { className: org.apache.tinkerpop.gremlin.driver.ser.GryoLiteMessageSerializerV1d0, config: {ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
 
- { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV1d0, config: { serializeResultToString: true }}
 
- { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerGremlinV1d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistryV1d0] }}
 
- { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerGremlinV2d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
 
- { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV1d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistryV1d0] }}




On Friday, 24 August 2018 22:10:52 UTC+5:30, Debasish Kanhar wrote:
Hi All,

So as you all know I've been developing Client drivers for JanusGraph. I was able to test my initial version of library against JanusGraph 0.3.0 which uses TP 3.3.3. I now want to test the same set of driver for JanusGraph 0.2.1 which used TP 3.2.9 but I feel I've hit a bump with respect to Serialization for Predicate class.

So, my existent features like Adding of GeoShapes, Querying GeoShape and RelationIdentifier serialization/deserialization works. But any of my features leveraging TinkerPop's Predicate (PSerializer) doesn't work. I feel like this is some sort of serialization issue.

So, when I've implemented TextContainsFuzzy, which was working against JG 0.3.0/TP 3.3.3 (Using GraphSON 3.0) but the same doesn't work on GraphSON 2.0.

The following is my serialized object being sent to Gremlin-Server (Gremlin-server logs specify this)

Using TextFuzzy:
Error message:

org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV2d0  - Request [PooledUnsafeDirectByteBuf(ridx: 303, widx: 303, cap: 337)] could not be deserialized by org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV2d0.
org
.apache.tinkerpop.shaded.jackson.databind.JsonMappingException: Could not deserialize the JSON value as required. Nested exception: org.apache.tinkerpop.shaded.jackson.databind.JsonMappingException: Could not deserialize the JSON value as required. Nested exception: java.lang.IllegalStateException: org.apache.tinkerpop.gremlin.process.traversal.P.textContainsFuzzy([Ljava.lang.Object;)

And the serialized object being sent:

{
 
"requestId": {
   
"@type": "g:UUID",
   
"@value": "7328b342-2ed3-43a5-b589-f6688b264ee7"
 
},
 
"processor": "traversal",
 
"op": "bytecode",
 
"args": {
   
"gremlin": {
     
"@type": "g:Bytecode",
     
"@value": {
       
"step": [
         
[
           
"V"
         
],
         
[
           
"has",
           
"name",
           
{
             
"@type": "g:P",
             
"@value": {
               
"predicate": "textContainsFuzzy",
               
"value": "herculeas"
             
}
           
}
         
]
       
]
     
}
   
},
   
"aliases": {
     
"g": "gg"
   
}
 
}
}

The serialized object really looks good, like the way it is expected. Doesn't it? Or else I cant seem to dechiper what's wrong.

I'm also facing similar errors which using the same PSerializer class for extending Geo Predicates like geoWithin, but I guess both are tied to same problem, and if one gets solved, other will also :-)

If needed, I can attach the serialized object for geoWithin too. But any help in debugging this will be really helpful, as this is last piece of development I'm expecting before releasing version 1 of drivers with support for both GraphSON 2.0 and GraphSON 3.0

Thanks.


Re: TinkerPop 3.2 vs TinkerPop 3.3 Serialization difference for Predicate class

Florian Hockmann <f...@...>
 

I'm not against merging this into 0.2, but I'm actually not sure whether we want to do that. My understanding is that new features go mainly into master and that the 0.2 branch mostly gets bug fixes and small improvements.
Users should upgrade to 0.3.0 in my opinion when they want to use new features like JanusGraph predicates in languages like Python and C#.

Also I'll need a small walkthrough on what things need to be done for the same.

I'm actually not sure about the workflow myself. Usually, PRs are merged into a feature branch and then from there up into master and not the other way around. One solution is probably to simply cherry-pick the commit. Can someone else shed some light on this?

I was unable to add You, Jason, or Misha to reviewers. Maybe because I'm not committer?

Yes, commit permissions are necessary to add reviewers as far as I know. But I will try to find some time to review this.

Am Sonntag, 2. September 2018 11:41:34 UTC+2 schrieb Debasish Kanhar:
Hi Florian,

Thanks for the link to PR. Didn't know I had stumbled upon the same issue already. Oops :-P

Anyways, quick questions, are there any plan to merge those changes to 0.2 branch? Reason I'm asking is because I feel a lot of users still haven't migrated to 0.3 series of JanusGraph, meaning that the drivers will still remain far from being usable. I did see your code changes in the PR, and they seem like few changes, which can be implemented. But wanted to know what are plans for merging it to 0.2 branch, and if no one else is working, I can take that up. Also I'll need a small walkthrough on what things need to be done for the same.

Also, I've create a PR #2 to push initial version of library to official repo. I was unable to add You, Jason, or Misha to reviewers. Maybe because I'm not committer? Can you check that out. We will need your comments to push that out.

On Monday, 27 August 2018 19:17:26 UTC+5:30, Florian Hockmann wrote:
JanusGraph simply lacks the required GraphSON serializers and deserializers for predicates. I created #1060 to track this issue and fixed it with PR #1061 which was merged into master and not into the 0.2 branch and is therefore not present in version 0.2.1.

That is also why I decided to only support JanusGraph 0.3.0 with JanusGraph.Net.

Am Freitag, 24. August 2018 18:46:25 UTC+2 schrieb Debasish Kanhar:
And, the following are serializers registered in my gremlin-server.yaml for JanusGraph 0.2.1 (I use the default packeged set of serializers):
serializers:
 
- { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV1d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
 
- { className: org.apache.tinkerpop.gremlin.driver.ser.GryoLiteMessageSerializerV1d0, config: {ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
 
- { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV1d0, config: { serializeResultToString: true }}
 
- { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerGremlinV1d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistryV1d0] }}
 
- { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerGremlinV2d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
 
- { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV1d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistryV1d0] }}




On Friday, 24 August 2018 22:10:52 UTC+5:30, Debasish Kanhar wrote:
Hi All,

So as you all know I've been developing Client drivers for JanusGraph. I was able to test my initial version of library against JanusGraph 0.3.0 which uses TP 3.3.3. I now want to test the same set of driver for JanusGraph 0.2.1 which used TP 3.2.9 but I feel I've hit a bump with respect to Serialization for Predicate class.

So, my existent features like Adding of GeoShapes, Querying GeoShape and RelationIdentifier serialization/deserialization works. But any of my features leveraging TinkerPop's Predicate (PSerializer) doesn't work. I feel like this is some sort of serialization issue.

So, when I've implemented TextContainsFuzzy, which was working against JG 0.3.0/TP 3.3.3 (Using GraphSON 3.0) but the same doesn't work on GraphSON 2.0.

The following is my serialized object being sent to Gremlin-Server (Gremlin-server logs specify this)

Using TextFuzzy:
Error message:

org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV2d0  - Request [PooledUnsafeDirectByteBuf(ridx: 303, widx: 303, cap: 337)] could not be deserialized by org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV2d0.
org
.apache.tinkerpop.shaded.jackson.databind.JsonMappingException: Could not deserialize the JSON value as required. Nested exception: org.apache.tinkerpop.shaded.jackson.databind.JsonMappingException: Could not deserialize the JSON value as required. Nested exception: java.lang.IllegalStateException: org.apache.tinkerpop.gremlin.process.traversal.P.textContainsFuzzy([Ljava.lang.Object;)

And the serialized object being sent:

{
 
"requestId": {
   
"@type": "g:UUID",
   
"@value": "7328b342-2ed3-43a5-b589-f6688b264ee7"
 
},
 
"processor": "traversal",
 
"op": "bytecode",
 
"args": {
   
"gremlin": {
     
"@type": "g:Bytecode",
     
"@value": {
       
"step": [
         
[
           
"V"
         
],
         
[
           
"has",
           
"name",
           
{
             
"@type": "g:P",
             
"@value": {
               
"predicate": "textContainsFuzzy",
               
"value": "herculeas"
             
}
           
}
         
]
       
]
     
}
   
},
   
"aliases": {
     
"g": "gg"
   
}
 
}
}

The serialized object really looks good, like the way it is expected. Doesn't it? Or else I cant seem to dechiper what's wrong.

I'm also facing similar errors which using the same PSerializer class for extending Geo Predicates like geoWithin, but I guess both are tied to same problem, and if one gets solved, other will also :-)

If needed, I can attach the serialized object for geoWithin too. But any help in debugging this will be really helpful, as this is last piece of development I'm expecting before releasing version 1 of drivers with support for both GraphSON 2.0 and GraphSON 3.0

Thanks.


Re: [PROPOSAL] new repository for .NET driver

Debasish Kanhar <d.k...@...>
 

@Jason:

I created a initial PR#2 to merge my codebase into official JanusGraph-Python repo you created. While the PR was successful, I wasn't able to add any reviewers, you or Florian or Misha. I'll need your inputs regarding copyright statements, liscence texts and other legal/similar formalities as well as other comments.

Also, had a query regarding status of merging fix #1060 into 0.2 stream of branched for JanusGraph, and if needed, I can work on it as that is currently only merged into master and 0.3. The discussion here will need your comments too :-)

Thanks.

On Wednesday, 15 August 2018 04:15:27 UTC+5:30, Jason Plurad wrote:
Apologies for the delay, but I've created

* https://github.com/JanusGraph/janusgraph-dotnet
* https://github.com/JanusGraph/janusgraph-python

Committers and maintainers should have access to it.


On Sunday, August 5, 2018 at 9:25:48 AM UTC-4, Florian Hockmann wrote:
Looks like we have reached consensus here. I have already created a stand-alone version of the library in its own repo:


I can create a PR to transfer it to JanusGraph as soon as we have a repo ready to target the PR against.

Am Dienstag, 31. Juli 2018 20:47:11 UTC+2 schrieb Misha Brukman:
+1

Makes sense that client libraries should be in a separate repo than the server:
  • enables a separate release cadence
  • allows for a different set of maintainers, even in the same org
  • each client library will be written in a different language
Having said that, this means that we'll have to keep track of version compatibilities between client library versions + server versions. Hopefully, they will be mostly backwards-compatible until a major version change.

On Sat, Jul 28, 2018 at 11:06 AM, Jason Plurad <p...@...> wrote:
Florian Hockmann is ready to contribute a .NET driver for JanusGraph. The pull request is currently found  in the main janusgraph repo.

The driver extends the TinkerPop .NET driver with additional capability for JanusGraph types and predicates. In a previous dev list discussion thread, he and I discussed making a separate repository for this body of work. I propose the name janusgraph-dotnet for the repository.

Let's give it 72 hours for lazy consensus.

I'm +1 for this. Looking forward to getting better driver support as part of the broader project.

--
You received this message because you are subscribed to the Google Groups "JanusGraph developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-dev/91f5a3bc-a220-45ff-a466-57c198f43506%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: TinkerPop 3.2 vs TinkerPop 3.3 Serialization difference for Predicate class

Debasish Kanhar <d.k...@...>
 

Hi Florian,

Thanks for the link to PR. Didn't know I had stumbled upon the same issue already. Oops :-P

Anyways, quick questions, are there any plan to merge those changes to 0.2 branch? Reason I'm asking is because I feel a lot of users still haven't migrated to 0.3 series of JanusGraph, meaning that the drivers will still remain far from being usable. I did see your code changes in the PR, and they seem like few changes, which can be implemented. But wanted to know what are plans for merging it to 0.2 branch, and if no one else is working, I can take that up. Also I'll need a small walkthrough on what things need to be done for the same.

Also, I've create a PR #2 to push initial version of library to official repo. I was unable to add You, Jason, or Misha to reviewers. Maybe because I'm not committer? Can you check that out. We will need your comments to push that out.

On Monday, 27 August 2018 19:17:26 UTC+5:30, Florian Hockmann wrote:
JanusGraph simply lacks the required GraphSON serializers and deserializers for predicates. I created #1060 to track this issue and fixed it with PR #1061 which was merged into master and not into the 0.2 branch and is therefore not present in version 0.2.1.

That is also why I decided to only support JanusGraph 0.3.0 with JanusGraph.Net.

Am Freitag, 24. August 2018 18:46:25 UTC+2 schrieb Debasish Kanhar:
And, the following are serializers registered in my gremlin-server.yaml for JanusGraph 0.2.1 (I use the default packeged set of serializers):
serializers:
 
- { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV1d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
 
- { className: org.apache.tinkerpop.gremlin.driver.ser.GryoLiteMessageSerializerV1d0, config: {ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
 
- { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV1d0, config: { serializeResultToString: true }}
 
- { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerGremlinV1d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistryV1d0] }}
 
- { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerGremlinV2d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
 
- { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV1d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistryV1d0] }}




On Friday, 24 August 2018 22:10:52 UTC+5:30, Debasish Kanhar wrote:
Hi All,

So as you all know I've been developing Client drivers for JanusGraph. I was able to test my initial version of library against JanusGraph 0.3.0 which uses TP 3.3.3. I now want to test the same set of driver for JanusGraph 0.2.1 which used TP 3.2.9 but I feel I've hit a bump with respect to Serialization for Predicate class.

So, my existent features like Adding of GeoShapes, Querying GeoShape and RelationIdentifier serialization/deserialization works. But any of my features leveraging TinkerPop's Predicate (PSerializer) doesn't work. I feel like this is some sort of serialization issue.

So, when I've implemented TextContainsFuzzy, which was working against JG 0.3.0/TP 3.3.3 (Using GraphSON 3.0) but the same doesn't work on GraphSON 2.0.

The following is my serialized object being sent to Gremlin-Server (Gremlin-server logs specify this)

Using TextFuzzy:
Error message:

org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV2d0  - Request [PooledUnsafeDirectByteBuf(ridx: 303, widx: 303, cap: 337)] could not be deserialized by org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV2d0.
org
.apache.tinkerpop.shaded.jackson.databind.JsonMappingException: Could not deserialize the JSON value as required. Nested exception: org.apache.tinkerpop.shaded.jackson.databind.JsonMappingException: Could not deserialize the JSON value as required. Nested exception: java.lang.IllegalStateException: org.apache.tinkerpop.gremlin.process.traversal.P.textContainsFuzzy([Ljava.lang.Object;)

And the serialized object being sent:

{
 
"requestId": {
   
"@type": "g:UUID",
   
"@value": "7328b342-2ed3-43a5-b589-f6688b264ee7"
 
},
 
"processor": "traversal",
 
"op": "bytecode",
 
"args": {
   
"gremlin": {
     
"@type": "g:Bytecode",
     
"@value": {
       
"step": [
         
[
           
"V"
         
],
         
[
           
"has",
           
"name",
           
{
             
"@type": "g:P",
             
"@value": {
               
"predicate": "textContainsFuzzy",
               
"value": "herculeas"
             
}
           
}
         
]
       
]
     
}
   
},
   
"aliases": {
     
"g": "gg"
   
}
 
}
}

The serialized object really looks good, like the way it is expected. Doesn't it? Or else I cant seem to dechiper what's wrong.

I'm also facing similar errors which using the same PSerializer class for extending Geo Predicates like geoWithin, but I guess both are tied to same problem, and if one gets solved, other will also :-)

If needed, I can attach the serialized object for geoWithin too. But any help in debugging this will be really helpful, as this is last piece of development I'm expecting before releasing version 1 of drivers with support for both GraphSON 2.0 and GraphSON 3.0

Thanks.


Re: [DISCUSS] Official JanusGraph docker image repo

Ted Wilmes <twi...@...>
 

Ok, I moved Mr. Pounds code over into my fork of the new repo and put a "scratch" PR in: https://github.com/JanusGraph/janusgraph-docker/pull/1

Let's move our discussion/feedback on approach over to that PR unless you all think it would be beneficial to continue here on the list.

--Ted


On Monday, August 27, 2018 at 3:11:12 PM UTC-5, Chris Hupman wrote:
I think that makes sense as a starting point. I'm assuming there will be a fair amount of testing, discussions,  and iterations before it gets pushed up as an official image. 

On Monday, August 27, 2018 at 1:05:49 PM UTC-7, Ted Wilmes wrote:
I think I can get to that later this week. I could push Chris' work into a new branch on the new repo and then put a PR in so we could come to some consensus on if that's a good starting point?

On Mon, Aug 27, 2018 at 2:57 PM Chris Hupman <ch...@...> wrote:
One last question, mainly for Expero folks. Is anyone already intending to make the initial commit? I have a conference this week, but can work on this next week if not.

On Monday, August 27, 2018 at 8:07:24 AM UTC-7, Jason Plurad wrote:
https://github.com/JanusGraph/janusgraph-docker is created

On Tuesday, August 21, 2018 at 3:17:00 AM UTC-4, Florian Hockmann wrote:
Yes, +1 for the separate repo.

Am Dienstag, 21. August 2018 03:04:23 UTC+2 schrieb Ted Wilmes:
Hey Chris,
Appreciate the extra stir and yes, I'm in agreement. +1 for a separate repo.

Thanks,
Ted

On Mon, Aug 20, 2018, 3:14 PM Chris Hupman <ch...@...> wrote:
Stirring one more time. Ted and Florian are you guys in favor of the creation of a repo for an official JanusGraph docker image?


On Thursday, August 16, 2018 at 8:19:05 AM UTC-7, Jason Plurad wrote:
+1 for a single janusgraph-docker repo.

Wouldn't you be able to use the directory structure to manage the different flavors of a Docker image that we want to publish to Docker Hub? I wouldn't want to create multiple JanusGraph repos for Docker technology.


On Wednesday, August 15, 2018 at 2:25:32 PM UTC-4, Chris Hupman wrote:
Since we seem to be making repos right now I wanted to stir this up a bit. I think right now would be a good time to make a janusgraph-docker repo for an official Docker image. 

I mainly wanted to ask if a single repo would be adequate. I personally like the approach of a single image defaulting to in-memory so the image can be deployed standalone and then configuring through -e flags at deploy. In one of the PRs it was suggested to default to using es and Cassandra. I am assuming, quite possibly incorrectly,  that would mean throwing es, Cassandra, and JanusGraph in a single container for ease of deploy. If that was the case an additional repo would probably be required. 

Hoping to get some +1s for a consensus on adding a janusgraph-docker repo.

Chris

On Thursday, June 7, 2018 at 12:34:30 AM UTC-7, Florian Hockmann wrote:
what if we included a minimal Docker build as part of the repo for client library testing and then left the separate repo to serve only the purpose of producing official, published Docker images?

That makes perfectly sense to me. Seems to be the best solution overall to me.

Am Mittwoch, 6. Juni 2018 22:05:18 UTC+2 schrieb Ted Wilmes:
Thanks for the responses Chris and Florian. I'm in agreement that we should first discuss the separate repo issue separate from the implementation details. That was my intent, but by including those two links I conflated the two. I would expect either approach we take to go through the usual PR process, or perhaps even another round of discussion. My Alpine base image example isn't a great one, may main point is that for whatever reason, there may be a community desire to support a variety of different configurations for the images (not Janus configs themselves to be clear).

Your point about client library development is a good one. My inclination is to treat the JanusGraph developer Docker environment use case differently than the JanusGraph end-user use cases (user dev & integration testing, production deployment). There would undoubtedly be some overlap, but what if we included a minimal Docker build as part of the repo for client library testing and then left the separate repo to serve only the purpose of producing official, published Docker images?

Thanks,
Ted

On Wednesday, June 6, 2018 at 6:41:50 AM UTC-5, Florian Hockmann wrote:
Hi,

first of all, big thanks for bringing this up again! I'm really looking forward to an official JanusGraph image.

I can think of one advantage of including the Docker images in the main JanusGraph repo: When you want to test a client library like the ones we will probably create for the different languages (I'm currently working on one for .NET) then you need an instance of the server and this should be a version of the server that will be released with the same version number. A JanusGraph docker image would make testing of such a client library much easier but when we put the Dockerfile in another repo then we would only have already published images available in the main repo. So you couldn't test such a client library with a Docker image of the server as it is at that moment in the repo, you would rather have to use an already released version.

A big advantage of a separate repo would be that we could make use of DockerHub's automated builds which can then also include the README.md and the Dockerfile.

One thing I'm wondering about:

potentially supporting a variety of different linux distros including alpine

What's the advantage of having images for other distros than alpine? Having alpine as the base image means that the image will be really small but do users also want a debian based image or something like that?

I haven't looked into the new repo Chris created in detail as I think that we should first only discuss about whether or not a separate repo should be used and I assume that there will be the usual review process afterwards.

Am Dienstag, 5. Juni 2018 22:52:42 UTC+2 schrieb Chris Hupman:
Hello,
I'm regularly using containers for Cassandra and Elasticsearch for my JanusGraph work and I had a couple of thoughts on this. 
  • In pull 700 only janusgraph is being run, in the experoinc repo es, cassandra, and janusgraph are all running in the same container. I personally think JanusGraph only running in-memory by default is the right approach. While an all-in-one is great for development it wouldn't really make sense for a real deployment. 
  • Right now both containers are using JanusGraphFactory(JGF) instead of ConfiguredGraphFactory(CGF). Since this container is for running JanusGraph server I think CGF would make more sense than JGF. 
  • To be more modular and avoid passing as many -e flags it should support multiple gremlin server files through ENVs. 
  • For JGF deployments we should add in a script to bind g = graph.traversal() to improve user experience as referenced in the docs
  • For CGF deployments it would be nice if the container tried to create a template using the configuration parameters it already had access to. 
  • I think it makes sense to house the docker components and tests in a separate repo. With the approach of downloading the release and moving it to /opt/janusgraph it would be easy to run a script afterwards to update the yaml and property files to take ENVs. 
I think this would be great for the overall health of the project and I'm willing to help out with code, docs, and/or testing.

On Tuesday, June 5, 2018 at 9:33:42 AM UTC-7, Ted Wilmes wrote:
Hello,
I wanted to restart a discussion on producing an official JanusGraph Docker image. There is lots of community interest and progress has been made but we're not quite there yet. There is an existing PR [1] that a coworker of mine, Chris Pounds, had been working on. After some brainstorming with Misha Brukman, we were wondering if a change in approach may make sense. Namely, that it might be a good idea to host the artifacts required to maintain Janus docker images separate from the main JanusGraph repo. With that in mind, Chris has recently created this repo: https://github.com/experoinc/janusgraph-images. It's sitting under out GitHub at the moment but is Apache 2 licensed, and if this plan makes sense to folks, we'd just transition it over to the Janus org and put it through the usual PR and approval process. With all that said, I think there are a number of benefits to maintaining the Docker plumbing separate from JanusGraph itself. I would imagine a matrix of different versions and image options at some point, potentially supporting a variety of different linux distros including alpine. Also, it would be nice to be able to version the Docker artifacts separately from JanusGraph so that we can rev them at different speeds. So, first step is I'd like to get some community feedback on this approach and see what folks thought about setting up a separate repo under the Janus org to host the Docker plumbing?

Thanks,
Ted

--
You received this message because you are subscribed to a topic in the Google Groups "JanusGraph developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/janusgraph-dev/V70uVxh9who/unsubscribe.
To unsubscribe from this group and all its topics, send an email to janusgraph-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-dev/d2da37b0-44c1-4c3b-b51b-58e279bdeefa%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "JanusGraph developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/janusgraph-dev/V70uVxh9who/unsubscribe.
To unsubscribe from this group and all its topics, send an email to janusgraph-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-dev/1da5c589-aef2-437c-b89e-9064108f0e7c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


[DISCUSS] Adding static code analysis into our review process

Florian Hockmann <f...@...>
 

JanusGraph currently uses Coverity as a static code analysis tool. While this already led to some findings, it can unfortunately not be used to analyse code changes from pull requests. I wondered whether we could improve our review process with a service that analyses changes in pull requests and directly comments on the PRs to report its findings. This hopefully makes reviews more efficient as reviewers don't have to comment on obvious style issues for example and it could find problems before they are added to production branches.

I already searched for a service that we could use. In my opinion, it should fulfil these requirements:
  1. Support for the languages we currently use / might add in the near future: Java, C#, Python, JavaScript / TypeScript.
  2. Integration into code reviews in GitHub.
  3. Free for open source projects with enough scans per day (some services limit these to 5 per day which could be not enough).
GitHub already lists services that can be used for code reviews here. Codacy seems to be the only service of those listed there that fulfils all three requirements. I wanted to try out Codacy and therefore already let it analyse forks of JanusGraph and the version of JanusGraph.Net that is currently in review:
So, what do others think about the idea to integrate a code analysis service into our review process in general? And are there any other suggestions for such a service besides Codacy?


Re: Using custom properties in ElasticSearchIndex

Alexandr Porunov <alexand...@...>
 

The PR for this issue is here: https://github.com/JanusGraph/janusgraph/pull/1231

I would appropriate if someone could review it.

Best regards,
Alexandr


On Tuesday, August 21, 2018 at 10:21:20 PM UTC+3, Alexandr Porunov wrote:
Hello,

I see that in the class "ElasticSearchIndex" in the method "pushMapping(String store, String key, KeyInformation information)" we ignore any custom properties from "KeyInformation information". 

As for me it is a drawback because we can't use specific features from ElasticSearch like on-disk index sorting, changing `similarity` algorithms for specific fields, using multi-fields and so on.

What do you think about adding a feature to allow custom properties?

I would appreciate if anyone could implement this feature. Is anyone interested?

If no one is able right now to implement this feature I can try to implement it.

Is there anything I should be aware of?

Best regards,
Alexandr


Re: [DISCUSS] Official JanusGraph docker image repo

Chris Hupman <chris...@...>
 

I think that makes sense as a starting point. I'm assuming there will be a fair amount of testing, discussions,  and iterations before it gets pushed up as an official image. 


On Monday, August 27, 2018 at 1:05:49 PM UTC-7, Ted Wilmes wrote:
I think I can get to that later this week. I could push Chris' work into a new branch on the new repo and then put a PR in so we could come to some consensus on if that's a good starting point?

On Mon, Aug 27, 2018 at 2:57 PM Chris Hupman <ch...@...> wrote:
One last question, mainly for Expero folks. Is anyone already intending to make the initial commit? I have a conference this week, but can work on this next week if not.

On Monday, August 27, 2018 at 8:07:24 AM UTC-7, Jason Plurad wrote:
https://github.com/JanusGraph/janusgraph-docker is created

On Tuesday, August 21, 2018 at 3:17:00 AM UTC-4, Florian Hockmann wrote:
Yes, +1 for the separate repo.

Am Dienstag, 21. August 2018 03:04:23 UTC+2 schrieb Ted Wilmes:
Hey Chris,
Appreciate the extra stir and yes, I'm in agreement. +1 for a separate repo.

Thanks,
Ted

On Mon, Aug 20, 2018, 3:14 PM Chris Hupman <ch...@...> wrote:
Stirring one more time. Ted and Florian are you guys in favor of the creation of a repo for an official JanusGraph docker image?


On Thursday, August 16, 2018 at 8:19:05 AM UTC-7, Jason Plurad wrote:
+1 for a single janusgraph-docker repo.

Wouldn't you be able to use the directory structure to manage the different flavors of a Docker image that we want to publish to Docker Hub? I wouldn't want to create multiple JanusGraph repos for Docker technology.


On Wednesday, August 15, 2018 at 2:25:32 PM UTC-4, Chris Hupman wrote:
Since we seem to be making repos right now I wanted to stir this up a bit. I think right now would be a good time to make a janusgraph-docker repo for an official Docker image. 

I mainly wanted to ask if a single repo would be adequate. I personally like the approach of a single image defaulting to in-memory so the image can be deployed standalone and then configuring through -e flags at deploy. In one of the PRs it was suggested to default to using es and Cassandra. I am assuming, quite possibly incorrectly,  that would mean throwing es, Cassandra, and JanusGraph in a single container for ease of deploy. If that was the case an additional repo would probably be required. 

Hoping to get some +1s for a consensus on adding a janusgraph-docker repo.

Chris

On Thursday, June 7, 2018 at 12:34:30 AM UTC-7, Florian Hockmann wrote:
what if we included a minimal Docker build as part of the repo for client library testing and then left the separate repo to serve only the purpose of producing official, published Docker images?

That makes perfectly sense to me. Seems to be the best solution overall to me.

Am Mittwoch, 6. Juni 2018 22:05:18 UTC+2 schrieb Ted Wilmes:
Thanks for the responses Chris and Florian. I'm in agreement that we should first discuss the separate repo issue separate from the implementation details. That was my intent, but by including those two links I conflated the two. I would expect either approach we take to go through the usual PR process, or perhaps even another round of discussion. My Alpine base image example isn't a great one, may main point is that for whatever reason, there may be a community desire to support a variety of different configurations for the images (not Janus configs themselves to be clear).

Your point about client library development is a good one. My inclination is to treat the JanusGraph developer Docker environment use case differently than the JanusGraph end-user use cases (user dev & integration testing, production deployment). There would undoubtedly be some overlap, but what if we included a minimal Docker build as part of the repo for client library testing and then left the separate repo to serve only the purpose of producing official, published Docker images?

Thanks,
Ted

On Wednesday, June 6, 2018 at 6:41:50 AM UTC-5, Florian Hockmann wrote:
Hi,

first of all, big thanks for bringing this up again! I'm really looking forward to an official JanusGraph image.

I can think of one advantage of including the Docker images in the main JanusGraph repo: When you want to test a client library like the ones we will probably create for the different languages (I'm currently working on one for .NET) then you need an instance of the server and this should be a version of the server that will be released with the same version number. A JanusGraph docker image would make testing of such a client library much easier but when we put the Dockerfile in another repo then we would only have already published images available in the main repo. So you couldn't test such a client library with a Docker image of the server as it is at that moment in the repo, you would rather have to use an already released version.

A big advantage of a separate repo would be that we could make use of DockerHub's automated builds which can then also include the README.md and the Dockerfile.

One thing I'm wondering about:

potentially supporting a variety of different linux distros including alpine

What's the advantage of having images for other distros than alpine? Having alpine as the base image means that the image will be really small but do users also want a debian based image or something like that?

I haven't looked into the new repo Chris created in detail as I think that we should first only discuss about whether or not a separate repo should be used and I assume that there will be the usual review process afterwards.

Am Dienstag, 5. Juni 2018 22:52:42 UTC+2 schrieb Chris Hupman:
Hello,
I'm regularly using containers for Cassandra and Elasticsearch for my JanusGraph work and I had a couple of thoughts on this. 
  • In pull 700 only janusgraph is being run, in the experoinc repo es, cassandra, and janusgraph are all running in the same container. I personally think JanusGraph only running in-memory by default is the right approach. While an all-in-one is great for development it wouldn't really make sense for a real deployment. 
  • Right now both containers are using JanusGraphFactory(JGF) instead of ConfiguredGraphFactory(CGF). Since this container is for running JanusGraph server I think CGF would make more sense than JGF. 
  • To be more modular and avoid passing as many -e flags it should support multiple gremlin server files through ENVs. 
  • For JGF deployments we should add in a script to bind g = graph.traversal() to improve user experience as referenced in the docs
  • For CGF deployments it would be nice if the container tried to create a template using the configuration parameters it already had access to. 
  • I think it makes sense to house the docker components and tests in a separate repo. With the approach of downloading the release and moving it to /opt/janusgraph it would be easy to run a script afterwards to update the yaml and property files to take ENVs. 
I think this would be great for the overall health of the project and I'm willing to help out with code, docs, and/or testing.

On Tuesday, June 5, 2018 at 9:33:42 AM UTC-7, Ted Wilmes wrote:
Hello,
I wanted to restart a discussion on producing an official JanusGraph Docker image. There is lots of community interest and progress has been made but we're not quite there yet. There is an existing PR [1] that a coworker of mine, Chris Pounds, had been working on. After some brainstorming with Misha Brukman, we were wondering if a change in approach may make sense. Namely, that it might be a good idea to host the artifacts required to maintain Janus docker images separate from the main JanusGraph repo. With that in mind, Chris has recently created this repo: https://github.com/experoinc/janusgraph-images. It's sitting under out GitHub at the moment but is Apache 2 licensed, and if this plan makes sense to folks, we'd just transition it over to the Janus org and put it through the usual PR and approval process. With all that said, I think there are a number of benefits to maintaining the Docker plumbing separate from JanusGraph itself. I would imagine a matrix of different versions and image options at some point, potentially supporting a variety of different linux distros including alpine. Also, it would be nice to be able to version the Docker artifacts separately from JanusGraph so that we can rev them at different speeds. So, first step is I'd like to get some community feedback on this approach and see what folks thought about setting up a separate repo under the Janus org to host the Docker plumbing?

Thanks,
Ted

--
You received this message because you are subscribed to a topic in the Google Groups "JanusGraph developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/janusgraph-dev/V70uVxh9who/unsubscribe.
To unsubscribe from this group and all its topics, send an email to janusgraph-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-dev/d2da37b0-44c1-4c3b-b51b-58e279bdeefa%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "JanusGraph developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/janusgraph-dev/V70uVxh9who/unsubscribe.
To unsubscribe from this group and all its topics, send an email to janusgraph-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-dev/1da5c589-aef2-437c-b89e-9064108f0e7c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [DISCUSS] Official JanusGraph docker image repo

Ted Wilmes <twi...@...>
 

I think I can get to that later this week. I could push Chris' work into a new branch on the new repo and then put a PR in so we could come to some consensus on if that's a good starting point?


On Mon, Aug 27, 2018 at 2:57 PM Chris Hupman <chris...@...> wrote:
One last question, mainly for Expero folks. Is anyone already intending to make the initial commit? I have a conference this week, but can work on this next week if not.

On Monday, August 27, 2018 at 8:07:24 AM UTC-7, Jason Plurad wrote:
https://github.com/JanusGraph/janusgraph-docker is created

On Tuesday, August 21, 2018 at 3:17:00 AM UTC-4, Florian Hockmann wrote:
Yes, +1 for the separate repo.

Am Dienstag, 21. August 2018 03:04:23 UTC+2 schrieb Ted Wilmes:
Hey Chris,
Appreciate the extra stir and yes, I'm in agreement. +1 for a separate repo.

Thanks,
Ted

On Mon, Aug 20, 2018, 3:14 PM Chris Hupman <ch...@...> wrote:
Stirring one more time. Ted and Florian are you guys in favor of the creation of a repo for an official JanusGraph docker image?


On Thursday, August 16, 2018 at 8:19:05 AM UTC-7, Jason Plurad wrote:
+1 for a single janusgraph-docker repo.

Wouldn't you be able to use the directory structure to manage the different flavors of a Docker image that we want to publish to Docker Hub? I wouldn't want to create multiple JanusGraph repos for Docker technology.


On Wednesday, August 15, 2018 at 2:25:32 PM UTC-4, Chris Hupman wrote:
Since we seem to be making repos right now I wanted to stir this up a bit. I think right now would be a good time to make a janusgraph-docker repo for an official Docker image. 

I mainly wanted to ask if a single repo would be adequate. I personally like the approach of a single image defaulting to in-memory so the image can be deployed standalone and then configuring through -e flags at deploy. In one of the PRs it was suggested to default to using es and Cassandra. I am assuming, quite possibly incorrectly,  that would mean throwing es, Cassandra, and JanusGraph in a single container for ease of deploy. If that was the case an additional repo would probably be required. 

Hoping to get some +1s for a consensus on adding a janusgraph-docker repo.

Chris

On Thursday, June 7, 2018 at 12:34:30 AM UTC-7, Florian Hockmann wrote:
what if we included a minimal Docker build as part of the repo for client library testing and then left the separate repo to serve only the purpose of producing official, published Docker images?

That makes perfectly sense to me. Seems to be the best solution overall to me.

Am Mittwoch, 6. Juni 2018 22:05:18 UTC+2 schrieb Ted Wilmes:
Thanks for the responses Chris and Florian. I'm in agreement that we should first discuss the separate repo issue separate from the implementation details. That was my intent, but by including those two links I conflated the two. I would expect either approach we take to go through the usual PR process, or perhaps even another round of discussion. My Alpine base image example isn't a great one, may main point is that for whatever reason, there may be a community desire to support a variety of different configurations for the images (not Janus configs themselves to be clear).

Your point about client library development is a good one. My inclination is to treat the JanusGraph developer Docker environment use case differently than the JanusGraph end-user use cases (user dev & integration testing, production deployment). There would undoubtedly be some overlap, but what if we included a minimal Docker build as part of the repo for client library testing and then left the separate repo to serve only the purpose of producing official, published Docker images?

Thanks,
Ted

On Wednesday, June 6, 2018 at 6:41:50 AM UTC-5, Florian Hockmann wrote:
Hi,

first of all, big thanks for bringing this up again! I'm really looking forward to an official JanusGraph image.

I can think of one advantage of including the Docker images in the main JanusGraph repo: When you want to test a client library like the ones we will probably create for the different languages (I'm currently working on one for .NET) then you need an instance of the server and this should be a version of the server that will be released with the same version number. A JanusGraph docker image would make testing of such a client library much easier but when we put the Dockerfile in another repo then we would only have already published images available in the main repo. So you couldn't test such a client library with a Docker image of the server as it is at that moment in the repo, you would rather have to use an already released version.

A big advantage of a separate repo would be that we could make use of DockerHub's automated builds which can then also include the README.md and the Dockerfile.

One thing I'm wondering about:

potentially supporting a variety of different linux distros including alpine

What's the advantage of having images for other distros than alpine? Having alpine as the base image means that the image will be really small but do users also want a debian based image or something like that?

I haven't looked into the new repo Chris created in detail as I think that we should first only discuss about whether or not a separate repo should be used and I assume that there will be the usual review process afterwards.

Am Dienstag, 5. Juni 2018 22:52:42 UTC+2 schrieb Chris Hupman:
Hello,
I'm regularly using containers for Cassandra and Elasticsearch for my JanusGraph work and I had a couple of thoughts on this. 
  • In pull 700 only janusgraph is being run, in the experoinc repo es, cassandra, and janusgraph are all running in the same container. I personally think JanusGraph only running in-memory by default is the right approach. While an all-in-one is great for development it wouldn't really make sense for a real deployment. 
  • Right now both containers are using JanusGraphFactory(JGF) instead of ConfiguredGraphFactory(CGF). Since this container is for running JanusGraph server I think CGF would make more sense than JGF. 
  • To be more modular and avoid passing as many -e flags it should support multiple gremlin server files through ENVs. 
  • For JGF deployments we should add in a script to bind g = graph.traversal() to improve user experience as referenced in the docs
  • For CGF deployments it would be nice if the container tried to create a template using the configuration parameters it already had access to. 
  • I think it makes sense to house the docker components and tests in a separate repo. With the approach of downloading the release and moving it to /opt/janusgraph it would be easy to run a script afterwards to update the yaml and property files to take ENVs. 
I think this would be great for the overall health of the project and I'm willing to help out with code, docs, and/or testing.

On Tuesday, June 5, 2018 at 9:33:42 AM UTC-7, Ted Wilmes wrote:
Hello,
I wanted to restart a discussion on producing an official JanusGraph Docker image. There is lots of community interest and progress has been made but we're not quite there yet. There is an existing PR [1] that a coworker of mine, Chris Pounds, had been working on. After some brainstorming with Misha Brukman, we were wondering if a change in approach may make sense. Namely, that it might be a good idea to host the artifacts required to maintain Janus docker images separate from the main JanusGraph repo. With that in mind, Chris has recently created this repo: https://github.com/experoinc/janusgraph-images. It's sitting under out GitHub at the moment but is Apache 2 licensed, and if this plan makes sense to folks, we'd just transition it over to the Janus org and put it through the usual PR and approval process. With all that said, I think there are a number of benefits to maintaining the Docker plumbing separate from JanusGraph itself. I would imagine a matrix of different versions and image options at some point, potentially supporting a variety of different linux distros including alpine. Also, it would be nice to be able to version the Docker artifacts separately from JanusGraph so that we can rev them at different speeds. So, first step is I'd like to get some community feedback on this approach and see what folks thought about setting up a separate repo under the Janus org to host the Docker plumbing?

Thanks,
Ted

--
You received this message because you are subscribed to a topic in the Google Groups "JanusGraph developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/janusgraph-dev/V70uVxh9who/unsubscribe.
To unsubscribe from this group and all its topics, send an email to janusg...@....
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-dev/d2da37b0-44c1-4c3b-b51b-58e279bdeefa%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "JanusGraph developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/janusgraph-dev/V70uVxh9who/unsubscribe.
To unsubscribe from this group and all its topics, send an email to janusgr...@....
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-dev/1da5c589-aef2-437c-b89e-9064108f0e7c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [DISCUSS] Official JanusGraph docker image repo

Chris Hupman <chris...@...>
 

One last question, mainly for Expero folks. Is anyone already intending to make the initial commit? I have a conference this week, but can work on this next week if not.


On Monday, August 27, 2018 at 8:07:24 AM UTC-7, Jason Plurad wrote:
https://github.com/JanusGraph/janusgraph-docker is created

On Tuesday, August 21, 2018 at 3:17:00 AM UTC-4, Florian Hockmann wrote:
Yes, +1 for the separate repo.

Am Dienstag, 21. August 2018 03:04:23 UTC+2 schrieb Ted Wilmes:
Hey Chris,
Appreciate the extra stir and yes, I'm in agreement. +1 for a separate repo.

Thanks,
Ted

On Mon, Aug 20, 2018, 3:14 PM Chris Hupman <ch...@...> wrote:
Stirring one more time. Ted and Florian are you guys in favor of the creation of a repo for an official JanusGraph docker image?


On Thursday, August 16, 2018 at 8:19:05 AM UTC-7, Jason Plurad wrote:
+1 for a single janusgraph-docker repo.

Wouldn't you be able to use the directory structure to manage the different flavors of a Docker image that we want to publish to Docker Hub? I wouldn't want to create multiple JanusGraph repos for Docker technology.


On Wednesday, August 15, 2018 at 2:25:32 PM UTC-4, Chris Hupman wrote:
Since we seem to be making repos right now I wanted to stir this up a bit. I think right now would be a good time to make a janusgraph-docker repo for an official Docker image. 

I mainly wanted to ask if a single repo would be adequate. I personally like the approach of a single image defaulting to in-memory so the image can be deployed standalone and then configuring through -e flags at deploy. In one of the PRs it was suggested to default to using es and Cassandra. I am assuming, quite possibly incorrectly,  that would mean throwing es, Cassandra, and JanusGraph in a single container for ease of deploy. If that was the case an additional repo would probably be required. 

Hoping to get some +1s for a consensus on adding a janusgraph-docker repo.

Chris

On Thursday, June 7, 2018 at 12:34:30 AM UTC-7, Florian Hockmann wrote:
what if we included a minimal Docker build as part of the repo for client library testing and then left the separate repo to serve only the purpose of producing official, published Docker images?

That makes perfectly sense to me. Seems to be the best solution overall to me.

Am Mittwoch, 6. Juni 2018 22:05:18 UTC+2 schrieb Ted Wilmes:
Thanks for the responses Chris and Florian. I'm in agreement that we should first discuss the separate repo issue separate from the implementation details. That was my intent, but by including those two links I conflated the two. I would expect either approach we take to go through the usual PR process, or perhaps even another round of discussion. My Alpine base image example isn't a great one, may main point is that for whatever reason, there may be a community desire to support a variety of different configurations for the images (not Janus configs themselves to be clear).

Your point about client library development is a good one. My inclination is to treat the JanusGraph developer Docker environment use case differently than the JanusGraph end-user use cases (user dev & integration testing, production deployment). There would undoubtedly be some overlap, but what if we included a minimal Docker build as part of the repo for client library testing and then left the separate repo to serve only the purpose of producing official, published Docker images?

Thanks,
Ted

On Wednesday, June 6, 2018 at 6:41:50 AM UTC-5, Florian Hockmann wrote:
Hi,

first of all, big thanks for bringing this up again! I'm really looking forward to an official JanusGraph image.

I can think of one advantage of including the Docker images in the main JanusGraph repo: When you want to test a client library like the ones we will probably create for the different languages (I'm currently working on one for .NET) then you need an instance of the server and this should be a version of the server that will be released with the same version number. A JanusGraph docker image would make testing of such a client library much easier but when we put the Dockerfile in another repo then we would only have already published images available in the main repo. So you couldn't test such a client library with a Docker image of the server as it is at that moment in the repo, you would rather have to use an already released version.

A big advantage of a separate repo would be that we could make use of DockerHub's automated builds which can then also include the README.md and the Dockerfile.

One thing I'm wondering about:

potentially supporting a variety of different linux distros including alpine

What's the advantage of having images for other distros than alpine? Having alpine as the base image means that the image will be really small but do users also want a debian based image or something like that?

I haven't looked into the new repo Chris created in detail as I think that we should first only discuss about whether or not a separate repo should be used and I assume that there will be the usual review process afterwards.

Am Dienstag, 5. Juni 2018 22:52:42 UTC+2 schrieb Chris Hupman:
Hello,
I'm regularly using containers for Cassandra and Elasticsearch for my JanusGraph work and I had a couple of thoughts on this. 
  • In pull 700 only janusgraph is being run, in the experoinc repo es, cassandra, and janusgraph are all running in the same container. I personally think JanusGraph only running in-memory by default is the right approach. While an all-in-one is great for development it wouldn't really make sense for a real deployment. 
  • Right now both containers are using JanusGraphFactory(JGF) instead of ConfiguredGraphFactory(CGF). Since this container is for running JanusGraph server I think CGF would make more sense than JGF. 
  • To be more modular and avoid passing as many -e flags it should support multiple gremlin server files through ENVs. 
  • For JGF deployments we should add in a script to bind g = graph.traversal() to improve user experience as referenced in the docs
  • For CGF deployments it would be nice if the container tried to create a template using the configuration parameters it already had access to. 
  • I think it makes sense to house the docker components and tests in a separate repo. With the approach of downloading the release and moving it to /opt/janusgraph it would be easy to run a script afterwards to update the yaml and property files to take ENVs. 
I think this would be great for the overall health of the project and I'm willing to help out with code, docs, and/or testing.

On Tuesday, June 5, 2018 at 9:33:42 AM UTC-7, Ted Wilmes wrote:
Hello,
I wanted to restart a discussion on producing an official JanusGraph Docker image. There is lots of community interest and progress has been made but we're not quite there yet. There is an existing PR [1] that a coworker of mine, Chris Pounds, had been working on. After some brainstorming with Misha Brukman, we were wondering if a change in approach may make sense. Namely, that it might be a good idea to host the artifacts required to maintain Janus docker images separate from the main JanusGraph repo. With that in mind, Chris has recently created this repo: https://github.com/experoinc/janusgraph-images. It's sitting under out GitHub at the moment but is Apache 2 licensed, and if this plan makes sense to folks, we'd just transition it over to the Janus org and put it through the usual PR and approval process. With all that said, I think there are a number of benefits to maintaining the Docker plumbing separate from JanusGraph itself. I would imagine a matrix of different versions and image options at some point, potentially supporting a variety of different linux distros including alpine. Also, it would be nice to be able to version the Docker artifacts separately from JanusGraph so that we can rev them at different speeds. So, first step is I'd like to get some community feedback on this approach and see what folks thought about setting up a separate repo under the Janus org to host the Docker plumbing?

Thanks,
Ted

--
You received this message because you are subscribed to a topic in the Google Groups "JanusGraph developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/janusgraph-dev/V70uVxh9who/unsubscribe.
To unsubscribe from this group and all its topics, send an email to janusgraph-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-dev/d2da37b0-44c1-4c3b-b51b-58e279bdeefa%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [DISCUSS] Pull Request review and approval policy

Chris Hupman <chris...@...>
 

+1. 

I like CTR for upstream merges to help streamline the process. 

I personally really like getting a second set of eyes on my commits and enjoy the feedback. That being said I've gotten some great review recommendations from non-committers in the past (Thanks Porunov) and agree that it makes sense to assume committers will request for additional review when warranted. 

On Monday, August 27, 2018 at 7:55:56 AM UTC-7, Jason Plurad wrote:
There hasn't been any negative responses on this thread. I will submit a PR today to update the PR docs to outline these updates in policy.

1. CTR is already part of our policy
  a. Committers are trusted to decide when a PR can be Commit-Then-Review (CTR) vs Review-Then-Commit (RTC)
  b. Include merging PRs upstream as one of the CTR scenarios
  c. GitHub will not be used to enforce the policy (disable approval reviews) because it does not allow committers to approve their own PRs

2. Our RTC policy requires either
  a. 2 approval votes from committers
  b. 1 approval vote from a committer followed by a one week review period
  c. Committers are trusted to seek 2nd approval for complex changes that require more review



On Friday, August 24, 2018 at 11:35:51 AM UTC-4, Alexandr Porunov wrote:
Will there be a new review and approval policy?
I believe it should be implemented.

+1 for all discussed above.

On Thursday, August 9, 2018 at 7:57:16 PM UTC+3, Jason Plurad wrote:
Perhaps a weekly update on the PR queue posted on the dev mailing list would be a good idea.

Something like:

# 0.2.2 queue
PR a - needs one more approval
PR b - waiting on revisions
PR c - unreviewed for 16 days
PR d - new this week


On Thursday, August 9, 2018 at 1:17:40 AM UTC-4, Jerry He wrote:
Finding the right balance between more review/more approvals and being more efficient is what we aim for.  I have been in situations in my PRs where another pair of eyes caught some error I made.  I was also in situations where my PRs sitting there for a long time waiting for reviews.

I think I am ok with adopting 2-4 from the TinkerPop process. 
But adding some extra guard (e.g an extra reminder or ping before commit) or time (more time than a week to accommodate busy professionals?)  will make it easier to relieve concerns.

Thanks,

Jerry


On Wednesday, August 8, 2018 at 1:14:06 PM UTC-7, Alexandr Porunov wrote:
I am not a committer but here are my thoughts:

1. I think it is enough to maintain 2 active branches right now. Latest release + future release (master). There is plainly no enough committers to maintain more active branches.

2. I think it is worth adding steps 2-4 because there are some PRs with huge delays.

On Wednesday, August 8, 2018 at 6:25:49 AM UTC+3, Jason Plurad wrote:
I have a couple discussion topics to bring up based on our recent history of getting releases out.

1. Based on what we have written in the docs (Chapter 44. Pull Requests), we have a commit-then-review (CTR) policy, however it does not work in practice. GitHub does not allow a committer to approve their own PRs. This makes it impossible to merge PRs without at least 1 other approval, and this has caused delays on some PRs to get merged. For example, the master branch is not yet open for business because the PR is still waiting for approval. Another situation was when we needed to merge a fix from 0.2 upstream into master. The 0.2 PR got approved and merged, but either the committer didn't open a separate PR to merge to master or the committer opened the PR to merge to master but that PR stagnated on getting approval. If we end up in a situation where we are maintaining more than 2 active branches, there would be even more friction involved getting changes merged upstream.

Based on these docs, we can disable restriction checks for repository administrators. If we still want to allow CTR, it seems like this needs to be done and all committers would need to be in a repository administrator group.


2. Currently our policy is for 2 committer approvals to merge a PR. Toward the end of the release cycle, I had gotten in the habit of performing my review approval first and then requesting reviews from the JanusGraph/committers group via GitHub. This should send a notification to all the committers in the group. I have no idea how other committers manage their GitHub notifications, but it seemed like a better approach than spamming the dev list or backchanneling individuals every time a PR needed a second review.

TinkerPop recently introduced a new approach to their PR review process:
We seem to agree that in the future we will go with the following review-the-commit (RTC) process:

1. Each change to TinkerPop release branches requires 3 +1s from committers and no -1s OR
2. A single +1 from a committer and a 1 week review period for objections (i.e. "cool down period") at which point we will assume a lazy consensus
3. The single +1 may come from the committer who submitted the PR in the first place
4. Committers are trusted with their changes but are expected to request reviews for complex changes as necessary and not rely strictly on lazy consensus
5. commit-the-review (CTR) procedures remain unchanged
By adopting bullets 2-4 from TinkerPop's process, I think it would enable us to merge changes into the codebase more efficiently. If committers were concerned about what PRs might get merged, they would need to check the PR queue at least once a week.


Thoughts?


Re: [PROPOSAL] new repository for a JanusGraph plugin for Apache Ambari

Jason Plurad <plu...@...>
 

https://github.com/JanusGraph/janusgraph-ambari is created


On Thursday, August 16, 2018 at 11:30:48 AM UTC-4, Misha Brukman wrote:
+1 to the proposal (and +1 on the name).

On Thu, Aug 16, 2018 at 11:20 AM, Jason Plurad wrote:
+1 for janusgraph-ambari

On Wednesday, August 15, 2018 at 6:31:34 PM UTC-4, Chris Hupman wrote:
Awhile back I wrote a plugin for Ambari to install JanusGraph. Right now it just installs and runs, but doesn't have any support for authentication or monitoring. So I wouldn't currently recommend it for production. 

Since JanusGraph isn't an Apache project I don't think it would work to add it directly to Ambari.  At least few people have found it and used it for PoCs and I'm wondering if it would make sense to house it under the JanusGraph umbrella. 

I was waiting for JanusGraph 0.3.0 to come out before I continued development and plan to start working on it again in the near future. 

My thought is an official docker container with some examples for Kubernetes Pods would give us a good deployment story for JanusGraph + Cassandra + es  and Ambari would give us a good deployment story for JanusGraph + HBase + Solr.

Chris

--
You received this message because you are subscribed to the Google Groups "JanusGraph developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-dev+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-dev/d2bccebe-1e51-4fea-9f69-ce4d34e4c4e1%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.


Re: [DISCUSS] Official JanusGraph docker image repo

Jason Plurad <plu...@...>
 

https://github.com/JanusGraph/janusgraph-docker is created


On Tuesday, August 21, 2018 at 3:17:00 AM UTC-4, Florian Hockmann wrote:
Yes, +1 for the separate repo.

Am Dienstag, 21. August 2018 03:04:23 UTC+2 schrieb Ted Wilmes:
Hey Chris,
Appreciate the extra stir and yes, I'm in agreement. +1 for a separate repo.

Thanks,
Ted

On Mon, Aug 20, 2018, 3:14 PM Chris Hupman <ch...@...> wrote:
Stirring one more time. Ted and Florian are you guys in favor of the creation of a repo for an official JanusGraph docker image?


On Thursday, August 16, 2018 at 8:19:05 AM UTC-7, Jason Plurad wrote:
+1 for a single janusgraph-docker repo.

Wouldn't you be able to use the directory structure to manage the different flavors of a Docker image that we want to publish to Docker Hub? I wouldn't want to create multiple JanusGraph repos for Docker technology.


On Wednesday, August 15, 2018 at 2:25:32 PM UTC-4, Chris Hupman wrote:
Since we seem to be making repos right now I wanted to stir this up a bit. I think right now would be a good time to make a janusgraph-docker repo for an official Docker image. 

I mainly wanted to ask if a single repo would be adequate. I personally like the approach of a single image defaulting to in-memory so the image can be deployed standalone and then configuring through -e flags at deploy. In one of the PRs it was suggested to default to using es and Cassandra. I am assuming, quite possibly incorrectly,  that would mean throwing es, Cassandra, and JanusGraph in a single container for ease of deploy. If that was the case an additional repo would probably be required. 

Hoping to get some +1s for a consensus on adding a janusgraph-docker repo.

Chris

On Thursday, June 7, 2018 at 12:34:30 AM UTC-7, Florian Hockmann wrote:
what if we included a minimal Docker build as part of the repo for client library testing and then left the separate repo to serve only the purpose of producing official, published Docker images?

That makes perfectly sense to me. Seems to be the best solution overall to me.

Am Mittwoch, 6. Juni 2018 22:05:18 UTC+2 schrieb Ted Wilmes:
Thanks for the responses Chris and Florian. I'm in agreement that we should first discuss the separate repo issue separate from the implementation details. That was my intent, but by including those two links I conflated the two. I would expect either approach we take to go through the usual PR process, or perhaps even another round of discussion. My Alpine base image example isn't a great one, may main point is that for whatever reason, there may be a community desire to support a variety of different configurations for the images (not Janus configs themselves to be clear).

Your point about client library development is a good one. My inclination is to treat the JanusGraph developer Docker environment use case differently than the JanusGraph end-user use cases (user dev & integration testing, production deployment). There would undoubtedly be some overlap, but what if we included a minimal Docker build as part of the repo for client library testing and then left the separate repo to serve only the purpose of producing official, published Docker images?

Thanks,
Ted

On Wednesday, June 6, 2018 at 6:41:50 AM UTC-5, Florian Hockmann wrote:
Hi,

first of all, big thanks for bringing this up again! I'm really looking forward to an official JanusGraph image.

I can think of one advantage of including the Docker images in the main JanusGraph repo: When you want to test a client library like the ones we will probably create for the different languages (I'm currently working on one for .NET) then you need an instance of the server and this should be a version of the server that will be released with the same version number. A JanusGraph docker image would make testing of such a client library much easier but when we put the Dockerfile in another repo then we would only have already published images available in the main repo. So you couldn't test such a client library with a Docker image of the server as it is at that moment in the repo, you would rather have to use an already released version.

A big advantage of a separate repo would be that we could make use of DockerHub's automated builds which can then also include the README.md and the Dockerfile.

One thing I'm wondering about:

potentially supporting a variety of different linux distros including alpine

What's the advantage of having images for other distros than alpine? Having alpine as the base image means that the image will be really small but do users also want a debian based image or something like that?

I haven't looked into the new repo Chris created in detail as I think that we should first only discuss about whether or not a separate repo should be used and I assume that there will be the usual review process afterwards.

Am Dienstag, 5. Juni 2018 22:52:42 UTC+2 schrieb Chris Hupman:
Hello,
I'm regularly using containers for Cassandra and Elasticsearch for my JanusGraph work and I had a couple of thoughts on this. 
  • In pull 700 only janusgraph is being run, in the experoinc repo es, cassandra, and janusgraph are all running in the same container. I personally think JanusGraph only running in-memory by default is the right approach. While an all-in-one is great for development it wouldn't really make sense for a real deployment. 
  • Right now both containers are using JanusGraphFactory(JGF) instead of ConfiguredGraphFactory(CGF). Since this container is for running JanusGraph server I think CGF would make more sense than JGF. 
  • To be more modular and avoid passing as many -e flags it should support multiple gremlin server files through ENVs. 
  • For JGF deployments we should add in a script to bind g = graph.traversal() to improve user experience as referenced in the docs
  • For CGF deployments it would be nice if the container tried to create a template using the configuration parameters it already had access to. 
  • I think it makes sense to house the docker components and tests in a separate repo. With the approach of downloading the release and moving it to /opt/janusgraph it would be easy to run a script afterwards to update the yaml and property files to take ENVs. 
I think this would be great for the overall health of the project and I'm willing to help out with code, docs, and/or testing.

On Tuesday, June 5, 2018 at 9:33:42 AM UTC-7, Ted Wilmes wrote:
Hello,
I wanted to restart a discussion on producing an official JanusGraph Docker image. There is lots of community interest and progress has been made but we're not quite there yet. There is an existing PR [1] that a coworker of mine, Chris Pounds, had been working on. After some brainstorming with Misha Brukman, we were wondering if a change in approach may make sense. Namely, that it might be a good idea to host the artifacts required to maintain Janus docker images separate from the main JanusGraph repo. With that in mind, Chris has recently created this repo: https://github.com/experoinc/janusgraph-images. It's sitting under out GitHub at the moment but is Apache 2 licensed, and if this plan makes sense to folks, we'd just transition it over to the Janus org and put it through the usual PR and approval process. With all that said, I think there are a number of benefits to maintaining the Docker plumbing separate from JanusGraph itself. I would imagine a matrix of different versions and image options at some point, potentially supporting a variety of different linux distros including alpine. Also, it would be nice to be able to version the Docker artifacts separately from JanusGraph so that we can rev them at different speeds. So, first step is I'd like to get some community feedback on this approach and see what folks thought about setting up a separate repo under the Janus org to host the Docker plumbing?

Thanks,
Ted

--
You received this message because you are subscribed to a topic in the Google Groups "JanusGraph developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/janusgraph-dev/V70uVxh9who/unsubscribe.
To unsubscribe from this group and all its topics, send an email to janusgraph-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-dev/d2da37b0-44c1-4c3b-b51b-58e279bdeefa%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [DISCUSS] Pull Request review and approval policy

Jason Plurad <plu...@...>
 

There hasn't been any negative responses on this thread. I will submit a PR today to update the PR docs to outline these updates in policy.

1. CTR is already part of our policy
  a. Committers are trusted to decide when a PR can be Commit-Then-Review (CTR) vs Review-Then-Commit (RTC)
  b. Include merging PRs upstream as one of the CTR scenarios
  c. GitHub will not be used to enforce the policy (disable approval reviews) because it does not allow committers to approve their own PRs

2. Our RTC policy requires either
  a. 2 approval votes from committers
  b. 1 approval vote from a committer followed by a one week review period
  c. Committers are trusted to seek 2nd approval for complex changes that require more review



On Friday, August 24, 2018 at 11:35:51 AM UTC-4, Alexandr Porunov wrote:
Will there be a new review and approval policy?
I believe it should be implemented.

+1 for all discussed above.

On Thursday, August 9, 2018 at 7:57:16 PM UTC+3, Jason Plurad wrote:
Perhaps a weekly update on the PR queue posted on the dev mailing list would be a good idea.

Something like:

# 0.2.2 queue
PR a - needs one more approval
PR b - waiting on revisions
PR c - unreviewed for 16 days
PR d - new this week


On Thursday, August 9, 2018 at 1:17:40 AM UTC-4, Jerry He wrote:
Finding the right balance between more review/more approvals and being more efficient is what we aim for.  I have been in situations in my PRs where another pair of eyes caught some error I made.  I was also in situations where my PRs sitting there for a long time waiting for reviews.

I think I am ok with adopting 2-4 from the TinkerPop process. 
But adding some extra guard (e.g an extra reminder or ping before commit) or time (more time than a week to accommodate busy professionals?)  will make it easier to relieve concerns.

Thanks,

Jerry


On Wednesday, August 8, 2018 at 1:14:06 PM UTC-7, Alexandr Porunov wrote:
I am not a committer but here are my thoughts:

1. I think it is enough to maintain 2 active branches right now. Latest release + future release (master). There is plainly no enough committers to maintain more active branches.

2. I think it is worth adding steps 2-4 because there are some PRs with huge delays.

On Wednesday, August 8, 2018 at 6:25:49 AM UTC+3, Jason Plurad wrote:
I have a couple discussion topics to bring up based on our recent history of getting releases out.

1. Based on what we have written in the docs (Chapter 44. Pull Requests), we have a commit-then-review (CTR) policy, however it does not work in practice. GitHub does not allow a committer to approve their own PRs. This makes it impossible to merge PRs without at least 1 other approval, and this has caused delays on some PRs to get merged. For example, the master branch is not yet open for business because the PR is still waiting for approval. Another situation was when we needed to merge a fix from 0.2 upstream into master. The 0.2 PR got approved and merged, but either the committer didn't open a separate PR to merge to master or the committer opened the PR to merge to master but that PR stagnated on getting approval. If we end up in a situation where we are maintaining more than 2 active branches, there would be even more friction involved getting changes merged upstream.

Based on these docs, we can disable restriction checks for repository administrators. If we still want to allow CTR, it seems like this needs to be done and all committers would need to be in a repository administrator group.


2. Currently our policy is for 2 committer approvals to merge a PR. Toward the end of the release cycle, I had gotten in the habit of performing my review approval first and then requesting reviews from the JanusGraph/committers group via GitHub. This should send a notification to all the committers in the group. I have no idea how other committers manage their GitHub notifications, but it seemed like a better approach than spamming the dev list or backchanneling individuals every time a PR needed a second review.

TinkerPop recently introduced a new approach to their PR review process:
We seem to agree that in the future we will go with the following review-the-commit (RTC) process:

1. Each change to TinkerPop release branches requires 3 +1s from committers and no -1s OR
2. A single +1 from a committer and a 1 week review period for objections (i.e. "cool down period") at which point we will assume a lazy consensus
3. The single +1 may come from the committer who submitted the PR in the first place
4. Committers are trusted with their changes but are expected to request reviews for complex changes as necessary and not rely strictly on lazy consensus
5. commit-the-review (CTR) procedures remain unchanged
By adopting bullets 2-4 from TinkerPop's process, I think it would enable us to merge changes into the codebase more efficiently. If committers were concerned about what PRs might get merged, they would need to check the PR queue at least once a week.


Thoughts?


Re: TinkerPop 3.2 vs TinkerPop 3.3 Serialization difference for Predicate class

Florian Hockmann <f...@...>
 

JanusGraph simply lacks the required GraphSON serializers and deserializers for predicates. I created #1060 to track this issue and fixed it with PR #1061 which was merged into master and not into the 0.2 branch and is therefore not present in version 0.2.1.

That is also why I decided to only support JanusGraph 0.3.0 with JanusGraph.Net.

Am Freitag, 24. August 2018 18:46:25 UTC+2 schrieb Debasish Kanhar:

And, the following are serializers registered in my gremlin-server.yaml for JanusGraph 0.2.1 (I use the default packeged set of serializers):
serializers:
 
- { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV1d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
 
- { className: org.apache.tinkerpop.gremlin.driver.ser.GryoLiteMessageSerializerV1d0, config: {ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
 
- { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV1d0, config: { serializeResultToString: true }}
 
- { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerGremlinV1d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistryV1d0] }}
 
- { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerGremlinV2d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
 
- { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV1d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistryV1d0] }}




On Friday, 24 August 2018 22:10:52 UTC+5:30, Debasish Kanhar wrote:
Hi All,

So as you all know I've been developing Client drivers for JanusGraph. I was able to test my initial version of library against JanusGraph 0.3.0 which uses TP 3.3.3. I now want to test the same set of driver for JanusGraph 0.2.1 which used TP 3.2.9 but I feel I've hit a bump with respect to Serialization for Predicate class.

So, my existent features like Adding of GeoShapes, Querying GeoShape and RelationIdentifier serialization/deserialization works. But any of my features leveraging TinkerPop's Predicate (PSerializer) doesn't work. I feel like this is some sort of serialization issue.

So, when I've implemented TextContainsFuzzy, which was working against JG 0.3.0/TP 3.3.3 (Using GraphSON 3.0) but the same doesn't work on GraphSON 2.0.

The following is my serialized object being sent to Gremlin-Server (Gremlin-server logs specify this)

Using TextFuzzy:
Error message:

org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV2d0  - Request [PooledUnsafeDirectByteBuf(ridx: 303, widx: 303, cap: 337)] could not be deserialized by org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV2d0.
org
.apache.tinkerpop.shaded.jackson.databind.JsonMappingException: Could not deserialize the JSON value as required. Nested exception: org.apache.tinkerpop.shaded.jackson.databind.JsonMappingException: Could not deserialize the JSON value as required. Nested exception: java.lang.IllegalStateException: org.apache.tinkerpop.gremlin.process.traversal.P.textContainsFuzzy([Ljava.lang.Object;)

And the serialized object being sent:

{
 
"requestId": {
   
"@type": "g:UUID",
   
"@value": "7328b342-2ed3-43a5-b589-f6688b264ee7"
 
},
 
"processor": "traversal",
 
"op": "bytecode",
 
"args": {
   
"gremlin": {
     
"@type": "g:Bytecode",
     
"@value": {
       
"step": [
         
[
           
"V"
         
],
         
[
           
"has",
           
"name",
           
{
             
"@type": "g:P",
             
"@value": {
               
"predicate": "textContainsFuzzy",
               
"value": "herculeas"
             
}
           
}
         
]
       
]
     
}
   
},
   
"aliases": {
     
"g": "gg"
   
}
 
}
}

The serialized object really looks good, like the way it is expected. Doesn't it? Or else I cant seem to dechiper what's wrong.

I'm also facing similar errors which using the same PSerializer class for extending Geo Predicates like geoWithin, but I guess both are tied to same problem, and if one gets solved, other will also :-)

If needed, I can attach the serialized object for geoWithin too. But any help in debugging this will be really helpful, as this is last piece of development I'm expecting before releasing version 1 of drivers with support for both GraphSON 2.0 and GraphSON 3.0

Thanks.


Re: Janusgraph + Spark standalone without hadoop

Jerry He <jerr...@...>
 

Yeah, a stack trace from Gremlin will us help to see what is going on. That should not be a dependency in that case. 

Thanks,

Jerry

On Fri, Aug 24, 2018 at 8:41 AM, Debasish Kanhar <d.k...@...> wrote:
Ah. I think we might be wrong in our understanding there. As I was trying to read the graph data from my underlaying backend (Cassandra) and not any Graph stored on HDFS using JanusGraph's Cassandra3InputFormat class. The same was also failing when I was trying to run OLAP using a Spark Cluster without setting HADOOP_CONF_DIR variable.

Ideally that should not have been scenario, as TP > 3.3 doesn't need intermediate HDFS storage, but doesn't look like that's happening. Well we can track this thing if needed. :-)


On Friday, 24 August 2018 21:04:24 UTC+5:30, Jerry He wrote:
The need for Hadoop conf (only the hdfs conf) is to read from or write to graph data files on hdfs. Direct interacting with JanusGraph backend without involving any graph data files on hdfs won't need that, I think.

Thanks

Jerry

On Fri, Aug 24, 2018 at 5:05 AM Debasish Kanhar <d...@...> wrote:
@Jerry:

JanusGraph doesn't need Hadoop Cluster to run OLAP yes, but doesn't JanusGraph needs to point to a live Hadoop Cluster by setting HADOOP_CONF_DIR in CLASSPATH? I guess that was my understanding, and that was missing piece in docs for which it took me really long time to crack OLAP using Spark cluster.

On Friday, 24 August 2018 06:53:20 UTC+5:30, Jerry He wrote:
That being said, to be clear, you don't need a Hadoop cluster or any kind if that is what you mean. JanusGraph packages the Hadoop jars it needs. That is all you need to run SparkComputer on JansGraph.

Thanks

Jerry

On Thu, Aug 23, 2018 at 5:56 PM Jerry He <je...@...> wrote:
I don't think it will work. Spark needs input (to read graph data) and output (to write graph data).  JanusGraph currently only provides Hadoop InputFormat based reading from JanusGraph for OLAP.
In Tinkerpop, there are InputRDD and OutputRDD interfaces, which are by Spark (SpackGraphComputer).  (Search for Tinkerpop InputRDD.)
Unfortunately, JanusGraph provides no implementations other than the InputFormat based at the moment.

Thanks,

Jerry
 


On Wed, Aug 22, 2018 at 8:46 AM, Wei Ding <dw...@...> wrote:
Hi All,
   I am pretty new to Janusgraph and want to get some suggestions from you. Previously I posted a question about using ES as backend storage, and got some good feedback from Jason (Thanks!).  Now here comes another question: If I want to use janusgraph spark standalone without Hadoop for OLAP,  can some one point me a direction? Basically I have spark standalone deployed on kubernetes, how could that be used for OLAP? 

Thanks a lot!

Wei

--
You received this message because you are subscribed to the Google Groups "JanusGraph developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-dev/512224e3-5b20-4e31-afa5-e2fd74591182%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "JanusGraph developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-dev/d5561b80-bd25-4529-8698-cd605f5bab0a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "JanusGraph developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-dev+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-dev/042b6daa-ff2f-4230-80b4-c10ba5a740cf%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.


Re: TinkerPop 3.2 vs TinkerPop 3.3 Serialization difference for Predicate class

Debasish Kanhar <d.k...@...>
 

And, the following are serializers registered in my gremlin-server.yaml for JanusGraph 0.2.1 (I use the default packeged set of serializers):
serializers:
 
- { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV1d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
 
- { className: org.apache.tinkerpop.gremlin.driver.ser.GryoLiteMessageSerializerV1d0, config: {ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
 
- { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV1d0, config: { serializeResultToString: true }}
 
- { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerGremlinV1d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistryV1d0] }}
 
- { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerGremlinV2d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
 
- { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV1d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistryV1d0] }}




On Friday, 24 August 2018 22:10:52 UTC+5:30, Debasish Kanhar wrote:
Hi All,

So as you all know I've been developing Client drivers for JanusGraph. I was able to test my initial version of library against JanusGraph 0.3.0 which uses TP 3.3.3. I now want to test the same set of driver for JanusGraph 0.2.1 which used TP 3.2.9 but I feel I've hit a bump with respect to Serialization for Predicate class.

So, my existent features like Adding of GeoShapes, Querying GeoShape and RelationIdentifier serialization/deserialization works. But any of my features leveraging TinkerPop's Predicate (PSerializer) doesn't work. I feel like this is some sort of serialization issue.

So, when I've implemented TextContainsFuzzy, which was working against JG 0.3.0/TP 3.3.3 (Using GraphSON 3.0) but the same doesn't work on GraphSON 2.0.

The following is my serialized object being sent to Gremlin-Server (Gremlin-server logs specify this)

Using TextFuzzy:
Error message:

org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV2d0  - Request [PooledUnsafeDirectByteBuf(ridx: 303, widx: 303, cap: 337)] could not be deserialized by org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV2d0.
org
.apache.tinkerpop.shaded.jackson.databind.JsonMappingException: Could not deserialize the JSON value as required. Nested exception: org.apache.tinkerpop.shaded.jackson.databind.JsonMappingException: Could not deserialize the JSON value as required. Nested exception: java.lang.IllegalStateException: org.apache.tinkerpop.gremlin.process.traversal.P.textContainsFuzzy([Ljava.lang.Object;)

And the serialized object being sent:

{
 
"requestId": {
   
"@type": "g:UUID",
   
"@value": "7328b342-2ed3-43a5-b589-f6688b264ee7"
 
},
 
"processor": "traversal",
 
"op": "bytecode",
 
"args": {
   
"gremlin": {
     
"@type": "g:Bytecode",
     
"@value": {
       
"step": [
         
[
           
"V"
         
],
         
[
           
"has",
           
"name",
           
{
             
"@type": "g:P",
             
"@value": {
               
"predicate": "textContainsFuzzy",
               
"value": "herculeas"
             
}
           
}
         
]
       
]
     
}
   
},
   
"aliases": {
     
"g": "gg"
   
}
 
}
}

The serialized object really looks good, like the way it is expected. Doesn't it? Or else I cant seem to dechiper what's wrong.

I'm also facing similar errors which using the same PSerializer class for extending Geo Predicates like geoWithin, but I guess both are tied to same problem, and if one gets solved, other will also :-)

If needed, I can attach the serialized object for geoWithin too. But any help in debugging this will be really helpful, as this is last piece of development I'm expecting before releasing version 1 of drivers with support for both GraphSON 2.0 and GraphSON 3.0

Thanks.


TinkerPop 3.2 vs TinkerPop 3.3 Serialization difference for Predicate class

Debasish Kanhar <d.k...@...>
 

Hi All,

So as you all know I've been developing Client drivers for JanusGraph. I was able to test my initial version of library against JanusGraph 0.3.0 which uses TP 3.3.3. I now want to test the same set of driver for JanusGraph 0.2.1 which used TP 3.2.9 but I feel I've hit a bump with respect to Serialization for Predicate class.

So, my existent features like Adding of GeoShapes, Querying GeoShape and RelationIdentifier serialization/deserialization works. But any of my features leveraging TinkerPop's Predicate (PSerializer) doesn't work. I feel like this is some sort of serialization issue.

So, when I've implemented TextContainsFuzzy, which was working against JG 0.3.0/TP 3.3.3 (Using GraphSON 3.0) but the same doesn't work on GraphSON 2.0.

The following is my serialized object being sent to Gremlin-Server (Gremlin-server logs specify this)

Using TextFuzzy:
Error message:

org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV2d0  - Request [PooledUnsafeDirectByteBuf(ridx: 303, widx: 303, cap: 337)] could not be deserialized by org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV2d0.
org
.apache.tinkerpop.shaded.jackson.databind.JsonMappingException: Could not deserialize the JSON value as required. Nested exception: org.apache.tinkerpop.shaded.jackson.databind.JsonMappingException: Could not deserialize the JSON value as required. Nested exception: java.lang.IllegalStateException: org.apache.tinkerpop.gremlin.process.traversal.P.textContainsFuzzy([Ljava.lang.Object;)

And the serialized object being sent:

{
 
"requestId": {
   
"@type": "g:UUID",
   
"@value": "7328b342-2ed3-43a5-b589-f6688b264ee7"
 
},
 
"processor": "traversal",
 
"op": "bytecode",
 
"args": {
   
"gremlin": {
     
"@type": "g:Bytecode",
     
"@value": {
       
"step": [
         
[
           
"V"
         
],
         
[
           
"has",
           
"name",
           
{
             
"@type": "g:P",
             
"@value": {
               
"predicate": "textContainsFuzzy",
               
"value": "herculeas"
             
}
           
}
         
]
       
]
     
}
   
},
   
"aliases": {
     
"g": "gg"
   
}
 
}
}

The serialized object really looks good, like the way it is expected. Doesn't it? Or else I cant seem to dechiper what's wrong.

I'm also facing similar errors which using the same PSerializer class for extending Geo Predicates like geoWithin, but I guess both are tied to same problem, and if one gets solved, other will also :-)

If needed, I can attach the serialized object for geoWithin too. But any help in debugging this will be really helpful, as this is last piece of development I'm expecting before releasing version 1 of drivers with support for both GraphSON 2.0 and GraphSON 3.0

Thanks.


Re: Janusgraph + Spark standalone without hadoop

Debasish Kanhar <d.k...@...>
 

Ah. I think we might be wrong in our understanding there. As I was trying to read the graph data from my underlaying backend (Cassandra) and not any Graph stored on HDFS using JanusGraph's Cassandra3InputFormat class. The same was also failing when I was trying to run OLAP using a Spark Cluster without setting HADOOP_CONF_DIR variable.

Ideally that should not have been scenario, as TP > 3.3 doesn't need intermediate HDFS storage, but doesn't look like that's happening. Well we can track this thing if needed. :-)


On Friday, 24 August 2018 21:04:24 UTC+5:30, Jerry He wrote:
The need for Hadoop conf (only the hdfs conf) is to read from or write to graph data files on hdfs. Direct interacting with JanusGraph backend without involving any graph data files on hdfs won't need that, I think.

Thanks

Jerry

On Fri, Aug 24, 2018 at 5:05 AM Debasish Kanhar <d...@...> wrote:
@Jerry:

JanusGraph doesn't need Hadoop Cluster to run OLAP yes, but doesn't JanusGraph needs to point to a live Hadoop Cluster by setting HADOOP_CONF_DIR in CLASSPATH? I guess that was my understanding, and that was missing piece in docs for which it took me really long time to crack OLAP using Spark cluster.

On Friday, 24 August 2018 06:53:20 UTC+5:30, Jerry He wrote:
That being said, to be clear, you don't need a Hadoop cluster or any kind if that is what you mean. JanusGraph packages the Hadoop jars it needs. That is all you need to run SparkComputer on JansGraph.

Thanks

Jerry

On Thu, Aug 23, 2018 at 5:56 PM Jerry He <je...@...> wrote:
I don't think it will work. Spark needs input (to read graph data) and output (to write graph data).  JanusGraph currently only provides Hadoop InputFormat based reading from JanusGraph for OLAP.
In Tinkerpop, there are InputRDD and OutputRDD interfaces, which are by Spark (SpackGraphComputer).  (Search for Tinkerpop InputRDD.)
Unfortunately, JanusGraph provides no implementations other than the InputFormat based at the moment.

Thanks,

Jerry
 


On Wed, Aug 22, 2018 at 8:46 AM, Wei Ding <dw...@...> wrote:
Hi All,
   I am pretty new to Janusgraph and want to get some suggestions from you. Previously I posted a question about using ES as backend storage, and got some good feedback from Jason (Thanks!).  Now here comes another question: If I want to use janusgraph spark standalone without Hadoop for OLAP,  can some one point me a direction? Basically I have spark standalone deployed on kubernetes, how could that be used for OLAP? 

Thanks a lot!

Wei

--
You received this message because you are subscribed to the Google Groups "JanusGraph developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-dev/512224e3-5b20-4e31-afa5-e2fd74591182%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "JanusGraph developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-dev/d5561b80-bd25-4529-8698-cd605f5bab0a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

821 - 840 of 1601