Date
1 - 20 of 22
[DISCUSS] Moving toward an initial release
|
sjudeng <sju...@...>
Thanks for creating the branch. I'm starting working through TP tests now. You might consider changing the branch name to 0.1-SNAPSHOT or jg01 or something since presumably it would service all potential 0.1.x releases and also to avoid confusion with the eventual 0.1.0 tag. |
|
|
|
Ted Wilmes <twi...@...>
That's a helpful summation of the Apache release process Taylor, thanks for that. I say yes, we should get the process started. I'll kick things off with a DISCUSS thread and create a 0.1.0 release branch that we can work out of. Thanks, Ted On Saturday, March 25, 2017 at 2:01:30 AM UTC-5, Samik R wrote:
|
|
|
|
Samik Raychaudhuri <sam...@...>
On hosting the zip file, I think we should just
use the github hosting. It's easy and comes bundled with github,
so why not use it?
toggle quoted message
Show quoted text
Thanks. -Samik On 23-Mar-17 5:03 PM, sjudeng wrote:
|
|
|
|
sjudeng <sju...@...>
The process you describe sounds good to me. I would call out tests specifically in there somewhere. For a JanusGraph release I
think this should include running the full TinkerPop test suite before
the candidate release is generated. Also some of the configuration/process could be manual for the initial release and then simplified/automated for the second release. I'm really interested in getting master freed up for post-initial release development. But this could happen in parallel to release preparation once we create the release branch. Given there has been some discussion here on features to include (#80, #159, #132), can we start with master as it is now and then start the discuss thread in parallel? In the discussion I hope we can keep focus that this is an initial compatibility release only. Then hopefully we can agree that any new features, even those with complete PRs, can wait for the second release. There's nothing saying that the second release can't happen a month after the first. If this is acceptable I'd like to get started at least with testing (which takes 2-3 days) as soon as possible. Once the tests are confirmed good hopefully others can help out with the documentation (changelog, etc.) updates and definitely to serve as the RM to actually tag and push out the release artifacts. Again in my opinion we could do some of the configuration/process steps manually the first time and then simplify/automate them later maybe as part of the second release. Is this approach too aggressive? |
|
|
|
"P. Taylor Goetz" <ptg...@...>
I would say, like Apache projects, the release should consist of the following: * Source tar.gz/zip, with associated GPG signature of the release manager. * Convenience binary tar.gz/zip + GPG sig. * Maven artifacts published to mirrors. The typical process works like this: 1. DISCUSS thread to form consensus on what to include. 2. Create a branch to release from, merge patches for the release. 3. Create build the release with `mvn release:prepare && mvn release:perform`. This will generate the tar.gz/zip files and stage the maven artifacts in Nexus. 4. Release Manager closes the Nexus repository, making the jar files available for review. 5. RM uploads tar.gz/zip files to staging area for review 5. RM calls a VOTE on janusgraph-dev@ for the release with links to: The tag/SHA for the release, CHANGELOG, tar.gz/zip files, Nexus staging repository. If the VOTE is successful, release the Maven artifacts from Nexus, and move the staged tar.gz/zip files to the public download location. Then announce the release on user@ and dev@ mailing lists. The Apache parent pom [1] can help automating this, though we may need to change it for whatever infrastructure we use. I don’t have an answer on where to various download sites should be, aside from Nexus. Misha? When I looked at the TitanDB/JanusGraph build system, it seemed pretty convoluted and IMHO is ripe for refactoring and simplification. I’d be happy to help with this, but my availability is going to limited for the next two weeks. -Taylor
|
|
|
|
sjudeng <sju...@...>
Now that the initial snapshot deployment is behind us what are the next steps towards an initial release? PR#80 and #159 have been merged so it seems to me that the tip of master right now might be a good starting point for a release. We could work on release steps directly on master or create a release branch. Below is one possible approach. Or is there another process already defined or other alternatives? * Create a release branch, 0.1-SNAPSHOT based on current tip of master (8914cae) * On master branch: Bump versions from 0.1.0-SNAPSHOT to 0.2.0-SNAPSHOT * On 0.1-SNAPSHOT branch - Run full test suite including TP tests - Update documentation as described in RELEASING.md - Validate release (what are the steps here? running janusgraph.sh?) - Tag version 0.1.0 as 0.1.0-RC1 - Call vote on 0.1.0 release - (Optional) Repeat previous steps (0.1.0-RC2, etc.) as necessary until vote passes - Push artifacts to Nexus - Upload release distribution zipfile (where?) * Merge 0.1-SNAPSHOT into master One question is where release distribution zipfile will be hosted, assuming this should be provided somewhere aside from (or in addition to) Nexus. Titan used S3 I think and janusgraph-dist/src/release/release.sh looks like it might still be assuming this? Another option would be GitHub releases, which would provide them on https://github.com/JanusGraph/janusgraph/releases (see this page for an example). |
|
|
|
Jason Plurad <plu...@...>
Agreed, creating a separate branch for the initial release and ongoing development makes sense. I'll be reviewing #159 this afternoon. Others are encouraged to review it also. > help make this more manageable for committers (which I'm not, so I can't help on this part). Disagree on the part in parens... this is an open community project and anybody who wants to contribute to JanusGraph can help move the project forward can do so by communicating on the mailing lists, on issues, and on pull requests. Your participation in all venues has done a lot for JanusGraph so far and is greatly appreciated! On Friday, March 10, 2017 at 4:22:22 PM UTC-5, sjudeng wrote:
|
|
|
|
sjudeng <sju...@...>
I'm not familiar with the proposed released process. The JanusGraph release docs appear to start with a candidate release already ready to be deployed but not how it's prepared/vetted. Personally I'd like to see ongoing development and release preparation work be broken out into different branches. On the release side it puts good visibility on what exactly will be the basis for the release and then also allows non-release development to continue without getting held up. It sounds like #159 should definitely be included. I'd hope after this long that any other critical features/fixes for an initial release have already been identified on GitHub issues/PRs and there should be some resistance to delaying the release at all at this point except for really critical blockers. Hopefully keeping it scoped down to just a couple PRs to review/merge will help make this more manageable for committers (which I'm not, so I can't help on this part). |
|
|
|
Ted Wilmes <twi...@...>
I think that's a solid list. If #159 gets through review, I'd be interested in having it included. WIthout it, bulk loads via the BulkLoaderVertexProgram become untenable due to unnecessary full vertex scans. If we can come to consensus on this list, shall we set a freeze date at some point in the not too distant future and then start working through the release steps? Maybe step 1 at that point should be publishing a snapshot followed by going through the full process. --Ted On Thu, Mar 9, 2017 at 9:32 PM, sjudeng <sju...@...> wrote:
|
|
|
|
sjudeng <sju...@...>
That's a good point that titan11 was already at Spark 1.5 via TinkerPop 3.1.0. At that point including TinkerPop 3.2.3 probably isn't going to make much difference in terms of migration. If that's fine then a release branch (0.1-SNAPSHOT) could be created right now off the tip of master. If going with TinkerPop 3.2.3 it might be worth considering releasing JanusGraph as 0.2.0 if there was intention to tie versioning to TinkerPop. If the goal is to try and move out relatively quickly on an initial release then I wouldn't wait for the update to TinkerPop 3.2.4 or really any other new feature/dependency update unless it's a true blocker. Looking over existing PRs and 0.1.0 release milestone issues it appears to me that resolving the below items would be sufficient for the initial "compatibility" release. The other issues are either done or in my opinion don't need to block a release if they can't be immediately addressed.
|
|
|
|
Ted Wilmes <twi...@...>
In that case, since the disk storage format is unchanged, could we simply suggest that for the time being they could continue running OLAP jobs using their Titan 1.0 install? There isn't a lot of interaction between OLAP and OLTP at this point aside from bulk loading/exporting. I agree we should minimize leaving folks behind with version jumps that will require infrastructure upgrades (search & storage), but the titan11 branch that Janus forked off of, though it wasn't at TinkerPop 3.2.3, had progressed beyond the 3.0.2 release and I feel that there was a general consensus that it was a reasonable starting point. I suppose we could cut a release from pre-upgrade to 3.2.3, but at that point, we'd still be ahead of folks running Titan 1.0 with the TP version 3.1.1. I thinking getting Janus up to TinkerPop 3.2.3 or possibly even 3.2.4 depending on the timing is a big value add for people interested in kicking the tires and production users. Having said that, perhaps we should start a thread that includes concrete examples of what a TinkerPop upgrade would mean to a Janus user coming from Titan and get feedback from the community on the level of pain this would cause them. --Ted On Thu, Mar 9, 2017 at 6:06 PM, sjudeng <sju...@...> wrote:
|
|
|
|
"P. Taylor Goetz" <ptg...@...>
My guess would be that end users might want as close to drop in replacement for Titan as possible, at least initially. That would at least give them the opportunity to move from an essentially EOL'd solution (Titan) to one with a future (JanusGraph). In other words, it would send the message that JanusGraph is a serious and viable continuation of Titan. In terms of community development, IMHO it would be best to get users moved over to JanusGraph as quickly as possible. The sooner that happens, the sooner we get actual user feedback that can help better inform the project's technical direction. -Taylor On Mar 9, 2017, at 7:06 PM, sjudeng <sju...@...> wrote:
|
|
|
|
sjudeng <sju...@...>
The TinkerPop update is relevant because it would require production users of OLAP capabilities to upgrade their compute clusters (e.g. from Spark 1.2 to Spark 1.6). |
|
|
|
Adam Phelps <a...@...>
Yeah, I agree completely here.
toggle quoted message
Show quoted text
The TinkerPop update to me is a very different breaking change, since it just requires some minor code changes vs needing to upgrade clusters. - Adam Phelps On 3/9/17 10:35 AM, P. Taylor Goetz wrote:
I think that’s a sound approach. Having an easy migration path from |
|
|
|
"P. Taylor Goetz" <ptg...@...>
I think that’s a sound approach. Having an easy migration path from Titan to JanusGraph will encourage adoption and help build community. -Taylor
|
|
|
|
sjudeng <sju...@...>
Let me know if this should be moved to another thread but is there any interest in scoping down the initial JanusGraph release to just be a "compatibility" release? There has been concern raised in discussions (for example here) that some users on production Titan systems would benefit from an initial release that has no breaking changes, especially to core storage, indexing and compute system components. There are already potentially breaking changes in master such as the TinkerPop update and others are in work. Beyond mitigating concerns regarding having an initially easy migration step from Titan to JanusGraph, pursing a release branch without the additional features and dependency updates currently in work would allow for a relatively more rapid (and hopefully more stable) initial JanusGraph release. Any feedback on inevitable compatibility issues could then be addressed to the benefit of the subsequent JanusGraph release. |
|
|
|
"P. Taylor Goetz" <ptg...@...>
I created the following issue to get Sonatype Nexus setup so we can publish artifacts to maven central. Any committers interested in having the ability to perform a release should follow those instructions. Once we get a list of usernames, I can go ahead and setup the Nexus account for the “org.janusgraph” group ID. Thanks, -Taylor*
|
|
|
|
Misha Brukman <mbru...@...>
FYI, I created a milestone for the 0.1.0 release (since we're on 0.1.0-SNAPSHOT now): https://github.com/JanusGraph/janusgraph/milestone/1 Let's start adding all blockers to that milestone so that it's easy to track where we stand. For example, I just added the following issue:
I think some of these should also be blockers:
though what it's really trying to get to is to get a stable set of tests that we can rely on, that Travis also runs for every PR. Right now, we're just verifying builds, but since the full test suite times out, we are not running any tests continuously at all, which is dangerous. On Sat, Jan 28, 2017 at 11:46 PM, Jerry He <jerr...@...> wrote:
|
|
|
|
Jerry He <jerr...@...>
+1 We are on the same page. The first release will be an introduction release. New face, updated dependencies as appropriate, on par with Titan 1.0/1.1 in term of testing and quality, with consideration/ documentation for migration from Titan. Jerry |
|
|
|
Ted Wilmes <twi...@...>
Apache release process sounds good to me. Perhaps we can discuss what to include on this list and then mark the relevant issues in github with the 0.1 milestone. As far as what to include, I think it would be good to at a minimum get the build & tests in a repeatedly passing state and then possibly update things to TinkerPop 3.2.4 which will be released shortly. I just entered this other issue which is serious, but I'm torn if it's serious enough require its inclusion in the first release. Essentially if batch queries to the backends are flipped on, certain traversals will fail. At this point, getting a release that is at least at parity with the Titan 1.1 branch + a TinkerPop bump is probably a good first step. What are you all's thoughts? --Ted On Friday, January 27, 2017 at 5:22:34 PM UTC-6, Misha Brukman wrote:
|
|
|