Date   

Re: [DISCUSS] 0.2.3 release and 0.3 branch

Oleksandr Porunov <alexand...@...>
 

Notice, that PRs 1547 and 1588 are targeting master branch (which is 0.4.0 release). If you want those PRs to be included into 0.3.2 and 0.4.0 releases you should rebase your PRs and target 0.3 branch.


On Friday, May 17, 2019 at 9:46:28 PM UTC+3, Christopher Jackson wrote:
What is the current timeframe for the next 0.3 release? erinc just submitted 2 PRs that are crucial for our product release slated for June end. Would it be possible to have an early June 0.3.x release?

On Friday, February 8, 2019 at 8:47:30 AM UTC-5, Florian Hockmann wrote:
I just created a 0.3 branch and bumped master to 0.4.0-SNAPSHOT so we can make breaking changes and upgrade TinkerPop to version 3.4.0. This means that we now have 3 active release branches:
  • master
  • 0.3
  • 0.2
For the 0.2 branch, we decided already that it hits EOL with the next 0.2.3 version. This leaves us with the question whether we want to merge bug fixes also into the 0.2 branch until we release 0.2.3 or whether we just want to freeze it as is and release 0.2.3 in its current state. In that case we could also release 0.2.3 very shortly and then close the 0.2 branch.

My suggestion here is that we don't target 0.2 for any new pull requests and release 0.2.3 shortly so we can close that branch as maintaining 3 release branches requires too much effort in my opinion. There are currently 6 pull requests open with bug fixes of which two already target the 0.2 branch:
  • issue-357: Fix NullPointerException in loadRelations() #362
  • Prevent deadlock when vertices are removed in parallel #1380
We're waiting in both PRs for the contributor to address review comments. If those are addressed until we start the VOTE for the 0.2.3, then I'd say that we merge them in and otherwise we can just merge them any time into 0.3.

The next question is which contributions should go into the 0.3 branch, in addition to master. When the 0.2 branch was created, there was a consensus to only merge contributions into master and use the 0.2 branch solely for bug fixes. I recently also documented this in our CONTRIBUTING.md:

First, you need to decide which release branch (e.g., master, 0.2) to create the feature branch from. If you intend to add a new feature, then master is the right branch. Bug fixes however should also be applied to other releases, so you should create your feature branch from the release branch with the lowest version number that is still active (e.g., 0.2).

However, we didn't strictly follow this decision and also merged some new features into 0.2 so we might want to discuss this branching and release strategy in general again for the 0.3 branch.

Do we want to stay with our earlier decision to only apply contributions to master and use other release branches only for bug fixes? Or are there good arguments to also apply other contributions like new features or dependency updates to other release branches, like 0.3?
In case we decide to only apply bug fixes to the 0.3 branch, will that take effect now or only after the 0.4.0 release?

I'm personally in favour of staying with our policy of applying contributions only to master, except for bug fixes which should also go to other active release branches, like 0.3 right now. Applying contributions also to other release branches just increases the overhead for contributors and since we usually don't have that many breaking changes in a new minor version, I also don't see why we shouldn't expect users to just update to the newest version.
Lastly regarding whether we want to still merge all contributions into the 0.3 branch until we have a 0.4.0 release, I'm leaning against that as it would require all contributors of open PR to change the target branch and it would also lead to confusion why our usual policy of only applying contributions to master doesn't apply here.

So, in short I propose the following:
  1. We start release preparations for 0.2.3 shortly and close the 0.2 branch afterwards.
  2. 0.3 only gets bug fixes from now on.
  3. All other contributions only go into master.
Does that sound ok to everybody?


Re: [DISCUSS] 0.2.3 release and 0.3 branch

Oleksandr Porunov <alexand...@...>
 

I am planning to release 0.3.2 version just after Chris releases 0.2.3 version (If everything goes smoothly 22 May is the day of starting 0.2.3 release). I think 0.3.2 release will be available either at the end of this month or at the beginning of June. If you need your fixes urgently you can build jars of `0.3` branch and go live with them. After `0.3.2` is released you will be able to replace your jars with maven dependencies.


On Friday, May 17, 2019 at 9:46:28 PM UTC+3, Christopher Jackson wrote:
What is the current timeframe for the next 0.3 release? erinc just submitted 2 PRs that are crucial for our product release slated for June end. Would it be possible to have an early June 0.3.x release?

On Friday, February 8, 2019 at 8:47:30 AM UTC-5, Florian Hockmann wrote:
I just created a 0.3 branch and bumped master to 0.4.0-SNAPSHOT so we can make breaking changes and upgrade TinkerPop to version 3.4.0. This means that we now have 3 active release branches:
  • master
  • 0.3
  • 0.2
For the 0.2 branch, we decided already that it hits EOL with the next 0.2.3 version. This leaves us with the question whether we want to merge bug fixes also into the 0.2 branch until we release 0.2.3 or whether we just want to freeze it as is and release 0.2.3 in its current state. In that case we could also release 0.2.3 very shortly and then close the 0.2 branch.

My suggestion here is that we don't target 0.2 for any new pull requests and release 0.2.3 shortly so we can close that branch as maintaining 3 release branches requires too much effort in my opinion. There are currently 6 pull requests open with bug fixes of which two already target the 0.2 branch:
  • issue-357: Fix NullPointerException in loadRelations() #362
  • Prevent deadlock when vertices are removed in parallel #1380
We're waiting in both PRs for the contributor to address review comments. If those are addressed until we start the VOTE for the 0.2.3, then I'd say that we merge them in and otherwise we can just merge them any time into 0.3.

The next question is which contributions should go into the 0.3 branch, in addition to master. When the 0.2 branch was created, there was a consensus to only merge contributions into master and use the 0.2 branch solely for bug fixes. I recently also documented this in our CONTRIBUTING.md:

First, you need to decide which release branch (e.g., master, 0.2) to create the feature branch from. If you intend to add a new feature, then master is the right branch. Bug fixes however should also be applied to other releases, so you should create your feature branch from the release branch with the lowest version number that is still active (e.g., 0.2).

However, we didn't strictly follow this decision and also merged some new features into 0.2 so we might want to discuss this branching and release strategy in general again for the 0.3 branch.

Do we want to stay with our earlier decision to only apply contributions to master and use other release branches only for bug fixes? Or are there good arguments to also apply other contributions like new features or dependency updates to other release branches, like 0.3?
In case we decide to only apply bug fixes to the 0.3 branch, will that take effect now or only after the 0.4.0 release?

I'm personally in favour of staying with our policy of applying contributions only to master, except for bug fixes which should also go to other active release branches, like 0.3 right now. Applying contributions also to other release branches just increases the overhead for contributors and since we usually don't have that many breaking changes in a new minor version, I also don't see why we shouldn't expect users to just update to the newest version.
Lastly regarding whether we want to still merge all contributions into the 0.3 branch until we have a 0.4.0 release, I'm leaning against that as it would require all contributors of open PR to change the target branch and it would also lead to confusion why our usual policy of only applying contributions to master doesn't apply here.

So, in short I propose the following:
  1. We start release preparations for 0.2.3 shortly and close the 0.2 branch afterwards.
  2. 0.3 only gets bug fixes from now on.
  3. All other contributions only go into master.
Does that sound ok to everybody?


Re: [DISCUSS] 0.2.3 release and 0.3 branch

Christopher Jackson <jackson.ch...@...>
 

What is the current timeframe for the next 0.3 release? erinc just submitted 2 PRs that are crucial for our product release slated for June end. Would it be possible to have an early June 0.3.x release?


On Friday, February 8, 2019 at 8:47:30 AM UTC-5, Florian Hockmann wrote:
I just created a 0.3 branch and bumped master to 0.4.0-SNAPSHOT so we can make breaking changes and upgrade TinkerPop to version 3.4.0. This means that we now have 3 active release branches:
  • master
  • 0.3
  • 0.2
For the 0.2 branch, we decided already that it hits EOL with the next 0.2.3 version. This leaves us with the question whether we want to merge bug fixes also into the 0.2 branch until we release 0.2.3 or whether we just want to freeze it as is and release 0.2.3 in its current state. In that case we could also release 0.2.3 very shortly and then close the 0.2 branch.

My suggestion here is that we don't target 0.2 for any new pull requests and release 0.2.3 shortly so we can close that branch as maintaining 3 release branches requires too much effort in my opinion. There are currently 6 pull requests open with bug fixes of which two already target the 0.2 branch:
  • issue-357: Fix NullPointerException in loadRelations() #362
  • Prevent deadlock when vertices are removed in parallel #1380
We're waiting in both PRs for the contributor to address review comments. If those are addressed until we start the VOTE for the 0.2.3, then I'd say that we merge them in and otherwise we can just merge them any time into 0.3.

The next question is which contributions should go into the 0.3 branch, in addition to master. When the 0.2 branch was created, there was a consensus to only merge contributions into master and use the 0.2 branch solely for bug fixes. I recently also documented this in our CONTRIBUTING.md:

First, you need to decide which release branch (e.g., master, 0.2) to create the feature branch from. If you intend to add a new feature, then master is the right branch. Bug fixes however should also be applied to other releases, so you should create your feature branch from the release branch with the lowest version number that is still active (e.g., 0.2).

However, we didn't strictly follow this decision and also merged some new features into 0.2 so we might want to discuss this branching and release strategy in general again for the 0.3 branch.

Do we want to stay with our earlier decision to only apply contributions to master and use other release branches only for bug fixes? Or are there good arguments to also apply other contributions like new features or dependency updates to other release branches, like 0.3?
In case we decide to only apply bug fixes to the 0.3 branch, will that take effect now or only after the 0.4.0 release?

I'm personally in favour of staying with our policy of applying contributions only to master, except for bug fixes which should also go to other active release branches, like 0.3 right now. Applying contributions also to other release branches just increases the overhead for contributors and since we usually don't have that many breaking changes in a new minor version, I also don't see why we shouldn't expect users to just update to the newest version.
Lastly regarding whether we want to still merge all contributions into the 0.3 branch until we have a 0.4.0 release, I'm leaning against that as it would require all contributors of open PR to change the target branch and it would also lead to confusion why our usual policy of only applying contributions to master doesn't apply here.

So, in short I propose the following:
  1. We start release preparations for 0.2.3 shortly and close the 0.2 branch afterwards.
  2. 0.3 only gets bug fixes from now on.
  3. All other contributions only go into master.
Does that sound ok to everybody?


Janusgraph 0.3.x release?

Christopher Jackson <jackson.ch...@...>
 

We've just submitted two prs which have been approved against the 3.x line. We'd like to make sure they get included in our next product release but we can only target officially released versions of janusgraph. We would like to know when the next expected release is and if we can push to have it done no later then early June?


Re: CLA signing automation

Misha Brukman <mbru...@...>
 

Hi Chris,

Yes, please add me to the thread and I'll take it from there.

Thanks,
Misha

On Tue, May 14, 2019 at 10:49 PM Chris Hupman <chris...@...> wrote:
I reached out to John Mertic from the linux foundation about the CLA automation tool and was told it is now ready for Prime time. He also said he could help get us onboarded. Would it be easiest if I just add you on the email thread Misha?

On Saturday, March 30, 2019 at 9:11:38 AM UTC-7, Oleksandr Porunov wrote:
Misha, Jason,

Could you please check if "fufler" signed a CLA? If so, could you create a PR to include him into janusgraph-legal repository?

Regards,
Oleksandr

On Sunday, March 24, 2019 at 5:08:34 PM UTC+2, Oleksandr Porunov wrote:
Misha, Jason,

Could you please check if VladimirBogomolov signed a CLA? Looks like he signed it 19 days ago but the CLA PR wasn't been submitted to janusgraph-legal repository still.

Regards,
Oleksandr

On Thursday, March 21, 2019 at 11:20:31 PM UTC+2, Misha Brukman wrote:
[ bcc: janusgraph-dev ]

No trouble at all, Chris! You're on the invite.

On Thu, Mar 21, 2019 at 3:15 PM Chris Hupman <ch...@...> wrote:
Hey Misha,

If it's not too much trouble I'd be interested in joining the call. 

Regards,

Chris

On Thursday, March 21, 2019 at 12:31:27 AM UTC-7, Oleksandr Porunov wrote:
Misha,

I really love to hear that! I am happy that there are progress in the CLA automation direction.
Hope it will be implemented soon.

On Thursday, March 21, 2019 at 12:34:43 AM UTC+2, Misha Brukman wrote:
In emailing he...@... to open a ticket (which I believe was the process in the past?), I've learned that (a) Linux Foundation no longer supports that email address, and (b) they are now using Jira for their ticket system. I logged in and created an account and found this really old ticket: https://jira.linuxfoundation.org/browse/CL-92 (subject: "get JanusGraph CLA template") which I pinged to get some update on the status.

I've also followed-up on our thread from February (!!) with some folks from The Linux Foundation to set up a meeting to discuss the onboarding to the new CLA process, and we now have a meeting scheduled on Mon, Mar 25 at 11am PDT / 2pm EDT. If any JanusGraph TSC member or committer is interested in joining the call (phone or video call, both are supported), please let me know, and I'll be happy to add you.

Looking forward to getting the CLA process fully automated soon!

On Sun, Mar 17, 2019 at 10:30 AM Oleksandr Porunov <alex...@...> wrote:
Florian, 

It is great news! I really happy to hear that!
Waiting for the bot :)

On Sunday, March 17, 2019 at 4:18:28 PM UTC+2, Florian Hockmann wrote:
I just asked in the issue I linked above whether it will be possible for us to use the CLA bot of the Linux Foundation and got a very promising response from Dan Kohn:

Yes, JanusGraph will be very welcome to use it with the LF managing the CLAbot. We're just a little backed up with the 2 week private beta of CommunityBridge and then we can hopefully move forward with the gRPC CLAbot rollout. https://www.linuxfoundation.org/press-release/2019/03/the-linux-foundation-launches-new-communitybridge-platform-to-help-sustain-open-source-communities/

The gRPC project will beta test the bot and then they want to make it accessible to other projects afterwards.

So, it sounds like we can use this automation in a few weeks / months :-)

Am Dienstag, 5. März 2019 14:12:21 UTC+1 schrieb Oleksandr Porunov:
Asking to check CLAs for the next contributors:

- VladimirBogomolov
- wangzheng90

On Tuesday, March 5, 2019 at 1:57:00 PM UTC+2, Oleksandr Porunov wrote:
Thanks for pointing to this problem. I didn't think about CCLAs before. We need to investigate it.

On Tuesday, March 5, 2019 at 12:56:53 PM UTC+2, Florian Hockmann wrote:
I agree with you that the CLA process takes too long right now and scares away some possible contributors which is really problematic.

The only problem I see with these tools is that they all only seem to handle individual CLAs (ICLA), but not corporate CLAs (CCLA). Since most of our contributors use a CCLA, we also need to find a solution for that.

Out of the 3 tools you listed, I found two that had issues for CCLAs:
For cla-assistant, closed without a fix: https://github.com/cla-assistant/cla-assistant/issues/15

Am Dienstag, 5. März 2019 10:43:30 UTC+1 schrieb Oleksandr Porunov:
Florian, I believe we can move with 3rd party CLA bot now because there is no any news from LF. 

There are contributors who would like to contribute, but CLA verification is taking too much time to accept new contributors. It really slows down the process because most of the time new contributors either don't want to go through the process where they need to print then sign then scan then send CLA or even when they do it they are losing interest after 1+ months of verification.

As for me, I think we should move to automate CLA signing ASAP. I think we can discuss a CLA bot for JanusGraph.

Some possible solutions could be either CLA automation tool like these tools:

Or signing with Google form like this one:

It would be nice to move to automate CLA signing on this or the next week.

On Friday, March 1, 2019 at 11:13:01 AM UTC+2, Oleksandr Porunov wrote:
Jason, thank you for submitting the PR. 
Also reporting that wangzheng90 from github sent a CLA. Related PR-1456.

Misha, could you please add necessary permissions to Jason?

On Friday, March 1, 2019 at 7:50:45 AM UTC+2, Jason Plurad wrote:
I've submitted a PR for the ICLA.

Regarding adding other committers to the CLA Google Group, it doesn't appear that I have authority to do that, only Misha.

I've noticed that I can't see the CLA archive via the Google Group. I do have a record of every email I've received since being added to the group (April 2018), so I have access to only a partial archive.


On Thursday, February 28, 2019 at 4:05:36 AM UTC-5, Oleksandr Porunov wrote:
I agree. If there is no response, we should go with 3rd party CLA bot.

On Thursday, February 28, 2019 at 10:15:28 AM UTC+2, Florian Hockmann wrote:
Also, is there any news from The Linux Foundation?

From my side, no. The Linux Foundation directed me to two members of the Kubernetes community who I contacted, but haven't heard back from them.

@Misha: Were you more successful?

If we don't hear anything from the LF any time soon, then I suggest that we just go ahead with a 3rd party CLA bot like you (Oleksandr) originally suggested in the first post here.

Am Donnerstag, 28. Februar 2019 00:31:28 UTC+1 schrieb Oleksandr Porunov:
I've noticed that it is already about 1 month has gone when `mad` from github send a CLA. Is there any news about including him into janusgraph/legal? Next PRs are stuck because of this:
#1406

Also, is there any news from The Linux Foundation?

On Thursday, February 14, 2019 at 6:54:02 PM UTC+2, Misha Brukman wrote:
I've reached out to The Linux Foundation folks I've previously spoken to about CLA automation (which wasn't implemented yet, as you can tell, and I haven't heard back from for a while). I believe they already have a GitHub bot we can reuse. I'll let you know what I find out.

On Fri, Feb 8, 2019 at 9:19 AM Florian Hockmann <f...@...> wrote:
Thanks for bringing this up, Oleksandr. I agree that it's important to make this process easier for contributors as each additional loop they have to jump through could prevent them from contributing to JanusGraph in the first place.

Apparently, Kubernetes has automated its CLA workflow. They are hosted within the Cloud Native Computing Foundation (CNCF) but it seems to use only infrastructure from the Linux Foundation and since the CNCF is also a Linux Foundation project, we should be able to use the same system in general.
They also have docs on how to set up the automation for a new repo. At first glance, it looks like we only need a JanusGraph group for identity.linuxfoundation.org.

While we're at it we could also automate the DCO check with a DCO bot. We currently have to check this manually while reviewing a PR which we tend to forget sometimes.

Am Freitag, 8. Februar 2019 08:16:31 UTC+1 schrieb Oleksandr Porunov:
Hello,

I have a concern about current CLA signing workflow. Currently we need to print a CLA agreement then fill it, then scan it then send it via email and then wait for an unknown amount of time while it is checked then wait while the PR with your signing details is merged into https://github.com/JanusGraph/legal.
It causes very huge delays sometimes more than 1 month. I think it is a big disadvantage for accepting new contributions.

I think we would be able to use either Google form for CLA accepting like this one:

Or maybe use some tool for CLA automation like this one:

There are a lot of CLA automation tools. I think we could use some.

Can we automate this process to digitally sign CLAs or we need real scanned paper copy?

--
You received this message because you are subscribed to the Google Groups "JanusGraph developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusg...@....
To post to this group, send email to jan...@....
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-dev/d40dbef3-8d4f-474b-9125-e301b37e563b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "JanusGraph developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusg...@....
To post to this group, send email to jan...@....
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-dev/50813679-4e44-43c3-9890-853f56a402bf%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "JanusGraph developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusg...@....
To post to this group, send email to jan...@....
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-dev/3da7826f-f526-4985-9b87-9f2f4e23435e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "JanusGraph developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgr...@....
To post to this group, send email to janusgr...@....
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-dev/9ef27267-52bf-4a24-b0c5-c3134000a67a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: CLA signing automation

Chris Hupman <chris...@...>
 

I reached out to John Mertic from the linux foundation about the CLA automation tool and was told it is now ready for Prime time. He also said he could help get us onboarded. Would it be easiest if I just add you on the email thread Misha?


On Saturday, March 30, 2019 at 9:11:38 AM UTC-7, Oleksandr Porunov wrote:
Misha, Jason,

Could you please check if "fufler" signed a CLA? If so, could you create a PR to include him into janusgraph-legal repository?

Regards,
Oleksandr

On Sunday, March 24, 2019 at 5:08:34 PM UTC+2, Oleksandr Porunov wrote:
Misha, Jason,

Could you please check if VladimirBogomolov signed a CLA? Looks like he signed it 19 days ago but the CLA PR wasn't been submitted to janusgraph-legal repository still.

Regards,
Oleksandr

On Thursday, March 21, 2019 at 11:20:31 PM UTC+2, Misha Brukman wrote:
[ bcc: janusgraph-dev ]

No trouble at all, Chris! You're on the invite.

On Thu, Mar 21, 2019 at 3:15 PM Chris Hupman <ch...@...> wrote:
Hey Misha,

If it's not too much trouble I'd be interested in joining the call. 

Regards,

Chris

On Thursday, March 21, 2019 at 12:31:27 AM UTC-7, Oleksandr Porunov wrote:
Misha,

I really love to hear that! I am happy that there are progress in the CLA automation direction.
Hope it will be implemented soon.

On Thursday, March 21, 2019 at 12:34:43 AM UTC+2, Misha Brukman wrote:
In emailing help...@rt.linuxfoundation.org to open a ticket (which I believe was the process in the past?), I've learned that (a) Linux Foundation no longer supports that email address, and (b) they are now using Jira for their ticket system. I logged in and created an account and found this really old ticket: https://jira.linuxfoundation.org/browse/CL-92 (subject: "get JanusGraph CLA template") which I pinged to get some update on the status.

I've also followed-up on our thread from February (!!) with some folks from The Linux Foundation to set up a meeting to discuss the onboarding to the new CLA process, and we now have a meeting scheduled on Mon, Mar 25 at 11am PDT / 2pm EDT. If any JanusGraph TSC member or committer is interested in joining the call (phone or video call, both are supported), please let me know, and I'll be happy to add you.

Looking forward to getting the CLA process fully automated soon!

On Sun, Mar 17, 2019 at 10:30 AM Oleksandr Porunov <alex...@...> wrote:
Florian, 

It is great news! I really happy to hear that!
Waiting for the bot :)

On Sunday, March 17, 2019 at 4:18:28 PM UTC+2, Florian Hockmann wrote:
I just asked in the issue I linked above whether it will be possible for us to use the CLA bot of the Linux Foundation and got a very promising response from Dan Kohn:

Yes, JanusGraph will be very welcome to use it with the LF managing the CLAbot. We're just a little backed up with the 2 week private beta of CommunityBridge and then we can hopefully move forward with the gRPC CLAbot rollout. https://www.linuxfoundation.org/press-release/2019/03/the-linux-foundation-launches-new-communitybridge-platform-to-help-sustain-open-source-communities/

The gRPC project will beta test the bot and then they want to make it accessible to other projects afterwards.

So, it sounds like we can use this automation in a few weeks / months :-)

Am Dienstag, 5. März 2019 14:12:21 UTC+1 schrieb Oleksandr Porunov:
Asking to check CLAs for the next contributors:

- VladimirBogomolov
- wangzheng90

On Tuesday, March 5, 2019 at 1:57:00 PM UTC+2, Oleksandr Porunov wrote:
Thanks for pointing to this problem. I didn't think about CCLAs before. We need to investigate it.

On Tuesday, March 5, 2019 at 12:56:53 PM UTC+2, Florian Hockmann wrote:
I agree with you that the CLA process takes too long right now and scares away some possible contributors which is really problematic.

The only problem I see with these tools is that they all only seem to handle individual CLAs (ICLA), but not corporate CLAs (CCLA). Since most of our contributors use a CCLA, we also need to find a solution for that.

Out of the 3 tools you listed, I found two that had issues for CCLAs:
For cla-assistant, closed without a fix: https://github.com/cla-assistant/cla-assistant/issues/15

Am Dienstag, 5. März 2019 10:43:30 UTC+1 schrieb Oleksandr Porunov:
Florian, I believe we can move with 3rd party CLA bot now because there is no any news from LF. 

There are contributors who would like to contribute, but CLA verification is taking too much time to accept new contributors. It really slows down the process because most of the time new contributors either don't want to go through the process where they need to print then sign then scan then send CLA or even when they do it they are losing interest after 1+ months of verification.

As for me, I think we should move to automate CLA signing ASAP. I think we can discuss a CLA bot for JanusGraph.

Some possible solutions could be either CLA automation tool like these tools:

Or signing with Google form like this one:

It would be nice to move to automate CLA signing on this or the next week.

On Friday, March 1, 2019 at 11:13:01 AM UTC+2, Oleksandr Porunov wrote:
Jason, thank you for submitting the PR. 
Also reporting that wangzheng90 from github sent a CLA. Related PR-1456.

Misha, could you please add necessary permissions to Jason?

On Friday, March 1, 2019 at 7:50:45 AM UTC+2, Jason Plurad wrote:
I've submitted a PR for the ICLA.

Regarding adding other committers to the CLA Google Group, it doesn't appear that I have authority to do that, only Misha.

I've noticed that I can't see the CLA archive via the Google Group. I do have a record of every email I've received since being added to the group (April 2018), so I have access to only a partial archive.


On Thursday, February 28, 2019 at 4:05:36 AM UTC-5, Oleksandr Porunov wrote:
I agree. If there is no response, we should go with 3rd party CLA bot.

On Thursday, February 28, 2019 at 10:15:28 AM UTC+2, Florian Hockmann wrote:
Also, is there any news from The Linux Foundation?

From my side, no. The Linux Foundation directed me to two members of the Kubernetes community who I contacted, but haven't heard back from them.

@Misha: Were you more successful?

If we don't hear anything from the LF any time soon, then I suggest that we just go ahead with a 3rd party CLA bot like you (Oleksandr) originally suggested in the first post here.

Am Donnerstag, 28. Februar 2019 00:31:28 UTC+1 schrieb Oleksandr Porunov:
I've noticed that it is already about 1 month has gone when `mad` from github send a CLA. Is there any news about including him into janusgraph/legal? Next PRs are stuck because of this:
#1406

Also, is there any news from The Linux Foundation?

On Thursday, February 14, 2019 at 6:54:02 PM UTC+2, Misha Brukman wrote:
I've reached out to The Linux Foundation folks I've previously spoken to about CLA automation (which wasn't implemented yet, as you can tell, and I haven't heard back from for a while). I believe they already have a GitHub bot we can reuse. I'll let you know what I find out.

On Fri, Feb 8, 2019 at 9:19 AM Florian Hockmann <f...@...> wrote:
Thanks for bringing this up, Oleksandr. I agree that it's important to make this process easier for contributors as each additional loop they have to jump through could prevent them from contributing to JanusGraph in the first place.

Apparently, Kubernetes has automated its CLA workflow. They are hosted within the Cloud Native Computing Foundation (CNCF) but it seems to use only infrastructure from the Linux Foundation and since the CNCF is also a Linux Foundation project, we should be able to use the same system in general.
They also have docs on how to set up the automation for a new repo. At first glance, it looks like we only need a JanusGraph group for identity.linuxfoundation.org.

While we're at it we could also automate the DCO check with a DCO bot. We currently have to check this manually while reviewing a PR which we tend to forget sometimes.

Am Freitag, 8. Februar 2019 08:16:31 UTC+1 schrieb Oleksandr Porunov:
Hello,

I have a concern about current CLA signing workflow. Currently we need to print a CLA agreement then fill it, then scan it then send it via email and then wait for an unknown amount of time while it is checked then wait while the PR with your signing details is merged into https://github.com/JanusGraph/legal.
It causes very huge delays sometimes more than 1 month. I think it is a big disadvantage for accepting new contributions.

I think we would be able to use either Google form for CLA accepting like this one:

Or maybe use some tool for CLA automation like this one:

There are a lot of CLA automation tools. I think we could use some.

Can we automate this process to digitally sign CLAs or we need real scanned paper copy?

--
You received this message because you are subscribed to the Google Groups "JanusGraph developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-de...@googlegroups.com.
To post to this group, send email to jan...@....
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-dev/d40dbef3-8d4f-474b-9125-e301b37e563b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "JanusGraph developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-de...@googlegroups.com.
To post to this group, send email to jan...@....
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-dev/50813679-4e44-43c3-9890-853f56a402bf%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "JanusGraph developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-de...@googlegroups.com.
To post to this group, send email to jan...@....
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-dev/3da7826f-f526-4985-9b87-9f2f4e23435e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: Ports/Backports

Chris Hupman <chris...@...>
 

I looked at all the PRs targeting 0.2 and currently I'm only really optimistic about 1575 making it in for the release. Right now the only thing I would consider delaying for is 1479, the service-wrapper PR, but I'm hoping to close that out this week. Here are my notes on the 4 PRs.

I did a full pass on this, but I want a second binding approval before this gets merged. I did some digging, but I still don't have a clear picture in my head of what acquireLocks() should look like.

I approved, but it will require a second approval to merge. At least the test will be easy to merge upstream.

This PR should probably be targeting master, otherwise you'll need to address the test failures on es1 and es2. I didn't do a review since it wasn't passing CI.

I did a pass and there were two items to resolve. The main one being adding a test. Since it requires a mock I would rather it goes into 0.3 or 0.4 so that we won't have to convert the test from easyMock to Mockito and junit 4 to junit 5.

Just so I understand better is there a push to get into 0.2.3 so you won't have to update to 0.3.x to get access to the features? Or did you just target 0.2 so the changes would make it into all the releases?

Cheers,

Chris


On Tuesday, May 14, 2019 at 5:10:10 AM UTC-7, Florian Hockmann wrote:
The problem is just that we recently decided to close branch 0.2 and only make a last release of that branch and then close it for development. Any new pull requests that now target 0.2 could therefore either postpone that release or if they are merged after the release, we need to merge them into 0.3 and from their into master. That is why I suggest that you try to target 0.3 instead for bug fixes or master for new features / enhancements. It shouldn't usually be a problem for you to rebase a feature branch on one of these release branches.

Am Montag, 13. Mai 2019 23:13:32 UTC+2 schrieb Chris Hupman:
We regularly merge commits upstream. You just need to worry about submitting the Pull Request you want to get in. If you want to take care of merging it into all branches here's a comment with an example.

 Changes to all release branches will also be merged into master

On Monday, May 13, 2019 at 1:29:32 PM UTC-7, wri...@... wrote:
Hi Janusgraph Team, 

We've just become a contributors and would like to push our changes to public repository.
The thing is that all changes are based on 0.2 version - the version we use in production.
How it'd suitable to review and (back)port the changes?
- Start from 0.2 branch and then port to master?
- (Vice versa) Start from master and then backport to 0.2?

Thanks


Re: Ports/Backports

Florian Hockmann <f...@...>
 

The problem is just that we recently decided to close branch 0.2 and only make a last release of that branch and then close it for development. Any new pull requests that now target 0.2 could therefore either postpone that release or if they are merged after the release, we need to merge them into 0.3 and from their into master. That is why I suggest that you try to target 0.3 instead for bug fixes or master for new features / enhancements. It shouldn't usually be a problem for you to rebase a feature branch on one of these release branches.

Am Montag, 13. Mai 2019 23:13:32 UTC+2 schrieb Chris Hupman:

We regularly merge commits upstream. You just need to worry about submitting the Pull Request you want to get in. If you want to take care of merging it into all branches here's a comment with an example.

 Changes to all release branches will also be merged into master

On Monday, May 13, 2019 at 1:29:32 PM UTC-7, wri...@... wrote:
Hi Janusgraph Team, 

We've just become a contributors and would like to push our changes to public repository.
The thing is that all changes are based on 0.2 version - the version we use in production.
How it'd suitable to review and (back)port the changes?
- Start from 0.2 branch and then port to master?
- (Vice versa) Start from master and then backport to 0.2?

Thanks


Re: [DISCUSS] JanusGraph 0.2.3 release

Chris Hupman <chris...@...>
 

Hey Pylyp,

I'll see if I can do a review on all 4 PRs tomorrow, but it'll require a second approver and all the comments resolved to merge. The size looks reasonable and they all have tests so I think they have a good chance of making it in 0.2.3, but if any of them can't get resolved in time we'll have to retarget at the next release, which will be 0.3.2 in this case. 

Chris


On Monday, May 13, 2019 at 2:56:31 PM UTC-7, writetoph1lm wrote:
Thanks Chris, 

One more PR https://github.com/JanusGraph/legal/pull/137 needs to be merged to get cla:yes.

Thanks

понедельник, 13 мая 2019 г., 16:52:05 UTC-4 пользователь Chris Hupman написал:
Hello,

My current intention is still to push the 0.2.3 release on May 22nd and I'm actively working on the 2 remaining pull requests to meet that deadline. If there's something you would like to get into 0.2.3 you'll want to submit your PR as soon as you can since reviews take time. Also just in case you didn't already know 0.2.3 will be the last 0.2.x release.

Regards,

Chris

On Monday, May 13, 2019 at 1:44:50 PM UTC-7, writetoph1lm wrote:
Hi All, 

We're currently using JG 0.2.2 in production and would like to push out changes to the next 0.2.x release.
CLA is signed and we're good to go. 
If everybody is ok I can open PR for 0.2 branch and then port changes to master in separate PR.

Thanks

вторник, 23 апреля 2019 г., 16:40:08 UTC-4 пользователь Chris Hupman написал:
Hi all,

This morning I went through and tagged all the issues and pull requests that I feel belong in the 0.2.3 release. Currently there are only 2 open pull requests targeting the 0.2 branch. I can't think of anything else that needs to be in the final 0.2.x release, but would love some additional input about whether or not anything else should be included. Currently I'm targeting May 22nd for the release date.

I also added issues to the milestone for my stretch goal of generating rpm and deb packages as release artifacts.  

Here is a link to the release milestone. Please provide any and all feedback.

Chris


Re: [DISCUSS] JanusGraph 0.2.3 release

writetoph1lm <writet...@...>
 

Thanks Chris, 

One more PR https://github.com/JanusGraph/legal/pull/137 needs to be merged to get cla:yes.

Thanks

понедельник, 13 мая 2019 г., 16:52:05 UTC-4 пользователь Chris Hupman написал:

Hello,

My current intention is still to push the 0.2.3 release on May 22nd and I'm actively working on the 2 remaining pull requests to meet that deadline. If there's something you would like to get into 0.2.3 you'll want to submit your PR as soon as you can since reviews take time. Also just in case you didn't already know 0.2.3 will be the last 0.2.x release.

Regards,

Chris

On Monday, May 13, 2019 at 1:44:50 PM UTC-7, writetoph1lm wrote:
Hi All, 

We're currently using JG 0.2.2 in production and would like to push out changes to the next 0.2.x release.
CLA is signed and we're good to go. 
If everybody is ok I can open PR for 0.2 branch and then port changes to master in separate PR.

Thanks

вторник, 23 апреля 2019 г., 16:40:08 UTC-4 пользователь Chris Hupman написал:
Hi all,

This morning I went through and tagged all the issues and pull requests that I feel belong in the 0.2.3 release. Currently there are only 2 open pull requests targeting the 0.2 branch. I can't think of anything else that needs to be in the final 0.2.x release, but would love some additional input about whether or not anything else should be included. Currently I'm targeting May 22nd for the release date.

I also added issues to the milestone for my stretch goal of generating rpm and deb packages as release artifacts.  

Here is a link to the release milestone. Please provide any and all feedback.

Chris


Re: Ports/Backports

Chris Hupman <chris...@...>
 

We regularly merge commits upstream. You just need to worry about submitting the Pull Request you want to get in. If you want to take care of merging it into all branches here's a comment with an example.

 Changes to all release branches will also be merged into master

On Monday, May 13, 2019 at 1:29:32 PM UTC-7, wri...@... wrote:
Hi Janusgraph Team, 

We've just become a contributors and would like to push our changes to public repository.
The thing is that all changes are based on 0.2 version - the version we use in production.
How it'd suitable to review and (back)port the changes?
- Start from 0.2 branch and then port to master?
- (Vice versa) Start from master and then backport to 0.2?

Thanks


Re: [DISCUSS] JanusGraph 0.2.3 release

Chris Hupman <chris...@...>
 

Hello,

My current intention is still to push the 0.2.3 release on May 22nd and I'm actively working on the 2 remaining pull requests to meet that deadline. If there's something you would like to get into 0.2.3 you'll want to submit your PR as soon as you can since reviews take time. Also just in case you didn't already know 0.2.3 will be the last 0.2.x release.

Regards,

Chris


On Monday, May 13, 2019 at 1:44:50 PM UTC-7, writetoph1lm wrote:
Hi All, 

We're currently using JG 0.2.2 in production and would like to push out changes to the next 0.2.x release.
CLA is signed and we're good to go. 
If everybody is ok I can open PR for 0.2 branch and then port changes to master in separate PR.

Thanks

вторник, 23 апреля 2019 г., 16:40:08 UTC-4 пользователь Chris Hupman написал:
Hi all,

This morning I went through and tagged all the issues and pull requests that I feel belong in the 0.2.3 release. Currently there are only 2 open pull requests targeting the 0.2 branch. I can't think of anything else that needs to be in the final 0.2.x release, but would love some additional input about whether or not anything else should be included. Currently I'm targeting May 22nd for the release date.

I also added issues to the milestone for my stretch goal of generating rpm and deb packages as release artifacts.  

Here is a link to the release milestone. Please provide any and all feedback.

Chris


Re: [DISCUSS] JanusGraph 0.2.3 release

writetoph1lm <writet...@...>
 

Hi All, 

We're currently using JG 0.2.2 in production and would like to push out changes to the next 0.2.x release.
CLA is signed and we're good to go. 
If everybody is ok I can open PR for 0.2 branch and then port changes to master in separate PR.

Thanks

вторник, 23 апреля 2019 г., 16:40:08 UTC-4 пользователь Chris Hupman написал:

Hi all,

This morning I went through and tagged all the issues and pull requests that I feel belong in the 0.2.3 release. Currently there are only 2 open pull requests targeting the 0.2 branch. I can't think of anything else that needs to be in the final 0.2.x release, but would love some additional input about whether or not anything else should be included. Currently I'm targeting May 22nd for the release date.

I also added issues to the milestone for my stretch goal of generating rpm and deb packages as release artifacts.  

Here is a link to the release milestone. Please provide any and all feedback.

Chris


Ports/Backports

writet...@...
 

Hi Janusgraph Team, 

We've just become a contributors and would like to push our changes to public repository.
The thing is that all changes are based on 0.2 version - the version we use in production.
How it'd suitable to review and (back)port the changes?
- Start from 0.2 branch and then port to master?
- (Vice versa) Start from master and then backport to 0.2?

Thanks


Re: Improve JanusGraph main page

Oleksandr Porunov <alexand...@...>
 

Asking for the review of this PR: https://github.com/JanusGraph/janusgraph.org/pull/64


Re: [DISCUSS] JanusGraph 0.2.3 release

Chris Hupman <chris...@...>
 

If anyone has cycles I would really like another set of eyes on PR #1479. I pretty much completely rewrote gremlin-server.sh so that it would be fully backwards compatible, but additionally support start, stop, and being run as a service in systemV and SystemD. 

I've done testing on centOS 6 and 7 as well as Ubuntu18 to cover testing on SystemD and SystemV. Even if it's just downloading the gremlin-server.sh script and making sure it works in place the way you're used to I would greatly appreciate it. 

I started to work on creating a noarch rpm using the changes made in this PR. I'm really looking forward to replacing a ton of the setup in the dockerfile with a single apt install. 


On Sunday, May 5, 2019 at 7:09:45 AM UTC-7, Oleksandr Porunov wrote:
Thank you Chris for starting this discussion. The milestone and release day looks good to me. I am back from vacation so I will be able to review and test release artifacts when they are ready.

Oleksandr

On Tuesday, April 23, 2019 at 11:40:08 PM UTC+3, Chris Hupman wrote:
Hi all,

This morning I went through and tagged all the issues and pull requests that I feel belong in the 0.2.3 release. Currently there are only 2 open pull requests targeting the 0.2 branch. I can't think of anything else that needs to be in the final 0.2.x release, but would love some additional input about whether or not anything else should be included. Currently I'm targeting May 22nd for the release date.

I also added issues to the milestone for my stretch goal of generating rpm and deb packages as release artifacts.  

Here is a link to the release milestone. Please provide any and all feedback.

Chris


Re: [DISCUSS] JanusGraph 0.2.3 release

Oleksandr Porunov <alexand...@...>
 

Thank you Chris for starting this discussion. The milestone and release day looks good to me. I am back from vacation so I will be able to review and test release artifacts when they are ready.

Oleksandr


On Tuesday, April 23, 2019 at 11:40:08 PM UTC+3, Chris Hupman wrote:
Hi all,

This morning I went through and tagged all the issues and pull requests that I feel belong in the 0.2.3 release. Currently there are only 2 open pull requests targeting the 0.2 branch. I can't think of anything else that needs to be in the final 0.2.x release, but would love some additional input about whether or not anything else should be included. Currently I'm targeting May 22nd for the release date.

I also added issues to the milestone for my stretch goal of generating rpm and deb packages as release artifacts.  

Here is a link to the release milestone. Please provide any and all feedback.

Chris


Re: CQL for OLAP issue with Syclla as backed both Local and Yarn Mode

Chris Hupman <chris...@...>
 

Hello,

This channel is really just to discuss matters relevant to the development of JanusGraph. We mainly discuss things like proposals for new features, release planning, and general administration of the codebase. You'll have much better luck posting this in janusgraph-users.

Cheers,

Chris


On Thursday, May 2, 2019 at 4:37:00 AM UTC-7, rak...@... wrote:
Hi All,

I am unable to run any analytics (OLAP) on JanusGraph with Syclla as backend. 
I tried both in Local and Yarn mode on AWS EMR cluster
  • In Yarn mode, it Throws an exception 10:07:58 ERROR org.apache.spark.SparkContext  - Error initializing SparkContext.
  • In Local mode, It runs perfectly around 500 tasks and gives an empty output (This I tried with SparkGraphComputer it gives the result)
I build the distribution archive from here (from branch Issue_985_spark_via_cql)

Following are the properties given in conf/hadoop-graph/read-cql.properties:

# Copyright 2019 JanusGraph Authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

#
# Hadoop Graph Configuration
#
gremlin.graph=org.apache.tinkerpop.gremlin.hadoop.structure.HadoopGraph
gremlin.hadoop.graphReader=org.janusgraph.hadoop.formats.cql.CqlInputFormat
gremlin.hadoop.graphWriter=org.apache.tinkerpop.gremlin.hadoop.structure.io.gryo.GryoOutputFormat
gremlin.hadoop.defaultGraphComputer=org.apache.tinkerpop.gremlin.spark.process.computer.SparkGraphComputer

gremlin.hadoop.jarsInDistributedCache=true
gremlin.hadoop.inputLocation=none
gremlin.hadoop.outputLocation=output
gremlin.spark.persistContext=true
#
# JanusGraph Cassandra InputFormat configuration
#
janusgraphmr.ioformat.conf.storage.backend=cql
janusgraphmr.ioformat.conf.storage.hostname=X.0.X.1
janusgraphmr.ioformat.conf.storage.port=9042
janusgraphmr.ioformat.conf.storage.cassandra.keyspace=graph1
storage.cassandra.keyspace=graph1

#
# Apache Cassandra InputFormat configuration
#
cassandra.input.partitioner.class=org.apache.cassandra.dht.Murmur3Partitioner

#
# SparkGraphComputer Configuration
#
#spark.master=spark://X.X.X.X:7077
spark.master=yarn
spark.submit.deployMode=client
spark.yarn.jars=/usr/lib/spark/jars/
spark.serializer=org.apache.spark.serializer.KryoSerializer
spark.kryo.registrator=org.janusgraph.hadoop.serialize.JanusGraphKryoRegistrator




Full stack error while running in yarn mode:

ava.lang.IllegalStateException: org.apache.spark.SparkException: Unable to load YARN support
at org.apache.tinkerpop.gremlin.process.computer.traversal.step.map.VertexProgramStep.processNextStart(VertexProgramStep.java:88)
at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:143)
at org.apache.tinkerpop.gremlin.process.traversal.step.util.ExpandableStepIterator.next(ExpandableStepIterator.java:50)
at org.apache.tinkerpop.gremlin.process.computer.traversal.step.map.ComputerResultStep.processNextStart(ComputerResultStep.java:68)
at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:143)
at org.apache.tinkerpop.gremlin.process.traversal.util.DefaultTraversal.hasNext(DefaultTraversal.java:192)
at org.codehaus.groovy.vmplugin.v7.IndyInterface.selectMethod(IndyInterface.java:236)
at org.apache.tinkerpop.gremlin.console.Console$_closure3.doCall(Console.groovy:214)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:98)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:264)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1034)
at org.codehaus.groovy.tools.shell.Groovysh.setLastResult(Groovysh.groovy:460)
at org.codehaus.groovy.vmplugin.v7.IndyInterface.selectMethod(IndyInterface.java:236)
at org.codehaus.groovy.tools.shell.Groovysh.execute(Groovysh.groovy:196)
at org.apache.tinkerpop.gremlin.console.GremlinGroovysh.super$3$execute(GremlinGroovysh.groovy)
at sun.reflect.GeneratedMethodAccessor14.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:98)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1225)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuperN(ScriptBytecodeAdapter.java:145)
at org.apache.tinkerpop.gremlin.console.GremlinGroovysh.execute(GremlinGroovysh.groovy:72)
at org.codehaus.groovy.tools.shell.Shell.leftShift(Shell.groovy:122)
at org.codehaus.groovy.tools.shell.InteractiveShellRunner.super$2$work(InteractiveShellRunner.groovy)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:98)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1225)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuperN(ScriptBytecodeAdapter.java:145)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuper0(ScriptBytecodeAdapter.java:165)
at org.codehaus.groovy.tools.shell.ShellRunner.run(ShellRunner.groovy:59)
at org.codehaus.groovy.tools.shell.InteractiveShellRunner.super$2$run(InteractiveShellRunner.groovy)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:98)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1225)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuperN(ScriptBytecodeAdapter.java:145)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuper0(ScriptBytecodeAdapter.java:165)
at org.codehaus.groovy.tools.shell.InteractiveShellRunner.run(InteractiveShellRunner.groovy:89)
at org.codehaus.groovy.vmplugin.v7.IndyInterface.selectMethod(IndyInterface.java:236)
at org.apache.tinkerpop.gremlin.console.Console.<init>(Console.groovy:146)
at org.codehaus.groovy.vmplugin.v7.IndyInterface.selectMethod(IndyInterface.java:236)
at org.apache.tinkerpop.gremlin.console.Console.main(Console.groovy:453)
Caused by: java.util.concurrent.ExecutionException: org.apache.spark.SparkException: Unable to load YARN support
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at org.apache.tinkerpop.gremlin.process.computer.traversal.step.map.VertexProgramStep.processNextStart(VertexProgramStep.java:68)
... 56 more
Caused by: org.apache.spark.SparkException: Unable to load YARN support
at org.apache.spark.deploy.SparkHadoopUtil$.liftedTree1$1(SparkHadoopUtil.scala:405)
at org.apache.spark.deploy.SparkHadoopUtil$.yarn$lzycompute(SparkHadoopUtil.scala:400)
at org.apache.spark.deploy.SparkHadoopUtil$.yarn(SparkHadoopUtil.scala:400)
at org.apache.spark.deploy.SparkHadoopUtil$.get(SparkHadoopUtil.scala:425)
at org.apache.spark.util.Utils$.getSparkOrYarnConfig(Utils.scala:2387)
at org.apache.spark.storage.BlockManager.<init>(BlockManager.scala:156)
at org.apache.spark.SparkEnv$.create(SparkEnv.scala:351)
at org.apache.spark.SparkEnv$.createDriverEnv(SparkEnv.scala:175)
at org.apache.spark.SparkContext.createSparkEnv(SparkContext.scala:257)
at org.apache.spark.SparkContext.<init>(SparkContext.scala:432)
at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2509)
at org.apache.spark.SparkContext.getOrCreate(SparkContext.scala)
at org.apache.tinkerpop.gremlin.spark.structure.Spark.create(Spark.java:52)
at org.apache.tinkerpop.gremlin.spark.structure.Spark.create(Spark.java:60)
at org.apache.tinkerpop.gremlin.spark.process.computer.SparkGraphComputer.lambda$submitWithExecutor$1(SparkGraphComputer.java:233)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: org.apache.spark.deploy.yarn.YarnSparkHadoopUtil
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at org.apache.spark.util.Utils$.classForName(Utils.scala:230)
at org.apache.spark.deploy.SparkHadoopUtil$.liftedTree1$1(SparkHadoopUtil.scala:401)
... 18 more


Is there anything required as classpath or required jars? also whats the problem with local mode? 
Do we have any alternative for this purpose (analytics on Janusgraph using spark), Currently I am running connected component using graphframes.

you help is appreciated, thanks in advance :)  




CQL for OLAP issue with Syclla as backed both Local and Yarn Mode

rakesh...@...
 

Hi All,

I am unable to run any analytics (OLAP) on JanusGraph with Syclla as backend. 
I tried both in Local and Yarn mode on AWS EMR cluster
  • In Yarn mode, it Throws an exception 10:07:58 ERROR org.apache.spark.SparkContext  - Error initializing SparkContext.
  • In Local mode, It runs perfectly around 500 tasks and gives an empty output (This I tried with SparkGraphComputer it gives the result)
I build the distribution archive from here (from branch Issue_985_spark_via_cql)

Following are the properties given in conf/hadoop-graph/read-cql.properties:

# Copyright 2019 JanusGraph Authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#      http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

#
# Hadoop Graph Configuration
#
gremlin.graph=org.apache.tinkerpop.gremlin.hadoop.structure.HadoopGraph
gremlin.hadoop.graphReader=org.janusgraph.hadoop.formats.cql.CqlInputFormat
gremlin.hadoop.graphWriter=org.apache.tinkerpop.gremlin.hadoop.structure.io.gryo.GryoOutputFormat
gremlin.hadoop.defaultGraphComputer=org.apache.tinkerpop.gremlin.spark.process.computer.SparkGraphComputer

gremlin.hadoop.jarsInDistributedCache=true
gremlin.hadoop.inputLocation=none
gremlin.hadoop.outputLocation=output
gremlin.spark.persistContext=true
#
# JanusGraph Cassandra InputFormat configuration
#
janusgraphmr.ioformat.conf.storage.backend=cql
janusgraphmr.ioformat.conf.storage.hostname=X.0.X.1
janusgraphmr.ioformat.conf.storage.port=9042
janusgraphmr.ioformat.conf.storage.cassandra.keyspace=graph1
storage.cassandra.keyspace=graph1

#
# Apache Cassandra InputFormat configuration
#
cassandra.input.partitioner.class=org.apache.cassandra.dht.Murmur3Partitioner

#
# SparkGraphComputer Configuration
#
#spark.master=spark://X.X.X.X:7077
spark.master=yarn
spark.submit.deployMode=client
spark.yarn.jars=/usr/lib/spark/jars/
spark.serializer=org.apache.spark.serializer.KryoSerializer
spark.kryo.registrator=org.janusgraph.hadoop.serialize.JanusGraphKryoRegistrator




Full stack error while running in yarn mode:

ava.lang.IllegalStateException: org.apache.spark.SparkException: Unable to load YARN support
at org.apache.tinkerpop.gremlin.process.computer.traversal.step.map.VertexProgramStep.processNextStart(VertexProgramStep.java:88)
at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:143)
at org.apache.tinkerpop.gremlin.process.traversal.step.util.ExpandableStepIterator.next(ExpandableStepIterator.java:50)
at org.apache.tinkerpop.gremlin.process.computer.traversal.step.map.ComputerResultStep.processNextStart(ComputerResultStep.java:68)
at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:143)
at org.apache.tinkerpop.gremlin.process.traversal.util.DefaultTraversal.hasNext(DefaultTraversal.java:192)
at org.codehaus.groovy.vmplugin.v7.IndyInterface.selectMethod(IndyInterface.java:236)
at org.apache.tinkerpop.gremlin.console.Console$_closure3.doCall(Console.groovy:214)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:98)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:264)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1034)
at org.codehaus.groovy.tools.shell.Groovysh.setLastResult(Groovysh.groovy:460)
at org.codehaus.groovy.vmplugin.v7.IndyInterface.selectMethod(IndyInterface.java:236)
at org.codehaus.groovy.tools.shell.Groovysh.execute(Groovysh.groovy:196)
at org.apache.tinkerpop.gremlin.console.GremlinGroovysh.super$3$execute(GremlinGroovysh.groovy)
at sun.reflect.GeneratedMethodAccessor14.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:98)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1225)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuperN(ScriptBytecodeAdapter.java:145)
at org.apache.tinkerpop.gremlin.console.GremlinGroovysh.execute(GremlinGroovysh.groovy:72)
at org.codehaus.groovy.tools.shell.Shell.leftShift(Shell.groovy:122)
at org.codehaus.groovy.tools.shell.ShellRunner.work(ShellRunner.groovy:95)
at org.codehaus.groovy.tools.shell.InteractiveShellRunner.super$2$work(InteractiveShellRunner.groovy)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:98)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1225)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuperN(ScriptBytecodeAdapter.java:145)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuper0(ScriptBytecodeAdapter.java:165)
at org.codehaus.groovy.tools.shell.InteractiveShellRunner.work(InteractiveShellRunner.groovy:130)
at org.codehaus.groovy.tools.shell.ShellRunner.run(ShellRunner.groovy:59)
at org.codehaus.groovy.tools.shell.InteractiveShellRunner.super$2$run(InteractiveShellRunner.groovy)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:98)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1225)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuperN(ScriptBytecodeAdapter.java:145)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuper0(ScriptBytecodeAdapter.java:165)
at org.codehaus.groovy.tools.shell.InteractiveShellRunner.run(InteractiveShellRunner.groovy:89)
at org.codehaus.groovy.vmplugin.v7.IndyInterface.selectMethod(IndyInterface.java:236)
at org.apache.tinkerpop.gremlin.console.Console.<init>(Console.groovy:146)
at org.codehaus.groovy.vmplugin.v7.IndyInterface.selectMethod(IndyInterface.java:236)
at org.apache.tinkerpop.gremlin.console.Console.main(Console.groovy:453)
Caused by: java.util.concurrent.ExecutionException: org.apache.spark.SparkException: Unable to load YARN support
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at org.apache.tinkerpop.gremlin.process.computer.traversal.step.map.VertexProgramStep.processNextStart(VertexProgramStep.java:68)
... 56 more
Caused by: org.apache.spark.SparkException: Unable to load YARN support
at org.apache.spark.deploy.SparkHadoopUtil$.liftedTree1$1(SparkHadoopUtil.scala:405)
at org.apache.spark.deploy.SparkHadoopUtil$.yarn$lzycompute(SparkHadoopUtil.scala:400)
at org.apache.spark.deploy.SparkHadoopUtil$.yarn(SparkHadoopUtil.scala:400)
at org.apache.spark.deploy.SparkHadoopUtil$.get(SparkHadoopUtil.scala:425)
at org.apache.spark.util.Utils$.getSparkOrYarnConfig(Utils.scala:2387)
at org.apache.spark.storage.BlockManager.<init>(BlockManager.scala:156)
at org.apache.spark.SparkEnv$.create(SparkEnv.scala:351)
at org.apache.spark.SparkEnv$.createDriverEnv(SparkEnv.scala:175)
at org.apache.spark.SparkContext.createSparkEnv(SparkContext.scala:257)
at org.apache.spark.SparkContext.<init>(SparkContext.scala:432)
at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2509)
at org.apache.spark.SparkContext.getOrCreate(SparkContext.scala)
at org.apache.tinkerpop.gremlin.spark.structure.Spark.create(Spark.java:52)
at org.apache.tinkerpop.gremlin.spark.structure.Spark.create(Spark.java:60)
at org.apache.tinkerpop.gremlin.spark.process.computer.SparkGraphComputer.lambda$submitWithExecutor$1(SparkGraphComputer.java:233)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: org.apache.spark.deploy.yarn.YarnSparkHadoopUtil
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at org.apache.spark.util.Utils$.classForName(Utils.scala:230)
at org.apache.spark.deploy.SparkHadoopUtil$.liftedTree1$1(SparkHadoopUtil.scala:401)
... 18 more


Is there anything required as classpath or required jars? also whats the problem with local mode? 
Do we have any alternative for this purpose (analytics on Janusgraph using spark), Currently I am running connected component using graphframes.

you help is appreciated, thanks in advance :)  




JanusGraph doesn't remove threadlocal tx after a transaction commit?

huangx...@...
 

By Java VisualVm, I found some StandardJanusGraphTx  "isOpen" field is false while they still live in memory.

I read the source code. A graph has a threadLocal var named txs and the reference chain is graph -> txs -> StandardJanusGraphTx.

When a transaction commit, JanusGraph doesn't call txs.remove() .In memory, the worst result:  txNum = threadNum × graphNum.

We usually remove threadlocal var in order to avoid OOM, but JanusGraph doesn't do that ,why?



Thanks.

501 - 520 of 1593