1. Janusgraph-Dev
  2. Topics

Make test more resilient

faro...@...
 

Hi

It would be cool to reduce the number test retries. Currently, basically each test needed to fail three times to be called broken. I would like to see removing these technique.

I have some idea to reach this goal:
  • Reduce the number of dependencies to run all tests, for example, use Testcontainers https://www.testcontainers.org/ (You only need java and docker). During the development of the PR: https://github.com/JanusGraph/janusgraph/pull/923, I had problems to run all tests locally (I developing on windows).
  • Reduce size of a single test, I needed to fix tests with more than 800 lines and they are called basic test, which confused me a bit.
  • Collect a list of unstable and only run these test multiple time. This could be a first step.

A benefit is also that test run time will be reduce by reducing the number of multiple executed tests.


Any ideas?


Dear Jan Jansen
Jason Plurad <plu...@...>
 

This sounds like an interesting approach. I'd be in favor of anything that makes testing easier for contributors. What else specifically do you need to start working on your approach?


On Thursday, March 29, 2018 at 10:41:44 AM UTC-4, Jan Jansen wrote:
Hi

It would be cool to reduce the number test retries. Currently, basically each test needed to fail three times to be called broken. I would like to see removing these technique.

I have some idea to reach this goal:
  • Reduce the number of dependencies to run all tests, for example, use Testcontainers https://www.testcontainers.org/ (You only need java and docker). During the development of the PR: https://github.com/JanusGraph/janusgraph/pull/923, I had problems to run all tests locally (I developing on windows).
  • Reduce size of a single test, I needed to fix tests with more than 800 lines and they are called basic test, which confused me a bit.
  • Collect a list of unstable and only run these test multiple time. This could be a first step.

A benefit is also that test run time will be reduce by reducing the number of multiple executed tests.


Any ideas?


Dear Jan Jansen
Jan Jansen <faro...@...>
 

I'm contributing to different rust project and they are using https://bors.tech/ which allows to merge PRs with more consistent. If you bors to merge a PR, it will merge the commits into a staging branch in top of the master wait until all build job are passed afterwards the PR is merged.

Am Donnerstag, 29. März 2018 16:41:44 UTC+2 schrieb Jan Jansen:

Hi

It would be cool to reduce the number test retries. Currently, basically each test needed to fail three times to be called broken. I would like to see removing these technique.

I have some idea to reach this goal:
  • Reduce the number of dependencies to run all tests, for example, use Testcontainers https://www.testcontainers.org/ (You only need java and docker). During the development of the PR: https://github.com/JanusGraph/janusgraph/pull/923, I had problems to run all tests locally (I developing on windows).
  • Reduce size of a single test, I needed to fix tests with more than 800 lines and they are called basic test, which confused me a bit.
  • Collect a list of unstable and only run these test multiple time. This could be a first step.

A benefit is also that test run time will be reduce by reducing the number of multiple executed tests.


Any ideas?


Dear Jan Jansen

1 - 3 of 3