1. Janusgraph-Dev
  2. Topics

[DISCUSS] Default test suite is too dang slow #1136

Jason Plurad <plu...@...>
 

Buck, this is a community project. It is open for new ideas and contributions.

I haven't worked on the build optimizations previously, so I don't have a strong opinion on what needs to be done here, but your ideas sound reasonable to me. I think it would be fine for you to move forward and see if those ideas can fly. Open up a pull request when you have something, then we can review and iterate.


On Monday, July 2, 2018 at 2:06:42 PM UTC-4, Buck Evan wrote:
I'm cross-posting this from my github issue, as advised by pluradj.

---

There's two separate documents that tell newcomers how to build and test the source code:
They both recommend `mvn clean install` for people that want to see the test suite pass (that's me). The first doc (only) mentions `-P comprehensive` for running the comprehensive set of tests, but warms that it will take "a significant amount of of time (> 1 hour)."

This strongly implies that by default the tests should take less than an hour, or at least that was true at time of writing. I'm only able to run a fraction of the (non-comprehensive) test suite in three hours.

As a potential contributor, I always want to see the test suite passing before I make any change, so I can make a change, run the tests, and be sure (or at least reasonably confident) that any errors are due to me. The above behavior presents a large barrier for me though. For more context, see #1086 and #1134.

My naive suggested implementation:
  1. Create a new `-P slow` that moves the slow tests out of the default set.
  2. Edit pom.xml to enable test parallelism wherever possible.
If you (plural) will explain to me what you actually want to do about this, I'll work on contributing it.
Buck Evan <buk...@...>
 

I'm cross-posting this from my github issue, as advised by pluradj.

---

There's two separate documents that tell newcomers how to build and test the source code:
  • https://docs.janusgraph.org/latest/building.html
  • https://github.com/JanusGraph/janusgraph/blob/master/BUILDING.md
They both recommend `mvn clean install` for people that want to see the test suite pass (that's me). The first doc (only) mentions `-P comprehensive` for running the comprehensive set of tests, but warms that it will take "a significant amount of of time (> 1 hour)."

This strongly implies that by default the tests should take less than an hour, or at least that was true at time of writing. I'm only able to run a fraction of the (non-comprehensive) test suite in three hours.

As a potential contributor, I always want to see the test suite passing before I make any change, so I can make a change, run the tests, and be sure (or at least reasonably confident) that any errors are due to me. The above behavior presents a large barrier for me though. For more context, see #1086 and #1134.

My naive suggested implementation:
  1. Create a new `-P slow` that moves the slow tests out of the default set.
  2. Edit pom.xml to enable test parallelism wherever possible.
If you (plural) will explain to me what you actually want to do about this, I'll work on contributing it.

1 - 2 of 2