[DISCUSS] Default test suite is too dang slow #1136

Buck Evan <buk...@...>

I'm cross-posting this from my github issue, as advised by pluradj.


There's two separate documents that tell newcomers how to build and test the source code:
  • https://docs.janusgraph.org/latest/building.html
  • https://github.com/JanusGraph/janusgraph/blob/master/BUILDING.md
They both recommend `mvn clean install` for people that want to see the test suite pass (that's me). The first doc (only) mentions `-P comprehensive` for running the comprehensive set of tests, but warms that it will take "a significant amount of of time (> 1 hour)."

This strongly implies that by default the tests should take less than an hour, or at least that was true at time of writing. I'm only able to run a fraction of the (non-comprehensive) test suite in three hours.

As a potential contributor, I always want to see the test suite passing before I make any change, so I can make a change, run the tests, and be sure (or at least reasonably confident) that any errors are due to me. The above behavior presents a large barrier for me though. For more context, see #1086 and #1134.

My naive suggested implementation:
  1. Create a new `-P slow` that moves the slow tests out of the default set.
  2. Edit pom.xml to enable test parallelism wherever possible.
If you (plural) will explain to me what you actually want to do about this, I'll work on contributing it.

Join janusgraph-dev@lists.lfaidata.foundation to automatically receive all group messages.