Perl’s prove is a great tool for running TAP-based test suites. I use it for a large project with about 20,000 tests. Our test suite runs multiple times per day. At that scale and frequency, the run time is a concern.
To address that,
-j option, allowing you to run tests in parallel. Unfortunately, in our large test suite not all tests are “parallel ready”. I needed a way to run most tests in parallel, but mark a few to be run in sequence. Ideally, I’d like these tests to be part of the same run as the parallel tests.
After much digging around, I found that
prove has had support for adding exceptions to parallel tests runs since in 2008. The feature has just never been documented in
prove, making it difficult to discover. Here’s an example which works with the currently released version of
# All tests are allowed to run in parallel, except those starting with "p" --rules='seq=t/p*.t' --rules='par=**'
For more details and documentation, you can see my related pull request. You can leave a comment there to advocate for or against merging the the patch, or you can download the raw patch directly to apply to your own copy of the Test-Harness-3.25 distribution.
*UPDATE 9/14/2012: * It turns out it doesn’t work quite as I thought. I tried tweaking some of the internals of
prove further, but hit a bug. See the details in my follow-up to the Perl-QA list. I changed approached and started working on making my exceptional tests parallel-safe. Perhaps I can cover some of the techniques I used for that in a future post.