"Continuous Testing" - Never Stop Testing

As I type this I am actively monitoring four running automated test projects my project team is leveraging (I should add there are more than four automated test projects for the project team) and the thought hit me in the face like a brick. We have so much resting running simultaneously that I was feeling 'stressed' keeping track of all of it.  Additionally, how this would stress our execution infrastructure. These are very a good problems to be stressed about -- much like "too many good pitchers" in baseball.   It indicates that we had reached an extremely important attitude shift on the project team.

 

Does the number of tests matter?

On an internal team Wiki I had written that the project team had over 10,000 automated test cases. Subsequent to that I have heard comments reacting to this fact with a 'it doesn't matter how many tests you have!". I agree with the speaker - far more important is that you value the tests that you do have.  How do you value them?

• Do you you understand the feedback they give?
• Do you react to that feedback?
• Do your team members understand and react?
• Do you find it so valuable that you wish you had more?

 If you answered "yes" to more than one of these, then you have the 'right' number.

 

If not, can you see yourself getting there?

If you have 0 tests, that number does matter and you have spoken volumes about you attitude about testing. However, if you can see the value of Continuous Testing. but can't see yourself getting "there", What do you do?  It's a lot like writer's block. The toughest part of writing is staring at the blank piece of paper. To get passed it, you just have to start writing. The same goes with testing, what is the first test you can write that you will value? To capture the essence of this idea, I tweeted the following mantra, "Test as you can, not as you can't"

 

Does automation matter?

In our resource starved environment (money, people, machines), I don't see how a software product can be successfully tested in any other fashion. See my next post -- "Automate Everything"