I was recently invited to be a guest speaker at the 2014 German Perl Workshop and I had a fantastic time. Not only was the workshop well-organized and with great food, but it was also my favorite Perl Workshop to date. Amongst other things, one of the organizers offered me his spare bedroom instead of a hotel room and it made for a very pleasant change (though the hotels are great, too, as there are usually groups of people I know who hang out at the hotels).
This time I gave a talk on a subject I have long wanted to speak about: the lies we tell about testing. Below is the video of my talk.
The video has a bit of a break partway through due to a medical emergency which occurred (everything turned out OK), but it certainly made it one of the more difficult situations to recover from in a talk. I’ve been speaking at conferences for years and nothing like this has happened before.
Regarding the video, you may notice a slide on all pairs testing where I refer to a hypothetical 10 argument subroutine as having 10 billion possible combinations of arguments. I didn’t explain that well enough because we’re also talking about taking multiple combinations of one argument, two arguments, and so on, up to ten arguments, and also testing each of the possible interesting values for each argument. I should have been more clear.
There’s much more I could have said in the talk, but as usual, there’s only so much time available. However, it boils down to this:
- Don’t trust snake-oil salesmen who use words like “always” or “never”
- You want to break your code with tests
- Don’t blindly trust your tools
- Code coverage of unit tests can be very misleading
- Writing more tests does not guarantee better software
- Claiming that tests are documentation is often a stretch
- Testing needs to be focused more on business needs
- TDD enthusiasts need to calm down
- The Perl community isn’t stealing enough testing ideas from others
- 100% code coverage is useless if customer’s don’t use your software (A/B testing, anyone?)