Creating Trustworthy Tests
If tests are designed to verify that the production code works correctly, how we can ensure that the code of tests works correctly?
That for me has been a big conundrum during a long time. Why we trust in the code of tests, but we do not trust in the production code? Tests are code, so they can have bugs and may fail in several ways. The test can say that the code is wrong, although it works, but it can say that the code is right, although it is not. If we should trust a test, it should be bug free; the test should ensure that it fails when the code fails, and passes when the code passes.
The problem is that the test code fails, and quite often. During my career, I have seen plenty of tests with bugs, and of all kinds. One typical is the test built with Selenium that fails randomly because of a timeout. That is unacceptable, and I already heard the same excuse a hundred times. You should fix them, that kind of error cannot detect a race condition. I have seen plenty of race-conditions hidden under those faulty tests. But there are plenty of other kinds of tests as well. The most repeated bug in tests is the test that does not fail when you break the code. It usually happens when the test is created after testing the code manually; because the code works, and the test passes, you…
