The Only Utility For Code Coverage

Several companies and governments set a minim for the code coverage, but does it make sense?

David Rodenas PhD

--

An uncovered plot of grass
Photo by Markus Spiske on Unsplash

A few years ago, a colleague was working in a company developing a product. One day, their organization saw the possibility of licensing it to a government agency. The fit was almost perfect, except for one metric: the code coverage. I do not remember exactly the percentage, let's say 80%, but that government agency required that minimum of code coverage. The problem was that the software has a near to 0% code coverage.

The remarkable thing is that the product was working, working well, with several actual customers. Although it had almost no automatic tests, the quality of the code and the product was outstanding. But that was no sufficient for that government agency, the agency did not trust any proof except a number generated by a suite of tests. So, in order to proceed with the plan, they decided to increase the code coverage.

How do you increase the code coverage of an existing program? Just add tests that execute your code.

He explained how the whole process went. They started adding tests for the functionalities that they knew that were present. Day by day, the code coverage was slowly growing. Several weeks later, they realized that it was too slow, so they changed tactics. They started to read the coverage reports, looking for big uncovered chunks of code, and then create the tests to cover those. That accelerated the process, but it was still slow. Every week that passed, the quality of the tests decayed, loosing intention and capability of real verification. The last weeks the tests were almost useless, but they served the objective; tests were executing the code, and that increased the coverage. It took months, but they achieved the demanded ratio.

He told that it was, by far, the worst experience that he had in the profession.

I remember that when he started to explain his experience I though: Wow, a government agency that care about testing, that is great. But minutes later, as he was explaining all their experience to achieve that demanded ratio, I got…

--

--

David Rodenas PhD

Passionate software engineer & storyteller. Sharing knowledge to advance our skills. Join me on a journey of discovery in the world of software engineering.