Monday, May 21, 2012

Unit Testing - Is 100% Code Coverage Useful?

I recently had a short twitter debate with @unclebobmartin of clean code fame about 100% code coverage. He is of course much smarter than I as well, is vastly more experienced and enjoys a well-earned reputation in the software community that I could only dream of. However, I felt I had a point. I disagree with the 100% dogma as far as Unit Tests go. In fact, I disagree with it strongly. I argue it is more important to unit test some of your code and ship it quickly and iterate - certainly in web development. That does not mean your application is not thoroughly tested by the QA Engineers through automated and manual testing on several environments before hitting production - we are talking about Unit Testing here.

 In my experience it can be quite difficult to achieve 100% code coverage in the real world (remembering what a unit test actually is and keeping in mind when it stops becoming a unit test and starts being a system test - database, file system, network). Think of all the plumbing and infrastructure code involved, think of all the getters and setters, think of all the external interaction with databases etc. Developers, if they are tasked with 100% code coverage, will probably find a way to hit the mark. Will those tests be useful? Does 100% actually guarantee anything? Should the tests not focus on key behavior and key user scenarios at the function level? Unit tests should cover some of the code and a good company will have great test engineers who write great automated system tests and do great manual testing and a great company will have constant feedback loop with product owners demonstrating the software as it progresses.

Of all the code out there running some of the biggest sites and most popular applications in the world, does anybody really think they have 100% unit test code coverage? Of course there are some situations when perhaps 100% code coverage may make perfect sense such as extremely critical systems in finance or medicine. And of course it also can depend on the culture and size of the company - startups are going to focus less on code coverage and more on just shipping code and features whereas enterprises are going to focus on quality more.

But I question the engineering fixation on 100% and it's actual value and ROI. It is NOT something I would blindly follow or suggest for every situation.

P.S. I like Kent Beck's answer on Quora.