Antipattern: test internals

Test Driven Development (TDD) is one of the most fundamental practices of agile development. It is claimed to offer numerous benefits, such as:

• Better for incremental development
• Results in better coverage of the tests
• Tests document the requirements
• Leads to improved software designs
• Developers are more domain centric or customer centric

However there is little empirical evidence of these benefits - particularly the claim that it leads to improved software designs. Using TDD to come up with a good architecture doesn't make sense. You can't expect incremental changes to let you transition from a bad architecture to a good one.

A significant issue is that there is a tendency to test the internals of a module. This is because TDD tends to emphasise the unit testing of every method of every class, or every free function.

This is very detrimental for the following reasons:

• The are lots of tests to write - often there is more test code than real production code
• Tests of internals of modules tends to be obscure because they depend on obscure implementation details, they are not testing the well understood, carefully designed, carefully documented public APIs
• It's harders to refactor code - because the tests break
• It puts developers in a mindset of exposing internal details, so they are testable. Programmers are supposed to develop modules which hide internal implementation details - this is one of the most fundamental programming principles, but under TDD they tend not to do that.

It's better to test at the level of the public API of whole modules. This reduces the amount of test code that needs to be written, for the same code coverage. Tests should relate back to the domain requirements, not the artefacts of the particular implementation.

Heavy mocking of tests is a good sign that the design approach is poor. Good tests don't need to mock things.

It's sometimes claimed that a lot of effort is put into writing automated tests which are intended to be studied by the client - i.e. User Acceptance Testing (UAT) using tools such as FitNesse, but the client isn't interested in studying hundreds of verbose and intricate automated tests. They are more interested demonstrations showing the software being used by end users.

It's generally better to emphasise tests which are as easy to write as possible, and that usually means writing them in the same language as the code being tested.

Design by contract

There is a way to heavily test the internals of modules, and it doesn't involve writing tests! It is to make heavy use of assertions which:

• Validate pre-conditions of functions and methods
• Validate post-conditions of functions and methods
• Validate class invariants

Hopefully the language has a concept of debug versus release build variants, so the assertion checks are only paid for in the debug variant. Better still, a language supports a compiler which is capable of proving that the assertions cannot trip, eliminating the need to check them at run time.

Unlike separate, dedicated tests, these assertions appear in the production code to which they relate. They are a great documentation aid for the programmer - they document the design assumptions, and help the programmer reason about the correctness of the code.

When the entire module is tested, there will be comprehensive testing of the internals via these assertion checks.