Chapter 2. PHPUnit's Goals

So far, we only have two tests for the array built-in and the count() function. When we start to test the numerous array_*() functions PHP offers, we will need to write a test for each of them. We could write the infrastructure for all these tests from scratch. However, it is much better to write a testing infrastructure once and then write only the unique parts of each test. PHPUnit is such an infrastructure.

A framework such as PHPUnit has to resolve a set of constraints, some of which seem always to conflict with each other. Simultaneously, tests should be:

Easy to learn to write.: If it's hard to learn how to write tests, developers will not learn to write them.
Easy to write.: If tests are not easy to write, developers will not write them.
Easy to read.: Test code should contain no extraneous overhead so that the test itself does not get lost in noise that surrounds it.
Easy to execute.: The tests should run at the touch of a button and present their results in a clear and unambiguous format.
Quick to execute.: Tests should run fast so so they can be run hundreds or thousands of times a day.
Isolated.: The tests should not affect each other. If the order in which the tests are run changes, the results of the tests should not change.
Composable.: We should be able to run any number or combination of tests together. This is a corollary of isolation.

There are two main clashes between these constraints:

Easy to learn to write versus easy to write.: Tests do not generally require all the flexibility of a programming language. Many testing tools provide their own scripting language that only includes the minimum necessary features for writing tests. The resulting tests are easy to read and write because they have no noise to distract you from the content of the tests. However, learning yet another programming language and set of programming tools is inconvenient and clutters the mind.
Isolated versus quick to execute.: If you want the results of one test to have no effect on the results of another test, each test should create the full state of the world before it begins to execute and return the world to its original state when it finishes. However, setting up the world can take a long time: for example connecting to a database and initializing it to a known state using realistic data.

PHPUnit attempts to resolve these conflicts by using PHP as the testing language. Sometimes the full power of PHP is overkill for writing little straight-line tests, but by using PHP we leverage all the experience and tools programmers already have in place. Since we are trying to convince reluctant testers, lowering the barrier to writing those initial tests is particularly important.

PHPUnit errs on the side of isolation over quick execution. Isolated tests are valuable because they provide high-quality feedback. You do not get a report with a bunch of test failures, which were really caused because one test at the beginning of the suite failed and left the world messed up for the rest of the tests. This orientation towards isolated tests encourages designs with a large number of simple objects. Each object can be tested quickly in isolation. The result is better designs and faster tests.

PHPUnit assumes that most tests succeed and it is not worth reporting the details of successful tests. When a test fails, that fact is worth noting and reporting. The vast majority of tests should succeed and are not worth commenting on except to count the number of tests that run. This is an assumption that is really built into the reporting classes, and not into the core of PHPUnit. When the results of a test run are reported, you see how many tests were executed, but you only see details for those that failed.

Tests are expected to be fine-grained, testing one aspect of one object. Hence, the first time a test fails, execution of the test halts, and PHPUnit reports the failure. It is an art to test by running in many small tests. Fine-grained tests improve the overall design of the system.

When you test an object with PHPUnit, you do so only through the object's public interface. Testing based only on publicly visible behaviour encourages you to confront and solve difficult design problems earlier, before the results of poor design can infect large parts of the system.

PHPUnit Manual

Chapter 2. PHPUnit's Goals