PHPUnit Manual

setUp() and tearDown() are nicely symmetrical in theory but not in practice. In practice, you only need to implement tearDown() if you have allocated external resources like files or sockets in setUp(). If your setUp() just creates plain PHP objects, you can generally ignore tearDown(). However, if you create many objects in your setUp(), you might want to unset() the variables pointing to those objects in your tearDown() so they can be garbage collected. The garbage collection of test case objects is not predictable.

What happens when you have two tests with slightly different setups? There are two possibilities:

There are few good reasons to share fixtures between tests, but in most cases the need to share a fixture between tests stems from an unresolved design problem.

A good example of a fixture that makes sense to share across several tests is a database connection: you log into the database once and reuse the database connection instead of creating a new connection for each test. This makes your tests run faster.

Example 6.2 uses the setUp() and tearDown() template methods of the PHPUnit_Framework_TestSuite class (see the section called “Suite-Level Setup”) to connect to the database before the test suite's first test and to disconnect from the database after the last test of the test suite, respectively. The $sharedFixture attribute of an PHPUnit_Framework_TestSuite object is available in the object's aggregated PHPUnit_Framework_TestSuite and PHPUnit_Framework_TestCase objects.

<?php
require_once 'PHPUnit/Framework.php';
 
class DatabaseTestSuite extends PHPUnit_Framework_TestSuite
{
    protected function setUp()
    {
        $this->sharedFixture = new PDO(
          'mysql:host=wopr;dbname=test',
          'root',
          ''
        );
    }
 
    protected function tearDown()
    {
        $this->sharedFixture = NULL;
    }
}
?>

It cannot be emphasized enough that sharing fixtures between tests reduces the value of the tests. The underlying design problem is that objects are not loosely coupled. You will achieve better results solving the underlying design problem and then writing tests using stubs (see Chapter 10), than by creating dependencies between tests at runtime and ignoring the opportunity to improve your design.

The PHPUnit_Framework_TestSuite class offers two template methods, setUp() and tearDown(), that are called before the first test of the test suite and after the last test of the test suite, respectively.

<?php
require_once 'MyTest.php';
 
class MySuite extends PHPUnit_Framework_TestSuite
{
    public static function suite()
    {
        return new MySuite('MyTest');
    }
 
    protected function setUp()
    {
        print "\nMySuite::setUp()";
    }
 
    protected function tearDown()
    {
        print "\nMySuite::tearDown()";
    }
}
?>

The MyTest test case class that is added to the test suite MySuite in Example 7.3 has two test methods, testOne() and testTwo() as well as the setUp() and tearDown() methods. The output shows in which order these eight methods are called:

MySuite::setUp()
MyTest::setUp()
MyTest::testOne()
MyTest::tearDown()
MyTest::setUp()
MyTest::testTwo()
MyTest::tearDown()
MySuite::tearDown()

Variables stored in $this->sharedFixture by the setUp() method of the PHPUnit_Framework_TestSuite class are available as $this->sharedFixture in all the test that are aggregated by the test suite object (see the section called “Sharing Fixture”).

Please note that a TestSuite's setUp() and tearDown() methods will be called even if no test of the test suite is run because it is, for instance, filtered.

You can extend your test class from PHPUnit_Extensions_ExceptionTestCase to test whether an exception is thrown inside the tested code. Example 8.1 shows how to subclass PHPUnit_Extensions_ExceptionTestCase and use its setExpectedException() method to set the expected exception. If this expected exception is not thrown, the test will be counted as a failure.

<?php
require_once 'PHPUnit/Extensions/ExceptionTestCase.php';
 
class ExceptionTest extends PHPUnit_Extensions_ExceptionTestCase
{
    public function testException()
    {
        $this->setExpectedException('Exception');
    }
}
?>

phpunit ExceptionTest
PHPUnit 3.1.9 by Sebastian Bergmann.

F

Time: 0 seconds

There was 1 failure:

1) testException(ExceptionTest)
Expected exception Exception

FAILURES!
Tests: 1, Failures: 1.

Table 8.1 shows the methods provided by PHPUnit_Extensions_ExceptionTestCase.

Alternatively, you can use the approach shown in Example 8.2 to test exceptions.

<?php
require_once 'PHPUnit/Framework.php';
 
class ExceptionTest extends PHPUnit_Framework_TestCase {
    public function testException() {
        try {
            // ... Code that is expected to raise an Exception ...
        }
 
        catch (Exception $expected) {
            return;
        }
 
        $this->fail('An expected Exception has not been raised.');
    }
}
?>

If the code that is expected to raise an exception in Example 8.2 does not raise the expected exception, the subsequent call to fail() (see Table 20.3) will halt the test and signal a problem with the test. If the expected exception is raised, the catch block will be executed, and the test will end successfully.

Sometimes you want to assert that the execution of a method, for instance, generates an expected output (via echo or print, for example). The PHPUnit_Extensions_OutputTestCase class uses PHP's Output Buffering feature to provide the functionality that is necessary for this.

Example 8.3 shows how to subclass PHPUnit_Extensions_OutputTestCase and use its expectOutputString() method to set the expected output. If this expected output is not generated, the test will be counted as a failure.

<?php
require_once 'PHPUnit/Extensions/OutputTestCase.php';
 
class OutputTest extends PHPUnit_Extensions_OutputTestCase
{
    public function testExpectFooActualFoo()
    {
        $this->expectOutputString('foo');
        print 'foo';
    }
 
    public function testExpectBarActualBaz()
    {
        $this->expectOutputString('bar');
        print 'baz';
    }
}
?>

phpunit OutputTest
PHPUnit 3.1.9 by Sebastian Bergmann.

.F

Time: 0 seconds

There was 1 failure:

1) testExpectBarActualBaz(OutputTest)
Failed asserting that two strings are equal.
expected string <bar>
difference      <  x>
got string      <baz>

FAILURES!
Tests: 2, Failures: 1.

Table 8.2 shows the methods provided by PHPUnit_Extensions_OutputTestCase.

You can extend your test class from PHPUnit_Extensions_PerformanceTestCase to test whether the execution of a function or method call, for instance, exceeds a specified time limit.

Example 8.4 shows how to subclass PHPUnit_Extensions_PerformanceTestCase and use its setMaxRunningTime() method to set the maximum running time for the test. If the test is not executed within this time limit, it will be counted as a failure.

<?php
require_once 'PHPUnit/Extensions/PerformanceTestCase.php';
 
class PerformanceTest extends PHPUnit_Extensions_PerformanceTestCase
{
    public function testPerformance()
    {
        $this->setMaxRunningTime(2);
        sleep(1);
    }
}
?>

Table 8.3 shows the methods provided by PHPUnit_Extensions_PerformanceTestCase.

Chapter 17 deals with another extension, the PHPUnit_Extensions_SeleniumTestCase class.

When you are working on a new test-case class, you might want to begin by writing empty test methods such as:

public function testSomething()
{
}

to keep track of the tests that you have to write. The problem with empty test methods is that they are interpreted as a success by the PHPUnit framework. This misinterpretation leads to the test reports being useless -- you cannot see whether a test is actually successful or just not yet implemented. Calling $this->fail() in the unimplemented test method does not help either, since then the test will be interpreted as a failure. This would be just as wrong as interpreting an unimplemented test as a success.

If we think of a successful test as a green light and a test failure as a red light, we need an additional yellow light to mark a test as being incomplete or not yet implemented. PHPUnit_Framework_IncompleteTest is a marker interface for marking an exception that is raised by a test method as the result of the test being incomplete or currently not implemented. PHPUnit_Framework_IncompleteTestError is the standard implementation of this interface.

Example 9.1 shows a test-case class, SampleTest, that contains one test method, testSomething(). By calling the convenience method markTestIncomplete() (which automatically raises an PHPUnit_Framework_IncompleteTestError exception) in the test method, we mark the test as being incomplete.

<?php
require_once 'PHPUnit/Framework.php';
 
class SampleTest extends PHPUnit_Framework_TestCase
{
    public function testSomething()
    {
        // Optional: Test anything here, if you want.
        $this->assertTrue(TRUE, 'This should already work.');
 
        // Stop here and mark this test as incomplete.
        $this->markTestIncomplete(
          'This test has not been implemented yet.'
        );
    }
}
?>

An incomplete test is denoted by an I in the output of the PHPUnit command-line test runner, as shown in the following example:

phpunit --verbose SampleTest
PHPUnit 3.1.9 by Sebastian Bergmann.

SampleTest
I


Time: 0 seconds

There was 1 incomplete test:

1) testSomething(SampleTest)
This test has not been implemented yet.
/home/sb/SampleTest.php:14

OK, but incomplete or skipped tests!
Tests: 1, Incomplete: 1.

Table 9.1 shows the API for marking tests as incomplete.

Not all tests can be run in every environment. Consider, for instance, a database abstraction layer that has several drivers for the different database systems it supports. The tests for the MySQL driver can of course only be run if a MySQL server is available.

Example 9.2 shows a test-case class, DatabaseTest, that contains one test method, testConnection(). In the test-case class' setUp() template method we check whether the MySQLi extension is available and use the markTestSkipped() method to skip the test if it is not.

<?php
require_once 'PHPUnit/Framework.php';
 
class DatabaseTest extends PHPUnit_Framework_TestCase
{
    protected function setUp()
    {
        if (!extension_loaded('mysqli')) {
            $this->markTestSkipped(
              'The MySQLi extension is not available.'
            );
        }
    }
 
    public function testConnection()
    {
        // ...
    }
}
?>

A test that has been skipped is denoted by an S in the output of the PHPUnit command-line test runner, as shown in the following example:

phpunit --verbose DatabaseTest
PHPUnit 3.1.9 by Sebastian Bergmann.

DatabaseTest
S


Time: 0 seconds

There was 1 skipped test:

1) testConnection(DatabaseTest)
The MySQLi extension is not available.
/home/sb/DatabaseTest.php:11

OK, but incomplete or skipped tests!
Tests: 1, Skipped: 1.

Table 9.2 shows the API for skipping tests.

Alternatively, you can test the Subject implementation without the use of Mock Objects and apply the Self Shunt Pattern instead. This uses the test-case object itself as a stub. The term self-shunting is taken from the medical practice of installing a tube that takes blood from an artery and returns it to a vein to provide a convenient place for injecting drugs.

First, we make our test-case class an implementor of Observer, an interface to be implemented by objects that want to observe Subject:

class ObserverTest extends PHPUnit_Framework_TestCase implements Observer
{
}

Next, we implement the one Observer method, update(), to check that it is called when the state of the observed Subject object changes:

public $wasCalled = FALSE;

public function update(Subject $subject)
{
  $this->wasCalled = TRUE;
}

Now, we can write our test. We create a new Subject object and attach the test object to it as an observer. When the state of the Subject changes -- for instance, by calling its doSomething() method -- the Subject object has to call the update() method on all objects that are registered as observers. We use the $wasCalled instance variable that is set by our implementation of update() to check whether the Subject object does what it is supposed to do:

public function testUpdate()
{
  $subject = new Subject;
  $subject->attach($this);
  $subject->doSomething();

  $this->assertTrue($this->wasCalled);
}

Notice that we create a new Subject object instead of relying on a global instance. Stubbing encourages this style of design. It reduces the coupling between objects and improves reuse.

If you are not familiar with the self-shunt pattern, the tests can be hard to read. What is going on here? Why is a test case also an observer? Once you get used to the idiom, the tests are easy to read. Everything you need to understand the test is in one class.

Tests that only test one thing are more informative than tests where failure can come from many sources. How can you isolate your tests from external influences? Simply put, by replacing the expensive, messy, unreliable, slow, complicated resources with stubs that are automatically generated for the purpose of your tests. For example, you can replace what is in reality a complicated computation with a constant, at least for the purposes of a single test.

Stubs solve the problem of allocating expensive external resources. For example, sharing a resource, such as a database connection, between tests helps but not using the database for the purposes of the tests at all is even better.

Example 10.2 shows how to stub method calls and set up return values.

<?php
require_once 'PHPUnit/Framework.php';
 
class StubTest extends PHPUnit_Framework_TestCase
{
    public function testStub()
    {
        $stub = $this->getMock('SomeClass', array('doSomething'));
        $stub->expects($this->any())
             ->method('doSomething')
             ->will($this->returnValue('foo'));
 
        // Calling $stub->doSomething() will now return 'foo'.
    }
}
?>

Table 10.2 lists the methods that are available to configure the return values for stubbed method calls.

Alternatively, you can write the stub yourself and improve your design along the way. Widely used resources are accessed through a single façade, so you can easily replace the resource with the stub. For example, instead of having direct database calls scattered throughout the code, you have a single Database object, an implementor of the IDatabase interface. Then, you can create a stub implementation of IDatabase and use it for your tests. You can even create an option for running the tests with the stub database or the real database, so you can use your tests for both local testing during development and integration testing with the real database.

Functionality that needs to be stubbed out tends to cluster in the same object, improving cohesion. By presenting the functionality with a single, coherent interface, you reduce the coupling with the rest of the system.

When you need to make a change to the internal structure of the software you are working on to make it easier to understand and cheaper to modify without changing its observable behavior, a test suite is invaluable in applying these so called refactorings safely. Otherwise, you might not notice the system breaking while you are carrying out the restructuring.

The following conditions will help you to improve the code and design of your project, while using unit tests to verify that the refactoring's transformation steps are, indeed, behavior-preserving and do not introduce errors:

When you need to add new functionality to the system, write the tests first. Then, you will be done developing when the test runs. This practice will be discussed in detail in the next chapter.

When you get a defect report, your impulse might be to fix the defect as quickly as possible. Experience shows that this impulse will not serve you well; it is likely that the fix for the defect causes another defect.

You can hold your impulse in check by doing the following:

Finding the smallest reliable reproduction of the defect gives you the opportunity to really examine the cause of the defect. The test you write will improve the chances that when you fix the defect, you really fix it, because the new test reduces the likelihood of undoing the fix with future code changes. All the tests you wrote before reduce the likelihood of inadvertently causing a different problem.

In this chapter, we will look at the example of a class that represents a bank account. The contract for the BankAccount class not only requires methods to get and set the bank account's balance, as well as methods to deposit and withdraw money. It also specifies the following two conditions that must be ensured:

Following the Test-First Programming approach, we write the tests for the BankAccount class before we write the code for the class itself. We use the contract conditions as the basis for the tests and name the test methods accordingly, as shown in Example 12.1.

<?php
require_once 'PHPUnit/Framework.php';
require_once 'BankAccount.php';
 
class BankAccountTest extends PHPUnit_Framework_TestCase
{
    protected $ba;
 
    protected function setUp()
    {
        $this->ba = new BankAccount;
    }
 
    public function testBalanceIsInitiallyZero()
    {
        $this->assertEquals(0, $this->ba->getBalance());
    }
 
    public function testBalanceCannotBecomeNegative()
    {
        try {
            $this->ba->withdrawMoney(1);
        }
 
        catch (BankAccountException $e) {
            $this->assertEquals(0, $this->ba->getBalance());
 
            return;
        }
 
        $this->fail();
    }
 
    public function testBalanceCannotBecomeNegative2()
    {
        try {
            $this->ba->depositMoney(-1);
        }
 
        catch (BankAccountException $e) {
            $this->assertEquals(0, $this->ba->getBalance());
 
            return;
        }
 
        $this->fail();
    }
}
?>

We now write the minimal amount of code needed for the first test, testBalanceIsInitiallyZero(), to pass. In our example this amounts to implementing the getBalance() method of the BankAccount class, as shown in Example 12.2.

<?php
class BankAccount
{
    protected $balance = 0;
 
    public function getBalance()
    {
        return $this->balance;
    }
}
?>

The test for the first contract condition now passes, but the tests for the second contract condition fail because we have yet to implement the methods that these tests call.

phpunit BankAccountTest
PHPUnit 3.1.9 by Sebastian Bergmann.

.
Fatal error: Call to undefined method BankAccount::withdrawMoney()

For the tests that ensure the second contract condition to pass, we now need to implement the withdrawMoney(), depositMoney(), and setBalance() methods, as shown in Example 12.3. These methods are written in a such a way that they raise an BankAccountException when they are called with illegal values that would violate the contract conditions.

<?php
class BankAccount
{
    protected $balance = 0;
 
    public function getBalance()
    {
        return $this->balance;
    }
 
    protected function setBalance($balance)
    {
        if ($balance >= 0) {
            $this->balance = $balance;
        } else {
            throw new BankAccountException;
        }
    }
 
    public function depositMoney($balance)
    {
        $this->setBalance($this->getBalance() + $balance);
 
        return $this->getBalance();
    }
 
    public function withdrawMoney($balance)
    {
        $this->setBalance($this->getBalance() - $balance);
 
        return $this->getBalance();
    }
}
?>

The tests that ensure the second contract condition now pass, too:

phpunit BankAccountTest
PHPUnit 3.1.9 by Sebastian Bergmann.

...

Time: 0 seconds


OK (3 tests)

Alternatively, you can use the static assertion methods provided by the PHPUnit_Framework_Assert class to write the contract conditions as design-by-contract style assertions into your code, as shown in Example 12.4. When one of these assertions fails, an PHPUnit_Framework_AssertionFailedError exception will be raised. With this approach, you write less code for the contract condition checks and the tests become more readable. However, you add a runtime dependency on PHPUnit to your project.

<?php
require_once 'PHPUnit/Framework.php';
 
class BankAccount
{
    private $balance = 0;
 
    public function getBalance()
    {
        return $this->balance;
    }
 
    protected function setBalance($balance)
    {
        PHPUnit_Framework_Assert::assertTrue($balance >= 0);
 
        $this->balance = $balance;
    }
 
    public function depositMoney($amount)
    {
        PHPUnit_Framework_Assert::assertTrue($amount >= 0);
 
        $this->setBalance($this->getBalance() + $amount);
 
        return $this->getBalance();
    }
 
    public function withdrawMoney($amount)
    {
        PHPUnit_Framework_Assert::assertTrue($amount >= 0);
        PHPUnit_Framework_Assert::assertTrue($this->balance >= $amount);
 
        $this->setBalance($this->getBalance() - $amount);
 
        return $this->getBalance();
    }
}
?>

By writing the contract conditions into the tests, we have used Design-by-Contract to program the BankAccount class. We then wrote, following the Test-First Programming approach, the code needed to make the tests pass. However, we forgot to write tests that call setBalance(), depositMoney(), and withdrawMoney() with legal values that do not violate the contract conditions. We need a means to test our tests or at least to measure their quality. Such a means is the analysis of code-coverage information that we will discuss next.

By default, all sourcecode files that contain at least one line of code that has been executed (and only these files) are included in the report. You can configure the sourcecode files that are included in the report using the PHPUnit_Util_Filter API (see Table 13.1).

The blacklist is pre-filled with all sourcecode files of PHPUnit itself and the tests. When the whitelist is empty (default), blacklisting is used. When the whitelist is not empty, whitelisting is used. When whitelisting is used, each file on the whitelist is added to the code coverage report regardless of whether or not it was executed.

Typically, in a project that is developed using an agile process, such as Extreme Programming, the documentation cannot keep up with the frequent changes to the project's design and code. Extreme Programming demands collective code ownership, so all developers need to know how the entire system works. If you are disciplined enough to consequently use "speaking names" for your tests that describe what a class should do, you can use PHPUnit's TestDox functionality to generate automated documentation for your project based on its tests. This documentation gives developers an overview of what each class of the project is supposed to do.

PHPUnit's TestDox functionality looks at a test class and all the test method names and converts them from camel case PHP names to sentences: testBalanceIsInitiallyZero() becomes "Balance is initially zero". If there are several test methods whose names only differ in a suffix of one or more digits, such as testBalanceCannotBecomeNegative() and testBalanceCannotBecomeNegative2(), the sentence "Balance cannot become negative" will appear only once, assuming that all of these tests succeed.

Let us take a look at the agile documentation generated for the BankAccount class (from Example 12.1):

phpunit --testdox BankAccountTest
PHPUnit 3.1.9 by Sebastian Bergmann.

BankAccount
 - Balance is initially zero
 - Balance cannot become negative

Alternatively, the agile documentation can be generated in HTML or plain text format and written to a file using the --testdox-html and --testdox-text arguments.

Agile Documentation can be used to document the assumptions you make about the external packages that you use in your project. When you use an external package, you are exposed to the risks that the package will not behave as you expect, and that future versions of the package will change in subtle ways that will break your code, without you knowing it. You can address these risks by writing a test every time you make an assumption. If your test succeeds, your assumption is valid. If you document all your assumptions with tests, future releases of the external package will be no cause for concern: if the tests succeed, your system should continue working.

When you document assumptions with tests, you own the tests. The supplier of the package -- who you make assumptions about -- knows nothing about your tests. If you want to have a closer relationship with the supplier of a package, you can use the tests to communicate and coordinate your activities.

When you agree on coordinating your activities with the supplier of a package, you can write the tests together. Do this in such a way that the tests reveal as many assumptions as possible. Hidden assumptions are the death of cooperation. With the tests, you document exactly what you expect from the supplied package. The supplier will know the package is complete when all the tests run.

By using stubs (see the chapter on "Mock Objects", earlier in this book), you can further decouple yourself from the supplier: The job of the supplier is to make the tests run with the real implementation of the package. Your job is to make the tests run for your own code. Until such time as you have the real implementation of the supplied package, you use stub objects. Following this approach, the two teams can develop independently.

The XML format supported by PHPUnit is loosely based upon the one used by the JUnit task for Apache Ant. The following example shows the XML logfile generated for the tests in ArrayTest:

<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
  <testsuite name="ArrayTest"
             file="/home/sb/ArrayTest.php"
             tests="2"
             failures="0"
             errors="0"
             time="0.016030">
    <testcase name="testNewArrayIsEmpty"
              class="ArrayTest"
              file="/home/sb/ArrayTest.php"
              line="6"
              time="0.008044"/>
    <testcase name="testArrayContainsAnElement"
              class="ArrayTest"
              file="/home/sb/ArrayTest.php"
              line="15"
              time="0.007986"/>
  </testsuite>
</testsuites>

The following XML logfile was generated for two tests, testFailure and testError, of a test-case class named FailureErrorTest and shows how failures and errors are denoted.

<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
  <testsuite name="FailureErrorTest"
             file="/home/sb/FailureErrorTest.php"
             tests="2"
             failures="1"
             errors="1"
             time="0.019744">
    <testcase name="testFailure"
              class="FailureErrorTest"
              file="/home/sb/FailureErrorTest.php"
              line="6"
              time="0.011456">
      <failure type="PHPUnit_Framework_ExpectationFailedException">
testFailure(FailureErrorTest)
Failed asserting that &lt;integer:2&gt; matches expected value &lt;integer:1&gt;.

/home/sb/FailureErrorTest.php:8
</failure>
    </testcase>
    <testcase name="testError"
              class="FailureErrorTest"
              file="/home/sb/FailureErrorTest.php"
              line="11"
              time="0.008288">
      <error type="Exception">testError(FailureErrorTest)
Exception: 

/home/sb/FailureErrorTest.php:13
</error>
    </testcase>
  </testsuite>
</testsuites>

The XML format for code coverage information logging supported by PHPUnit is loosely based upon the one used by Clover. The following example shows the XML logfile generated for the tests in BankAccountTest:

<?xml version="1.0" encoding="UTF-8"?>
<coverage generated="1184835473" phpunit="3.1.9">
  <project name="BankAccountTest" timestamp="1184835473">
    <file name="/home/sb/BankAccount.php">
      <class name="BankAccountException">
        <metrics methods="0" coveredmethods="0" statements="0"
                 coveredstatements="0" elements="0" coveredelements="0"/>
      </class>
      <class name="BankAccount">
        <metrics methods="4" coveredmethods="4" statements="13"
                 coveredstatements="5" elements="17" coveredelements="9"/>
      </class>
      <line num="77" type="method" count="3"/>
      <line num="79" type="stmt" count="3"/>
      <line num="89" type="method" count="2"/>
      <line num="91" type="stmt" count="2"/>
      <line num="92" type="stmt" count="0"/>
      <line num="93" type="stmt" count="0"/>
      <line num="94" type="stmt" count="2"/>
      <line num="96" type="stmt" count="0"/>
      <line num="105" type="method" count="1"/>
      <line num="107" type="stmt" count="1"/>
      <line num="109" type="stmt" count="0"/>
      <line num="119" type="method" count="1"/>
      <line num="121" type="stmt" count="1"/>
      <line num="123" type="stmt" count="0"/>
      <metrics loc="126" ncloc="37" classes="2" methods="4" coveredmethods="4"
               statements="13" coveredstatements="5" elements="17"
               coveredelements="9"/>
    </file>
    <metrics files="1" loc="126" ncloc="37" classes="2" methods="4"
             coveredmethods="4" statements="13" coveredstatements="5"
             elements="17" coveredelements="9"/>
  </project>
</coverage>

The JavaScript Object Notation (JSON) is a lightweight data-interchange format. The following example shows the JSON messages generated for the tests in ArrayTest:

{"event":"suiteStart","suite":"ArrayTest","tests":2}
{"event":"test","suite":"ArrayTest",
 "test":"testNewArrayIsEmpty(ArrayTest)","status":"pass",
 "time":0.000460147858,"trace":[],"message":""}
{"event":"test","suite":"ArrayTest",
 "test":"testArrayContainsAnElement(ArrayTest)","status":"pass",
 "time":0.000422954559,"trace":[],"message":""}

The following JSON messages were generated for two tests, testFailure and testError, of a test-case class named FailureErrorTest and show how failures and errors are denoted.

{"event":"suiteStart","suite":"FailureErrorTest","tests":2}
{"event":"test","suite":"FailureErrorTest",
 "test":"testFailure(FailureErrorTest)","status":"fail",
 "time":0.000483989716,"trace":[],"message":""}
{"event":"test","suite":"FailureErrorTest",
 "test":"testError(FailureErrorTest)","status":"error",
 "time":0.000466108322,"trace":[],"message":""}

The Test Anything Protocol (TAP) is Perl's simple text-based interface between testing modules. The following example shows the TAP logfile generated for the tests in ArrayTest:

1..2
# TestSuite "ArrayTest" started.
ok 1 - testNewArrayIsEmpty(ArrayTest)
ok 2 - testArrayContainsAnElement(ArrayTest)
# TestSuite "ArrayTest" ended.

The following TAP logfile was generated for two tests, testFailure and testError, of a test-case class named FailureErrorTest and shows how failures and errors are denoted.

1..2
# TestSuite "FailureErrorTest" started.
not ok 1 - Failure: testFailure(FailureErrorTest)
not ok 2 - Error: testError(FailureErrorTest)
# TestSuite "FailureErrorTest" ended.

PHPUnit can generate a description of the test result as a graph that can be rendered to diagrams in several useful formats such as images using the GraphViz tools.

phpunit --log-graphviz BankAccount.dot BankAccountTest
PHPUnit 3.1.9 by Sebastian Bergmann.

...

Time: 0 seconds

OK (3 tests)

The following example shows the GraphViz markup generated (and saved to the BankAccount.dot file in the current directory) for the tests in BankAccountTest:

digraph G {
graph [ overlap="scale",splines="true",sep=".1",fontsize="8" ];
"BankAccountTest" [ color="green" ];
subgraph "cluster_BankAccountTest" {
label="";
"testBalanceIsInitiallyZero" [ color="green" ];
"testBalanceCannotBecomeNegative" [ color="green" ];
"testBalanceCannotBecomeNegative2" [ color="green" ];
}
"BankAccountTest" -> "testBalanceIsInitiallyZero";
"BankAccountTest" -> "testBalanceCannotBecomeNegative";
"BankAccountTest" -> "testBalanceCannotBecomeNegative2";
}

We can now use the dot command-line tool that is part of the GraphViz software suite to generate a graphic from this markup:

dot -T png -o BankAccount.png BankAccount.dot

Figure 15.1 shows the graph representation of the test result rendered from the GraphViz markup above.

Figure 15.1. Graph representation of the test result

Successful tests are displayed with a green border, failures and errors with a red border, and incomplete or skipped tests with a yellow border. A parent node, such as test suite, is displayed with a non-green border if it has a child node, such as a test, that was not successful.

PHPUnit can write test result and code coverage data to a test database. Several ideas for future features depend on this data.

Before we can write test result and code coverage data to a database, we need to create a database using one of the supplied schema definitions:

sqlite3 BankAccount.db < PHPUnit/Util/Log/Database/SQLite3.sql

Now we can execute a test suite and write the test result and code coverage data to a database:

phpunit --test-db-dsn sqlite:///home/sb/BankAccount.db --test-db-log-rev 1 BankAccountTest
PHPUnit 3.1.9 by Sebastian Bergmann.

...

Time: 0 seconds


OK (3 tests)

Storing code coverage data in database, this may take a moment.

Table 15.1 shows the arguments understood by the TextUI test runner (see Chapter 5) for the test database.

You can use @assert annotation in the documentation block of a method to automatically generate simple, yet meaningful tests instead of incomplete test-cases. Example 16.2 shows an example.

<?php
class Calculator
{
    /**
     * @assert (0, 0) == 0
     * @assert (0, 1) == 1
     * @assert (1, 0) == 1
     * @assert (1, 1) == 2
     */
    public function add($a, $b)
    {
        return $a + $b;
    }
}
?>

Each method in the original class is checked for @assert annotations. These are transformed into test code such as

    /**
     * Generated from @assert (0, 0) == 0.
     */
    public function testAdd() {
        $o = new Calculator;
        $this->assertEquals(0, $o->add(0, 0));
    }

Below is the output of running the generated test-case class.

phpunit CalculatorTest
PHPUnit 3.1.9 by Sebastian Bergmann.

....

Time: 0 seconds


OK (4 tests)

Table 16.1 shows the supported variations of the @assert annotation and how they are transformed into test code.

Selenium RC is a test tool that allows you to write automated user-interface tests for web applications in any programming language against any HTTP website using any mainstream browser. It uses Selenium Core, a library that performs automated browser tasks using JavaScript. Selenium tests run directly in a browser, just as real users do. These tests can be used for both acceptance testing (by performing higher-level tests on the integrated system instead of just testing each unit of the system independently) and browser compatibility testing (by testing the web application on different operating systems and browsers).

Let us take a look at how Selenium RC is installed:

Now we can send commands to the Selenium RC server using its client/server protocol.

The PHPUnit_Extensions_SeleniumTestCase test case extension implements the client/server protocol to talk to Selenium RC as well as specialized assertion methods for web testing.

Example 17.1 shows how to test the contents of the <title> element of the http://www.example.com/ website.

<?php
require_once 'PHPUnit/Extensions/SeleniumTestCase.php';
 
class WebTest extends PHPUnit_Extensions_SeleniumTestCase
{
    protected function setUp()
    {
        $this->setBrowser('*firefox');
        $this->setBrowserUrl('http://www.example.com/');
    }
 
    public function testTitle()
    {
        $this->open('http://www.example.com/');
        $this->assertTitleEquals('Example Web Page');
    }
}
?>

Unlike with the PHPUnit_Framework_TestCase class, test case classes that extend PHPUnit_Extensions_SeleniumTestCase have to provide a setUp() method. This method is used to configure the Selenium RC session. See Table 17.1 for the list of methods that are available for this.

Table 17.2 lists the various assertion methods that PHPUnit_Extensions_SeleniumTestCase provides.

Table 17.3 shows the two template methods of PHPUnit_Extensions_SeleniumTestCase:

Please refer to the documentation of Selenium Core for a reference of the commands available and how they are used.

Continuous Integration demands a fully automated and reproducible build, including testing, that runs many times a day. This allows each developer to integrate daily thus reducing integration problems. While this can be achieved by setting up a cronjob that makes a fresh checkout from the project's source code repository at regular intervals, runs the tests, and publishes the results a more comfortable solution may be desired.

This is where a framework for continuous build processes such as CruiseControl comes into play. It includes, but is not limited to, plugins for email notification, Apache Ant, and various source control tools. A web interface is provided to view the details of the current and previous builds.

The following example assumes that CruiseControl has been installed into /usr/local/cruisecontrol.

<project name="BankAccount" default="build" basedir=".">
 <target name="checkout">
  <exec dir="${basedir}/source/" executable="svn">
   <arg line="up"/>
  </exec>
 </target>

 <target name="test">
  <exec dir="${basedir}/source" executable="phpunit" failonerror="true">
   <arg line="--log-xml ${basedir}/build/logs/bankaccount.xml BankAccountTest"/>
  </exec>
 </target>

 <target name="build" depends="checkout,test"/>
</project>

<cruisecontrol>
  <project name="BankAccount" buildafterfailed="false">
    <plugin
    name="svnbootstrapper"
    classname="net.sourceforge.cruisecontrol.bootstrappers.SVNBootstrapper"/>
    <plugin
    name="svn"
    classname="net.sourceforge.cruisecontrol.sourcecontrols.SVN"/>

    <listeners>
      <currentbuildstatuslistener file="logs/${project.name}/status.txt"/>
    </listeners>

    <bootstrappers>
      <svnbootstrapper localWorkingCopy="projects/${project.name}/source/"/>
    </bootstrappers>

    <modificationset>
      <svn localWorkingCopy="projects/${project.name}/source/"/>
    </modificationset>

    <schedule interval="300">
      <ant
      anthome="apache-ant-1.7.0"
      buildfile="projects/${project.name}/build.xml"/>
    </schedule>

    <log dir="logs/${project.name}">
      <merge dir="projects/${project.name}/build/logs/"/>
    </log>

    <publishers>
      <currentbuildstatuspublisher
      file="logs/${project.name}/buildstatus.txt"/>

      <email
      mailhost="localhost"
      buildresultsurl="http://cruise.example.com/buildresults/${project.name}"
      skipusers="true"
      spamwhilebroken="true"
      returnaddress="project@example.com">
        <failure address="dev@lists.example.com" reportWhenFixed="true"/>
      </email>
    </publishers>
  </project>
</cruisecontrol>

Apache Maven is a software project management and comprehension tool. Based on the concept of a Project Object Model (POM), Apache Maven can manage a project's build, reporting and documentation from a central place of information.

The single XML logfile that PHPUnit's XML logging facility (see the section called “XML Format”) generates needs to be split into separate XML logfiles, one for each test suite, before it can be processed by Apache Maven's surefire plugin. This plugin is used during the test phase of the build lifecycle to execute the unit tests of an application. Example 18.4 shows an XSLT stylesheet that performs this splitting. Example 18.3 shows an example pom.xml configuration file.

<?xml version="1.0" encoding="UTF-8"?>

<project xmlns="http://maven.apache.org/POM/4.0.0"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
  http://maven.apache.org/maven-v4_0_0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <!-- ... -->
  <prerequisites>
    <maven>2.0.7</maven>
  </prerequisites>    
  <!-- ... -->
  <build>
  <!-- ... -->
    <plugins>
      <plugin>
      <dependencies>
        <dependency>
          <groupId>org.apache.ant</groupId>
          <artifactId>ant-trax</artifactId>
          <version>1.7.0</version>
        </dependency>
        <dependency>
          <groupId>net.sf.saxon</groupId>
          <artifactId>saxon</artifactId>
          <version>8.7</version>
        </dependency>
      </dependencies>
      <groupId>org.apache.maven.plugins</groupId>
      <artifactId>maven-antrun-plugin</artifactId>
      <version>1.2-SNAPSHOT</version>
      <executions>
        <execution>
          <id>codecoverage</id>
          <phase>pre-site</phase>
          <goals>
            <goal>run</goal>
          </goals>
          <configuration>
            <tasks>
              <property name="phpunit.codecoverage"
                        location="${project.reporting.outputDirectory}/phpunit/codecoverage" />
              <property name="surefire.reports"
                        location="${project.build.directory}/surefire-reports" />

              <mkdir dir="${phpunit.codecoverage}"/>
              <mkdir dir="${surefire.reports}"/>

              <!-- ${ant.phpunit} path to PHPUnit executable -->
              <!-- ${ant.pear};... this is the include path for your test execution -->
              <!-- ${test.AllTests} PHPUnit cmd line args like 'AllTests de/dmc/dashboard/AllTests.php' -->
              <exec executable="${ant.phpunit}" dir="${basedir}">
                <arg line="-d include_path=${ant.pear};${project.build.sourceDirectory};${project.build.testSourceDirectory}
                           --report ${phpunit.codecoverage} ${test.AllTests}" />
              </exec>

              <xslt in="${phpunit.codecoverage}/logfile.xml"
                    out="${surefire.reports}/xslt.info"
                    style="src/test/config/phpunit_to_surefire.xslt"
                    processor="trax">
                <!-- this is the output folder for surefire like XML Reports -->
                <param name="outputDir" expression="${surefire.reports}"/>
              </xslt>
            </tasks>
          </configuration>
        </execution>
      </executions>
      </plugin>
    </plugins>
  </build>
  <reporting>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-report-plugin</artifactId>
        <version>2.4-SNAPSHOT</version>
        <reportSets>
          <reportSet>
            <reports>
              <report>report-only</report>
            </reports>
          </reportSet>
        </reportSets>
      </plugin>
    </plugins>
  </reporting>
</project>

<?xml version="1.0" encoding="UTF-8"?>

<xsl:stylesheet version="2.0"
  xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
  xmlns:xs="http://www.w3.org/2001/XMLSchema"
  xmlns:fn="http://www.w3.org/2005/xpath-functions">
  <xsl:output method="xml" version="1.0" encoding="UTF-8" indent="yes"/>
  <xsl:param name="outputDir">.</xsl:param>

  <xsl:template match="testsuites">
    <xsl:apply-templates select="testsuite"/>
  </xsl:template>

  <xsl:template match="testsuite">
    <xsl:if test="testcase">
      <xsl:variable name="outputName" select="./@name"/>
      <xsl:result-document href="file:///{$outputDir}/{$outputName}.xml" method="xml">
        <xsl:copy-of select="."/>
      </xsl:result-document>
    </xsl:if>

    <xsl:apply-templates select="testsuite"/>
  </xsl:template>
</xsl:stylesheet>

Most of the time, you will encounter five classes or interfaces when you are using PHPUnit:

Figure 20.1 shows the relationship of the five basic classes and interfaces in PHPUnit: PHPUnit_Framework_Assert, PHPUnit_Framework_Test, PHPUnit_Framework_TestCase, PHPUnit_Framework_TestSuite, and PHPUnit_Framework_TestResult.

Figure 20.1. The five basic classes and interfaces in PHPUnit

Most test cases written for PHPUnit are derived indirectly from the class PHPUnit_Framework_Assert, which contains methods for automatically checking values and reporting discrepancies. The methods are declared static, so you can write design-by-contract style assertions in your methods and have them reported through PHPUnit (Example 20.1).

<?php
require_once 'PHPUnit/Framework.php';
 
class Sample
{
    public function aSampleMethod($object)
    {
        PHPUnit_Framework_Assert::assertNotNull($object);
    }
}
 
$sample = new Sample;
$sample->aSampleMethod(NULL);
?>

Fatal error: Uncaught exception 'PHPUnit_Framework_ExpectationFailedException'
with message 'Failed asserting that <null> is not identical to <null>.'

Most of the time, though, you'll be checking the assertions inside of tests.

There are two variants of each of the assertion methods: one takes a message to be displayed with the error as a parameter, and one does not. The optional message is typically displayed when a failure is displayed, which can make debugging easier.

<?php
require_once 'PHPUnit/Framework.php';
 
class MessageTest extends PHPUnit_Framework_TestCase
{
    public function testMessage()
    {
        $this->assertTrue(FALSE, 'This is a custom message.');
    }
}
?>

The following example shows the output you get when you run the testMessage() test from Example 20.2, using assertions with messages:

phpunit MessageTest
PHPUnit 3.1.6 by Sebastian Bergmann.

F

Time: 0 seconds

There was 1 failure:

1) testMessage(MessageTest)
This is a custom message.
Failed asserting that <boolean:false> is true.
/home/sb/MessageTest.php:8

FAILURES!
Tests: 1, Failures: 1.

Table 20.1 shows all the varieties of assertions.

More complex assertions can be formulated using the PHPUnit_Framework_Constraint classes. They can be evaluated using the assertThat() method as shown in Example 20.3.

<?php
require_once 'PHPUnit/Framework.php';
 
class ConstraintTest extends PHPUnit_Framework_TestCase
{
    public function testNotEquals()
    {
        $constraint = $this->logicalNot(
          $this->equalTo('foo')
        );
 
        $this->assertThat('foo', $constraint);
    }
}
?>

phpunit ConstraintTest
PHPUnit 3.1.9 by Sebastian Bergmann.

F

Time: 0 seconds

There was 1 failure:

1) testNotEquals(ConstraintTest)
Failed asserting that <string:foo> is not equal to <string:foo>.
/home/sb/ConstraintTest.php:12

FAILURES!
Tests: 1, Failures: 1.

Table 20.2 shows the available PHPUnit_Framework_Constraint implementations.

You may find that you need other assertions than these to compare objects specific to your project. Create your own Assert class to contain these assertions to simplify your tests.

Failing assertions all call a single bottleneck method, fail(string $message), which throws an PHPUnit_Framework_AssertionFailedError. There is also a variant which takes no parameters. Call fail() explicitly when your test encounters an error. The test for an expected exception is an example. Table 20.3 lists the bottlenext methods in PHPUnit.

markTestIncomplete() and markTestSkipped() are convenience methods for marking a test as being incomplete or skipped.

Although unit tests are about testing the public interface of a class, you may sometimes want to test the values of non-public attributes. The readAttribute() method enables you to do this and returns the value of a given (static) attribute from a given class or object.

PHPUnit_Framework_Test is the generic interface used by all objects that can act as tests. Implementors may represent one or more tests. The two methods are shown in Table 20.6.

PHPUnit_Framework_TestCase and PHPUnit_Framework_TestSuite are the two most prominent implementors of PHPUnit_Framework_Test. You can implement PHPUnit_Framework_Test yourself. The interface is kept small intentionally so it will be easy to implement.

Your test-case classes will inherit from PHPUnit_Framework_TestCase. Most of the time, you will run tests from automatically created test suites. In this case, each of your tests should be represented by a method named test* (by convention).

PHPUnit_Framework_TestCase implements PHPUnit_Framework_Test::count() so that it always returns 1. The implementation of PHPUnit_Framework_Test::run(PHPUnit_Framework_TestResult $result) in this class runs setUp(), runs the test method, and then runs tearDown(), reporting any exceptions to the PHPUnit_Framework_TestResult.

Table 20.7 shows the methods provided by PHPUnit_Framework_TestCase.

There are two template methods -- setUp() and tearDown() -- you can override to create and dispose of the objects against which you are going to test. Table 20.8 shows these methods. A third template method, sharedAssertions(), allows to define assertions that should be performed by all tests of a test case.

A PHPUnit_Framework_TestSuite is a composite of PHPUnit_Framework_Tests. At its simplest, it contains a bunch of test cases, all of which are run when the suite is run. Since it is a composite, however, a suite can contain suites which can contain suites and so on, making it easy to combine tests from various sources and run them together.

In addition to the PHPUnit_Framework_Test methods -- run(PHPUnit_Framework_TestResult $result) and count() -- PHPUnit_Framework_TestSuite provides methods to create named or unnamed instances. Table 20.9 shows the methods to create PHPUnit_Framework_TestSuite instances.

PHPUnit_Framework_TestSuite also provides methods for adding and retrieving PHPUnit_Framework_Tests, as shown in Table 20.10.

Example 20.4 shows how to create and run a test suite.

<?php
require_once 'PHPUnit/Framework.php';
 
require_once 'ArrayTest.php';
 
// Create a test suite that contains the tests
// from the ArrayTest class.
$suite = new PHPUnit_Framework_TestSuite('ArrayTest');
 
// Run the tests.
$suite->run();
?>

Chapter 7 shows how to use the PHPUnit_Framework_TestSuite class to organize test suites by hierarchically composing test cases.

The PHPUnit_Framework_TestSuite class provides two template methods -- setUp() and tearDown() -- that are called before and after the tests of a test suite are run, respectively.

While you are running all these tests, you need somewhere to store all the results: how many tests ran, which failed, and how long they took. PHPUnit_Framework_TestResult collects these results. A single PHPUnit_Framework_TestResult is passed around the whole tree of tests; when a test runs or fails, the fact is noted in the PHPUnit_Framework_TestResult. At the end of the run, PHPUnit_Framework_TestResult contains a summary of all the tests.

PHPUnit_Framework_TestResult is also a subject than can be observed by other objects wanting to report test progress. For example, a graphical test runner might observe the PHPUnit_Framework_TestResult and update a progress bar every time a test starts.

Table 20.12 summarizes the API of PHPUnit_Framework_TestResult.

If you want to register as an observer of a PHPUnit_Framework_TestResult, you need to implement PHPUnit_Framework_TestListener. To register, call addListener(), as shown in Table 20.13.

Table 20.14 shows the methods that test listeners implement; also see Example 21.3.

Many of the classes mentioned so far in this book come from PHPUnit/Framework. Here are all the packages in PHPUnit:

Write utility methods in an abstract subclass of PHPUnit_Framework_TestCase and derive your test case classes from that class. This is one of the easiest ways to extend PHPUnit.

Write your own class with assertions special to your purpose.

You can wrap test cases or test suites in a subclass of PHPUnit_Extensions_TestDecorator and use the Decorator design pattern to perform some actions before and after the test runs.

PHPUnit ships with two concrete test decorators: PHPUnit_Extensions_RepeatedTest and PHPUnit_Extensions_TestSetup. The former is used to run a test repeatedly and only count it as a success if all iterations are successful. The latter was discussed in Chapter 6.

Example 21.1 shows a cut-down version of the PHPUnit_Extensions_RepeatedTest test decorator that illustrates how to write your own test decorators.

<?php
require_once 'PHPUnit/Extensions/TestDecorator.php';
 
class PHPUnit_Extensions_RepeatedTest extends PHPUnit_Extensions_TestDecorator
{
    private $timesRepeat = 1;
 
    public function __construct(PHPUnit_Framework_Test $test, $timesRepeat = 1)
    {
        parent::__construct($test);
 
        if (is_integer($timesRepeat) &&
            $timesRepeat >= 0) {
            $this->timesRepeat = $timesRepeat;
        }
    }
 
    public function count()
    {
        return $this->timesRepeat * $this->test->count();
    }
 
    public function run(PHPUnit_Framework_TestResult $result = NULL)
    {
        if ($result === NULL) {
            $result = $this->createResult();
        }
 
        for ($i = 0; $i < $this->timesRepeat && !$result->shouldStop(); $i++) {
            $this->test->run($result);
        }
 
        return $result;
    }
}
?>

The PHPUnit_Framework_Test interface is narrow and easy to implement. You can write an implementation of PHPUnit_Framework_Test that is simpler than PHPUnit_Framework_TestCase and that runs data-driven tests, for instance.

Example 21.2 shows a data-driven test-case class that compares values from a file with Comma-Separated Values (CSV). Each line of such a file looks like foo;bar, where the first value is the one we expect and the second value is the actual one.

<?php
require_once 'PHPUnit/Framework.php';
require_once 'PHPUnit/Util/Timer.php';
require_once 'PHPUnit/TextUI/TestRunner.php';
 
class DataDrivenTest implements PHPUnit_Framework_Test
{
    private $lines;
 
    public function __construct($dataFile)
    {
        $this->lines = file($dataFile);
    }
 
    public function count()
    {
        return 1;
    }
 
    public function run(PHPUnit_Framework_TestResult $result = NULL)
    {
        if ($result === NULL) {
            $result = new PHPUnit_Framework_TestResult;
        }
 
        foreach ($this->lines as $line) {
            $result->startTest($this);
            PHPUnit_Util_Timer::start();
 
            list($expected, $actual) = explode(';', $line);
 
            try {
                PHPUnit_Framework_Assert::assertEquals(trim($expected), trim($actual));
            }
 
            catch (PHPUnit_Framework_AssertionFailedError $e) {
                $result->addFailure($this, $e, PHPUnit_Util_Timer::stop());
            }
 
            catch (Exception $e) {
                $result->addError($this, $e, PHPUnit_Util_Timer::stop());
            }
 
            $result->endTest($this, PHPUnit_Util_Timer::stop());
        }
 
        return $result;
    }
}
 
$test = new DataDrivenTest('data_file.csv');
$result = PHPUnit_TextUI_TestRunner::run($test);
?>