Prev Next

Chapter 1. Automating Tests

Even good programmers make mistakes. The difference between a good programmer and a bad programmer is that the good programmer uses tests to detect his mistakes as soon as possible. The sooner you test for a mistake the greater your chance of finding it and the less it will cost to find and fix. This explains why leaving testing until just before releasing software is so problematic. Most errors do not get caught at all, and the cost of fixing the ones you do catch is so high that you have to perform triage with the errors because you just cannot afford to fix them all.

Testing with PHPUnit is not a totally different activity from what you should already be doing. It is just a different way of doing it. The difference is between testing, that is, checking that your program behaves as expected, and performing a battery of tests, runnable code-fragments that automatically test the correctness of parts (units) of the software. These runnable code-fragments are called unit tests.

In this chapter we will go from simple print-based testing code to a fully automated test. Imagine that we have been asked to test PHP's built-in array. One bit of functionality to test is the function count(). For a newly created array we expect the count() function to return 0. After we add an element, count() should return 1. Example 1.1 shows what we want to test.

Example 1.1: Testing array operations

<?php
$fixture = array();
// $fixture is expected to be empty.

$fixture[] = 'element';
// $fixture is expected to contain one element.
?>

A really simple way to check whether we are getting the results we expect is to print the result of count() before and after adding the element (see Example 1.2). If we get 0 and then 1, array and count() behave as expected.

Example 1.2: Using print to test array operations

<?php
$fixture = array();
print count($fixture) . "\n";

$fixture[] = 'element';
print count($fixture) . "\n";
?>
0
1

Now, we would like to move from tests that require manual interpretation to tests that can run automatically. In Example 1.3, we write the comparison of the expected and actual values into the test code and print ok if the values are equal. If we ever see a not ok message, we know something is wrong.

Example 1.3: Comparing expected and actual values to test array operations

<?php
$fixture = array();
print count($fixture) == 0 ? "ok\n" : "not ok\n";

$fixture[] = 'element';
print count($fixture) == 1 ? "ok\n" : "not ok\n";
?>
ok
ok

We now factor out the comparison of expected and actual values into a function that raises an Exception when there is a discrepancy (Example 1.4). This gives us two benefits: the writing of tests becomes easier and we only get output when something is wrong.

Example 1.4: Using an assertion function to test array operations

<?php
$fixture = array();
assertTrue(count($fixture) == 0);

$fixture[] = 'element';
assertTrue(count($fixture) == 1);

function assertTrue($condition)
{
if (!$condition) {
throw new Exception('Assertion failed.');
}
}
?>

The test is now completely automated. Instead of just testing as we did with our first version, with this version we have an automated test.

The goal of using automated tests is to make fewer mistakes. While your code will still not be perfect, even with excellent tests, you will likely see a dramatic reduction in defects once you start automating tests. Automated tests give you justified confidence in your code. You can use this confidence to take more daring leaps in design (Refactoring), get along with your teammates better (Cross-Team Tests), improve relations with your customers, and go home every night with proof that the system is better now than it was this morning because of your efforts.

Prev Next
1. Automating Tests
2. PHPUnit's Goals
3. Installing PHPUnit
4. Writing Tests for PHPUnit
Test Dependencies
Data Providers
Testing Exceptions
Testing PHP Errors
Testing Output
Assertions
assertArrayHasKey()
assertClassHasAttribute()
assertClassHasStaticAttribute()
assertContains()
assertContainsOnly()
assertCount()
assertEmpty()
assertEqualXMLStructure()
assertEquals()
assertFalse()
assertFileEquals()
assertFileExists()
assertGreaterThan()
assertGreaterThanOrEqual()
assertInstanceOf()
assertInternalType()
assertLessThan()
assertLessThanOrEqual()
assertNull()
assertObjectHasAttribute()
assertRegExp()
assertStringMatchesFormat()
assertStringMatchesFormatFile()
assertSame()
assertSelectCount()
assertSelectEquals()
assertSelectRegExp()
assertStringEndsWith()
assertStringEqualsFile()
assertStringStartsWith()
assertTag()
assertThat()
assertTrue()
assertXmlFileEqualsXmlFile()
assertXmlStringEqualsXmlFile()
assertXmlStringEqualsXmlString()
5. The Command-Line Test Runner
Command-Line switches
6. Fixtures
More setUp() than tearDown()
Variations
Sharing Fixture
Global State
7. Organizing Tests
Composing a Test Suite Using the Filesystem
Composing a Test Suite Using XML Configuration
8. Database Testing
Supported Vendors for Database Testing
Difficulties in Database Testing
The four stages of a database test
1. Clean-Up Database
2. Set up fixture
3–5. Run Test, Verify outcome and Teardown
Configuration of a PHPUnit Database TestCase
Implementing getConnection()
Implementing getDataSet()
What about the Datatabase Schema (DDL)?
Tip: Use your own Abstract Database TestCase
Understanding DataSets and DataTables
Available Implementations
Beware of Foreign Keys
Implementing your own DataSets/DataTables
The Connection API
Database Assertions API
Asserting the Row-Count of a Table
Asserting the State of a Table
Asserting the Result of a Query
Asserting the State of Multiple Tables
Frequently Asked Questions
Will PHPUnit (re-)create the database schema for each test?
Am I required to use PDO in my application for the Database Extension to work?
What can I do, when I get a Too much Connections Error?
How to handle NULL with Flat XML / CSV Datasets?
9. Incomplete and Skipped Tests
Incomplete Tests
Skipping Tests
10. Test Doubles
Stubs
Mock Objects
Stubbing and Mocking Web Services
Mocking the Filesystem
11. Testing Practices
During Development
During Debugging
12. Test-Driven Development
BankAccount Example
13. Behaviour-Driven Development
BowlingGame Example
14. Code Coverage Analysis
Specifying Covered Methods
Ignoring Code Blocks
Including and Excluding Files
Edge cases
15. Other Uses for Tests
Agile Documentation
Cross-Team Tests
16. Skeleton Generator
Generating a Test Case Class Skeleton
Generating a Class Skeleton from a Test Case Class
17. PHPUnit and Selenium
Selenium Server
PHPUnit_Extensions_SeleniumTestCase
18. Logging
Test Results (XML)
Test Results (TAP)
Test Results (JSON)
Code Coverage (XML)
Code Coverage (TEXT)
19. Extending PHPUnit
Subclass PHPUnit_Framework_TestCase
Write custom assertions
Implement PHPUnit_Framework_TestListener
Subclass PHPUnit_Extensions_TestDecorator
Implement PHPUnit_Framework_Test
A. Assertions
B. Annotations
@assert
@author
@backupGlobals
@backupStaticAttributes
@codeCoverageIgnore*
@covers
@dataProvider
@depends
@expectedException
@expectedExceptionCode
@expectedExceptionMessage
@group
@outputBuffering
@runTestsInSeparateProcesses
@runInSeparateProcess
@test
@testdox
@ticket
C. The XML Configuration File
PHPUnit
Test Suites
Groups
Including and Excluding Files for Code Coverage
Logging
Test Listeners
Setting PHP INI settings, Constants and Global Variables
Configuring Browsers for Selenium RC
D. Index
E. Bibliography
F. Copyright