Chapter 15. Logging

PHPUnit supports the logging of test results in several formats.

XML Format

The XML format supported by PHPUnit is loosely based upon the one used by the JUnit task for Apache Ant. The following example shows the XML logfile generated for the tests in ArrayTest:

<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
  <testsuite name="ArrayTest"
             file="/home/sb/ArrayTest.php"
             tests="2"
             failures="0"
             errors="0"
             time="0.016030">
    <testcase name="testNewArrayIsEmpty"
              class="ArrayTest"
              file="/home/sb/ArrayTest.php"
              line="6"
              time="0.008044"/>
    <testcase name="testArrayContainsAnElement"
              class="ArrayTest"
              file="/home/sb/ArrayTest.php"
              line="15"
              time="0.007986"/>
  </testsuite>
</testsuites>

The following XML logfile was generated for two tests, testFailure and testError, of a test-case class named FailureErrorTest and shows how failures and errors are denoted.

<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
  <testsuite name="FailureErrorTest"
             file="/home/sb/FailureErrorTest.php"
             tests="2"
             failures="1"
             errors="1"
             time="0.019744">
    <testcase name="testFailure"
              class="FailureErrorTest"
              file="/home/sb/FailureErrorTest.php"
              line="6"
              time="0.011456">
      <failure type="PHPUnit_Framework_ExpectationFailedException">
testFailure(FailureErrorTest)
Failed asserting that &lt;integer:2&gt; matches expected value &lt;integer:1&gt;.

/home/sb/FailureErrorTest.php:8
</failure>
    </testcase>
    <testcase name="testError"
              class="FailureErrorTest"
              file="/home/sb/FailureErrorTest.php"
              line="11"
              time="0.008288">
      <error type="Exception">testError(FailureErrorTest)
Exception: 

/home/sb/FailureErrorTest.php:13
</error>
    </testcase>
  </testsuite>
</testsuites>

Code Coverage (XML)

The XML format for code coverage information logging supported by PHPUnit is loosely based upon the one used by Clover. The following example shows the XML logfile generated for the tests in BankAccountTest:

<?xml version="1.0" encoding="UTF-8"?>
<coverage generated="1184835473" phpunit="3.1.9">
  <project name="BankAccountTest" timestamp="1184835473">
    <file name="/home/sb/BankAccount.php">
      <class name="BankAccountException">
        <metrics methods="0" coveredmethods="0" statements="0"
                 coveredstatements="0" elements="0" coveredelements="0"/>
      </class>
      <class name="BankAccount">
        <metrics methods="4" coveredmethods="4" statements="13"
                 coveredstatements="5" elements="17" coveredelements="9"/>
      </class>
      <line num="77" type="method" count="3"/>
      <line num="79" type="stmt" count="3"/>
      <line num="89" type="method" count="2"/>
      <line num="91" type="stmt" count="2"/>
      <line num="92" type="stmt" count="0"/>
      <line num="93" type="stmt" count="0"/>
      <line num="94" type="stmt" count="2"/>
      <line num="96" type="stmt" count="0"/>
      <line num="105" type="method" count="1"/>
      <line num="107" type="stmt" count="1"/>
      <line num="109" type="stmt" count="0"/>
      <line num="119" type="method" count="1"/>
      <line num="121" type="stmt" count="1"/>
      <line num="123" type="stmt" count="0"/>
      <metrics loc="126" ncloc="37" classes="2" methods="4" coveredmethods="4"
               statements="13" coveredstatements="5" elements="17"
               coveredelements="9"/>
    </file>
    <metrics files="1" loc="126" ncloc="37" classes="2" methods="4"
             coveredmethods="4" statements="13" coveredstatements="5"
             elements="17" coveredelements="9"/>
  </project>
</coverage>

JavaScript Object Notation (JSON)

The JavaScript Object Notation (JSON) is a lightweight data-interchange format. The following example shows the JSON messages generated for the tests in ArrayTest:

{"event":"suiteStart","suite":"ArrayTest","tests":2}
{"event":"test","suite":"ArrayTest",
 "test":"testNewArrayIsEmpty(ArrayTest)","status":"pass",
 "time":0.000460147858,"trace":[],"message":""}
{"event":"test","suite":"ArrayTest",
 "test":"testArrayContainsAnElement(ArrayTest)","status":"pass",
 "time":0.000422954559,"trace":[],"message":""}

The following JSON messages were generated for two tests, testFailure and testError, of a test-case class named FailureErrorTest and show how failures and errors are denoted.

{"event":"suiteStart","suite":"FailureErrorTest","tests":2}
{"event":"test","suite":"FailureErrorTest",
 "test":"testFailure(FailureErrorTest)","status":"fail",
 "time":0.000483989716,"trace":[],"message":""}
{"event":"test","suite":"FailureErrorTest",
 "test":"testError(FailureErrorTest)","status":"error",
 "time":0.000466108322,"trace":[],"message":""}

Test Anything Protocol (TAP)

The Test Anything Protocol (TAP) is Perl's simple text-based interface between testing modules. The following example shows the TAP logfile generated for the tests in ArrayTest:

1..2
# TestSuite "ArrayTest" started.
ok 1 - testNewArrayIsEmpty(ArrayTest)
ok 2 - testArrayContainsAnElement(ArrayTest)
# TestSuite "ArrayTest" ended.

The following TAP logfile was generated for two tests, testFailure and testError, of a test-case class named FailureErrorTest and shows how failures and errors are denoted.

1..2
# TestSuite "FailureErrorTest" started.
not ok 1 - Failure: testFailure(FailureErrorTest)
not ok 2 - Error: testError(FailureErrorTest)
# TestSuite "FailureErrorTest" ended.

GraphViz Markup

PHPUnit can generate a description of the test result as a graph that can be rendered to diagrams in several useful formats such as images using the GraphViz tools.

phpunit --log-graphviz BankAccount.dot BankAccountTest
PHPUnit 3.1.9 by Sebastian Bergmann.

...

Time: 0 seconds

OK (3 tests)

The following example shows the GraphViz markup generated (and saved to the BankAccount.dot file in the current directory) for the tests in BankAccountTest:

digraph G {
graph [ overlap="scale",splines="true",sep=".1",fontsize="8" ];
"BankAccountTest" [ color="green" ];
subgraph "cluster_BankAccountTest" {
label="";
"testBalanceIsInitiallyZero" [ color="green" ];
"testBalanceCannotBecomeNegative" [ color="green" ];
"testBalanceCannotBecomeNegative2" [ color="green" ];
}
"BankAccountTest" -> "testBalanceIsInitiallyZero";
"BankAccountTest" -> "testBalanceCannotBecomeNegative";
"BankAccountTest" -> "testBalanceCannotBecomeNegative2";
}

We can now use the dot command-line tool that is part of the GraphViz software suite to generate a graphic from this markup:

dot -T png -o BankAccount.png BankAccount.dot

Figure 15.1 shows the graph representation of the test result rendered from the GraphViz markup above.

Figure 15.1. Graph representation of the test result

Successful tests are displayed with a green border, failures and errors with a red border, and incomplete or skipped tests with a yellow border. A parent node, such as test suite, is displayed with a non-green border if it has a child node, such as a test, that was not successful.

Test Database

PHPUnit can write test result and code coverage data to a test database. Several ideas for future features depend on this data.

For each run of the test suite there is a row in the run table.
For each test that is executed as part of a test run there is a row in the test table.
For each file that is part of a revision there is a row in the code_file table.
For each class that is declared in a file that is part of a revision there is a row in the code_class table.
For each method that is declared in a file that is part of a revision there is a row in the code_method table.
For each line of code that belongs to a file that is part of a revision there is a row in the code_line table.
The code_coverage table connects each test to the lines of code it covers.

Before we can write test result and code coverage data to a database, we need to create a database using one of the supplied schema definitions:

sqlite3 BankAccount.db < PHPUnit/Util/Log/Database/SQLite3.sql

Now we can execute a test suite and write the test result and code coverage data to a database:

phpunit --test-db-dsn sqlite:///home/sb/BankAccount.db --test-db-log-rev 1 BankAccountTest
PHPUnit 3.1.9 by Sebastian Bergmann.

...

Time: 0 seconds


OK (3 tests)

Storing code coverage data in database, this may take a moment.

Table 15.1 shows the arguments understood by the TextUI test runner (see Chapter 5) for the test database.

Table 15.1. TextUI Arguments for the Test Database

Argument	Meaning
`--test-db-dsn <dsn>`	The PDO data source name (DSN) for the database connection. In general, a DSN consists of the PDO driver name, followed by a colon, followed by the PDO driver-specific connection syntax.
`--test-db-log-rev <r>`	A number, for instance a Subversion Global Revision Number, that uniquely identifies the current revision of the codebase.
`--test-db-log-info ...`	Additional information on the test environment, for instance.