02: Running the test suite

Running the test suite with run-tests :: Writing tests for PHP source

Jul 13, 2017

Now that we've compiled PHP from source, we need to learn how to run the tests using PHP's black-box testing tool called run-tests.php.

Where the test files live

The test files live in various places throughout the PHP source code.

Running a test as plain-old PHP

We looked at the basic test at tests/basic/001.phpt and saw it was a normal PHP file so we ran it with our compiled binary.

$ sapi/cli/php tests/basic/001.phpt

Running the test suite

The two primary ways of running the test suites are 1) directly through run-tests.php and 2) the make target test.

Running run-tests directly

We first invoked the help screen for run-tests.

$ sapi/cli/php run-tests.php --help

The run-tests tool won't run unless we specify the PHP binary that will run the tests. There are three primary ways of specifying a PHP binary.

  1. You can set the environment variable TEST_PHP_EXECUTABLE.
$ export TEST_PHP_EXECUTABLE=`pwd`/sapi/cli/php
$ sapi/cli/php run-tests.php
  1. You can set the -p flag.
$ sapi/cli/php run-tests.php -p `pwd`/sapi/cli/php
  1. You can set the -P flag.
$ sapi/cli/php run-tests.php -P

By default run-tests will run the entire test suite so we specified a single test by passing it as an argument.

$ sapi/cli/php run-tests.php -P tests/basic/001.phpt

You can also specify a directory or list other tests or directories.

$ sapi/cli/php run-tests.php -P tests/basic/ ext/standard/tests/random/

Running run-tests with make

The easiest way to run run-tests is with the make target test.

$ make test

We specify the tests we want to run with TESTS.

$ make test TESTS=ext/standard/tests/random/
$ make test TESTS="ext/standard/tests/random/ ext/standard/tests/password"

Test statuses

After running the tests, run-tests will mark the test as one of the following statuses:


All posts in this series

If you found this guide helpful, say, "Hi" on twitter! I'd love to hear from you. :)