Now that we've compiled PHP from source, we need to learn how to run the tests using PHP's black-box testing tool called
Where the test files live
The test files live in various places throughout the PHP source code.
Zend/tests/Zend engine tests
tests/More Zend engine tests
Running a test as plain-old PHP
We looked at the basic test at
tests/basic/001.phpt and saw it was a normal PHP file so we ran it with our compiled binary.
$ sapi/cli/php tests/basic/001.phpt
Running the test suite
The two primary ways of running the test suites are 1) directly through
run-tests.php and 2) the make target
Running run-tests directly
We first invoked the help screen for run-tests.
$ sapi/cli/php run-tests.php --help
The run-tests tool won't run unless we specify the PHP binary that will run the tests. There are three primary ways of specifying a PHP binary.
- You can set the environment variable
$ export TEST_PHP_EXECUTABLE=`pwd`/sapi/cli/php $ sapi/cli/php run-tests.php
- You can set the
$ sapi/cli/php run-tests.php -p `pwd`/sapi/cli/php
- You can set the
$ sapi/cli/php run-tests.php -P
By default run-tests will run the entire test suite so we specified a single test by passing it as an argument.
$ sapi/cli/php run-tests.php -P tests/basic/001.phpt
You can also specify a directory or list other tests or directories.
$ sapi/cli/php run-tests.php -P tests/basic/ ext/standard/tests/random/
Running run-tests with make
The easiest way to run run-tests is with the make target
$ make test
We specify the tests we want to run with
$ make test TESTS=ext/standard/tests/random/ $ make test TESTS="ext/standard/tests/random/ ext/standard/tests/password"
After running the tests, run-tests will mark the test as one of the following statuses:
- Passed The test passed
- Failed The test failed
- Borked Invalid test file (this shouldn't happen out of the box)
Warned A warning was raised from the
--SKIPIF--section (more on this later)
Leaked A memory leak was detected (Valgrind must be enabled with the
- XFailed The test failed, but it was expected to (weird)
Slow Test took longer than timeout (defaults to 60 seconds or 300 seconds if testing for memory leaks; timeout can be customized with
- PHP Internals Book: Running the test suite
run-tests.phpsource on GitHub
Discussion around refactoring
- Take part in PHP TestFest 2017
- I'll be giving talks about writing tests for PHP source at NEPHP 2017 and ZendCon 2017. Join me! :)
- The Docker setup I used in these screencasts
All posts in this series
- 01: Compiling PHP from source
- 02: Running the test suite
03: All about
- 04: Debugging failed tests
- 05: Finding untested code
- 06: Submitting a PR to php-src