Now that we know how to create tests and debug them when they fail, let's make a useful test that actually covers some untested code.
Finding untested lines of code
The PHP gcov website shows what lines of C code are covered by the test suite.
We took a long tangent to talk about the
PHP_FUNCTION macro as well as the two ways Zend parse parameters (ZPP) works: with the
zend_parse_parameters() function and the multi-line macro.
Eventually we found some uncovered lines of code in
ext/json/json.c for the
json_decode() function that checked that value for the
depth param was greater than 0.
Creating a new test
We made sure that we had the
ext/json extension installed.
$ sapi/cli/php -m | grep json
We tried to create a new test.
$ vi ext/json/tests/json_decode_error.phpt
But quickly realized that there was already a test there so we created a new file variation.
$ vi ext/json/tests/json_decode_error001.phpt
Then we created our test.
--TEST-- json_decode() - depth error --CREDITS-- Sammy Kaye Powers me at sammyk dot me # TestFest Chicago PHP UG 2017-07-18 --SKIPIF-- <?php if (!extension_loaded('json')) die('skip ext/json required'); ?> --FILE-- <?php var_dump(json_decode('', false, 0)); ?> --EXPECTF-- Warning: json_decode(): Depth must be greater than zero in %s on line %d NULL
We ran our test to see it pass with flying colors.
$ make test TESTS=ext/json/tests/json_decode_error001.phpt
At this point we could totally send our new test as a PR to the main php-src repo, but we wanted to see that this test actually covered the untested lines.
Generating a code coverage report
Since the PHP gcov website isn't updated regularly, we took Elizabeth Smith's advice and generated the code coverage reports locally.
First we have to install lcov.
$ sudo apt-get update $ sudo apt-get install lcov
Then we can use the handy
config.nice script to run configure again with all the previous flags in addition to any new ones. So we ran it with
--enable-gcov since we already ran it with
$ ./config.nice --enable-gcov
Next we had to delete all the previously compiled files with
make clean so that everything could be recompiled with the appropriate flags that gcov needs.
$ make clean && make
Finally we were able to generate the lcov report but instead of running the entire test suite, we just wanted to run the
$ make lcov TESTS=ext/json/tests/
That generated all the HTML output for gcov and we were able to open the HTML file at
lcov_html/index.html. We browsed to the uncovered lines in
json.c to see that they were now covered by our new test.
In our next video, we'll be submitting our new test to PHP source on GitHub.
- Take part in PHP TestFest 2017
- I'll be giving talks about writing tests for PHP source at NEPHP 2017 and ZendCon 2017. Join me! :)
- The Docker setup I used in these screencasts
All posts in this series
- 01: Compiling PHP from source
- 02: Running the test suite
03: All about
- 04: Debugging failed tests
- 05: Finding untested code
- 06: Submitting a PR to php-src