05: Finding untested code

Finding untested code in PHP source :: Writing tests for PHP source

Jul 21, 2017

Now that we know how to create tests and debug them when they fail, let's make a useful test that actually covers some untested code.

Finding untested lines of code

The PHP gcov website shows what lines of C code are covered by the test suite.

We took a long tangent to talk about the PHP_FUNCTION macro as well as the two ways Zend parse parameters (ZPP) works: with the zend_parse_parameters() function and the multi-line macro.

Eventually we found some uncovered lines of code in ext/json/json.c for the json_decode() function that checked that value for the depth param was greater than 0.

Creating a new test

We made sure that we had the ext/json extension installed.

$ sapi/cli/php -m | grep json

We tried to create a new test.

$ vi ext/json/tests/json_decode_error.phpt

But quickly realized that there was already a test there so we created a new file variation.

$ vi ext/json/tests/json_decode_error001.phpt

Then we created our test.

json_decode() - depth error
Sammy Kaye Powers me at sammyk dot me
# TestFest Chicago PHP UG 2017-07-18
<?php if (!extension_loaded('json')) die('skip ext/json required'); ?>
var_dump(json_decode('[]', false, 0));
Warning: json_decode(): Depth must be greater than zero in %s on line %d

We ran our test to see it pass with flying colors.

$ make test TESTS=ext/json/tests/json_decode_error001.phpt

At this point we could totally send our new test as a PR to the main php-src repo, but we wanted to see that this test actually covered the untested lines.

Generating a code coverage report

Since the PHP gcov website isn't updated regularly, we took Elizabeth Smith's advice and generated the code coverage reports locally.

First we have to install lcov.

$ sudo apt-get update
$ sudo apt-get install lcov

Then we can use the handy config.nice script to run configure again with all the previous flags in addition to any new ones. So we ran it with --enable-gcov since we already ran it with --enable-debug previously.

$ ./config.nice --enable-gcov

Next we had to delete all the previously compiled files with make clean so that everything could be recompiled with the appropriate flags that gcov needs.

$ make clean && make

Finally we were able to generate the lcov report but instead of running the entire test suite, we just wanted to run the ext/json tests.

$ make lcov TESTS=ext/json/tests/

That generated all the HTML output for gcov and we were able to open the HTML file at lcov_html/index.html. We browsed to the uncovered lines in json.c to see that they were now covered by our new test.

In our next video, we'll be submitting our new test to PHP source on GitHub.


All posts in this series

If you found this guide helpful, say, "Hi" on twitter! I'd love to hear from you. :)