05: Finding untested code

Now that we know how to create tests and debug them when they fail, let's make a useful test that actually covers some untested code.

Finding untested lines of code

The PHP gcov website shows what lines of C code are covered by the test suite.

We took a long tangent to talk about the PHP_FUNCTION macro as well as the two ways Zend parse parameters (ZPP) works: with the zend_parse_parameters() function and the multi-line macro.

Eventually we found some uncovered lines of code in ext/json/json.c for the json_decode() function that checked that value for the depth param was greater than 0.

Creating a new test

We made sure that we had the ext/json extension installed.

$ sapi/cli/php -m | grep json

We tried to create a new test.

$ vi ext/json/tests/json_decode_error.phpt

But quickly realized that there was already a test there so we created a new file variation.

$ vi ext/json/tests/json_decode_error001.phpt

Then we created our test.

--TEST--
json_decode() - depth error
--CREDITS--
Sammy Kaye Powers me at sammyk dot me
# TestFest Chicago PHP UG 2017-07-18
--SKIPIF--
<?php if (!extension_loaded('json')) die('skip ext/json required'); ?>
--FILE--
<?php
var_dump(json_decode('[]', false, 0));
?>
--EXPECTF--
Warning: json_decode(): Depth must be greater than zero in %s on line %d
NULL

We ran our test to see it pass with flying colors.

$ make test TESTS=ext/json/tests/json_decode_error001.phpt

At this point we could totally send our new test as a PR to the main php-src repo, but we wanted to see that this test actually covered the untested lines.

Generating a code coverage report

Since the PHP gcov website isn't updated regularly, we took Elizabeth Smith's advice and generated the code coverage reports locally.

First we have to install lcov.

$ sudo apt-get update
$ sudo apt-get install lcov

Then we can use the handy config.nice script to run configure again with all the previous flags in addition to any new ones. So we ran it with --enable-gcov since we already ran it with --enable-debug previously.

$ ./config.nice --enable-gcov

Next we had to delete all the previously compiled files with make clean so that everything could be recompiled with the appropriate flags that gcov needs.

$ make clean && make

Finally we were able to generate the lcov report but instead of running the entire test suite, we just wanted to run the ext/json tests.

$ make lcov TESTS=ext/json/tests/

That generated all the HTML output for gcov and we were able to open the HTML file at lcov_html/index.html. We browsed to the uncovered lines in json.c to see that they were now covered by our new test.

In our next video, we'll be submitting our new test to PHP source on GitHub.

Resources

Take part in PHP TestFest 2017
I'll be giving talks about writing tests for PHP source at NEPHP 2017 and ZendCon 2017. Join me! :)
The Docker setup I used in these screencasts

All posts in this series

If you found this guide helpful, say, "Hi" on twitter! I'd love to hear from you. :)

Finding untested code in PHP source :: Writing tests for PHP source

Finding untested lines of code

Creating a new test

Generating a code coverage report

Resources

All posts in this series

Share

Need more help?