Actions

Difference between revisions of "Unit Testing -- Data Driven Example"

From Joomla! Documentation

m
m (added Testing and Bug Squad categories)
 
(7 intermediate revisions by 3 users not shown)
Line 1: Line 1:
 
== Anatomy of a Data-Driven Test ==
 
== Anatomy of a Data-Driven Test ==
This expands on the basics given in (add link). In this example, we're interested in running a number of tests on the JRequest::getVar() method, each with different parameters. While it's possible to just write one test and then copy it once for each test, this would create a lot of duplicate code, add a lot of needless lines to the test file, and make it harder to organize test cases.
+
This expands on the basics given in the [[Unit_Testing_--_a_Simple_Example|Simple Example]]. In this example, we're interested in running a number of tests on the JRequest::getVar() method, each with different parameters. While it's possible to just write one test and then copy it once for each test, this would create a lot of duplicate code, add a lot of needless lines to the test file, and make it harder to organize test cases.
  
 
We also need to introduce a mock class with this example. Let's dig into the test to see why.
 
We also need to introduce a mock class with this example. Let's dig into the test to see why.
  
 
=== Defining the Test Criteria for JRequest::getVar() ===
 
=== Defining the Test Criteria for JRequest::getVar() ===
We'll start by getting an understanding of what actually needs to be tested when we say "test getVar". Looking at the code, we can learn a few things:
+
We'll start by getting an understanding of what actually needs to be tested when we say "test getVar()". Looking at the code, we can learn a few things:
* The getVar method uses the $hash parameter to determine the source of the variable. This means we should test all expected values for $hash and a few invalid ones as well.
+
* The getVar() method uses the $hash parameter to determine the source of the variable. This means we should test all expected values for $hash and a few invalid ones as well. We'll need to initialize the data sources too.
 
* The class maintains a cache in $GLOBALS['_JREQUEST']. This means we should check to ensure that the hash works as expected, and that we need to pay some attention to ensuring that the cache is in a known state when we run a test.
 
* The class maintains a cache in $GLOBALS['_JREQUEST']. This means we should check to ensure that the hash works as expected, and that we need to pay some attention to ensuring that the cache is in a known state when we run a test.
 
* getVar() calls the protected method _cleanVar() to do the real work. Depending on the value of the $mask parameter, this might result in a call to JFilterInput::clean(). We learn two things. First, we need to test various values for $mask (now in combination with $hash). Second, since testing JFilterInput is the responsibility of a unit test for that class, we will need to create a mock JFilterInput class, so we can isolate JRequest.
 
* getVar() calls the protected method _cleanVar() to do the real work. Depending on the value of the $mask parameter, this might result in a call to JFilterInput::clean(). We learn two things. First, we need to test various values for $mask (now in combination with $hash). Second, since testing JFilterInput is the responsibility of a unit test for that class, we will need to create a mock JFilterInput class, so we can isolate JRequest.
  
=== Building a Mock Class ===
+
=== Building the Mock Class ===
(ed note: this should be its own page)
+
The process of building a mock JFilterInput class for this test is discussed in detail in the [[Unit_Testing_Mock_Objects|Mock Objects in Joomla]] article. It's a good idea to have an understanding of this before proceeding.
 
+
==== Defining a Mock Class for JFilterInput ====
+
There are two ways to look at mock classes. If a class is pretty simple, it's possible to write a common class that can be used by a number of unit tests. However if the class is more complex, it's better to write a test-specific mock that only implements the subset of the class that's needed.
+
 
+
The JFilterInput class falls into the second category. Writing a good general purpose mock for it is too complicated and time consuming, so we will create a subset that only does what we need for our JRequest tests.
+
 
+
JRequest uses two methods in JFilterInput: getInstance() and clean().
+
 
+
The getInstance method is straightforward. It returns an instance a JFilterInput object. Since we aren't interested in the internals of JFilterInput, we have no need to pay attention to the parameters passed to getInstance() in our mock.
+
 
+
For clean(), we're interested in making sure that JRequest calls clean() with the expected number parameter values, and that clean is called the appropriate number of times (this will verify the cache mechanism). Our mock also needs to return the clean version of the parameters.
+
 
+
==== Implementing the JFilterInput Mock Class ====
+
This class follows a common pattern for mock objects. For each test, we define expectations. An expectation is a combination of the expected parameters and the number of calls to a mock method.
+
 
+
This is a simple case because we have only one method to set expectations for: clean(). The below is the complete code for our mock, which can be found in /unittest/tests/libraries/joomla/environment/JFilterInput-mock-general.php
+
  
 +
Once the mock is defined, we replace the real JInputFilter class by injecting a mock just after the unit test platform is loaded:
  
 
<source lang="php">
 
<source lang="php">
/**
+
/*
  * Mock of JFilterInput for JRequest testing
+
  * Mock classes
 
  */
 
  */
class JFilterInput
+
JLoader::injectMock(
{
+
    dirname(__FILE__) . DS . 'JFilterInput-mock-general.php',
/**
+
    'joomla.filter.filterinput'
* Information on the calls expected to the mock object.
+
);
*
+
* This array is indexed by a hash of the source and type; each element is
+
* an array containing the source, type, expected response and number of
+
* expected calls.
+
*/
+
static private $_expectations = array();
+
 
+
/**
+
* Returns a reference to a mock input filter singleton
+
*/
+
function & getInstance()
+
{
+
static $instance;
+
 
+
if (! $instance) {
+
$instance = new JFilterInput();
+
}
+
return $instance;
+
}
+
 
+
/**
+
* Stub for the clean method.
+
*
+
* @access  public
+
* @param  mixed  Input string/array-of-string to be 'cleaned'
+
* @param  string  Return type for the variable (INT, FLOAT, BOOLEAN, WORD,
+
* ALNUM, CMD, BASE64, STRING, ARRAY, PATH, NONE)
+
* @return  mixed  Canned response based on table lookup.
+
* @static
+
*/
+
function clean($source, $type='string')
+
{
+
$hash = md5($source . '|' . strtoupper($type));
+
if (! isset($this -> _expectations[$hash])) {
+
$this -> _expectations[$hash] = array(
+
'source' => $source,
+
'type' => $type,
+
'result' => null,
+
'count' => 0,
+
);
+
}
+
--$this -> _expectations[$hash]['count'];
+
return $this -> _expectations[$hash]['result'];
+
}
+
 
+
function mockReset() {
+
$this -> _expectations = array();
+
}
+
 
+
function mockSetUp($source, $type, $result, $count = 1) {
+
$hash = md5($source . '|' . strtoupper($type));
+
$this -> _expectations[$hash] = array(
+
'source' => $source,
+
'type' => $type,
+
'result' => $result,
+
'count' => $count
+
);
+
}
+
 
+
function mockTearDown() {
+
foreach ($this -> _expectations as $hash => $info) {
+
if (! $info['count']) {
+
unset($this -> _expectations[$hash]);
+
}
+
}
+
if (count($this -> _expectations)) {
+
return $this -> _expectations;
+
}
+
return true;
+
}
+
 
+
}
+
 
</source>
 
</source>
 
The mockSetup() method defines the expectations for the calls to clean() during a test. This is simple. For each combination of clean($source, $type), we specify the what clean should return ($result) and the number of times the method should be called with those parameters ($count).
 
 
The mockTearDown() method is called at the end of the test. If an expectation has been met, the value of count will be zero. If our test ran exactly as expected, then all expectations should have a zero count. Expectations with a zero count are removed from the array, and if the resulting array is empty then the test passed. If there are non-zero values then either the method wasn't called as many times as expected (a count greater than zero), or it was called with parameters we didn't anticipate (a negative count). mockTearDown() either returns true on success, or returns a list of mismatched expectations.
 
 
Of the two methods we need to mock, getInstance() simply returns a new instance of our mock. The clean() method matches the input parameters with expectations, either decrementing the count for parameters we expected, or creating a record of parameters we didn't expect.
 
 
That's all we need to support our test for JRequest. Let's put it to use.
 
 
Sample general-purpose mock: /unittest/tests/libraries/joomla/plugin/JPluginHelper-mock-general.php
 
  
 
=== A Unit Test for JRequest::getVar() ===
 
=== A Unit Test for JRequest::getVar() ===
 
 
The core of our unit test is simple. For each combination of inputs we want to do this:
 
The core of our unit test is simple. For each combination of inputs we want to do this:
  
 
<source lang="php">
 
<source lang="php">
/*
+
//
* Get the variable and check the value.
+
        /*
*/
+
        * Get the variable and check the value.
$actual = JRequest::getVar($name, $default, $hash, $type, $mask);
+
        */
$this -> assertEquals($expect, $actual, 'Non-cached getVar');
+
        $actual = JRequest::getVar($name, $default, $hash, $type, $mask);
/*
+
        $this -> assertEquals($expect, $actual, 'Non-cached getVar');
* Repeat the process to check caching (JFilterInput should not
+
        /*
* get called unless the default is being used).
+
        * Repeat the process to check caching (JFilterInput should not
*/
+
        * get called unless the default is being used).
$actual = JRequest::getVar($name, $default, $hash, $type, $mask);
+
        */
$this -> assertEquals($expect, $actual, 'Cached getVar');
+
        $actual = JRequest::getVar($name, $default, $hash, $type, $mask);
 +
        $this -> assertEquals($expect, $actual, 'Cached getVar');
 
</source>
 
</source>
  
Line 147: Line 50:
 
In our unit test class, we'll define a setUp() method that calls the helper and clears the cache:
 
In our unit test class, we'll define a setUp() method that calls the helper and clears the cache:
 
<source lang="php">
 
<source lang="php">
function setUp() {
+
//
JRequestTest_DataSet::initSuperGlobals();
+
    function setUp() {
// Make sure the request hash is clean.
+
        JRequestTest_DataSet::initSuperGlobals();
$GLOBALS['_JREQUEST'] = array();
+
        // Make sure the request hash is clean.
}
+
        $GLOBALS['_JREQUEST'] = array();
 +
    }
 
</source>
 
</source>
 
This code is called by PHPUnit at the beginning of every test.
 
This code is called by PHPUnit at the beginning of every test.
Line 163: Line 67:
 
We define each of these tests in an array in our helper class, $getVarTests. Here is a sample entry:
 
We define each of these tests in an array in our helper class, $getVarTests. Here is a sample entry:
 
<source lang="php">
 
<source lang="php">
array(
+
//
'tag',  null,      'request',  'none', 0, 'from _REQUEST',
+
        array(
array(
+
            'tag',  null,      'request',  'none', 0, 'from _REQUEST',
array('from _REQUEST', 'NONE', 'from _REQUEST', 1),
+
            array(
),
+
                array('from _REQUEST', 'NONE', 'from _REQUEST', 1),
),
+
            ),
 +
        ),
 
</source>
 
</source>
 
In this entry, we're passing $name = 'tag', $default = null, $hash = 'request', $type = 'none', and $mask = 0.
 
In this entry, we're passing $name = 'tag', $default = null, $hash = 'request', $type = 'none', and $mask = 0.
Line 176: Line 81:
 
We also expect that this test will result in a single call to JInputFilter::clean() with $source = 'from _REQUEST' and $type = 'NONE'. Our mock of this method will return the string 'from _REQUEST', and we expect this method to be called with these parameters precisely once.
 
We also expect that this test will result in a single call to JInputFilter::clean() with $source = 'from _REQUEST' and $type = 'NONE'. Our mock of this method will return the string 'from _REQUEST', and we expect this method to be called with these parameters precisely once.
  
You can find the whole data set in /unittest/tests/libraries/joomla/environment/JRequest-helper-dataset.php.  
+
You can find the whole data set in /unittest/tests/libraries/joomla/environment/JRequest-helper-dataset.php. When you look at the full test set, you will see that calls where we are looking for the default variable are expected to be made twice. This is because the default is not cached. Our expectations will determine if the cache mechanism is functioning as expected.
  
 
Some readers may be surprised when they don't find tests to verify that strings like '123abc' are not numeric. There are no tests like this because although this is the most common use of getVar(), the actual filtering is done by the JFilterInput class, so test cases like that belong with the JFilterInput tests. Our tests merely need to verify that the data is coming from the right place.
 
Some readers may be surprised when they don't find tests to verify that strings like '123abc' are not numeric. There are no tests like this because although this is the most common use of getVar(), the actual filtering is done by the JFilterInput class, so test cases like that belong with the JFilterInput tests. Our tests merely need to verify that the data is coming from the right place.
 +
 +
'''Specifying Version Restrictions'''
 +
 +
The test facility provides a standard mechanism for masking tests by version. You can add version restrictions to the data set as follows: "jver_min" specifies the minimum version; "jver_max" specifies the maximum version; and "jver_below" specifies defines an upper limit to the version.
 +
 +
Examples:
 +
<source lang="php">
 +
'jver_below' => '1.5.0' // Specifies any test in the 1.x series before the 1.5.0 release.
 +
'jver_max' => '1.5.4'  // Specifies a test that will be disabled in versions 1.5.5 and above.
 +
'jver_min' => '1.6.0'  // Specifies a test that will only be active on version 1.6.0 and above.
 +
</source>
 +
 +
For data-driven tests, these values will be stored as part of the data set. In your tests, use JUnit_Setup::isTestEnabled(JVERSION, $range) to determine if the test should be run.
 +
 +
==== PHPUnit's Data Provider Mechanism ====
 +
PHPUnit has a very simple way to link a data set to a test. Here is an example:
 +
<source lang="php">
 +
//
 +
    /**
 +
    * @dataProvider myDataSet
 +
    */
 +
    function testExample($testVal) {
 +
        // test code
 +
    }
 +
 +
    static function myDataSet() {
 +
        return array(array(1), array(2), array(5), array(200));
 +
    }
 +
</source>
 +
This code will result in the testExample() method being called four times. The first time the value of $testVal will be 1, the second time it will be 2, and the fourth time it will be 200. PHPUnit recognizes the @dataProvider pragma in the doc block, calls the method named in the pragma to get the data set, and then calls the test method once for each element in the array, passing each element of the sub-array as a parameter to the test method. if the test method took two string parameters, the data set would look like array(array('p1', 'p2'), array('param1', 'param2'), ...).
 +
 +
PHPUnit will consider each of these calls to be a separate test, and if a test fails, it will output the current value of the paremeters used to call the method.
 +
 +
==== Our Data Provider ====
 +
Here is the code for the data provider we use in the test. We simply return the $getVarTests array from our helper class.
 +
<source lang="php">
 +
//
 +
    static public function getVarData() {
 +
        return JRequestTest_DataSet::$getVarTests;
 +
    }
 +
</source>
 +
 +
==== Putting It All Together ====
 +
Finally here is our test case with all the elements in place:
 +
<source lang="php">
 +
//
 +
    /**
 +
    * @dataProvider getVarData
 +
    */
 +
    function testGetVarFromDataSet(
 +
        $name, $default, $hash, $type, $mask, $expect, $filterCalls
 +
    ) {
 +
        $filter = JFilterInput::getInstance();
 +
        $filter -> mockReset();
 +
        if (count($filterCalls)) {
 +
            foreach ($filterCalls as $info) {
 +
                $filter -> mockSetUp(
 +
                    $info[0], $info[1], $info[2], $info[3]
 +
                );
 +
            }
 +
        }
 +
        /*
 +
        * Get the variable and check the value.
 +
        */
 +
        $actual = JRequest::getVar($name, $default, $hash, $type, $mask);
 +
        $this -> assertEquals($expect, $actual, 'Non-cached getVar');
 +
        /*
 +
        * Repeat the process to check caching (the JFilterInput mock should not
 +
        * get called unless the default is being used).
 +
        */
 +
        $actual = JRequest::getVar($name, $default, $hash, $type, $mask);
 +
        $this -> assertEquals($expect, $actual, 'Cached getVar');
 +
        if (($filterOK = $filter -> mockTearDown()) !== true) {
 +
            $this -> fail(
 +
                'JFilterInput not called as expected:'
 +
                . print_r($filterOK, true)
 +
            );
 +
        }
 +
    }
 +
</source>
 +
Each element in the sample data set is passed in as a parameter to our test.
 +
 +
The setUp() method has initialized the super-globals already. We start by instantiating a mock JFilterInput object, resetting it, and then defining our expectations for it by calling mockSetUp() for each element in the $filterCalls array.
 +
 +
Don't forget to add a require_once above your class definition to include your helper class.
 +
 +
Then we run the core of our test, calling once with the cache empty and one where there may be a cached value.
 +
 +
Finally, we call mockTearDown(). If this method returns true, then the mock was used as we expected. If not, we dump the details of the discrepancies while failing the test.
 +
 +
Return to [[Unit_Testing|Unit Testing]]
 +
 +
[[Category:Testing]][[Category:Bug Squad]]

Latest revision as of 14:52, 15 December 2011

Contents

Anatomy of a Data-Driven Test

This expands on the basics given in the Simple Example. In this example, we're interested in running a number of tests on the JRequest::getVar() method, each with different parameters. While it's possible to just write one test and then copy it once for each test, this would create a lot of duplicate code, add a lot of needless lines to the test file, and make it harder to organize test cases.

We also need to introduce a mock class with this example. Let's dig into the test to see why.

Defining the Test Criteria for JRequest::getVar()

We'll start by getting an understanding of what actually needs to be tested when we say "test getVar()". Looking at the code, we can learn a few things:

  • The getVar() method uses the $hash parameter to determine the source of the variable. This means we should test all expected values for $hash and a few invalid ones as well. We'll need to initialize the data sources too.
  • The class maintains a cache in $GLOBALS['_JREQUEST']. This means we should check to ensure that the hash works as expected, and that we need to pay some attention to ensuring that the cache is in a known state when we run a test.
  • getVar() calls the protected method _cleanVar() to do the real work. Depending on the value of the $mask parameter, this might result in a call to JFilterInput::clean(). We learn two things. First, we need to test various values for $mask (now in combination with $hash). Second, since testing JFilterInput is the responsibility of a unit test for that class, we will need to create a mock JFilterInput class, so we can isolate JRequest.

Building the Mock Class

The process of building a mock JFilterInput class for this test is discussed in detail in the Mock Objects in Joomla article. It's a good idea to have an understanding of this before proceeding.

Once the mock is defined, we replace the real JInputFilter class by injecting a mock just after the unit test platform is loaded:

/*
 * Mock classes
 */
JLoader::injectMock(
    dirname(__FILE__) . DS . 'JFilterInput-mock-general.php',
    'joomla.filter.filterinput'
);

A Unit Test for JRequest::getVar()

The core of our unit test is simple. For each combination of inputs we want to do this:

//
        /*
         * Get the variable and check the value.
         */
        $actual = JRequest::getVar($name, $default, $hash, $type, $mask);
        $this -> assertEquals($expect, $actual, 'Non-cached getVar');
        /*
         * Repeat the process to check caching (JFilterInput should not
         * get called unless the default is being used).
         */
        $actual = JRequest::getVar($name, $default, $hash, $type, $mask);
        $this -> assertEquals($expect, $actual, 'Cached getVar');

We check that we get the expected result, and we check that the cache works.

Defining the Test Environment

JRequest gets data from the PHP super-globals ($_GET, $_POST, etc.). We're going to define a helper class that both initializes these variables and holds our data set of tests (see /unittest/tests/libraries/joomla/environment/JRequest-helper-dataset.php). The initSuperGlobals() method will set up the environment. One of the things we do is define an entry called 'tag' in each of the super-globals, each with a value that identifies the source variable.

In our unit test class, we'll define a setUp() method that calls the helper and clears the cache:

//
    function setUp() {
        JRequestTest_DataSet::initSuperGlobals();
        // Make sure the request hash is clean.
        $GLOBALS['_JREQUEST'] = array();
    }

This code is called by PHPUnit at the beginning of every test.

Defining a Data Set

What we know about our data set is this:

  • We need to test many combinations of the parameters $name, $default, $hash, $type, and $mask.
  • For each of these combinations, getVar() should return a single expected result.
  • For some of these combinations, JRequest will use JInputFilter. In these cases we need to define the expected calls to our mock class, and the result.

We define each of these tests in an array in our helper class, $getVarTests. Here is a sample entry:

//
        array(
            'tag',  null,       'request',  'none', 0, 'from _REQUEST',
            array(
                array('from _REQUEST', 'NONE', 'from _REQUEST', 1),
            ),
        ),

In this entry, we're passing $name = 'tag', $default = null, $hash = 'request', $type = 'none', and $mask = 0.

We expect the return value for this combination of parameters to be the string 'from _REQUEST'.

We also expect that this test will result in a single call to JInputFilter::clean() with $source = 'from _REQUEST' and $type = 'NONE'. Our mock of this method will return the string 'from _REQUEST', and we expect this method to be called with these parameters precisely once.

You can find the whole data set in /unittest/tests/libraries/joomla/environment/JRequest-helper-dataset.php. When you look at the full test set, you will see that calls where we are looking for the default variable are expected to be made twice. This is because the default is not cached. Our expectations will determine if the cache mechanism is functioning as expected.

Some readers may be surprised when they don't find tests to verify that strings like '123abc' are not numeric. There are no tests like this because although this is the most common use of getVar(), the actual filtering is done by the JFilterInput class, so test cases like that belong with the JFilterInput tests. Our tests merely need to verify that the data is coming from the right place.

Specifying Version Restrictions

The test facility provides a standard mechanism for masking tests by version. You can add version restrictions to the data set as follows: "jver_min" specifies the minimum version; "jver_max" specifies the maximum version; and "jver_below" specifies defines an upper limit to the version.

Examples:

'jver_below' => '1.5.0' // Specifies any test in the 1.x series before the 1.5.0 release.
'jver_max' => '1.5.4'   // Specifies a test that will be disabled in versions 1.5.5 and above.
'jver_min' => '1.6.0'   // Specifies a test that will only be active on version 1.6.0 and above.

For data-driven tests, these values will be stored as part of the data set. In your tests, use JUnit_Setup::isTestEnabled(JVERSION, $range) to determine if the test should be run.

PHPUnit's Data Provider Mechanism

PHPUnit has a very simple way to link a data set to a test. Here is an example:

//
    /**
     * @dataProvider myDataSet
     */
    function testExample($testVal) {
        // test code
    }
 
    static function myDataSet() {
        return array(array(1), array(2), array(5), array(200));
    }

This code will result in the testExample() method being called four times. The first time the value of $testVal will be 1, the second time it will be 2, and the fourth time it will be 200. PHPUnit recognizes the @dataProvider pragma in the doc block, calls the method named in the pragma to get the data set, and then calls the test method once for each element in the array, passing each element of the sub-array as a parameter to the test method. if the test method took two string parameters, the data set would look like array(array('p1', 'p2'), array('param1', 'param2'), ...).

PHPUnit will consider each of these calls to be a separate test, and if a test fails, it will output the current value of the paremeters used to call the method.

Our Data Provider

Here is the code for the data provider we use in the test. We simply return the $getVarTests array from our helper class.

//
    static public function getVarData() {
        return JRequestTest_DataSet::$getVarTests;
    }

Putting It All Together

Finally here is our test case with all the elements in place:

//
    /**
     * @dataProvider getVarData
     */
    function testGetVarFromDataSet(
        $name, $default, $hash, $type, $mask, $expect, $filterCalls
    ) {
        $filter = JFilterInput::getInstance();
        $filter -> mockReset();
        if (count($filterCalls)) {
            foreach ($filterCalls as $info) {
                $filter -> mockSetUp(
                    $info[0], $info[1], $info[2], $info[3]
                );
            }
        }
        /*
         * Get the variable and check the value.
         */
        $actual = JRequest::getVar($name, $default, $hash, $type, $mask);
        $this -> assertEquals($expect, $actual, 'Non-cached getVar');
        /*
         * Repeat the process to check caching (the JFilterInput mock should not
         * get called unless the default is being used).
         */
        $actual = JRequest::getVar($name, $default, $hash, $type, $mask);
        $this -> assertEquals($expect, $actual, 'Cached getVar');
        if (($filterOK = $filter -> mockTearDown()) !== true) {
            $this -> fail(
                'JFilterInput not called as expected:'
                . print_r($filterOK, true)
            );
        }
    }

Each element in the sample data set is passed in as a parameter to our test.

The setUp() method has initialized the super-globals already. We start by instantiating a mock JFilterInput object, resetting it, and then defining our expectations for it by calling mockSetUp() for each element in the $filterCalls array.

Don't forget to add a require_once above your class definition to include your helper class.

Then we run the core of our test, calling once with the cache empty and one where there may be a cached value.

Finally, we call mockTearDown(). If this method returns true, then the mock was used as we expected. If not, we dump the details of the discrepancies while failing the test.

Return to Unit Testing