The Unit Tests Working Group is a Production Working Group. It is responsible for seeing that the Unit Tests for Joomla's code projects, primarily the CMS and Framework, are created and maintained.
The PLT Contact for this working group is Michael Babker.
It is a current goal within Joomla's development community that all contributions that affect the CMS libraries (code found in the libraries folder in the CMS) and all Framework contributions include unit tests. For bug fixes, these tests will help to ensure that a regression is not introduced into the code base while working on other issues. For new features, these tests help demonstrate the expected behavior of a feature and enable the project's maintainers to ensure the stability of this code.
While the general goal is that all code has unit test coverage (100% lines of code covered), the project's overall goal is that the CMS libraries and each Framework package reach 80% coverage of methods considered part of the public API, which unless specified otherwise can be assumed to mean all public and protected methods in the code base. Test coverage should make all reasonable attempts to test all possible routes within a method are supported with tests, to include error conditions as practical. It is understood that there may be some lines of code that may be unreachable with test coverage for various reasons and these will be addressed on a case-by-case basis.
Goals for improving the test structure can be summarized in three phases:
The initial phase of this "project" will be to review implemented tests and port tests to the Joomla Platform to improve test coverage as needed. The current unit test suite has essentially been untouched since the formal split of the Joomla Platform (between the releases of CMS 1.6 and 1.7), and the test suite itself is designed to run on PHPUnit 3.4 (current version as of April 2012 is 3.6). Rouven has a branch on his CMS repo which has removed all unimplemented test stubs, tests for removed methods, and tests that are already covered in the Platform. The remaining tests are available at https://github.com/realityking/joomla-cms/tree/unittest/tests/unit/suite/libraries/joomla.
Once the current structure has been combed through to ensure that any valid tests have been ported to the Joomla Platform, we will develop a new unit test framework for the CMS using current versions of all tools (i.e. PHPUnit). The testing will be focused on the CMS 3.0 release series, and as such should be written using PHP 5.3.1 as the minimum supported version (note: the minimum PHP version is based on the current Platform requirements and the previously announced drop of PHP 5.2 support for CMS 3.0). The Platform has a strong unit test infrastructure and can be used as a model for the CMS's framework.
Note that PHPUnit has internal support for database testing. The Platform utilizes an in-memory SQLite database for general unit testing with additional test classes to test specific database driver functionality. MySQL(i), PostgreSQL, and SQL Server are fully supported by PHPUnit.
Phase 3 will be an ongoing phase as new commits will require that the unit tests pass as well as new code needing testing. The primary focus will be on CMS specific library classes (those found in /libraries/cms and the FinderIndexer** classes). The tests should be written in a way to ensure that all methods work as intended, prevent backwards compatibility from being broken (unless otherwise intended and announced), and other use cases as necessary (such as security issues).
In addition to writing unit tests, the test structure will need to remain up-to-date with external tools. For example, test methods in PHPUnit that are deprecated in the current 3.7 release that have been removed from the in-development 3.8 release should be avoided to ease the transition to the new version. In instances where backwards compatibility will be broken between PHPUnit releases, an announced transition period must be made to allow for the test structure to be updated and all downstream contributors to update their systems.