Unit Tests for CMS Working Group
The Unit Tests for CMS Working Group is a Production Working Group. It is responsible for seeing that the Unit Tests for the CMS are created and maintained.
Coordinators & PLT Contact
The coordinator for this working group is Michael Babker.
- PHPUnit Manual (links to current version; as of April 2012, this is version 3.6)
- Joomla Platform unit test library
- GitHub repo where work will occur
- CMS Unit Test Coverage Report (note that this is a temporary host until integrated into the Jenkins CI server and hosted on the joomla.org domain)
Technical Work produced by this group
Reaching the desired goal of 100% unit test coverage of CMS library classes can be broken down into three phases:
Phase 1 - Review of current test structure
The initial phase of this "project" will be to review implemented tests and port tests to the Joomla Platform to improve test coverage as needed. The current unit test suite has essentially been untouched since the formal split of the Joomla Platform (between the releases of CMS 1.6 and 1.7), and the test suite itself is designed to run on PHPUnit 3.4 (current version as of April 2012 is 3.6). Rouven has a branch on his CMS repo which has removed all unimplemented test stubs, tests for removed methods, and tests that are already covered in the Platform. The remaining tests are available at https://github.com/realityking/joomla-cms/tree/unittest/tests/unit/suite/libraries/joomla.
Phase 2 - Develop unit test structure
Once the current structure has been combed through to ensure that any valid tests have been ported to the Joomla Platform, we will develop a new unit test framework for the CMS using current versions of all tools (i.e. PHPUnit). The testing will be focused on the CMS 3.0 release series, and as such should be written using PHP 5.3.1 as the minimum supported version (note: the minimum PHP version is based on the current Platform requirements and the previously announced drop of PHP 5.2 support for CMS 3.0). The Platform has a strong unit test infrastructure and can be used as a model for the CMS's framework.
Note that PHPUnit has internal support for database testing. The Platform utilizes an in-memory SQLite database for general unit testing with additional test classes to test specific database driver functionality. MySQL(i) and PostgreSQL are fully supported by PHPUnit, whereas SQL Server does not have this support. A long term goal could potentially be to write a PHPUnit class which enables SQL Server testing and could be pushed upstream to PHPUnit if desired.
Phase 3 - Write and maintain tests
Phase 3 will be an ongoing phase as new commits will require that the unit tests pass as well as new code needing testing. The primary focus will be on CMS specific library classes (those found in libraries/cms and the FinderIndexer** classes). The tests should be written in a way to ensure that all methods work as intended, prevent backwards compatibility from being broken (unless otherwise intended and announced), and other use cases as necessary (such as security issues).
In addition to writing unit tests, the test structure will need to remain up-to-date with external tools. For example, test methods in PHPUnit that are deprecated in the current 3.6 release that have been removed from the in-development 3.7 release should be avoided to ease the transition to the new version. In instances where backwards compatibility will be broken between PHPUnit releases, an announced transition period must be made to allow for the test structure to be updated and all downstream contributors to update their systems.
As part of J and Beyond 2012, there will be a session dedicated to discussing the CMS Unit Testing, its current status, and a plan for creating and maintaining a current test structure.