Testing Joomla Extensions with Codeception

Note: this page is under construction

What is Codeception?

Codeception PHP Testing Framework is designed to work just out of the box. This means its installation requires minimal steps and no external dependencies preinstalled (except PHP, of course). Only one configuration step should be taken and you are ready to test your web application from an eye of actual user.

See the following video of codeception in action testing Joomla:

Codeception provides possibilities to do Unit Testing or to Test APIs, however this page is focused in System Testing.

Why Use Codeception?

Why use Codeception when we already have Selenium IDE?

Codeception works great with Selenium. But with Codeception you can write your tests in PHP. The main reason is: Selenium IDE tests are tightly bound to XPath locators. If you ever change anything in layout tests will fall. Codeception locators are more stable. You can use names, labels, button names and CSS to match elements on page.

Features

• Selenium WebDriver integration
• Elements matched by name, CSS, XPath
• PageObjects and StepObjects included
• Powered by PHPUnit
• Data Cleanup
• Parallel Execution of tests
• BDD-style readable tests

Architecture of a test

The testing framework is built on the three layered testing architecture

Scenarios (Cept Class)

Cept or Test, it is the main class which contains steps of execution for the tests, Cept Class will call Step Class functions which will indirectly make use of page objects to perform operations

Example:

$I = new AcceptanceTester($scenario);
$I->wantTo('Install Joomla CMS');
$I->amOnPage('http://localhost/installation/index.php');
// I Wait for the text Main Configuration, meaning that the page is loaded
$I->waitForText('Main Configuration', 10, 'h3');
$I->click(JoomlaInstallationConfigurationPage::$elements['Language Selector']);
$I->click(JoomlaInstallationConfigurationPage::$elements['English (United Kingdom)']);
$I->fillField('Site Name', 'weblinks');
$I->fillField('Description', 'Site for testing Weblinks component');
$I->fillField('Admin Email', $cfg['Admin email']);
$I->fillField('Admin Username', $cfg['username']);
$I->fillField('Admin Password', $cfg['password']);
$I->fillField('Confirm Admin Password', $cfg['password']);
$I->click('Next');
...

Page Objects

Each page on your application has a class which we call page objects, each page class will have information about the URL, fields, Text present on the page. We use the best possible selector for any field on the page.

Example

class JoomlaAdministratorLoginManagerPage
{
	/**
	 * @var string Url of the page
	 */
	public static $URL = '/administrator/index.php';

	/**
	 * Array of Page elements indexed by descriptive name or label
	 *
	 * @var array
	 */
	public static $elements = array(
		'username' => "#mod-login-username",
		'password' => "#mod-login-password"
	);
}

Naming convention for Page Objects

When this document is created Codeception 2.0 was not having a PSR-4 compatible autoloader. Therefore we are creating a convention so with the future Codeception 2.1 with PSR-4 compatible autoloader (https://github.com/Codeception/Codeception/pull/1228) we are creating a naming convention for Pages so it will cost us a small effort to move to autoloaded classes in the future.

The naming convention works as follows:

APPLICATION (for example Joomla, Patchtester, Weblinks or your Extension name)
+ CLIENT (in Joomla we use it for defining if is Installation, Administrator or Frontend
+ CLIENT PART (In Joomla we use it following the menu are were it belongs: Components, Menus, System,...)
+ PAGE NAME (The Page name) 
+ Page

Examples:

JoomlaAdministratorLoginPage
JoomlaAdministratorSystemGlobalconfigurationPage
JoomlaInstallationConfigurationPage

And they are located with a folder estructure similar to:

Folder estructure for Pages in System Tests

Remind that in Codeception 2.0 you need to specify each folder in the acceptance/_bootstrap.php file:

<?php
// Here you can initialize variables that will be available to your tests
\Codeception\Util\Autoload::registerSuffix('Page', __DIR__.DIRECTORY_SEPARATOR.'_pages/joomla/installation');
\Codeception\Util\Autoload::registerSuffix('Page', __DIR__.DIRECTORY_SEPARATOR.'_pages/joomla/administrator');

Documentation

More info: http://codeception.com/docs/07-AdvancedUsage#PageObjects

Step Objects

Step Objects are classes which contains steps which are to be executed on the application to perform certain operations, step objects make use of page objects to perform these actions.

Example:

class JoomlaAdministratorLoginSteps extends \AcceptanceTester
{
	/**
	 * Function to execute an Admin Login for Joomla 3
	 *
	 * @return void
	 */
	public function doAdministratorLogin($user, $password)
	{
		$I = $this;
		$I->amOnPage(\JoomlaAdministratorLoginPage::$URL);
		$I->fillField(\JoomlaAdministratorLoginPage::$elements['username'], $user);
		$I->fillField(\JoomlaAdministratorLoginPage::$elements['password'], $password);
		$I->click('Log in');
		$I->see('Category Manager');
	}
}

More info: http://codeception.com/docs/07-AdvancedUsage#StepObjects

The principles for writing a test: Readability and Performance

In Joomla we wive priority to Readability over Performance, even when performance is a must.

To explain that let me show an example: Let's say that we want to fill a field Username in a login form.

<form action="administrator/index.php" method="post" id="form-login">
  <label for="username_field">Username</label>
  <input name="username_field" tabindex="1" id="username_field" type="text" size="15" />
  ...

the most performant way to fill that form field would be:

$I->fillField('#username_field', 'Mark');

And usually we would move that to a page object element ending in:

$I->fillField(\LoginPage::$username, 'Mark');

However Codeception is a smart framework and it adds some magic to it's methods (see: https://github.com/Codeception/Codeception/blob/2.0/src/Codeception/Module/WebDriver.php#L511-L513). It allows me to write more readable tests (keeping performance hight).

If we do this:

$I->fillField('Username', 'Mark');

Codeception will be smart and will search first in the page if there is any "LABEL" called Username. That is a fast operation, so performance will be affected just a little. And instead we have written less code than in previous examples.

In conclusion we have more readable tests and less code to maintain while keeping performance. That is a win!

Note: sometimes this tricks are not always possible, labels are not always there or there are some elements hidden that shows and hides depending on a Javascript. All these special cases would require a Page element.

Setting Up Codeception

setting up the repository

clone the directory

1. Fork com_localise
3. Git clone git@github.com:*your-github-profile-name*/com_localise.git

Testing with Codeception

There are two ways to get and run codeception, via PHAR or via Composer. Using Composer is recommended (because we are using in some tests the Codeception 2.1 development version not available in the composer.phar).

• Using Composer

   You need to have Composer in your system, if not download it from here: https://getcomposer.org/

   Step 1: composer update

   Step 2: run Codeception by doing: php vendor/bin/codecept build

• Codeception.phar

   Linux Machine  : wget http://codeception.com/codecept.phar

   Windows Machine: Download from codecept.phar

   Step1 : php ./codecept.phar build

Running the Test

Rename tests/acceptance.suite.dist.yml to tests/acceptance.suite.yml

Step 1: Modify the configuration at tests/acceptance.suite.yml to fit your server details.

Step 2: php codecept.phar run

with --steps to see a step-by-step report on the performed actions.

php vendor/bin/codecept run --steps

with --html. This command will run all tests for all suites, displaying the steps, and building HTML and XML reports. Reports will be store in tests/_output/ directory.

php vendor/bin/codecept run --html

To activate the debug mode in Codeception just add the parameter --debug when you run the tests:

php vendor/bin/codecept run acceptance --debug

Best Practices at Testing

How to use URLs

Codeception documentation suggests to use URL's as properties of a Page Object:

$I->amOnPage(JoomlaInstallationConfigurationPage::$URL);

Instead of hardcoding the url in the test:

$I->amOnPage('/administrator/index.php?option=com_banners&view...');

The pros of this approach is: - first way is more readable for non developers. - it helps to make tests more maintenable in the case of a url change in the system

However in joomla a change in an url is rare, and if it happens changing it will just be a Find&Replace task. I Joomla test we want to have the benefit of both, the url and a representative name that describes the page where we are going. That will make our tests more readable. Thus, we will write the urls the following way:

$I->amGoingTo('go to Create New Banner page at Joomla Administrator');
$I->amOnPage('/administrator/index.php?option=com_banners&view...');

Working with Hidden Elements

Selenium can't interact with elements that are not visible:

<a href="http://www.joomla.org" style="display: none;" id="joomla_site">go to joomla</a>

In this case you can't do:

$I->click('go to joomla');

Usually this happens when you need to wait for a Javascript animation that end up showing that element, so in this case you will need to do this:

$I->waitForElementVisible('#joomla_site');
$I->click('go to joomla');

When do not use contains

We do not recommend to use the function contains on XPath because is not specific and can create false positives at tests. See: https://github.com/joomla-projects/GSOC-Webdriver_system_tests_for_CMS/pull/99#issuecomment-69004191

Always add xPath locators to your tests

Performance in tests is very important. You can reduce dramatically the time spent on a test with just adding a few extra locators. For example, the following code pertains to a test:

// see($text, $selector = null)
$I->see('Your article has been published');

In the previous code we are not facilitating the "Selector" so Codeception will have to search for the element in all the HTML. Doing this instead:

$I->see('Your article has been published', '#messages');

the tests went from: Time: 1.17 minutes to: Time: 49.18 seconds

You can see this video of a real example: http://www.youtube.com/watch?v=0JCv0BR2yZY

Losing 30 seconds in every Assertion can delay your tests dramatically. Therefore, remind always to add xPath locators to all your Assertions.

Using locators in $I->see

Do not use see without locator:

// Wrong:
$I->see('Your article has been published');

// Right:
$I->see('Your article has been published', '#messages');

Using locators in $I->waitForText

Do not use waitForText without locator:

// Wrong:
$I->waitForText('Finalisation');

// Right:
$I->waitForText('Finalisation', 10, 'h3');

Quote and Double Quotes in Compex XPaths

Sometimes you will have to write XPaths that looks like:

//button[@onclick='Joomla.submitbutton("config.save.application.apply")']

When moved to a Page Object you will probably will face this situation:

Typical issue when you write a long XPath in a Page object

To avoid it remind to escape the quotes:

// Wrong
public static $elements = array(
   'Save' => "//button[@onclick="Joomla.submitbutton('config.save.application.apply')"]"
);
// Right
public static $elements = array(
   'Save' => "//button[@onclick=\"Joomla.submitbutton('config.save.application.apply')\"]"
);