Writing System Tests for Joomla! - Part 2

From Joomla! Documentation

(Redirected from Writing System Tests for Joomla! Part 2)

Creating an Intermediate System Test[edit]

Note: This is a continuation of the article Writing System Tests for Joomla! - Part 1. This article assumes that you have read that article.

In this test, we are going to execute the test described at Intermediate_Selenium_Example and integrate it into the system tests suite in 1.6.

1. Open Joomla! 1.6 in FireFox and navigate to the home page.

2. Open Selenium IDE by going to the Tools menu and select Selenium IDE. Selenium will start recording automatically and the red record button will appear highlighted (see image below).

3. Verify that the Base URL displayed in Selenium, next to the record button, is that of your test site.

4. Perform the test by following the instructions described above (Note: there are some slight variations between the test described above and the one we performed. This is due to variation of 1.6 during the development process).

5. When you have completed your test, click the the red record button (see image below) to stop recording.


Selenium-screen.png


6. Once recording has stopped, create a new PHP file in Eclipse (or your favourite code editor) and copy / paste the code from Selenium IDE to there. The contents of the example test is:

<?php

require_once 'PHPUnit/Extensions/SeleniumTestCase.php';

class Example extends PHPUnit_Extensions_SeleniumTestCase
{
  function setUp()
  {
    $this->setBrowser("*chrome");
    $this->setBrowserUrl("http://change-this-to-the-site-you-are-testing/");
  }

  function testMyTestCase()
  {
    $this->open("/workspace/joomla-1-6-source/administrator/index.php?option=com_login");
    $this->type("mod-login-username", "admin");
    $this->click("link=Log in");
    $this->waitForPageToLoad("30000");
    $this->click("link=User Manager");
    $this->waitForPageToLoad("30000");
    $this->click("//li[@id='toolbar-new']/a/span");
    $this->waitForPageToLoad("30000");
    $this->type("jform_name", "My Test User");
    $this->type("jform_username", "TestUser");
    $this->type("jform_password", "password");
    $this->type("jform_password2", "password");
    $this->type("jform_email", "test@example.com");
    $this->click("1group_2");
    $this->click("link=Save & Close");
    $this->waitForPageToLoad("30000");
    try {
        $this->assertTrue($this->isTextPresent("Item successfully saved."));
    } catch (PHPUnit_Framework_AssertionFailedError $e) {
        array_push($this->verificationErrors, $e->toString());
    }
    $this->type("search", "TestUser");
    $this->click("//button[@type='submit']");
    $this->waitForPageToLoad("30000");
    try {
        $this->assertTrue($this->isTextPresent("TestUser"));
    } catch (PHPUnit_Framework_AssertionFailedError $e) {
        array_push($this->verificationErrors, $e->toString());
    }
    $this->click("link=Log out");
    $this->waitForPageToLoad("30000");
    $this->click("link=Go to site home page.");
    $this->waitForPageToLoad("30000");
    $this->type("modlgn_username", "TestUser");
    $this->type("modlgn_passwd", "password");
    $this->click("Submit");
    $this->waitForPageToLoad("30000");
    try {
        $this->assertTrue($this->isTextPresent("My Test User"));
    } catch (PHPUnit_Framework_AssertionFailedError $e) {
        array_push($this->verificationErrors, $e->toString());
    }
    $this->click("link=Logout");
    $this->waitForPageToLoad("30000");
    $this->click("//button[@type='submit']");
    $this->waitForPageToLoad("30000");
    $this->click("link=Site Administrator");
    $this->waitForPageToLoad("30000");
    $this->type("mod-login-username", "admin");
    $this->click("link=Log in");
    $this->waitForPageToLoad("30000");
    $this->click("//img[@alt='User Manager']");
    $this->waitForPageToLoad("30000");
    $this->click("cb0");
    $this->click("link=Delete");
    $this->waitForPageToLoad("30000");
    try {
        $this->assertTrue($this->isTextPresent("1 item(s) successfully deleted."));
    } catch (PHPUnit_Framework_AssertionFailedError $e) {
        array_push($this->verificationErrors, $e->toString());
    }
    $this->click("link=Log out");
    $this->waitForPageToLoad("30000");
  }
}
?>

As noted in Intermediate_Selenium_Example Selenium does not record passwords that are entered. Additionally, we want to utilize Selenium_Test_Case_Methods to prevent coding path or login information in the test.


1. The first step in integrating this test into tests/suite is to call SeleniumJoomlaTestCase.php so that we can utilize the Selenium Test Case Methods. To do this, we will replace the first portion of our code:

<?php
require_once 'PHPUnit/Extensions/SeleniumTestCase.php';

with the following code

<?php
require_once 'SeleniumJoomlaTestCase.php';
  • We can now start utilizing the Selenium Joomla Test Case Methods.

2. We will now extend a class into our SeleniumJoomlaTestCase to implement the client/server protocol to talk to Selenium RC as well as specialized assertion methods for web testing.

This is done by replacing:

class Example extends PHPUnit_Extensions_SeleniumTestCase
{

with

class User0002Test extends SeleniumJoomlaTestCase
{
  • The top portion of our test now looks like:
<?php
require_once 'SeleniumJoomlaTestCase.php';

class User0002Test extends SeleniumJoomlaTestCase
{

  function testMyTestCase()
  {
  • At this point, running the test would successfully launch a two browsers instances, one for Selenium Remote Control and one for test itself, but it would fail due to the lack of the password being recorded. Additionally, we want to be sure that none of the path information is hard coded in the test.

3. To improve our code, and achieve what we noted above, we would delete the function below:

function setUp()
{
    $this->setBrowser("*chrome");
    $this->setBrowserUrl("http://change-this-to-the-site-you-are-testing/");
}

and add the following to our testMyTestCase function

 $this->setUp();

Additionaly, we can replace the following line of code:

$this->open("/workspace/joomla-1-6-source/administrator/index.php?option=com_login");

with

$this->gotoAdmin();

As you can see, this will prevent us from hard coding the path into the test.

We can now replace

$this->type("mod-login-username", "admin");
$this->click("link=Log in");
$this->waitForPageToLoad("30000");

with

$this->doAdminLogin();

At this point, running our test would succeed up until the point where we need to log into the administrative interface the second time. We'll fix that by replacing:

$this->type("mod-login-username", "admin");
$this->click("link=Log in");
$this->waitForPageToLoad("30000");

with

$this->doAdminLogin();

Running our test should now successfully create, verify the creation of, and delete our new test user.

  • To rely to the user what is happening, we want to add echo statements to our code.

Our final test looks like:

<?php

require_once 'SeleniumJoomlaTestCase.php';

class User0002Test extends SeleniumJoomlaTestCase
{
  function testMyTestCase()
  {
    echo("Starting testMyTestCase\n");
    $this->setUp();
    $this->gotoAdmin();
    $this->doAdminLogin();
    $this->click("link=User Manager");
    $this->waitForPageToLoad("30000");
    echo("Add new user.\n");
    $this->click("//li[@id='toolbar-new']/a/span");
    $this->waitForPageToLoad("30000");
    $this->type("jform_name", "My Test User");
    $this->type("jform_username", "TestUser");
    $this->type("jform_password", "password");
    $this->type("jform_password2", "password");
    $this->type("jform_email", "test@example.com");
    $this->click("1group_2");
    $this->click("link=Save & Close");
    $this->waitForPageToLoad("30000");
    echo("Verify existence of new user.\n");
    try {
        $this->assertTrue($this->isTextPresent("Item successfully saved."));
    } catch (PHPUnit_Framework_AssertionFailedError $e) {
        array_push($this->verificationErrors, $e->toString());
    }
    $this->type("search", "TestUser");
    $this->click("//button[@type='submit']");
    $this->waitForPageToLoad("30000");
    try {
        $this->assertTrue($this->isTextPresent("TestUser"));
    } catch (PHPUnit_Framework_AssertionFailedError $e) {
        array_push($this->verificationErrors, $e->toString());
    }
    $this->click("link=Log out");
    $this->waitForPageToLoad("30000");
    echo("Go to home page.\n");   
    $this->click("link=Go to site home page.");
    $this->waitForPageToLoad("30000");
    echo("Log in as TestUser.\n");   
    $this->type("modlgn_username", "TestUser");
    $this->type("modlgn_passwd", "password");
    $this->click("Submit");
    $this->waitForPageToLoad("30000");
    echo("Verify existence of new user.\n");   
    try {
        $this->assertTrue($this->isTextPresent("My Test User"));
    } catch (PHPUnit_Framework_AssertionFailedError $e) {
        array_push($this->verificationErrors, $e->toString());
    }
    $this->click("link=Logout");
    $this->waitForPageToLoad("30000");
    $this->click("//button[@type='submit']");
    $this->waitForPageToLoad("30000");   
    $this->click("link=Site Administrator");
    $this->waitForPageToLoad("30000");
    $this->doAdminLogin();
    $this->click("//img[@alt='User Manager']");
    $this->waitForPageToLoad("30000");
    $this->click("cb0");
    echo("Delete new user.\n");   
    $this->click("link=Delete");
    $this->waitForPageToLoad("30000");
    try {
        $this->assertTrue($this->isTextPresent("1 item(s) successfully deleted."));
    } catch (PHPUnit_Framework_AssertionFailedError $e) {
        array_push($this->verificationErrors, $e->toString());
    }
    $this->click("link=Log out");
    $this->waitForPageToLoad("30000");
  }
}
?>

Creating System Test in PHP[edit]

Previously, we showed how to create a test by recording a session in Selenium IDE, converting that to PHP, and then using that code as the basis of a PHP system test.

We can also simply create new test directly in PHP. For example, by leveraging the Selenium_Test_Case_Methods present in Joomla! 1.6, we can create the start of a system test with just a few lines of code.

For example, to test logging into and then out of the back end, we could use the following code:

<?php
require_once 'SeleniumJoomlaTestCase.php';

class ControlPanelExample extends SeleniumJoomlaTestCase
{
    function testExample()
    {
        $this->setUp();
        $this->doAdminLogin();
        $this->doAdminLogout();
    }
}
?>
  • require_once 'SeleniumJoomlaTestCase.php';? allows use to utilize the methods that belong to the Selenium Joomla Test Case class
  • class ControlPanelExample extends SeleniumJoomlaTestCase extends a class into our SeleniumJoomlaTestCase to implement the client/server protocol to talk to Selenium RC as well as specialized assertion methods for web testing.
  • Each test is written as a function.
  • $this->setUp(); envokes our configuration from tests/system/servers/configdef.php see Running_Automated_Tests_for_Version_1.6#Create_a_Selenium_Configuration_File
  • $this->doAdminLogin(); is a Selenium Joomla Test Case method that logs into the back end.
  • $this->doAdminLogoutn(); is a Selenium Joomla Test Case method that logs out of the back end.

Joomla Core Selenium testcases[edit]

As of February 2010, the unit and system tests for version 1.6 have been incorporated into the SVN trunk under a folder called "tests". More information about running the version 1.6 tests is available in the article Running Automated Tests for Version 1.6. The tests folder is included when you do a checkout of trunk using the SVN.

Running System Tests[edit]

Instructions for running the system tests for version 1.6 can be found here: Running Automated Tests for Version 1.6.

Troubleshooting[edit]

When writing a test or troubleshooting a test failure, it is convenient to be able to step through a test one line at a time. Using the Selenium IDE, it is very easy to do this. One caveat is that you can only do this with tests created with the Selenium IDE. Once a test has been converted to PHP and turned into a system test outside of the IDE, that test cannot be converted back (automatically at least) to the Selenium IDE HTML format.

However, in this case you can use the debugger inside your PHP IDE (Eclipse or Netbeans, for example) to debug the test.

To step through a test in the Selenium IDE, follow these steps:

  1. Make sure you have the format set to HTML (Options→Format→HTML) and the Table tab highlighted.
  2. Highlight the first line in the test and press the "X" key to execute that line. Highlight the next line and repeat.
  3. Alternatively, you can place a debug "break" in any line by highlighting that line and pressing the "B" key. It works as an on/off toggle. When you have a break, you can then press the Play button and it will stop at the break. Then you can use the blue Step button to continue the test one line at a time.

To debug a test in Eclipse, see Running_Automated_Tests_for_Version_1.6. Note that the setup for debugging tests inside Eclipse is more difficult than simply using the Selenium IDE. Therefore, to troubleshoot a PHP system test, it may be easier to isolate the portion of the test that is failing, recreate that portion inside the Selenium IDE, and debug the test there.

Frequently Asked Questions[edit]

Using XPath for Selecting Elements[edit]

When creating system tests, you need to be able to identify specific HTML elements on the page being tested. In some cases, you want to click on a link or a button. In other cases, you want to test that the elements are shown correctly. Or you may need to fill in a form. In all of these cases, a key aspect is identifying the correct element.

Selenium uses XPath as one way to find and work with HTML elements. When you record a test, this is often done for you. However, as you work with more complicated tests, it is very helpful to understand a little bit about how to use XPath, especially as it relates to specific situations in Joomla.

This section documents some examples of using XPath for specific situations in Joomla.

Finding an Element Based on Exact Value of an Attribute
XPath Example: //li[@id='toolbar-apply']/a/span
Explanation: Find the first li element with @id attribute that is equal to 'toolbar-apply'.
PHP Code to Click:
$this->click("//li[@id='toolbar-apply']/a/span");
Comment: This is often the method used when recording a test with the Selenium IDE and works well when the an element has a unique @id that won't change.
Finding an Element Based on Partial Match of an Attribute
XPath Example: //button[contains(@onclick, 'component.save')]
Explanation: Find the first button element that contains an "@onclick" attribute that is equal to 'component.save'.
PHP Code to Click:
$this->click("//button[contains(@onclick, 'component.save')]");
Comment: Using a partial text match can be handy when an attribute has a very long value or if the value can change (for example, based on row number).
Finding an Element Based on Partial Text Match of All Contained Text
XPath Example: //tr[contains(., 'Modules')]/td/a
Explanation: Find the first tr element that contains the text 'Modules' anywhere inside this element (including in child elements), then find the first td element that contains an anchor (a).
PHP Code to Click:
$this->click("//tr[contains(., 'Modules')]/td/a");
Comment: In many cases, using a partial text match is preferable to an exact text match, since it is less likely to be affected by minor changes in the layouts.
Finding an Element Based on Partial Text Match (That Element Only)
XPath Example: //tr/td/a[contains(text(), 'Modules')]
Explanation: Find the first tr element that contains a td element with an anchor (a), where the anchor contains the text 'Modules'.
PHP Code to Click:
$this->click("//tr/td/a[contains(text(), 'Modules')]");
Comment: In many cases, using a partial text match is preferable to an exact text match, since it is less likely to be affected by minor changes in the layouts.
Finding an Element Based on Partial Case-Insensitive Text Match (That Element Only)
XPath Example: //input[contains(translate(@value, 'ABCDEFGHIJKLMNOPQRSTUVWXYZ', 'abcdefghijklmnopqrstuvwxyz'),'installed successfully')]
Explanation: Find the first input element that contains the text 'installed successfully', regardless of case.
PHP Code to Click:
$this->click("//tr/td/a[contains(text(), 'Modules')]");
Comment: In many cases, using a case-insensitive text match is preferable to an exact text match, since it is less likely to be affected by minor changes in the layouts.
Finding and Element Based on Values of Two Attributes
XPath Example://select[contains(@id, 'edit.state')][contains(@title, 'Test Administrator')]
Explanation: Find the first element that contains a select with an id containing the text 'edit.state' and a title containing the text 'Test Administrator'.
PHP Code to Select:
$this->select("//select[contains(@id,'edit.state')][contains(@title,'Test Administrator')]", "label=Inherited");
Comment: In this example, we look for the first select element that meets two conditions. Putting the two attribute tests in square brackets next to each other tells XPath that both conditions must be true.
Finding a Link or Button in Table based on Text in the Same Row
XPath Example://table[@class='adminlist']/tbody//tr//td/a[contains(text(), 'Articles Modules')]/../../td[3]/a
Explanation: Find the first table with class='adminlist', then find the first 'tr' element in the 'tbody' element, then the first 'tr'tr element that has an anchor ('a') that has the text 'Articles Modules' in it. Next, go up two levels in the tree (to get back to the 'tr' element) and go over to the third 'td' element in this row.
PHP Code to Click:
$this->click("//table[@class='adminlist']/tbody//tr//td/a[contains(text(), '" . $articleTitle . "')]/../../td[3]/a");
Comment: This is useful for clicking on an icon in a row where the article title is shown. This example is used to click on the Published link in the Article Manager where $articleTitle is the title of the article. It is used in the togglePublished() method of the SeleniumJoomlaTestCase class.

Testing for Text in Tables[edit]

You can use the assertTable command to test for an exact match of text in a specific table cell. For example, in the Category Manager screen, there is a table with the class of "adminlist". You can test that a specific cell contains text using the following command:

Selenium IDE Example
Command: assertTable
Target: //table[@class='adminlist'].2.1
Value: Joomla! (Alias: joomla)
Explanation: Find the 3rd row, 2nd column from the first table with class 'adminlist' and check that it contains the text "Joomla! (Alias: joomla)".
Note that the row and columns start at 0 and that the <thead> element contains a <tr> element that counts as row zero.
PHP Code
$this->assertEquals("Joomla! (Alias: joomla)", $this->getTable("//table[@class='adminlist'].2.1"));

Tips & Tricks[edit]

  • It is recommended to update your local copy of Joomla! to the latest build before writing or using any tests.
  • Some of the tests written for Joomla! version 1.6 use the included sample data. You need to install the sample data to run these tests.
  • Some system tests are loosely dependent on the default templates. The systems tests written for version 1.6 use the Beez2 (front end) and Bluestork (back end) templates.
  • It is recommended that tests "undo" any changes made to the site. For example, if you create a new User Group for a test, or series of tests, be sure to delete the group before concluding your test or series of tests. This can be done as part of the test. Other tests may encounter issues if artifacts of other tests are present.
  • The test are written in PHP and we are using Selenium RC to execute them later. You will need to select the PHP Selenium RC Format in Selenium IDE. Go to Options -> Format -> PHP Selenium RC
  • Use the PHP echo command often in the test class to document what is happening inside the test. This is very helpful for troubleshooting test failures.
  • More information about writing test can be found at http://www.phpunit.de/manual/current/en/writing-tests-for-phpunit.html
  • Maximize flexibility through test methods: In many tests, similar or identical actions are repeated. In these cases, these actions can be reproduced as a granular test method added to /tests/system/SeleniumJoomlaTestCases.php. Good examples include:
    • doAdminLogin
    • gotoAdmin
    • gotoSite

(see Selenium Test Case Methods)


See Also[edit]