Joomla! Documentation™

In all software there are mistakes that need to be fixed, this is also the case in Open Source software like Joomla!. The source code of Joomla! is centrally located on the internet, making it accessible to everyone. Anyone who spots a bug or issue with Joomla! can report it. A guide to Filing bugs and issues can be found here: Filing bugs and issues

Reported bugs will be solved and eventually need testing. This page explains how you can contribute to the Joomla project by testing proposed solutions. You do not need to have any coding knowledge, you do need a Joomla installation to test on. 

A Quick Reference Guide for testing is available at: Testing Joomla! patches QuickReferenceGuide

The Joomla staging code

The Joomla staging code is the most recent code that is being worked on. All bugfixes and new features are applied to this code.

The Joomla staging code is located at https://github.com/joomla/joomla-cms, this is the code that needs to be installed. How that is done is explained a little further under Preparing to test.

The issue-tracker

The issue-tracker for Joomla is located here: https://issues.joomla.org

The list shows the tracker items from new to old but the order can be changed by using the filters under Search tools. All items in the tracker, that have a patch, have a version number beside the title.

• PR-staging: This is Joomla 3.9.x
• PR-4.0-dev: This is Joomla 4.0

These labels will change in the future as new versions come and old versions go.

The status is important and this is an explanation:

New - Immediately after an issue is reported it receives this status. When you find a bug, create a description how to reproduce the bug. People who are testing the issue then know how to test the patch.

Confirmed - The tracker team has been able to reproduce the issue. The coding team is going to look for a solution.

Pending - A solution has been found for the issue. A patch has been attached that can be tested. The issue needs to be tested. Pay attention to the label beside the title, this indicates for which Joomla version the patch is applicable.

Ready to commit - The patch has been successfully tested and the code can be committed.

Closed - This status is for all issues that need no further action.

Filtering is useful in the issue tracker. Monitoring can be useful to receive notifications of an issue that has been updated.

Basically you start testing issues that have the status "Pending".

Preparing to test

First the master code has to be retrieved from the Joomla GitHub repository. There are several different ways to download the Joomla master code but here we only discuss the download option.

Create folder

First we need to create a folder on either the local webserver or the hosting account that can be accessed via the browser. Create the folder bugtesting and go to this folder via Explorer or Finder.

Download Joomla!

Go to the Joomla GitHub location and click on Download ZIP

This will get you a copy of the Joomla master code in a ZIP file. Unzip the file in a temporary location, a folder named joomla-cms-staging will appear. Inside this folder is the Joomla master code. Go into the joomla-cms-staging folder and copy the complete contents to the folder bugtesting that you created in the previous step.

Now you can install Joomla by navigating to http://localhost/bugtesting if you are using a local webserver, if you use one on a hosting account, navigate to the appropriate URL for the hosting account. You will see the regular installation process of Joomla. For sample data choose: Test English (GB) sample data

This is important so everybody uses the same data to test!

The installation folder does not have to be removed, this way you can easily reinstall everything if needed.

Download Patch Tester

To help you with the process of testing patches, a Joomla extension called Patch Tester has been developed. The Patch Tester can be downloaded from the Patch Tester page and after that can be installed via the Extension manager in Joomla.

To use the Patch Tester a GitHub account is required, the reason for this that there is a strict limit on the number of anonymous requests that can be made via the Patch Tester. You can sign up for a GitHub account on this page.

Setup Patch Tester

The GitHub account details can be entered into the settings of the Patch Tester. After the Patch Tester has been installed go to Components -> Joomla! Patch Tester and click on Options in the top right corner. The Options page shows up and there are three tabs. On the first tab called GitHub Repository you can choose which repository you want to use. By default this is Joomla! CMS.

On the second tab called GitHub Authentication you can enter your GitHub username and password or token if you use a token.

Note: If you have enabled Two-Factor Authentication for your GitHub account, you will need to create a personal access token at https://github.com/settings/applications and enter that in the GitHub token field.

After entering your account details click on Save & Close. Once back in the Patch Tester click on Fetch Data in the top left corner.

You are now ready to start testing.

Find an issue in the issue tracker you want to test or use the "Feeling lucky" button. First check if you can reproduce the issue. If you can reproduce the issue, then you can install the patch.

Installing Patch

In the Patch Tester find the issue you are working on by (partial) title or ID. The title of a tracker always looks like this:

[#4308] - Fixed event dispatching inconsistencies leading to incomplete API

Beside the # is the ID and behind the - sign comes the title.

After the patch has been found click on the blue button Apply Patch. Check if the bug has been fixed and also check if no new bugs have been introduced

Recording test results

After testing, please add your test result in the test overview (use the button "Test this"). Now you only need to add a comment if you have extra information or if you test unsuccessful. As the Tracker now automatically add a comment that you test it.

It must also be noted that it is much better if you can test patches in a multilingual environment (with at least 3 content languages published) as there can be issues triggered only in such environment.

Revert patch

After a patch has been tested it is no longer needed and can be removed from your test installation, this way you are always testing with a clean system (You can test patches in conjunction to each other to make sure they don't conflict). Go back to the Patch Tester and click on the green button Revert Patch. This will remove the patch and you can start testing the next issue.

Testing patches using Git

Since the Patch Tester cannot be used for all patches to test, sometimes we must use another way. The Patch Tester replaces the complete file and does not only apply the changes of the pull request, so you may not be testing an issue in an isolated state.

To truly test a patch with only the changes made in the patch, we must use Git. The reason is that Git can apply a patch with only the changes in that patch. Git does this using a diff file.

The prerequisites to be able to use the below method are:

• Joomla is installed using Git
• cURL is installed

Getting the diff

To get a diff from a pull request is rather easy. Let's say the URL to the pull request is

https://github.com/joomla/joomla-cms/pull/1452

To get the diff, all you need to do is add .diff after the URL, so it becomes

https://github.com/joomla/joomla-cms/pull/1452.diff

Testing the diff

Before you apply the diff, you can first check if the diff can be applied, a sanity check. Go to the root of your Joomla installation and run the following command:

curl -L https://github.com/joomla/joomla-cms/pull/1452.diff | git apply --check

Applying the diff

After the diff has been tested and there are no errors, the diff can be applied by running the following command:

curl -L https://github.com/joomla/joomla-cms/pull/1452.diff | git apply

With the diff applied you can run your tests and record your test results as explained above. If needed, you can apply multiple diffs at once as long as they do not overlap each other. In some cases this is needed when there are multiple pull requests that need to be combined to be tested.

Reverting the diff

Once you are done with testing you can cleanup the system by revering the diff with the following command:

curl -L https://github.com/joomla/joomla-cms/pull/1452.diff | git apply -R

To verify that everything is clean you can run

git status

You should get the notice:
nothing to commit, working directory clean

In case the system is not clean you can get a notice like this:

There are changed files in the working directory

In case you see changed files but you have not made any changes you can cleanup the system with

git reset --hard HEAD

Now all should be clean

If you have become enthusiastic to test, give it a try. It takes a bit of effort to set everything up but once that is done, it is fun and easy! In addition you are also contributing to Joomla!.

Videos

Learn How to test patch with this video series How to "Patch Test" Joomla! code:

• Part 1: Introduction

• Part 2: Neard Web Server setup

• Part 3: Sample issue testing.

Link

Brian Teeman: A Dummies Guide to Joomla Bug Testing