Testing Joomla! patches

From Joomla! Documentation

Other languages:
Deutsch • ‎English • ‎español • ‎français • ‎Bahasa Indonesia • ‎Nederlands • ‎português do Brasil • ‎русский • ‎Türkçe

In all software there are mistakes that need to be fixed, this is also the case in Open Source software like Joomla!. The source code of Joomla! is centrally located on the internet, making it accessible to everyone. Anyone who spots a bug or issue with Joomla! can report it. A guide to Filing bugs and issues can be found here: Filing bugs and issues

Reported bugs will be solved and eventually need testing. This page explains how you can contribute to the Joomla project by testing proposed solutions. You do not need to have any coding knowledge, you do need a Joomla installation to test on. 

A Quick Reference Guide for testing is available at: Testing Joomla! patches QuickReferenceGuide

The Joomla staging code

The Joomla staging code is the most recent code that is being worked on. All bugfixes and new features are applied to this code.

The Joomla staging code is located at https://github.com/joomla/joomla-cms, this is the code that needs to be installed. How that is done is explained a little further under Preparing to test.

The issue-tracker

The issue-tracker for Joomla is located here: https://issues.joomla.org

Joomla issue tracker-en.png

The list shows the tracker items from new to old but the order can be changed by using the filters under Search tools. All items in the tracker, that have a patch, have a version number beside the title.

  • PR-staging: This is Joomla 3.8.x
  • PR-4.0-dev: This is Joomla 4.0

These labels will change in the future as new versions come and old versions go.

The status is important and this is an explanation:

New - Immediately after an issue is reported it receives this status. When you find a bug, create a description how to reproduce the bug. People who are testing the issue then know how to test the patch.

Confirmed - The tracker team has been able to reproduce the issue. The coding team is going to look for a solution.

Pending - A solution has been found for the issue. A patch has been attached that can be tested. The issue needs to be tested. Pay attention to the label beside the title, this indicates for which Joomla version the patch is applicable.

Ready to commit - The patch has been successfully tested and the code can be committed.

Closed - This status is for all issues that need no further action.

Filtering is useful in the issue tracker. Monitoring can be useful to receive notifications of an issue that has been updated.

Basically you start testing issues that have the status "Pending".


Preparing to test

First the master code has to be retrieved from the Joomla GitHub repository. There are several different ways to download the Joomla master code but here we only discuss the download option.

Create folder

First we need to create a folder on either the local webserver or the hosting account that can be accessed via the browser. Create the folder bugtesting and go to this folder via Explorer or Finder.

Download Joomla!

Go to the Joomla GitHub location and click on Download ZIP

Joomla github-en.png

This will get you a copy of the Joomla master code in a ZIP file. Unzip the file in a temporary location, a folder named joomla-cms-staging will appear. Inside this folder is the Joomla master code. Go into the joomla-cms-staging folder and copy the complete contents to the folder bugtesting that you created in the previous step.

Now you can install Joomla by navigating to http://localhost/bugtesting if you are using a local webserver, if you use one on a hosting account, navigate to the appropriate URL for the hosting account. You will see the regular installation process of Joomla. For sample data choose: Test English (GB) sample data

This is important so everybody uses the same data to test!

The installation folder does not have to be removed, this way you can easily reinstall everything if needed.


Download Patch Tester

To help you with the process of testing patches, a Joomla extension called Patch Tester has been developed. The Patch Tester can be downloaded from the Patch Tester page and after that can be installed via the Extension manager in Joomla.

To use the Patch Tester a GitHub account is required, the reason for this that there is a strict limit on the number of anonymous requests that can be made via the Patch Tester. You can sign up for a GitHub account on this page.

Setup Patch Tester

The GitHub account details can be entered into the settings of the Patch Tester. After the Patch Tester has been installed go to Components -> Joomla! Patch Tester and click on Options in the top right corner. The Options page shows up and there are three tabs. On the first tab called GitHub Repository you can choose which repository you want to use. By default this is Joomla! CMS.


Patchtester repo-en.png

On the second tab called GitHub Authentication you can enter your GitHub username and password or token if you use a token.

Patchtester details-en.png

Note: If you have enabled Two-Factor Authentication for your GitHub account, you will need to create a personal access token at https://github.com/settings/applications and enter that in the GitHub token field.

After entering your account details click on Save & Close. Once back in the Patch Tester click on Fetch Data in the top left corner.

You are now ready to start testing.

Find an issue in the issue tracker you want to test or use the "Feeling lucky" button. First check if you can reproduce the issue. If you can reproduce the issue, then you can install the patch.


Installing Patch

In the Patch Tester find the issue you are working on by (partial) title or ID. The title of a tracker always looks like this:

[#4308] - Fixed event dispatching inconsistencies leading to incomplete API

Beside the # is the ID and behind the - sign comes the title.

After the patch has been found click on the blue button Apply Patch. Check if the bug has been fixed and also check if no new bugs have been introduced

Recording test results

After testing, please add your test result in the test overview (use the button "Test this"). Now you only need to add a comment if you have extra information or if you test unsuccessful. As the Tracker now automatically add a comment that you test it.

It must also be noted that it is much better if you can test patches in a multilingual environment (with at least 3 content languages published) as there can be issues triggered only in such environment.

The new Test This button and how to use it

Revert patch

After a patch has been tested it is no longer needed and can be removed from your test installation, this way you are always testing with a clean system. Go back to the Patch Tester and click on the green button Revert Patch. This will remove the patch and you can start testing the next issue.

If you have become enthusiastic to test, give it a try. It takes a bit of effort to set everything up but once that is done, it is fun and easy! In addition you are also contributing to Joomla!.

Videos

Learn How to test patch with this video series How to "Patch Test" Joomla! code:

  • Part 1: Introduction

  • Part 2: Neard Web Server setup

  • Part 3: Sample issue testing.

Link

Brian Teeman: A Dummies Guide to Joomla Bug Testing