Actions

Difference between revisions of "Testing Checklists"

From Joomla! Documentation

(added 2 more points for pending list)
(14 intermediate revisions by 6 users not shown)
Line 2: Line 2:
  
 
=== Testing Open Issues ===
 
=== Testing Open Issues ===
The "Open" status indicates that no JBS member has evaluated this issue. So the goal is to learn enough about the issue to be able to change the status to one of the following: Unable to Confirm, Known Issue, Duplicate Report, Not Joomla! Core, Confirmed,  or Information Required. See [[Reporting issues]] for details about these statuses.
+
The "Open" status indicates that no JBS member has evaluated this issue. So the goal is to learn enough about the issue to be able to change the status to one of the following: Unable to Confirm, Known Issue, Duplicate Report, Not Joomla! Core, Confirmed,  or Information Required. See [[Bug Tracking Process#Resolving Issues | Bug Tracking Process]] for details about these status codes.
  
 
Here is a checklist for testing Open issues:
 
Here is a checklist for testing Open issues:
Line 11: Line 11:
 
* Is the issue a known issue or limitation? If so, change status to Known Issue.
 
* Is the issue a known issue or limitation? If so, change status to Known Issue.
 
* Is the issue an enhancement request? If so, change status to Not a Bug.
 
* Is the issue an enhancement request? If so, change status to Not a Bug.
* Before testing, did you update your local SVN repository to get the latest SVN Joomla! version?
+
* Before testing, did you update your local GIT repository to get the latest GIT Joomla! version? (see [[Git for Testers and Trackers]])
* If you couldn't duplicate the problem with the latest SVN version, did you test using the latest production version?
+
* If you couldn't duplicate the problem with the latest GIT version, did you test using the latest production version?
 +
* If the issue is still unconfirmed, try to to reproduce the issue with different cache settings.
 +
* Try to localize the problem by changing your cache settings.
 
* If you changed the issue to Confirmed, did you make sure there is a detailed test plan in the comments to allow someone who is not familiar with the issue to understand and test the issue?
 
* If you changed the issue to Confirmed, did you make sure there is a detailed test plan in the comments to allow someone who is not familiar with the issue to understand and test the issue?
  
 
=== Testing Pending Issues ===
 
=== Testing Pending Issues ===
A Pending issue is one that (a) has been confirmed, (b) has a detailed test plan, and (c) has a proposed patch to be tested. Here is a checklist for testing Pending issues:
+
A Pending issue is one that (a) has been confirmed, (b) has a detailed test plan, and (c) has a proposed patch to be tested ([[Learn more about patch files]]). Here is a checklist for testing Pending issues:
* Is there a test plan for the issue? If not, post a comment and change status to Information Required.
+
* Is there a test plan for the issue shown in the Test Instructions field? If not, post a comment and change status to Information Required.
* Before testing, did you update your local SVN repository to get the latest SVN Joomla! version?
+
* Before testing, did you update your local GIT repository to get the latest GIT Joomla! version?
 
* Did you test the issue ''before'' applying the patch to make sure you can duplicate the problem?
 
* Did you test the issue ''before'' applying the patch to make sure you can duplicate the problem?
 
* Did you test the issue after applying the patch to make sure the problem is fixed?
 
* Did you test the issue after applying the patch to make sure the problem is fixed?
Line 26: Line 28:
 
** language file issues (test with Debug Language set to Yes).
 
** language file issues (test with Debug Language set to Yes).
 
** PHP warnings (test with Error Reporting set to Maximum).
 
** PHP warnings (test with Error Reporting set to Maximum).
 +
* In your comment before you report the test results (positive or negative) add @test to the comment on a separate line (this is used for stats).
 
* If your test is successful and you are the first tester, indicate in your comment that a second test is needed. If you are the second tester, change the status to Ready to Commit.
 
* If your test is successful and you are the first tester, indicate in your comment that a second test is needed. If you are the second tester, change the status to Ready to Commit.
 
* Check the "Monitor Item" link on the left side if you want to receive an email when someone changes this tracker issue.
 
* Check the "Monitor Item" link on the left side if you want to receive an email when someone changes this tracker issue.
 
* Remember to revert the changes made by the patch after you are done testing.
 
* Remember to revert the changes made by the patch after you are done testing.
 +
 +
=== Testing Code Snippets ===
 +
Sometimes to test a patch you will need to test PHP code snippets. One very simple way to do this is as follows:
 +
# Create a new folder called "com_test" under the components folder.
 +
# Create a new file called "test.php" under this folder.
 +
# Put your PHP code snippet in this file.
 +
# To run the file, just enter the URL <your domain>/index.php?option=com_test. For example, <nowiki>http://localhost/joomla_development/index.php?option=com_test</nowiki>.
 +
 +
This will run the code snippet inside the Joomla! framework. This way, when you are done, you can just remove the com_test folder and no other files have been changed. (Sam, thanks for this tip!)
 +
 +
=== General Tips & Tricks ===
 +
* If you have the cache enabled (Global Configuration &rarr; System &rarr; Cache Settings), you will need to clean the cache each time you make a change to the code or parameters. Otherwise, you might be seeing the older cached version of the page. For this reason, unless you are specifically testing the cache feature, test with the Cache set to "No".
 +
 +
* When working it is best to set error reporting to maximum and turn on all debugging options (these are set in Global Configuration).
  
 
[[Category:Bug Squad]]
 
[[Category:Bug Squad]]

Revision as of 08:13, 22 March 2013

This article is designed to help Joomla! Bug Squad members as they test Open issues and Pending issues.

Contents

Testing Open Issues

The "Open" status indicates that no JBS member has evaluated this issue. So the goal is to learn enough about the issue to be able to change the status to one of the following: Unable to Confirm, Known Issue, Duplicate Report, Not Joomla! Core, Confirmed, or Information Required. See Bug Tracking Process for details about these status codes.

Here is a checklist for testing Open issues:

  • Did you carefully read the whole description and any other comments?
  • Is there enough information provided? If not, add a comment and change to Information Required.
  • Did you search the tracker to see if this issue has already been reported? If so, add a comment and change to Duplicate Report.
  • Does the issue relate to a Joomla! core program? If not, change status to Not Joomla! Core.
  • Is the issue a known issue or limitation? If so, change status to Known Issue.
  • Is the issue an enhancement request? If so, change status to Not a Bug.
  • Before testing, did you update your local GIT repository to get the latest GIT Joomla! version? (see Git for Testers and Trackers)
  • If you couldn't duplicate the problem with the latest GIT version, did you test using the latest production version?
  • If the issue is still unconfirmed, try to to reproduce the issue with different cache settings.
  • Try to localize the problem by changing your cache settings.
  • If you changed the issue to Confirmed, did you make sure there is a detailed test plan in the comments to allow someone who is not familiar with the issue to understand and test the issue?

Testing Pending Issues

A Pending issue is one that (a) has been confirmed, (b) has a detailed test plan, and (c) has a proposed patch to be tested (Learn more about patch files). Here is a checklist for testing Pending issues:

  • Is there a test plan for the issue shown in the Test Instructions field? If not, post a comment and change status to Information Required.
  • Before testing, did you update your local GIT repository to get the latest GIT Joomla! version?
  • Did you test the issue before applying the patch to make sure you can duplicate the problem?
  • Did you test the issue after applying the patch to make sure the problem is fixed?
  • Are there any other test cases that should be considered? Examples might include:
    • template override files (especially beez template)
    • testing with SEF or mod_rewrite enabled and disabled
    • language file issues (test with Debug Language set to Yes).
    • PHP warnings (test with Error Reporting set to Maximum).
  • In your comment before you report the test results (positive or negative) add @test to the comment on a separate line (this is used for stats).
  • If your test is successful and you are the first tester, indicate in your comment that a second test is needed. If you are the second tester, change the status to Ready to Commit.
  • Check the "Monitor Item" link on the left side if you want to receive an email when someone changes this tracker issue.
  • Remember to revert the changes made by the patch after you are done testing.

Testing Code Snippets

Sometimes to test a patch you will need to test PHP code snippets. One very simple way to do this is as follows:

  1. Create a new folder called "com_test" under the components folder.
  2. Create a new file called "test.php" under this folder.
  3. Put your PHP code snippet in this file.
  4. To run the file, just enter the URL <your domain>/index.php?option=com_test. For example, http://localhost/joomla_development/index.php?option=com_test.

This will run the code snippet inside the Joomla! framework. This way, when you are done, you can just remove the com_test folder and no other files have been changed. (Sam, thanks for this tip!)

General Tips & Tricks

  • If you have the cache enabled (Global Configuration → System → Cache Settings), you will need to clean the cache each time you make a change to the code or parameters. Otherwise, you might be seeing the older cached version of the page. For this reason, unless you are specifically testing the cache feature, test with the Cache set to "No".
  • When working it is best to set error reporting to maximum and turn on all debugging options (these are set in Global Configuration).