Difference between revisions of "Bug Tracking Process"

From Joomla! Documentation

(Remove the "Needs help" Message)
(48 intermediate revisions by 14 users not shown)
Line 1: Line 1:
=Overview=
+
This article documents the current Joomla! bug tracking process from the time a new bug is reported to the time it is closed.
The [http://joomlacode.org/gf/project/joomla/tracker/?action=TrackerItemBrowse&tracker_id=32 Joomla! Bug Tracker] is the place where all Joomla! bugs are tracked. This article documents the current Joomla! bug tracking process from the time a new bug is reported to the time it is closed.
 
  
=Reporting Issues=
+
Joomla! bugs are tracked in the [http://issues.joomla.org/ Joomla! Issue Tracker], this Tracker is for issues with supported Joomla Versions. As of June 2015, this is version 3.4+
  
[[Image:ReportingIssues.jpg|1024 px]]
+
==Helping out==
  
The process is normally started in one of two ways: the bug is added to the tracker, or a user reports the bug in the [http://forum.joomla.org/viewforum.php?f=199 Joomla! Bug Forum] for the given maintenance release.
+
You do ''not'' need to be a member of the JBS to help fix bugs in Joomla. [[Filing bugs and issues|Anyone can report bugs]], test patches, or submit patches. If you want to help with resolving bugs, go to the [http://issues.joomla.org/ Tracker]. You can help resolve Open issues as outlined below. You can create and submit patches for Confirmed issues. Or you can help test Pending issues. To report about what you have done, login with a Github account and add a comment. You'll be amazed at how much impact you can have and how good it feels to contribute to the Joomla! project.
  
== Issues reported on the forum ==
+
If you have any questions, or are interested in joining the [[Portal:Bug_Squad|JBS]], please contact [mailto:bugsquad@community.joomla.org The Bug Squad Coordinators].
 +
 
 +
===Testing Pre releases===
 +
 
 +
Before a stable release of Joomla is released the update needs testing.  This is a simple process:
 +
# Install the current stable release of Joomla in a subfolder or on localhost (you can use a copy of a production site but '''never test on a live production site''').
 +
# Navigate to the Options of the Joomla! Update component and set the 'Update server' to 'Test'. [[Help32:Components Joomla Update|Components Joomla Update]]
 +
# When a Joomla pre release is ready for testing, log into your test site and install the Update.  Then test the site works as expected.  Please report any errors/bugs on the [http://issues.joomla.org/ Joomla! Issue Tracker]
 +
 
 +
==Reporting Issues==
 +
 
 +
[[Image:ReportingIssues.png|ReportingIssues.png]]
 +
 
 +
The process is normally started in one of two ways: the bug is added to the respective tracker, or a user reports the bug in the [http://forum.joomla.org/viewforum.php?f=728 Joomla! Bug Forum] for the given maintenance release.
 +
 
 +
=== Issues reported on the forum ===
  
 
JBS members scan the forums to determine when issues need to be put into the tracker. If the issue can be reproduced, is clearly a bug, and there are step-by-step instructions for how to reproduce it, it can be entered into the tracker with a status of Confirmed. If it is not as clear-cut, it can be entered with a status of Open, so that other JBS members will know it needs further investigation.
 
JBS members scan the forums to determine when issues need to be put into the tracker. If the issue can be reproduced, is clearly a bug, and there are step-by-step instructions for how to reproduce it, it can be entered into the tracker with a status of Confirmed. If it is not as clear-cut, it can be entered with a status of Open, so that other JBS members will know it needs further investigation.
  
== Issues directly reported to the tracker ==
+
=== Issues directly reported to the tracker ===
  
When an issue is added to the tracker, the status will be either
+
When an issue is added to the tracker, the status could be depending on the situation;
1. Open  
+
# Open
2. Confirmed
+
# Confirmed
3. or Pending  
+
# Pending
depending on the situation. If the issue needs more investigation, then it should be set to Open. If the issue (1) is a bug and (2) can be reproduced and (3) has good test instructions, it should be set to Confirmed. If it meets the three Confirmed criteria and also has a good patch attached, it should be set to Pending. See below for more information about the status codes.
+
If the issue needs more investigation, then it should be set to Open. If the issue (1) is a bug and (2) can be reproduced and (3) has good test instructions, it should be set to Confirmed. If it meets the three Confirmed criteria and also has a good patch attached, it should be set to Pending. See below for more information about the status codes.
  
== Issue Priorities ==
+
=== Issue Priorities ===
  
Most issues are priority 3, or Normal. The artifacts are prioritized according to the following characteristics:
+
[[Bug and Issue Tracker Priority|Why most issues are priority 3]], or Normal. The artifacts are prioritized according to the following characteristics:
  
 
'''Critical (1):'''
 
'''Critical (1):'''
The trunk is not working at all. Significant parts of the source are broken preventing key operations.  Examples would be login, installation, extension installers, javascript errors that prevent you from moving a save or similar action, etc.  Also includes the generation of Fatal PHP errors.
+
The trunk is not working at all. Significant parts of the source are broken preventing key operations.  Examples would be login, installation, extension installers, javascript errors that prevent you from moving a save or similar action, etc.  Also includes the generation of Fatal PHP errors and major security issues in a prerelease (Security issues for a stable release should NOT be reported in the tracker but instead reported to the security team security@joomla.org).
  
'''Major (2):'''
+
'''Urgent (2):'''
Parts of the source are obstructing operation in a serious way or causing a major loss in advertised function.  Examples would includes PHP notices and warnings and reported javascript errors.
+
Parts of the source are obstructing operation in a serious way or causing a major loss in advertised function.  Examples would includes PHP notices and warnings and reported javascript errors.  Major issues will also typically prevent the release cycle from moving from Beta to Release Candidate (RC), or Release Candidate to General Availability (GA).
  
'''Normal: (3)'''
+
'''Medium (3):'''
 
Issues that are hindering advertised behavior but the application is still workable.  Examples would include parameters not working as advertised, language files not loading as expected, etc.
 
Issues that are hindering advertised behavior but the application is still workable.  Examples would include parameters not working as advertised, language files not loading as expected, etc.
  
'''Minor (4):'''
+
'''Low (4):'''
 
Minor loss of function and generally annoying behavior.  May include less common platform or browser specific problems that while they may be technically major in those environments, they represent a minority.  Also include missing translation strings.
 
Minor loss of function and generally annoying behavior.  May include less common platform or browser specific problems that while they may be technically major in those environments, they represent a minority.  Also include missing translation strings.
  
'''Trivial (5):'''
+
'''Very low (5):'''
 
Cosmetic problems, misspelled words, graphically misaligned object, less common issues with parameters, etc.
 
Cosmetic problems, misspelled words, graphically misaligned object, less common issues with parameters, etc.
  
= Resolving Issues =
+
== Resolving Issues ==
  
The bug squad takes care of the 1.5 release which is now in maintenance mode. That means getting the 1.5.1, 1.5.2, 1.5.3, 1.5.4 etc releases ready by fixing problems that come up. The idea is to make the release increasingly stable and take care of issues that come up. At the same time, it is vitally important not to break anything that is working. That's called software regression and it's not something you want at this stage.
+
The bug squad takes care of the Joomla releases. For example, that means getting the 3.4, 3.5, 3.6, 3.7 etc releases ready by fixing problems that come up. The idea is to make the release increasingly stable and take care of issues that come up. At the same time, it is vitally important not to break anything that is working. That's called software regression and it's not something you want at this stage.
  
In the 1.5 tracker there are several common statuses, mainly: open, confirmed, pending, ready to commit.
+
In the trackers there are several common statuses, mainly: open, confirmed, pending, ready to commit.
  
* '''Open''' means it's reported, but it hasn't been determined for sure whether it is a real bug or not. Many Open issues are not actually bugs. If the issue fits into one of the categories below, then the status is changed as indicated and the bug is closed:
+
* '''Open/New''' means it's reported, but it hasn't been determined for sure whether it is a real bug or not. Many Open issues are not actually bugs. If the issue fits into one of the categories below, then the status is changed as indicated and the bug is closed:
 
** Cannot be reproduced. We have tried the same thing the reported did but the software appears to work correctly. (In many cases, more information is needed to be able to reproduce a bug. See "Information Required" below.) Change status to '''Unable to confirm'''.
 
** Cannot be reproduced. We have tried the same thing the reported did but the software appears to work correctly. (In many cases, more information is needed to be able to reproduce a bug. See "Information Required" below.) Change status to '''Unable to confirm'''.
 
** Has already been reported in a different issue number. Change status to '''Duplicate report''' and add the number on the duplicates tab.
 
** Has already been reported in a different issue number. Change status to '''Duplicate report''' and add the number on the duplicates tab.
 
** Is a known limitation of the software. Change status to '''Known issue'''.
 
** Is a known limitation of the software. Change status to '''Known issue'''.
 
** Is a feature request, a mistake made by a user, or is the way the software is intended to work. Change status to '''Not a bug'''.
 
** Is a feature request, a mistake made by a user, or is the way the software is intended to work. Change status to '''Not a bug'''.
** Is a bug with an extension or some other external program. Change status to '''Not Joomla! core'''.
+
** Is a bug with an extension or some other external program or a server issue that will not be addressed. Change status to '''Not Joomla! core'''.
 
* '''Information Required''' is used if we need more information ''from the person who reported the issue'' to decide about the issue. For example, there are questions about how to reproduce the problem or other questions about the issue. If we get the information we need, then we can continue processing the issue. If we don't get the information within two weeks, then we can change the status to Unable to confirm (or another of the closed status codes if that is more applicable).
 
* '''Information Required''' is used if we need more information ''from the person who reported the issue'' to decide about the issue. For example, there are questions about how to reproduce the problem or other questions about the issue. If we get the information we need, then we can continue processing the issue. If we don't get the information within two weeks, then we can change the status to Unable to confirm (or another of the closed status codes if that is more applicable).
* '''Needs Review''' is used if we need a JBS Coordinator, Development Coordinator, or other experienced developer to review the issue. This is different from Information Required, which means that we need more information from the person who reported the issue.
+
* '''Needs Review''' is used if we need a PLT Member or CMS Maintainer for a review / decision. This is different from Information Required, which means that we need more information from the person who reported the issue.
* '''Confirmed''' means that JBS has confirmed that this issue is a bug in Joomla! that should be fixed. That's when the JBS tries to solve it or consults with the development team about a solution. ''At this point there should be clear step-by-step test instructions that indicate how to reproduce the problem.''  
+
* '''Confirmed''' means that JBS has confirmed that this issue is a bug in Joomla! that should be fixed. That's when the JBS tries to solve it or consults with the development team about a solution. ''At this point there should be clear step-by-step test instructions that indicate how to reproduce the problem. For version 1.6, use the Test Instructions field for this information.''
* '''Pending''' means that a patch has been submitted that the submitter believes fixes the bug. If needed, additional test instructions are entered for the issue. Every Pending issue should have instructions that tell the tester how to reproduce the problem and make sure the patch fixes the problem.
+
* '''Pending''' means that a patch has been submitted. Every Pending issue should have instructions that tell the tester how to reproduce the problem and make sure the patch fixes the problem.
* '''Ready to commit''' means that two separate people have successfully tested the same patch file, and it works correctly with the patch. Note that, for some issues that are more complex or higher impact, we may need more than two people to test or may need to test on multiple platforms.  
+
* '''Ready to commit''' means that (in general) two separate people have successfully tested the same patch file, and it works correctly with the patch. Note that, for some issues that are more complex or higher impact, we may need more than two people to test or may need to test on multiple platforms. For simple issues, such as fixing typos in language strings or comments, one tester is enough.
* '''Fixed in SVN''' means that, after reviewing the code, the JBS commit coordinators have determined that the patch is good and the change has been committed to the Joomla! codebase. At this point, it will be part of the next Joomla! maintenance release.
+
* '''Fixed in Code Base''' means that, after reviewing the code, the JBS commit coordinators have determined that the patch is good and the change has been committed to the Joomla! code-base. At this point, it will be part of the next Joomla! maintenance release.
 +
* '''Easy Test''' The Easy Test mark is not a dedicated status is is more a label to show Pending issues with easy test instructions.
 +
 
  
 
The flowchart below provides a visual guide to how the process for resolving bugs works.
 
The flowchart below provides a visual guide to how the process for resolving bugs works.
  
[[Image:ResolvingIssues.jpg|1024 px]]
+
[[Image:ResolvingIssues.png|ResolvingIssues.png]]
 
 
You do ''not'' need to be a member of the JBS to help fix bugs in Joomla. Anyone can report bugs, test patches, or submit patches. If you want to help with resolving bugs, go to the [http://joomlacode.org/gf/project/joomla/tracker/?action=TrackerItemBrowse&tracker_id=32 Tracker]. You can help resolve Open issues as outlined above. You can create and submit patches for Confirmed issues. Or you can help test Pending issues. To report about what you have done, login to joomlacode and add a comment. You'll be amazed at how much impact you can have and how good it feels to contribute to the Joomla! project.
 
 
 
If you have any questions, or are interested in joining the JBS, please contact the [http://community.joomla.org/magazine/author/75-mark-dexter.html JBS Coordinator].
 
  
<noinclude>[[Category:Development Working Group]][[Category:Bug Squad]]</noinclude>
+
<noinclude> [[Category:Bug Squad]]</noinclude>
 +
[[Category:Bug Tracker]]

Revision as of 04:51, 28 June 2015

This article documents the current Joomla! bug tracking process from the time a new bug is reported to the time it is closed.

Joomla! bugs are tracked in the Joomla! Issue Tracker, this Tracker is for issues with supported Joomla Versions. As of June 2015, this is version 3.4+

Helping out[edit]

You do not need to be a member of the JBS to help fix bugs in Joomla. Anyone can report bugs, test patches, or submit patches. If you want to help with resolving bugs, go to the Tracker. You can help resolve Open issues as outlined below. You can create and submit patches for Confirmed issues. Or you can help test Pending issues. To report about what you have done, login with a Github account and add a comment. You'll be amazed at how much impact you can have and how good it feels to contribute to the Joomla! project.

If you have any questions, or are interested in joining the JBS, please contact The Bug Squad Coordinators.

Testing Pre releases[edit]

Before a stable release of Joomla is released the update needs testing. This is a simple process:

  1. Install the current stable release of Joomla in a subfolder or on localhost (you can use a copy of a production site but never test on a live production site).
  2. Navigate to the Options of the Joomla! Update component and set the 'Update server' to 'Test'. Components Joomla Update
  3. When a Joomla pre release is ready for testing, log into your test site and install the Update. Then test the site works as expected. Please report any errors/bugs on the Joomla! Issue Tracker

Reporting Issues[edit]

ReportingIssues.png

The process is normally started in one of two ways: the bug is added to the respective tracker, or a user reports the bug in the Joomla! Bug Forum for the given maintenance release.

Issues reported on the forum[edit]

JBS members scan the forums to determine when issues need to be put into the tracker. If the issue can be reproduced, is clearly a bug, and there are step-by-step instructions for how to reproduce it, it can be entered into the tracker with a status of Confirmed. If it is not as clear-cut, it can be entered with a status of Open, so that other JBS members will know it needs further investigation.

Issues directly reported to the tracker[edit]

When an issue is added to the tracker, the status could be depending on the situation;

  1. Open
  2. Confirmed
  3. Pending

If the issue needs more investigation, then it should be set to Open. If the issue (1) is a bug and (2) can be reproduced and (3) has good test instructions, it should be set to Confirmed. If it meets the three Confirmed criteria and also has a good patch attached, it should be set to Pending. See below for more information about the status codes.

Issue Priorities[edit]

Why most issues are priority 3, or Normal. The artifacts are prioritized according to the following characteristics:

Critical (1): The trunk is not working at all. Significant parts of the source are broken preventing key operations. Examples would be login, installation, extension installers, javascript errors that prevent you from moving a save or similar action, etc. Also includes the generation of Fatal PHP errors and major security issues in a prerelease (Security issues for a stable release should NOT be reported in the tracker but instead reported to the security team security@joomla.org).

Urgent (2): Parts of the source are obstructing operation in a serious way or causing a major loss in advertised function. Examples would includes PHP notices and warnings and reported javascript errors. Major issues will also typically prevent the release cycle from moving from Beta to Release Candidate (RC), or Release Candidate to General Availability (GA).

Medium (3): Issues that are hindering advertised behavior but the application is still workable. Examples would include parameters not working as advertised, language files not loading as expected, etc.

Low (4): Minor loss of function and generally annoying behavior. May include less common platform or browser specific problems that while they may be technically major in those environments, they represent a minority. Also include missing translation strings.

Very low (5): Cosmetic problems, misspelled words, graphically misaligned object, less common issues with parameters, etc.

Resolving Issues[edit]

The bug squad takes care of the Joomla releases. For example, that means getting the 3.4, 3.5, 3.6, 3.7 etc releases ready by fixing problems that come up. The idea is to make the release increasingly stable and take care of issues that come up. At the same time, it is vitally important not to break anything that is working. That's called software regression and it's not something you want at this stage.

In the trackers there are several common statuses, mainly: open, confirmed, pending, ready to commit.

  • Open/New means it's reported, but it hasn't been determined for sure whether it is a real bug or not. Many Open issues are not actually bugs. If the issue fits into one of the categories below, then the status is changed as indicated and the bug is closed:
    • Cannot be reproduced. We have tried the same thing the reported did but the software appears to work correctly. (In many cases, more information is needed to be able to reproduce a bug. See "Information Required" below.) Change status to Unable to confirm.
    • Has already been reported in a different issue number. Change status to Duplicate report and add the number on the duplicates tab.
    • Is a known limitation of the software. Change status to Known issue.
    • Is a feature request, a mistake made by a user, or is the way the software is intended to work. Change status to Not a bug.
    • Is a bug with an extension or some other external program or a server issue that will not be addressed. Change status to Not Joomla! core.
  • Information Required is used if we need more information from the person who reported the issue to decide about the issue. For example, there are questions about how to reproduce the problem or other questions about the issue. If we get the information we need, then we can continue processing the issue. If we don't get the information within two weeks, then we can change the status to Unable to confirm (or another of the closed status codes if that is more applicable).
  • Needs Review is used if we need a PLT Member or CMS Maintainer for a review / decision. This is different from Information Required, which means that we need more information from the person who reported the issue.
  • Confirmed means that JBS has confirmed that this issue is a bug in Joomla! that should be fixed. That's when the JBS tries to solve it or consults with the development team about a solution. At this point there should be clear step-by-step test instructions that indicate how to reproduce the problem. For version 1.6, use the Test Instructions field for this information.
  • Pending means that a patch has been submitted. Every Pending issue should have instructions that tell the tester how to reproduce the problem and make sure the patch fixes the problem.
  • Ready to commit means that (in general) two separate people have successfully tested the same patch file, and it works correctly with the patch. Note that, for some issues that are more complex or higher impact, we may need more than two people to test or may need to test on multiple platforms. For simple issues, such as fixing typos in language strings or comments, one tester is enough.
  • Fixed in Code Base means that, after reviewing the code, the JBS commit coordinators have determined that the patch is good and the change has been committed to the Joomla! code-base. At this point, it will be part of the next Joomla! maintenance release.
  • Easy Test The Easy Test mark is not a dedicated status is is more a label to show Pending issues with easy test instructions.


The flowchart below provides a visual guide to how the process for resolving bugs works.

ResolvingIssues.png