Back to Problem Report Form Bug Description Check List for bugs that crash or occur intermittently
What kind of problems should you report?
Problem reports fall into five categories: program bugs, implementation errors, enhancements, new features, and documentation.
A program bug is a problem with the code that causes the program to do something in an incorrect way. This doesn't mean that you would prefer that the program work in a different way, or that it works in a way that you don't understand, or that it works in a way that is different from CANSARP 3.0 (see the section on feature requests below). Instead, it means that the program functions in a way other than its proscribed manner for this version or that it simply doesn't function at all when it should. User interface issues are also program bugs. These include display problems within the interface which prevent or interfere with your ability to perform certain tasks from within the system.
An implementation error is an error in the way that SAR theory and/or search planning practice is implemented in the program which prevents or interferes with your ability to plan a search in keeping with the procedures outlined in NSARM or IAMSAR.
Enhancements are requests for minor changes to existing features that will improve the usability of the tool. Examples include changing the names of fields or buttons to more accurately describe their function or moving buttons or check boxes to a new location so that they're closer to other related widgets.
Feature requests include suggestions for new features or changes to existing features beyond minor enhancements. Suggestions for old features or old behaviours (i.e. that something work like it did in CANSARP 3.0 rather than the way it does now) are other examples of feature requests. While this input is valuable and appreciated, there is no time or budget available to make major changes to CANSARP V5.0. Your suggestions will be recorded and considered for future versions of the program should time and money allow at that time.
It can be difficult to tell whether a change would be easy to implement (an enhancement) or require significant effort (a feature request) without being able to "see under the hood" of the program. Because of the interdepedence of various aspects of the program, a seemingly simple change in one place can have a cascade effect and produce or require complex changes in other places. Be advised that problem reports classified as enhancements by the reporter may be changed to feature requests by the programmers once they determine how much effort is required to implement the change. By the same token, if a feature request appears complex but could actually be implemented with only minimal effort, it might be upgraded to an enhancement report.
Finally, issues in the documentation that merit a problem report include errors or omissions in the user manuals and/or the online help (bearing in mind that such documentation is still being developed). Examples include information that is inaccurate or out of date, instructions that do not work (or have been omitted), terminology that is incorrect, unclear or undefined, or screen shots that do not match current versions of the software.
How to Investigate and Describe a Program Bug
The most important thing is that a report be filed when a problem occurs. If a sentence or two is all you have time for, by all means send us that. Any report, no matter how brief or incomplete, is better than no report at all.
Having said that, the more complete and clear the description of the problem is, the easier it will be for a programmer to reproduce the problem and then isolate and correct the error in the code, especially for bugs that cause the program to crash or bugs that only occur intermittently.
For example, a description such as "The wizard crashed in step 2" tells us something important, but it doesn't tell a programmer very much about what might have triggered the crash. What was happening before the wizard was opened? Was the wizard used to create a new scenario or to modify the active scenario? If it was a new scenario, was it being added to an active case or to a new one? Was any case open previously? If it was used to modify the active scenario, was it the first time it had been opened, or was it reopened? Were any of the fields changed in step 2 or earlier? Did it crash when the cancel button was clicked? The finish button? Before any buttons were clicked? Etc. etc.
As you can see, this report would require that someone spend time and energy imagining and then testing many possible situations in hopes of triggering the same crash. You can help us and speed up bug fixing considerably by including as much information as possible in your detailed description.
Be complete in your description. If you can't reproduce the bug or don't have time to try, describe the situation that led to it as completely as possible. The more information you can provide about what you did (or what you didn't do) the better, as it will give the programmer a head start on reproducing the bug himself by narrowing down all the different things he has to consider. Refer to the bug description check list and try to answer as many of the questions as you can.
One reason we ask you to describe what you'd been doing recently in the case of crashing or intermittent bugs is that many bugs actually turn up some time after the initial error occurs, once some other task or system that's performing correctly tries to use memory or data that was mishandled by an earlier task. The bug may appear to be connected to the second event, the one that produced the obvious error, but the source of the problem is actually the first task. Without knowing all the steps that lead up to a problem occuring, it can be impossible to isolate the underlying cause.
Another reason for the detail is that the programmer might not perform the same tasks in quite the same way that you do. Everyone develops habits when they use a tool for an extended period and there might be something unique about your habit that triggers the bug, or about a programmer's habit that avoids it. This is why the check list asks how you accessed the task that triggered the problem: if you always open an editor from the canvas but the programmer always opens it from a manager, it'll take him longer to find the bug if it's connected to the way the editor is opened.
Include the name of the dialog box/window/field/button/menu item/other widget, using the exact spelling that appears in the program. Keep in mind that CANSARP contains over 100,000 lines of programming code and many parts of it are very similar. If you use the exact spelling, a programmer can search through the code for that particular phrase and find it more quickly.
Take a screenshot. Sometimes a picture is worth 1000 words. If the problem is visible on the canvas in some way, a screenshot can be helpful and informative. Be sure to turn on display any of environmental data that's affecting drift or is otherwise connected to the problem as long as it doesn't obscure the problem you're describing. If it does and you've got time to take a couple of screenshots, take one with the data displayed and one without.
If in doubt, preserve the case or scenario. When the bug is really weird or only happens in one particular scenario, sometimes a programmer can only get to the root of the problem by opening the buggy scenario itself. If possible, stop making changes to the scenario; if you must keep working with the scenario, duplicate it in the Scenario Manager and then continue your work in the new copy. That will leave the original, buggy scenario untouched. In either event, include the case name and scenario number in your bug report so that the programming team can download the files and have a look.
If you have time, try to reproduce the error. If the bug doesn't happen all the time, document what you were doing when it happened and then try to make it happen again. If you can't reproduce it with direct experience of how it occured, imagine how hard it will be for someone else. If the bug can't be reproduced, it probably can't be fixed. If you can reproduce it, your description of what steps you take to make the bug happen repeatedly are even more helpful.
If you have even more time, try to isolate the trigger. If you can reproduce the bug, try different variations on the steps required to trigger it to see what's really a contributing factor and what's just coincidence. For instance, if you know that the bug happens if you do X, Y and Z, see if it happens if you do Z, Y and X instead, or only do X and Z, or only Y and Z. Try the same steps in different scenarios to see if it happens all the time or only in one scenario.
Also try related tasks. If you know that one thing triggers a bug and that there are other closely related systems in the software (another manager that does something similar, for instance), see if the bug also happens in that other system. Bear in mind that some things appear to be different systems but are actually the same thing reused in different places. The Time Series Editor, for instance, is exactly the same window whether it's opened from the Wind Manager, from the Current Manager or from the canvas, even though the label at the top of the window changes.
Explain your reasoning for an enhancement or a feature request. Understanding how a change would improve the software can have an impact on what priority it's given, and could make the difference between it being considered a P5 feature request or a P3 problem report. For instance, a request to "remove colour X from list of search object colours" might be deferred as a P5 enhancement, whereas "remove colour X from list of search object colours - printer always prints that colour as white so it's invisible in tasking forms" is a much more pressing problem that would be addressed more quickly.
Even if the explanation doesn't elevate a feature request to enhancement or problem report status, it might suggest an alternative short term solution that's quick and easy to implement. At the very least, the reason for the request will be recorded in the bug tracking system so that when the next version of the software is being developed the designers have a better idea what's wanted and why.
Problem reports are prioritized according to the impact that they have on the use of the program so that programmer effort is focused where it will have the most benefit. Every problem report is assigned a prioritization (P) rating from 1 to 5 in the bug tracking system. These priority ratings can be qualitatively described as urgent (P1), high priority (P2), routine (P3), low priority (P4) and deferred (P5).
Bugs that cause the program to crash, make it unstable or otherwise unusuable are given the highest priority. Generally speaking, other program bugs and implementation errors range from urgent to routine (P1 to P3), enhancements to existing features and updates to documentation are routine or low (P3 or P4), and feature requests are deferred (P5) but are kept in the system as a resource for the planning stages of future versions.