Providing Good Feedback for Bug Reporters
 by Tracey Clark, in Editorials - Sat, Jan 24th 2004 00:00 UTC

A comment on a bug I submitted recently spurred me to provide some feedback from an application user's perspective on bug reports. There are ways of responding to a bug report that encourage the types of responses that are helpful to developers, and there are ways of responding that only produce anger and frustration, without getting anything fixed. My hope is to encourage good communication between bug reporters and developers to enable better, quicker bugfixes.

My Background

I have been reporting bugs for a few years, on various pieces of software. I've had both positive and negative experiences with the responses I've received from developers. I come from a professional background of using computers, supporting computers, change control, and basic programming, so I've seen problem reporting from a few angles.

To begin, I'll outline responses I have received or seen that were unhelpful. Following that will be responses I have seen that were helpful. Finally, I'll outline some general guidelines which I feel promote good communication between developers and reporters.

Unhelpful Responses

It's not our product, it's some other software causing the problem (with no debugging or troubleshooting).

Trying to shift blame with no investigation does nobody and nothing any good. It only makes the developers look like they don't want to take responsibility for a possible bug or don't want to do any troubleshooting. Some reported bugs do wind up really being a problem with other software. If properly investigated, you've got the data to back the position that other software caused the bug. Posting the exact reason it's a problem with other software to the report is great feedback for the bug reporters, who can then report it to the correct place. They also know you cared enough about their bug to help them get it to the right place, making them more likely to report future bugs.

I can't reproduce it/works for me (with no other qualifications).

That doesn't help anyone, and it sounds like another "I don't wanna do work" response. The fact that person B cannot reproduce a bug does not negate the fact that person A can always reproduce it. Helpful information to add to a comment like this would be: Did you use the same version on the same OS, following exactly the same steps (using menus as opposed to an icon), etc.? The programmers may have done things exactly the same way, but maybe they didn't. The bug reporters don't know unless they are told. Also helpful are comments about what you're trying to do to reproduce the problem, such as "I'm continuing to try this on another machine" or "I'm working with a few people to see if we can reproduce this bug". One failed attempt to reproduce the bug with no further work isn't working the bug, it's akin to lip service. Tell the reporters instead what other troubleshooting information they need to give you to track the bug down.

The two responses above are unfortunately so common in the IT world, and so well-known by users (who aren't fooled), that a company made an IT-style Magic 8-Ball. The answers include:

• It's not a problem with my code.
• I can't reproduce the problem.
• Reboot your computer.
• Etc.

&!*#*&... We've said a million times not to report x because blah blah...

(And other not-so-subtle "You're stupid for having contacted us" or abusive types of responses).

A polite pointer to the FAQ or appropriate documentation is more appropriate. Telling people they shouldn't have written about one bug will discourage them from writing about any bug.

No response (when assigned).

Hello? Is anyone watching this thing? Even "Bug in queue, will get to this" would be helpful. (I like the note in Mozilla's Bugzilla which says "This means the bug is awaiting triage..."). Getting no response may lead reporters to feel that their time is being wasted.

Helpful Responses

Thanks for your submission.

It's always appropriate to thank someone for taking the time to report a perceived problem, even if it turns out not to be a problem with the software they thought was the problem. Even a badly-written bug report can be helpful with a bit of work (in most cases). The reporter can always be given a link to a "How to report bugs well" FAQ or your project's bug guidelines. Politeness is good. Happy bug reporters help us troubleshoot our software. They are a valuable resource to the Open Source community. Pissing them off means that you don't get as much testing from them. A simple "thank you" and a kind attitude go a long way to make someone feel respected and happy.

I can't reproduce the problem, but here's something you can do to provide me with more information....

Letting people know what else they can do to clarify a bug or provide more troubleshooting information lets them know that their response is not wasted, that the developer cares enough about the problem to want more information and actually wants to get the bug fixed.

Bug confirmed, working on this...

You may not yet know the cause of the problem, and don't have time to write much. A simple note to say you've duplicated the issue and you're looking into it lets the users know their bug report has been received and something is being done. This improves their confidence that their reports mean something.

Suggestions

Better communication and more questions rather than snarky answers make for happier bug reporters and better bug troubleshooting. Here are a few other, general suggestions that go toward improving communication:

Have a written, easily findable bug submission procedure for your project.

People who report bugs may not know what information is helpful for the developers. Your bug submission guidelines will tell them what that information is. You can't guarantee that everyone will read it or follow it. You can guarantee that you will get less of what you need in terms of helpful bug reports if you don't have one.

Try to speak in terms the bug reporter can understand.

Perhaps the meaning of "just run lsmod" is obvious to you, but it may not be obvious to the bug reporters. This doesn't make the reporters bad, nor does it justify getting angry at them. No one is born knowing all computer commands. Not everyone has the same level of expertise in the same areas as a developer. Communicating with the reporters in ways they understand means you will get more helpful answers from them. Sometimes, they can run a command they didn't know before if only you tell them how to do it or point them at the appropriate FAQ.

Don't respond in anger, respond when you are calm.

People are responding to your request to submit bug reports, and they deserve to be treated with a basic amount of respect. Responding in anger will increase the likelihood that you will say something inappropriate or something that you may regret. Wait until you have a cool head to respond to comments.

Respond with common courtesy, don't put people down.

Again, if you're asking people to submit bugs, putting them down when they do is counter-productive. It will only make you and your project look bad. Treating people respectfully increases their willingness to help you get your bugs solved and your programs running more stably. This shouldn't have to be said, but there it is.

There are many things we can do to improve communication in bug reporting. This article provides a few which can be used in most situations. By following practices of courtesy and good feedback, we can improve the rate at which bugs get fixed and encourage the volunteers waiting to help us in that process. The Open Source community as a whole can benefit.

This article is licensed under the Creative Commons Attribution-ShareAlike License. To view a copy of this license, visit http://creativecommons.org/licenses/by-sa/1.0/ or send a letter to Creative Commons, 559 Nathan Abbott Way, Stanford, California 94305, USA.

Author's bio:

Tracey Clark can be reached at grrliegeek@elenari.net.

T-Shirts and Fame!

We're eager to find people interested in writing articles on software-related topics. We're flexible on length, style, and topic, so long as you know what you're talking about and back up your opinions with facts. Anyone who writes an article gets a t-shirt from ThinkGeek in addition to 15 minutes of fame. If you think you'd like to try your hand at it, let jeff.covey@freshmeat.net know what you'd like to write about.

[Comments are disabled]

Referenced categories Topic :: Software Development :: Bug Tracking

 Comments

[»] Both sides
by Erik Ljungström - May 5th 2004 17:12:35

This sheds some light over a few problems which affects both sides. From a developers point of view, I want clear, direct and well worked bug reports. A hasty and unresearched report is as good as no report at all. Then there's the bug reporters angle, which is (as previously pointed out) almost the same as the developers. They take time and sometimes effort to improve something that I, the developer, has messed up. Not always to gain something for themselves. Another rather important issue here is, how far does "common people" go in order to get their bug reported. I mean, some projects actually requires that the user registers on some bug database (with all the mail, approval mail, confirmation mail etc. it generates), find the suitable category, put the bug there, unsubscripe from the list (back to e-mail hell). Of course I understand that larger projects needs this in order to maintain sanity. Now, going through that alone is worth a kiss imho.

Personally, I accept all bug reports myself, through common e-mail, and I try to be as polite as possible in my responses, regardless of how the initial contact was. But sometimes, deep down, I rage upon those who sends things such as "This doesn't work.. bad bad bad Erik!".

So, to sum it up, this is a bigger issue than it at first seems. All bug reporters should follow the steps earlier described in this thread, and all developers should act the way the initial poster means (or better yet, stop producing bugs :-)

[reply] [top]

[»] Code sollutions
by Noah Roberts - Feb 23rd 2004 10:47:00

You might also consider not talking crap about someone's patch submision in front of them. When someone takes the time to actually attempt fixing a problem and track you down to give you the patch, don't kick them in the teeth. If the patch is mistaken there are better ways to tell them then saying, "This is crap and I am not going to use it." There are certain projects I won't be helping because this is the responce I got when trying to be a helpful user.

This would be under "Common Courtesy" in the article, but I think it deserves special attention. When someone goes out of their way to attempt a sollution to the bug be nice to them.

NR

[reply] [top]

[»] Re: Code sollutions
by Buy Gifts - Apr 10th 2004 00:09:56


> You might also consider not talking crap
> about someone's patch submision in front
> of them. When someone takes the time to
> actually attempt fixing a problem and
> track you down to give you the patch,
> don't kick them in the teeth. If the
> patch is mistaken there are better ways
> to tell them then saying, "This is
> crap and I am not going to use it."
> There are certain projects I won't be
> helping because this is the responce I
> got when trying to be a helpful user.
>
> This would be under "Common
> Courtesy" in the article, but I
> think it deserves special attention.
> When someone goes out of their way to
> attempt a sollution to the bug be nice
> to them.
>
> NR I totally agree with you... Your right on the money!

--
Digital Cameras - Biz

[reply] [top]

---

[»] Reality Check
by KotH - Jan 24th 2004 15:13:38

Did you ever think about who those developers are you want to reach ? Those are OSS developer who work in their free time on a project. They work on it because it's fun to do it. But reading shitty bugreports and trying to read stupid "it does not work" peoples minds is no fun at all.

And mind you, i've done it, i've done it for a long time, every day 1 to 5 (!) hours. Most time telling people to read the bug submission procedure (which is neither hard to find nor hard to read) or those who claim to have read it to read again and try to understand what it really says.
When it wasn't this, i told them to read the FAQ as their "bug" was already covered there (mostly for several months) or to simply have a look at the documentation as it explains what they did wrong.
Overall not even 1% of the "bugreports" were actual bugs but simple PEBCAK which could be easily solved by just taking their time to read the documentation even a grep would have shown the solution in most cases.

As i said, i did this for a long time several hours a day. I never counted how much time i worked for others, solving their problems w/o being paid for it. I did it and i don't regret it, but i also stopped it because seeing that nobody ready your mails about how a bugreport should be send just plainly sucks.

Thus IMHO you should rethink what you wrote and rather adress those who send bugreports. To tell them to use their common sense and their brain. It's not hard to write a proper bugreport, but just a few seem to be able to do so.

1) Get a clue what happend.
Did your app crash ? Did it do something you did not expect ? What might have caused this ? Can you reproduce it ?

2) READ and UNDERSTAND the error message
I'm sure there is one. Also switching verbose mode on and trying again might be a good idea to know what the app was actualy doing.

3) Read the documentation
Start from the FAQ and read all parts of the documentation that seem even remotly relevant and most importantly try to understand them. In most cases you can solve your problem by this yourself.

4) Narrow down what the problem is
Try again with different settings/options/files and try to narrow down the submodule which might cause the problem. It's a difference to know whether it was the "whole program" or just a small part of it that might doing something wrong.

5) Type your error message/problem description into google
You might laugh, but just a c&p of the error message into google will get you a few hits of people who already had the same problem with possible solutions.

6) Find the bug submission procedure of your application
Most projects have a section of their documentation devoted to reporting bugs. Read it, understand it, read it again. Then follow it precisely and word for word, don't think about it, just do as it says whatever it might be. There is a reason why the developers wrote that.

7) Write the bugreport to the proper place
Most projects have a user mailinglist, a bugs mailinglist or a bugtracker. Find it and use it. NEVER write to a developer directly, never ever. They are already burried in thousands of mails, they dont need another one to send to /dev/null. Also follow the guidelines about proper mail formating (not more than 70 characters per line, no html, no big attachments) and NEVER press "reply" on another mail if you start a new thread, people will shot you for this and most probably your mail will get lost in the other thread.

8) Wait
Don't be hasty little hobbits. Those are humans with a life outside the world of bits and bytes. They have a work, a family, friends and lots of other things to do. It may take a few days or even weeks until someone responds to your bugreport.

9) Providing additional information
If you are asked to provide additional information or to try something, then you should write a reply to the mail which is a) properly formated (the things mentioned above plus proper quoting, NO TOFU!) and b) with correct mail headers that provide In-Reply-To and References fields. Don't use broken shit like Webmailers or Outlook, they just break the thread and the other side will have a hard time to get what your previous mail was about.

10) Send a thank you
If someone took a lot of time to help you with your problem than you should thank him


Sometimes i think that we should charge everyone sending a bugreport a few dollars, just to cover those 5 minutes it takes to quickly read his bugreport. Maybe then they'll think twice about writing "it does not work, please help me". Beside, it would cover the costs of buying new hardware for the servers or maybe even make us rich :)

[reply] [top]

[»] Re: Reality Check
by Greg Sharp - Jan 24th 2004 20:56:12

KotH, Mr. Clark is simply reminding us that the bug reporter is not paid for his effort. A polite, prompt, and thoughtful reply is generally appropriate.

Concerning your particular advice, I respectfully disagree on a few points. I have gotten many useful bug reports which were hastily written (8) or poorly researched (1-5,9). However, as you point out, using proper reporting procedures (6,7) and being courtious (10) are very much appreciated.

Cheers, Greg

[reply] [top]

---

[»] Re: Reality Check
by jeff covey - Jan 26th 2004 11:13:10


> Mr. Clark is simply reminding us [...]

Somehow, I don't think that Tracey (AKA "grrliegeek") should be called "Mister". :)

--
vs lbh pna ernq guvf, lbh'er n trrx.

[reply] [top]

---

[»] Re: Reality Check
by Greg Sharp - Jan 26th 2004 12:09:52


>
> Somehow, I don't think that Tracey (AKA
> "grrliegeek") should be called
> "Mister". :)
>
I'm sorry Tracey. No disrespect intended.

One more "unhelpful response" is when a developer gives you a positive initial response, but then after that you never hear anything again. This often happens with small and solo projects.


[reply] [top]

---

[»] Re: Reality Check
by Kirby Vandivort - Feb 6th 2004 07:37:58


>
> KotH,
>
> Mr. Clark is simply reminding us that
> the bug reporter is not paid for his
> effort.
On the contrary, a bug reporter typically _is_ expecting to be paid for the effort, in the form of a bug fix that fixes the problem that they are experiencing at that particular moment.



[reply] [top]

---

[»] Re: Reality Check
by David Eklund - Jan 26th 2004 23:49:28

I think this response is a good example of why this article was written in the first place.

I can understand being frustrated by unhelpful, uninformed bug reports, but in reality, anyone who bothers to do so is in fact doing you a favor. If a user complains about something that is explained clearly already in a FAQ somewhere, then it's no problem to politely redirect them to that FAQ - one need not get upset about it.

In any case, responding angrily to a bug report only makes for a greater rift between the developer(s) and the users. It's important in any software project - and particularly in an OSS project - to have healthy communication between the developer and user base, and in order to do so, one has to maintain a certain level of politeness when the person on the other end fails to understand.

Of course this is true on the user's end as well - you can't always expect the developer to respond to you within an hour, telling you all your needs and problems have been magically dealth with - but I think it's particularly a problem on the developers' end, who often come off as haughty and intimidating.

I think the final lesson is this - if your users are afraid to submit bug reports, then there's something wrong; and a little courtesy goes a long way in maintaining a healthy level of communication with your users.

[reply] [top]

---

[»] Re: Reality Check
by paolino - Jan 27th 2004 09:44:09

% 2) READ and UNDERSTAND the error
> message
> I'm sure there is one. Also switching
> verbose mode on and trying again might
> be a good idea to know what the app was
> actualy doing.
IMHO a user might not *understand* an error message, what is useful to a developer as an error message might be totally unmeaningful to the casual user: "Wrong pointer to param &frobnicate of method foobar" or even a more user-friendly "Passing bad parameter in configuration file. Found int, expecting string" might not be something people understand ("what is a string, anyway?").


> 4) Narrow down what the problem is
> Try again with different
> settings/options/files and try to narrow
> down the submodule which might cause the
> problem. It's a difference to know
> whether it was the "whole
> program" or just a small part of it
> that might doing something wrong.
This might help, but do you prefer to have a bug fixed or a lazy guy not reporting (or being devnulled) because he didn't try all the possible configuration/compiler/modules switches?


> 5) Type your error message/problem
> description into google
> You might laugh, but just a c&p of
> the error message into google will get
> you a few hits of people who already had
> the same problem with possible
> solutions.
IMHO, if other ppl solved the problem either you know or you want to know in order to put the solution in documentation. Then, reply writing "the solution is in FAQ 3... Thanks for reporting anyway". It's 1 minutes and makes people feel good about you and your project.


> 9) Providing additional information
> If you are asked to provide additional
> information or to try something, then
> you should write a reply to the mail
> which is a) properly formated (the
> things mentioned above plus proper
> quoting, NO TOFU!) and b) with correct
> mail headers that provide In-Reply-To
> and References fields. Don't use broken
> shit like Webmailers or Outlook, they
> just break the thread and the other side
> will have a hard time to get what your
> previous mail was about.
People might be less netiquette-prone than you. Do you think this is a good reason not to solve problems in/with your software?


> 10) Send a thank you
> If someone took a lot of time to help
> you with your problem than you should
> thank him
I agree, and think that the developer should thank the reporter as well... :)


> Sometimes i think that we should charge
> everyone sending a bugreport a few
> dollars, just to cover those 5 minutes
> it takes to quickly read his bugreport.
> Maybe then they'll think twice about
> writing "it does not work, please
> help me". Beside, it would cover
> the costs of buying new hardware for the
> servers or maybe even make us rich :)
They are (trying to) work for you: trying to let you know that your program has what they think is a bug. That is either because *it is* or because they didn't find out it wasn't. That's a problem, anyway. And the developer should try to fix it, imho.


cheers
paolino

--
I don't think so... -- as Descartes said just before disappearing prg

[reply] [top]

---

[»] Re: Reality Check
by Kathryn Andersen - Jan 30th 2004 20:22:29


> They are (trying to) work for you:
> trying to let you know that your program
> has what they think is a bug. That is
> either because *it is* or because they
> didn't find out it wasn't. That's a
> problem, anyway. And the developer
> should try to fix it, imho.
Indeed. A bug report for something which isn't a bug can be an indication of another kind of bug -- a bug in the documentation. Since mine tend to be one-person projects where I write the docs too, I want to know about those sort of bugs too. If one person misunderstood something, other people probably will too, and it's worth rewriting that section of the docs to make them clearer.

--
Say it with flowers -- send a Triffid.

[reply] [top]

---

[»] Re: Reality Check
by Tracey Clark - Feb 2nd 2004 10:56:34


> Did you ever think about who those
> developers are you want to reach ? (snip)

> Thus IMHO you should rethink what you
> wrote and rather adress those who send
> bugreports.

KotH, of course I had to think about developers when I wrote this article. I think about the developers every time I submit or work a bug (I work with the Mozilla Project & more recently with the Fedora Project). The article is written with both the developer and the person submitting the bug in mind. Note this from the first paragraph:

There are ways of responding to a bug report that encourage the types of responses that are helpful to developers

There are already Bug Submission HOWTO documents if you care to Google for them. I wrote my HOWTO because this end of bug reporting didn't seem to be covered as well.

Ms. Grrliegeek ;)

--
Tracey Clark

[reply] [top]

---

[»] Been there
by Macdaddy - Jan 24th 2004 12:09:45

I too have reported bugs for years. I also report web form errors and bad links. Usually I get a decent response or no response. Occasionally I get the damnedest responses. They are the rudest responses I've ever encountered. When I receive one of these I create a message that quotes all of our coorespondence and a well-formed inquiry directed to management-type as to whether this is the appropriate way to respond to a customer or potential customer. Then I send that message to every single address I can find for a given company. If you want to make an ass out of yourself to me in private email, you'd best be prepared for all your co-workers and supers to hear about it. I always get a good response to these messages from some sort of management-type person.

[reply] [top]

[»] Well said...
by Lina Inverse - Jan 24th 2004 11:55:41

Every developer should read this and keep this article in mind.

--
Regards, Lina

[reply] [top]

[»] Re: Well said...
by SystemMobile - Jan 24th 2004 21:06:22


> Every developer should read this and
> keep this article in mind.
I agree. This article has made me consider how I respond to questions about articles I've written as well as to queries about my OSS software.

[reply] [top]

---

[»] Re: Well said...
by Agent - May 8th 2004 01:40:11


> Every developer should read this and
> keep this article in mind. I double that, i could not have said it better my self. I would like to add another comment, i think the problem with todays programmers is more of an issue where they are working on the same darn thing for so long they just get sick of fixing the bugs. They want to see something new and start programming some more bugs :)

--
Real Estate - Rentals

[reply] [top]

---