What devs and doctors wish you knew about writing bug reports

Calendar icon   2022-10-16   Scroll icon  2121
Tag icon  communication , software , tech , technical writing , writing

Recently I gave a presentation at work about writing good bug reports and I've gotten quite positive responses from it so I figured I'd put it in writing here as well for anyone who would find it useful. Enjoy!

Why should I care about bug reports?

I'll start off with the carrot: you should care about writing good bug reports because it will help your problems get solved much faster. And I'm not just talking about actual software bugs but problems in general. A bug report is simply something that will help someone diagnose a problem you're having. And let me tell you, dear reader, if you make it easier for someone to understand and thus solve your problem, it is much more likely to happen. People who solve problems are busy, and when they have to inevitably start prioritising, you want to make it as easy possible for them to pay attention your problem.

The broadness of my definition is intentional. In my opinion, describing your symptoms to a doctor is fundementally the same process as opening an issue on GitHub is, they just use different lingo. I'll illustrate this with an example later.

It should be noted that writing good bug reports is hard, very hard. It takes practice, and this post is not giving you any hard and fast rules, just some rules of thumb to use and adapt as fits you.

Rule 1: Be to the point

The two most common types of error messages are: "Oops, something went wrong!" or

javax.servlet.ServletException: Something bad happened
    at com.example.myproject.OpenSessionInViewFilter.doFilter(OpenSessionInViewFilter.java:60)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
    at com.example.myproject.ExceptionHandlerFilter.doFilter(ExceptionHandlerFilter.java:28)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
    at com.example.myproject.OutputBufferFilter.doFilter(OutputBufferFilter.java:33)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
    at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388)
    at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
    at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
    at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
    at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418)
    at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
    at org.mortbay.jetty.Server.handle(Server.java:326)
    at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
    at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:943)
    at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:756)
    at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
    at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
    at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
    at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
Caused by: com.example.myproject.MyProjectServletException
    at com.example.myproject.MyServlet.doPost(MyServlet.java:169)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
    at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1166)
    at com.example.myproject.OpenSessionInViewFilter.doFilter(OpenSessionInViewFilter.java:30)
    ... 27 more

I cut out most of that java error message but you get the point. The problems with these is that they don't give you the right amount of information about exaclty what went wrong. The first gives you no information at all, and the second one technically does, but it burries it in so much unhelpfull bullshit that it's almost impossible to find. The amount of information to include has a sweet spot, and where that spot is, can vary wildely on a case by case basis

In case you still find that a little vague, let me illustrate a bit more concretely with a fictional example:

The case of Jan-Klaas Klompenmaker

Imagine you're a tech support for a company that has a webapp and you get the following email:

Hello,

I hope this email finds you well. My name is Jan-Klaas Klompenmaker. I was recently using your website but it doesn’t work for me. I am 87 years old so perhaps I am simply not understanding how it is meant to work, sorry if this is the case. Normally I would ask my daughter to help with this but she is currently on holiday in Cyprus so I cannot ask her for help you see. I am on Apple, if you could help me I’d really appreciate it. Thank you and enjoy the rest of your day.

Best wishes, Jan-Klaas Klompenmaker

Setting aside the politeness, the email contains only three bits of "relevant" information:

  1. I was recently using your website
  2. It doesn't work for me
  3. I am on Apple

This is why I dislike people trying to be overly polite when reporting issues. I appreciate the sentiment, but it distracts from the task at hand. I'll be less annoyed if I can do my job efficinetly and finish early than I am if someone doesn't ask me how my day is going.

Okay so that's on not including irrelevant information, but what about the info that is there? I think you'll agree that while my summary is more to the point, it's not that much more informative. With the power of fictional hindsight I can show you a better versoin:

Hello,

I was using your website on 2022-10-16 at 3:24 AM (when there was a scheduled maintenence outage). When I submitted my form I got sent to a 404 page, I was visiting your website on an iPhone with the oldest OS still supported.

Thank you

Much better isn't it? The fictional person of course wouldn't know about the outage so you would have to piece together yourself but the leap from my revised version to the answer is a lot more managable. In short: I'm here to help you, so let me.

Rule 2: Come to me with a problem, not a solution

Sometimes people will try to be helpful and do some of the diagnosis and problem solving for me. Sometimes because the answer seems much more straightforward than it actually is, sometimes because they genuinely have made some useful inferences already. That's all fine, even if it's a little grating when they turn out to be wrong.

The actual mistake they almost always make in that situation is only telling you the solution but not the original problem statement or how they arrived at that answer. The problem with this is that it does not give me what I need to assess the solution the way I would with any other solution: Does it actually solve the problem? Is it compatible with other parts of the system? Does it follow the necessary standards? so on and so forth.

In my professional experience it is also the case that the solution people come to you with rarely actually solves the issue they have. Therefore whenever a user or client comes to me with a solution instead of a problem description it's a red flag to me. I almost never take something like that at face value. Not because I think they are stupid, but because this is how you end up with solutions without problems. More often than not, there is an underlying problem they have not discussed, knowingly or not. If this is perhaps still a little vague to you, consider the following example:

The case of the red button

Hello,

The button on your contact form should be blue [instead of the red it is now].

Thank you

Again setting aside the possible rudeness of a message like this, it doesn't give me the information I need to assess whether this is a good idea. Do they prefer blue to red? Does the shade we have now make some kind of colour theoretic faux pas? How will other users of my system see that change?

Again, with the benifit of fictional insight I'll show you what they actually meant:

Hello,

I have Protanopia [red-green colourblindness] and so I can’t find the button on your contact form or read the text on it properly.

Thank you

Aaaahhhh, okay. Aside from being a more legitimate reason to ask for the change than simple preference, it opens up a lot of different options that won't require a potential overhaul of the entire colour pallete of the website, such as having a higher contrast between text and background. In short: if you want me to solve your problem, give me what I need to do that.

Rule 3: Be clear in your expectations

The final bit to consider in a good bug report is what you consider a solution to look like. Often, when someone has done a good job on the previous two points this one is fairly easy to infer, but not always. This step is especially critical if the solution you want is different from the first thing that people think of when you describe the problem, which could be the case for example when you communicate across cultures, or disciplines or with someone that doesn't have your disability or experience.

If you go back and read the previous examples carefully you'll notice that they don't actually state what the solution should be, but this can be quite easily inferred. In the first example, the website should have accepted the submission and possibly given confirmation that it was successful. In the second example it is the button being clearly legible to the user. However this is not always the case. Consider the following (admittedly somewhat contrived) example:

Hello,

When the users try to look at the flash animation on my website, they cannot do so anymore because Adobe depricated flash. It is an important part of my portfolio, can you bring it back online somehow?

Thanks for your help.

This follows the previous two points but leaves out a crucial bit: what requirements do the solutions have to satisfy? Is its being flash a hard requirement? Is a recording of the animation an acceptable substitution? Do we have copyright or licensing requirements?

If the point is only that the animation be shown because it's part of a portfolio, an embedded YouTube player showing the animation could very well suffice. Otherwise more complex considerations should be made. The client can usually not assess these things accurately so in any bug report it is important that you specify what completion means to you. In short: be clear on when I can give my solution back to you.

Putting it all together

Until now I've only described common mistakes that people make when writing bug reports, but in order for this to be actually useful, I ought to show you how to do it properly instead. In summary the best practices are:

  1. Describe what has happened
  2. Describe what should have happened
  3. Describe what an acceptable solution looks like to you
  4. Describe any relevant context and no more

That last bit is usually a tricky one, since, especially if you are a lay person, you cannot always accurately judge what constitutes as "relevant" context. This is where personal judgement and practice come in. Basically the rule of them here is that you want to describe anything necessary for another person to recreate the incident.

For a final demonstration I'll write a bug report in what I consider to be a very good manner. To also drive home the broadness of this subject I'm going to do it with a slightly unconventional subject.

Showcase: A medical diagnosis

Consider the following email:

Dear dr. Whitecoat,

I’ve been running long distances (5km+) for over a few years now which has always gone well. However for the past few months my performance has gone down steadily seemingly out of nowhere. I also get shortness of breath when I work out which I didn’t have before, and I have moments where out of nowhere my heart starts racing like crazy and I feel very nauseous. On hot days I also have chest pain. Any idea what could be wrong and what I can do to get back to running?

Thanks!

Hopefully you can see that very few words in the letter above are not spent talking about one of the three cases I outlined in the section. For reference we have:

  1. Problem statement: "However for the past few months my performance has gone down steadily seemingly out of nowhere. I also get shortness of breath when I work out which I didn’t have before, and I have moments where out of nowhere my heart starts racing like crazy and I feel very nauseous. On hot days I also have chest pain."
  2. Expectation: "Any idea what could be wrong and what I can do to get back to running?"
  3. Context: I’ve been running long distances (5km+) for over a few years now which has always gone well.

Of course this is not an exact taxonomy so there is a fair bit of overlap between the intent of these sentences, but hopefully you see what I'm getting at.

As a final remark I'd also like to note the following. The other person has responsibility here as well. As I already said, it's very likely that you don't have the full context needed to include all of the "relevant" information. These are just tools to help you do the best you can. After that it's on the other person to help you get them what they need, so they can help you, get what you need.