Categories
Experience Reports Ramblings

Why dev testing fails

During my time as a developer I generally produced a good quality of work. My knowledge of design patterns may not have been very good but I tested my stuff to ensure that it did what I expected it to and that is why there was generally a low bug count for my work… But there was a bug count.

Given that I am very confident in my testing skills (more than my dev skills!), how was it that I missed things in my dev testing? In this blog post I hope to explore that topic.

I believe there are four reasons why we may miss things in our dev testing and push code changes with bugs:

  • Blind Spots: Being human we will have cognitive biases or a tendency to miss certain edge cases. There are things that at first I would miss frequently and therefore got more attention in time. Mine was writing incomplete log messages where.
  • Laziness: Most developers want to be writing code so we can be tempted to get our code pushed as quickly as possible in order to move on to the next thing. Certain aspects of testing may be areas where we get slack or cut corners. I mostly got slack around install/upgrade. Yawn. (See also I get bored)
  • Iterative: My approach to development is very iterative. Sometimes I’ll use TDD and sometimes unit tests come a little later but I’ll always manually test my code as I’m working on it. In theory this is great but I’m not going to re-test everything on each iteration. Occasionally this meant that I missed finding out that I broke something.
  • Scope: I don’t believe dev testing should include things like system, load, soak or other more involving tests. If there’s a high time investment to test various scenarios, rather than having two people repeat the same testing, we will often cover the highest risk tests in dev testing and leave the rest for the test phase. There is some testing that is best done by the person testing a completed user story / feature. Consequently it is quite reasonable that developers can exclude certain testing from their dev testing.

(oh hey, BsLIS, or BLISS – lets say I meant that)

So there are some understandable reasons for us to end up blissfully pushing bugs. How do we solve it?

To some extent, I don’t think we need to do anything special.

This is the reason why we (should) have dedicated test phases of a user story’s life. Ideally by working within teams where testing is completed by the development team (possibly by a dedicated tester) then you can build up the relationships, understand each other and catch these things immediately.

As a tester I like to try and get to know my colleagues and how they work. I like to understand what mistakes they may make, as well as thinking about what mistakes I may make. If we can understand and appreciate how these bugs can slip through dev testing, we can catch them easier.

Obviously if you know your flaws then it is good to work on them. If testing exposes your blind spots, try to force yourself to be more aware of them. Using techniques like TDD allows us to ensure, at least at a unit test level, that our iterative changes aren’t breaking our previous work. Also lean on your colleagues and your testers. If bugs are raised against your software, ask yourself why you missed it. Mistakes happen. It’s fine – especially if we’ve caught it before shipping.

My one final note is that it is OK to knowingly not manually dev test an area of code, so long as we clearly communicate not only what we’ve tested and but also what we’ve knowingly not tested. Writing automated tests also lets you flex your scope on manual dev testing.

Dev testing doesn’t need to be perfect. Instead communicate, be clear and work together. The team can be way more than the sum of its parts.

Leave a Reply

Your email address will not be published. Required fields are marked *