Month: May 2023

Revenge of the Gatekeeper

For the past couple of months I’ve spent half my time, which is in itself reduced following complications, as a “QA Champion”. A title I dislike. In particular I dislike how QA tends to be associated with the testing best described as “checking” and “QA monkeys” running “test scripts”.

The organisation that I joined as “QA Champion” (I’ve died a little more inside when writing that) has had quality issues, especially since a re-org tried to move us into a LeSS structure with dev teams taking on quality and testing. This was first done by getting rid of testers (well not quite) and trying to shift the QA left to do QA earlier. Everyone needs to be doing some QA, not just the QA teams that still sort of exist. We plan to solve this by automating all the QA.

Ignoring the “automation solves everything” part and the fact that we are working on an old tech stack that lacks testability, I have real issues with our approach.

Let’s have a quick prequel first.

My understanding is that our old methods were that testing was thrown over the fence to teams who ran lots and lots of test cases then produced reports and would be the authority on whether we can release. In other words QA as the gatekeepers of quality. To me this is the dark side and rightfully needs defeating.

So why the ranting, rambling blog post? Surely I should be happy that we’re abandoning this? Well no. I’ve found that the QA Champion is sod all to do with championing quality and we’re the new gatekeepers of quality.

What have I become?

A New Hope

As bleak as it may seem, a shift in the balance may be happening. I’m hoping that I’ve awakened something within the QA Champion group so that we can become a force for good.

I’ve started something of a rebellion. Through a tech talk I shared what I think continuous testing, shift left and quality engineering, all that goodness, could look like for us. With passionate discussion (or constant whining), I’ve got discussions going where I think that there is a strong alliance to bring around change.

My goal is built on two pillars right now.

First is with the teams that I immediately work with, we are looking to use practices to help us test continuously and build quality into our day-to-day. This hasn’t been a hard sell because:

• There’s a strong desire to improve quality where possible.
• I’ve said that I’ll have their backs in rejecting the initiatives coming from above (i.e. the ways of the dark side).
• Many of us have worked together. Some of them taught me what I’m now passing back.

Secondly I am continuing to raise the discussion on what I believe to be the right practice and challenging things like “let’s get teams producing reports detailing all their testing for an epic”. I hope to change the language and expectations from management so that they aren’t looking for the same sort of gatekeeping as before. If they want us to be potentially shippable, instead of going through the arduous and prolonged hardening and release processes seemingly non-stop, lets focus our energies on employing quality engineering.

A Force Awakened

So far I’ve had mixed feelings about my progress. There are moments of hope and feeling like there’s positivity and receptiveness to this, followed by a request for dev teams to run hundreds of manual test cases each in a hardening phase for an internal release.

However I’ve now shared our new approach in the latest sprint review. We’ve had lively and positive discussions about how we can actually get to this stage. Things may actually start coming together.

I will wrap up this post by saying that if the next month or so goes well, I hope to get my job title changed from Senior Test Engineer and get the QA Champion program re-branded.

Hopefully this tale will end with the rise of the Senior Quality Engineer.

(sorry)

During my time as a developer I generally produced a good quality of work. My knowledge of design patterns may not have been very good but I tested my stuff to ensure that it did what I expected it to and that is why there was generally a low bug count for my work… But there was a bug count.

Given that I am very confident in my testing skills (more than my dev skills!), how was it that I missed things in my dev testing? In this blog post I hope to explore that topic.

I believe there are four reasons why we may miss things in our dev testing and push code changes with bugs:

• Blind Spots: Being human we will have cognitive biases or a tendency to miss certain edge cases. There are things that at first I would miss frequently and therefore got more attention in time. Mine was writing incomplete log messages where.

• Laziness: Most developers want to be writing code so we can be tempted to get our code pushed as quickly as possible in order to move on to the next thing. Certain aspects of testing may be areas where we get slack or cut corners. I mostly got slack around install/upgrade. Yawn. (See also I get bored)

• Iterative: My approach to development is very iterative. Sometimes I’ll use TDD and sometimes unit tests come a little later but I’ll always manually test my code as I’m working on it. In theory this is great but I’m not going to re-test everything on each iteration. Occasionally this meant that I missed finding out that I broke something.

• Scope: I don’t believe dev testing should include things like system, load, soak or other more involving tests. If there’s a high time investment to test various scenarios, rather than having two people repeat the same testing, we will often cover the highest risk tests in dev testing and leave the rest for the test phase. There is some testing that is best done by the person testing a completed user story / feature. Consequently it is quite reasonable that developers can exclude certain testing from their dev testing.

(oh hey, BsLIS, or BLISS – lets say I meant that)

So there are some understandable reasons for us to end up blissfully pushing bugs. How do we solve it?

To some extent, I don’t think we need to do anything special.

This is the reason why we (should) have dedicated test phases of a user story’s life. Ideally by working within teams where testing is completed by the development team (possibly by a dedicated tester) then you can build up the relationships, understand each other and catch these things immediately.

As a tester I like to try and get to know my colleagues and how they work. I like to understand what mistakes they may make, as well as thinking about what mistakes I may make. If we can understand and appreciate how these bugs can slip through dev testing, we can catch them easier.

Obviously if you know your flaws then it is good to work on them. If testing exposes your blind spots, try to force yourself to be more aware of them. Using techniques like TDD allows us to ensure, at least at a unit test level, that our iterative changes aren’t breaking our previous work. Also lean on your colleagues and your testers. If bugs are raised against your software, ask yourself why you missed it. Mistakes happen. It’s fine – especially if we’ve caught it before shipping.

My one final note is that it is OK to knowingly not manually dev test an area of code, so long as we clearly communicate not only what we’ve tested and but also what we’ve knowingly not tested. Writing automated tests also lets you flex your scope on manual dev testing.

Dev testing doesn’t need to be perfect. Instead communicate, be clear and work together. The team can be way more than the sum of its parts.