Category: Ramblings

Categories
Experience Reports Ramblings

Reflection: Key highlights as a Quality Coach

I recently reflected upon my failings as a Quality Coach but that wasn’t to say that I failed. Far from it. I am very proud of my work as a Quality Coach and it is a role that I hope to pick up again.

Initiatives

Analytics

As a group we had never really looked at analytics before. It was deemed “too much effort” for our previous application suite given it was often in air-gapped networks (i.e. no way for the data to be sent back to us. However we did bemoan the lack of understanding of our customers.

When picking up a cloud based web application, I latched on to this. First I spoke to people about their challenges and noted how prevalent understanding our customers were. I experimented with Google Analytics and learnt about using it. I then pitched us adapting it, organising some brainstorming and leading to user stories on the backlog… which were then implemented.

I then looked at reporting that data. When we weren’t sure about the importance of fixing a bug, I could point to usage data. When wondering about the impact of a new release, I could point to change in user behaviours. It was useful. We also were using it for our planning to understand when we can try and force customers to switch from an older view to another.

Whilst the developers who did the hard work got the credit (and rightly so), I know full well that I made a difference here.

Refinement & ATDD

Our refinement process had been “grand” for probably around 7-8 years (at least with the teams I worked in). We did lack a little bit of test input at times but generally, not too bad. However when working in a new product area, we had made a few mistakes. We’d misunderstood behaviours, requirements and what is involved. This is why it became a key area for me throughout my time as a Quality Coach.

As it is something I was quite pleased with, I’ll elaborate more in a blog post another time (I have a long “to do” list!), but what I wanted to highlight was that I got testing needs to be considered before starting work. More than that, we also started thinking about the impact of our changes and opportunities for improving quality (including tech debt).

I helped encourage teams to think of edge cases as well, partly by bringing my tester hat and leading by example on asking awkward questions but also I started leveraging my interest in examples, a concept that I’ve explored in manual testing (my first post on here!). Whilst full example mapping sessions felt optimistic, I started getting us to define examples of behaviour and this helped people think of different cases, ask new questions and find that what seemed like a simple change had way more variables. We even managed to take it further in a smaller “standalone” project and implement a process similar to ATDD. I’ll elaborate on this in my separate post but for me, it was a cracking win.

Others

There’s a number of other initiatives that I’m pleased with, which I won’t elaborate on but:

  • Kept unit testing, even in difficult projects where there’s temptation to abandon it, an active area.
  • Started a TDD initiative, including running a workshop with more to come, pairing with an architect (sadly never got a chance to see it through).
  • Meaningful RCAs
  • Collaborative Test Strategy
  • Assessing our levels with different quality attributes (NFRs)
  • We started caring about accessibility (even if unfortunately were told to stop caring)

Coaching & Leadership

Vision

As I started writing about this, I found I had plenty to say, which I’ll follow up in a separate post but one of my personal highlights was setting and sharing a quality vision. The initiatives that I described in more depth above? They are directly tied to part of the vision. We made tangible progress towards a vision. For me it wasn’t a load of fluffy words but integral to what we tried to achieve.

Coaching

Throughout my time as a Quality Coach I tried to develop my coaching skills. I would use retrospectives and RCAs to try and coach developers. Originally I started by trying to coach at a team level (albeit with resistance for taking up team’s time) but eventually got to start having 1:1s with the developers.

Not every session was gold dust but not only was I able to understand people’s challenges better but the developers themselves got plenty of benefit. I was able to explore topics and challenges that went unsaid all too often. As I developed my ability with coaching and asking questions, I was also able to help people solve challenges unrelated to my area of speciality. I got a real kick out of it when I learnt that I’d helped someone solve their problem and I felt it was a positive that people took value from them and were reluctant to decrease the frequency when I was under pressure to reduce the time I spent with the devs in this manner.

Side note: I struggled to sell the value of this coaching.

"When you do things right, people won't be sure you've done anything at all"

Summary

Whilst not without its challenges, I am proud of my work as a Quality Coach. Of course I could have done better in certain areas (this is always the case!) but the teams that I work with are definitely better off from my initiatives, leadership and coaching. It makes me excited to think about what I could achieve in my next role.

Categories
Experience Reports Ramblings

Reflection: Key mistakes as a Quality Coach

I enjoyed my time as a Quality Coach, working with between one and three teams at a time. I think it was a relatively successful stint. However I want to acknowledge some of the areas where I struggled.

Role Definition

As a role, quality coaching is definitely very different to anything that I’ve done before and it was also very new within our group. Unfortunately I never really got myself established into what I think would be the ideal Quality Coach role & responsibilities. It was only when I was on gardening leave, reflecting and taking on board a host of great resources that I felt like I had confidence in my own understanding of the role. If you don’t fully understand your place, let alone others understanding it, this doesn’t set you up for success.

Looking back, I just didn’t spend enough time getting people to talk through the challenges and how they can solve them. I always succumbed to pressure and would reduce my involvement, even if it was glaringly obvious to me that we were only going to suffer later.

Ownership

Coaching is largely about helping people make the right decisions and grow. However in my role I tended to own all of our quality initiatives. I owned our test environment & kit. Test strategies not working? I took responsibility to change that. RCAs only happened if I organised them. Talking about testing early, putting together test strategies etc, tended to only happen when I led it.

The knock on effect here is that whilst the team did view quality and testing as a shared responsibility and collaborated in the journey, they didn’t really lead it enough. I didn’t create enough opportunities to empower people. However “in my defense”, I would argue that a bunch of developers were never going to be super enthusiastic about leading change on test strategy.

I’m curious how they’ll get on without me. Have I led by example and left enough of a framework in place for people to carry it on?

Pairing

I’m a shy and socially anxious person. As no one has ever really paired with me, I wasn’t sure how to engage people. Consequently I did no where near enough to push pairing with people, in particular to push pairing with myself. I did do some mob approaches, which seemed easier from a social point of view, and had some successful sessions in those final few weeks and months but going forward this is arguably my biggest area for growth.

Automated Tests

I hate this topic. And this is a problem.

When it comes to quality and testing, most of the talk is around using these tools and I don’t have a lot of experience here. Don’t get me wrong, I am grand with every C# testing framework that I’ve used, explored TDD as a developer and have a good grasp of the theory & concepts. I’ve ran a TDD workshop (using C#) and introduced automated acceptance tests (in a C# windows app), so I’m definitely not clueless. However I don’t know the syntax or best practices with tools like Selenium, Playwright or Cypress.

Consequently as a coach I can really support people in understand what to test, layering and explain TDD & BDD but when a Playwright test isn’t behaving, I’m useless.

Metrics

Another topic that I’ve never quite gotten along with. It is odd as playing with spreadsheets makes me happy but I just couldn’t really see the value in them – at least the ones that I’ve seen. I was interested in using QPAM or QCTG but they had some subjectivity and it felt an expensive meeting to discuss.

However my biggest lightbulb moment came during a lean coffee session for a DORA Community Discussion on metrics. I’d been approaching metrics as an exploratory testing. Looking at data and seeing what I could discover. Instead I needed to be more of a politician. Have a story then gather data to prove it.

Summary

I believe that I was on the right path but needed to be bolder. I should have been establishing metrics to prove that we have challenges with our quality then using that to justify initiatives that I coach people through.

I need to have more confidence (and/or less fear) with pairing and unfortunately I will have to learn some of the modern automated testing frameworks in order to help me succeed in the future.

Categories
Experience Reports Ramblings

Experimenting for quick wins

Experimentation is important, if not essential, if we want to have successful high performing teams. It allows us to try things without needing months to review and phase in a new way of working. Instead encourage trying something different, reflect and start again.

In theory, I really like it.

In practice, it was difficult within our group.

I’d like to talk about my attempts to learn the drums. I was tempted to try drumming as I still believe there’s something musical that I can be good at. I started with a small experiment.

I bought a cheap electronic drum kit. I did some reading and I started trying to follow along to a few YouTube videos. I even got a little bit of success doing a not completely terrible job of playing some songs. It was kind of fun.

However I was pretty useless. My experiment didn’t transform me into a musician so I gave up. It joins my bass & guitar sitting idle. Occasionally I’ll have a quick play with one, maybe get a bit of fun but it isn’t yielding great results so I rarely bother.

And this was my frustration with experiments at my former work. We were very open to trying something but seeing things through is a challenge. Straight away we’re asking “is this giving us the results we want?”, potentially then spending more effort on analysing success/failure than the experiment itself and subsequently drifting or stepping off that path. Maybe the odd strum of the guitar so we can say that we play instruments, but not really.

If I want to become a musician, I know that it takes time and practice. The same can be said for becoming high performing teams.

Sometimes our experiments & initiatives can be about trying new things for quick wins but failure to succeed doesn’t necessarily mean the idea is bad. It may mean that it isn’t truly effective yet.

It is essential that we don’t just look for how quickly we can get end results. Some things will require building up skill and practice. If you never give yourself the opportunity to become good at something, you’ll never reach those standards.

Categories
Ramblings

What is quality?

This is my favourite question when meeting people for the first time.

For some people it is about the software doing what it is meant to do. It might be robustness & reliability or adherence to specifications. Other people spoke about the value and solving customer problems. Is it useful? Some people referred to the swishness & polish plus the experience of it.

I like to represent these as the three pillars of quality.

I think this is really interesting to share and convey because within software we often overly focus on correctness as a measure of quality.

Let’s consider this with an example other than software… food!

When I am going out for a meal, what I want is to be fed & getting nutritional value. If I come away from a meal out feeling hungry, I really won’t feel like my user needs have been met. It was lacking value. This has happened with a fine dining meal where sure, it was fancy but I got a snack as soon as I got home. The meal wasn’t quality.

However if I go to a fast food place, maybe it will fill me up, but it is hardly the greatest experience as there’s possibly screaming kids, the food is… fine… and . This is why despite being correctly cooked and filling me up, I wouldn’t describe it as quality.

Finally if I eat a meal but it hasn’t been correctly cooked then I am highly unlikely to regard it as quality. For example when a thai meal had way too much 5 spice, it wasn’t quality. When my dirty fries made my body crash (vomit), it wasn’t quality.

However it isn’t essential to achieve high levels in all of these.

I will still enjoy my fast food because it fills my needs and I can have confidence it won’t make me ill. I will sometimes eat food knowing it won’t meet my nutritional needs… but it will be gooooood too eat. However my red line would be correctness. Or at least aspects of it (don’t make me feel ill!).

Fast food lacks excellence but can still provide me with quality

It is important to know what matters to you and your customers. For the past 14+ years I’ve worked in an industry when we didn’t care about excellence. Our customers wanted value and correctness.

Similarly some customers will be happy with a lack of excellence and correctness because they are so happy that their problems are being solved.

However a word of caution. Building off one core pillar isn’t without risks. Even two pillars isn’t without risks. When a core pillar crumbles or the focus changes for the customer, suddenly your perceived quality will tumble.

You can build up quality with one pillar but this is not without its risks!

So in summary:

  • Quality is more than correctness. Excellence and Value are essential to think about.
  • Think about what matters most to your customers.
  • But don’t forget about the other pillars.

Side note: I originally started using this referring to it as Dan Ashby’s Three Pillars of Quality… but he never mentioned it. I’m now thinking that I took a concept from one of his (many amazing) talks and adjusted it into the pillars. If I stole this from someone, I’m sorry and please let me know!

Categories
Ramblings

Continuous testing or continually testing?

One of the first things I did when joining my previous organisation was pitch testing throughout the SDLC. I wanted to share the importance of testing everything that we do.

However not everyone shares that idea of continuous testing throughout the SDLC. For some continuous testing is continuously running the same tests over and over. If you don’t have them fully automated E2E, repeat them manually over and over.

Skipping over how expensive it is to focus on E2E tests, this really misses the entire point of continuous testing. Instead of continuously trying to discover more about our application and challenging everything that we are doing, all we are doing is repeating that same check… and getting the same answers.

Whilst there can definitely be value in repeating tests when you have fear of a risk of regressions (and automation makes that so much easier), the true value in testing is to be trying new things, new ideas and devising new tests.

This is partly why I think people wedded to writing traditional manual test cases are doomed to struggle with test cases. It is a slow, tedious and usually fruitless endeavour.

Testing is more than checking. We need to be continuously experimenting, learning and discovering behaviours. When looking to regression test, don’t ask “what test cases do we have?” but instead “what are the risks and concerns that I have?” Ask “what information do we need to uncover?”. Then devise tests, ideally exploratory or at least looser than rigid test cases in order to uncover the information that you need and the risks associated with that.

In a future post I intend to give a case study or two on some interesting approaches that I’ve looked at.

I originally meant to post something around this about 2 years ago – better late than never! What I won’t state is whether it was still an issue at time of writing…

Categories
Ramblings

Devs can test

I repeatedly hear the notion that developers can’t test. I’m not sure if this is because we’re protective of our role (testing has been a dying role for decades) or to excuse developers who can’t be bothered to test.

So let me be clear…

Devs can test.

In fact if I had a high risk feature that I needed someone to look at, there’s a few developers that I’d trust more than a tester I didn’t know (and some I do).

I worked as a developer for 5 years. I was alright at it. However, in that time I never forgot how to test. In fact I never stopped testing. Instead of functional test and system testing, I was unit testing and dev testing my code. I was writing documents where I’d identify edge cases and what the behaviour should be… then implementing it. When our tester was on holiday or overworked, we chipped in. The quality didn’t tumble. The world didn’t collapse.

In one of my testing roles, going back over a decade, I was bemoaning that our processes lent towards following test specs when the pass rate was ridiculously high. Instead I’d find the bugs when I wandered away from the defined test cases that I was meant to be executing and started performing what I called destructive testing (ad hoc). I was shocked to learn that the reason why these tests (nearly) always passed was because the devs try and test them first. There were possibly some automated tests in the mix as well.

What I’ve learnt is that when developers, or testers, say that devs can’t test, what they mean is developers would rather focus on writing code. Perhaps they’ve been tainted from only running the boring test specs that drive me mad. Perhaps they won’t invest time in learning about exploratory testing. However, they have to test. They will. They can.

In the coming weeks I’ll share some more detailed case studies on examples of how I’ve seen (& helped) developers perform excellent testing

As quality / testing specialists we should endeavor to include developers in the conversation. We can remain the specialists but with a little bit of support/encouragement, developers can do a solid job of testing. This all helps us build better software and for the dedicated testers, the opportunity to focus on areas where your elevated skillset can add value.

And the next time someone says developers can’t test, ask them if that means quality / testing specialists can’t write code?

Categories
Ramblings

A new era ahead

I’ve recently been made redundant after 14 years of service, the last 2 as a Quality Coach for Avigilon software (although the first 6 months was without a job title).

However I am not disappointed or upset. Quite the opposite. It has given me a new lease if anything. I am excited for what the future may bring. I am fortunate enough to be in a financial position where I can explore new opportunities at my own pace.

Assuming that I do take my time looking for the right new role, I hope to spend some of this time learning, soaking up knowledge and also articulating some of the things that I’ve learnt (both good and bad!) as well as reflecting upon my achievements. Where possible this will be done in the garden, hoping nice weather can add to my current positive attitude!

I look forward to renewing and sharing my passion for quality!

A view of a sunny garden full of colour
I’ll be taking my gardening leave literally!
Categories
Ramblings

2024: Glad That’s Over

It has been a trying year, both personally and professionally. However before I flesh that out, I thought I’d try and articulate everything that has happened for me within the world of testing & quality:

  • I got a new benefit at work: 10% time! This led to me taking time to learn more during working hours.
  • My teams started work on a web browser based product, having been pretty much only working on a thick client talking to devices for years and years.
  • Got to do my first 5Ws & 1H.
  • I created and abandoned a heuristics based app called Test Addict. We used it a couple of times at work.
  • I started a new role as a Quality Coach (which in truth is similar to what I was doing previously, but I cunningly added extra pressure and expectations on myself).
  • We did our first Risk Storming.
  • I attended TestBash UK – this was great!
  • Started creating something cool (but not available yet).
  • Was a guest on my first podcast (but not available yet).
  • Experimented with different ways of doing test strategies, retros & RCAs to get teams engaged.

Big Fat Failure

In my annual review I was quite scathing of myself.

I failed the teams

We started a new project that whilst not greenfield, was full of opportunity. However we’ve struggled with quality with the teams quite damning in their own assessment. Our automated tests are flakey, we don’t have confidence in how we test and the bug count is beyond anything I’ve worked on before (and that is without me doing the testing!). This is without even touching on technical challenges.

Whilst quality is a shared responsibility and there’s plenty mitigating factors (excuses or reasons?), as the Quality Coach you can hardly walk away feeling like you’ve done a good job when it feels like everything is on fire.

Defeated

Throughout the year I’ve tried bringing new ideas and approaches to the teams but I’m not sure how much has really stuck. The second I stopped being the one to run RCAs or write a test strategy, they largely stop. I did get the teams testing for one quality attribute (I won’t name) for the first time, but we were told to side line it. After resisting taking on the state of our lab & kit, I succumbed, did a load of work and 2 months later its a mess again. Our flakey tests are addressed by “oh just run it again”. We did exercises like using heuristics, risk storming and 5Ws & 1H and the feedback was “that was really useful”… but they won’t get repeated. Finally I had a goal around teaching teams exploratory testing and to date I don’t think anyone has written an Explore, With, To Discover charter.

This isn’t me griping. It is a failing on me. I’m meant to be helping the teams have good habits but they aren’t there.

As for why I called this section defeated, well in Q1 & the start of Q2 I was introducing new ideas for planning testing, thinking about testing and testing techniques. Then over the week or so following my late wife’s birthday I got a wave of requests to “back off” and not take up so much of the team’s time on testing/quality stuff. And I did. I gave up.

Broken

As I’d worried a little in last year’s reflection, I’ve been feeling burnt out and haven’t been able to “over achieve” my capacity as I perhaps did in 2023. With the exception of the odd podcast when gardening, I’ve stopped exploring testing & quality in my spare time. I think the fact that I’ve only written two posts during 2024, one about grief and one written one evening during a conference, is quite telling.

Hope

I’ve been pretty bad at acknowledging my successes this year but there have been a few:

  • Teams might not do exploratory testing as Elizabeth Hendrickson describes in “Explore It!” but they are going beyond the ACs and checklists.
  • Teams are engaged when I organise activities like collaborative test strategies, round tables & RCAs (in the past that has felt like a struggle).
    • In fact in Q1 2025 I’m switching teams that I support and the teams expressed disappointment (and concern!) that I wouldn’t be there to lead on quality discussions.
  • Testing is always a topic that we consider throughout the SDLC.
  • I pushed for us to adopt analytics and we did!
  • I’ve gotten better at being a coach as opposed to a teacher.
  • Whilst deprioritized (for now), I did get us looking at new types of testing in our day-to-day.

Generally speaking, I’ve felt like over the past few months I’ve been collaborating with the teams in a positive way. We’re working to solve challenges and started making in roads on the many challenges we’ve faced. When I was previously feeling “unwanted”, I’m now feeling like a valuable resource. Our group of Quality Coaches, spread across the world, are collaborating more too.

I may have felt like everything had fallen to pieces but I’ve been enjoying the collaboration on how we build back.

Perhaps most importantly, I will definitely get myself the counselling I need. I will recover from the neurological effects of spousal grief and hopefully my energy levels too. With that, perhaps we’ll see a Rich resembling something like what existed 740 days ago…

Categories
Experience Reports Ramblings

Sketchnoting Adventures

Several A4 pages are scattered on a table. They all contain notes taken from a talk in a sketchnoting format

I’ve been meaning to write this for a while but after a really interesting conversation at TestBash, I decided to finally get something written down!

In June 2023 I attended Testing Atelier X and it included a workshop by Marianne Duijst on sketchnoting. I’ve seen some very attractive looking sketchnotes from many people, including the wonderful Louise Gibbs, however I didn’t get how people could take their notes, make them pretty and informative. I also didn’t entirely get the point. However I quite enjoyed the exercise and have since gone on to use this approach a lot when watching talks in my spare or personal development time.

The value that they bring is that you can provide a structure to your notes that makes them way more readable when you look back at them later. I rarely found my notes from talks to be worth looking back on in the past but since I’ve started sketchnoting, I’ve found myself picking them back up to remind me of the topics and key points. Importantly they are something that I could share whilst my previous scribbles in a notepad would be hidden (I used to intentionally write in such an illegible manner that only I could decipher the text).

I am no where near sharp enough to do them live but I’ve learnt some techniques:

  • For live talks, consider post it notes or just quickly writing things in a jotter. Then revisit them later.
    • Post it notes are great as you can reorganise them!
  • If you’re able to pause, try and avoid stopping too often. Listen, absorb then rewind to take the notes.
  • Avoid too much bloat to make it easy to parse.
  • Focus on the speaker’s words rather than “oh I could go do this”.
  • Don’t fret about being messy.
Several A4 pages are scattered on a table. They all contain notes taken from a talk in a sketchnoting format.

See more examples of people’s sketchnotes on the Ministry of Testing Club

Categories
Ramblings

Success & Grief

I know this isn’t related to testing or quality but I wanted to air it as it does affect my interactions with people and the community.

It has been a year since my wife passed away unexpectedly. Unsurprisingly it was a difficult year and I’ve really struggled. At the same time I’ve had what many might consider a successful year. So how do these conflicting matters combine?

Not great.

My lowest points (since those first few weeks) were coming back from successful conferences and I’ve felt similar after having a really great day with work.

When we succeed, as humans we want to share our achievements and what makes us excited. The natural person would be your closest friend or family member. Both of these were my wife. Consequently when I’ve succeeded, that natural want to share and celebrate has kicked in but I can’t do that with the one person who matters. They are gone. Forever. The high is met by an equal reality crash. It makes me want to run and hide after a good day. From a good day.

This leads to a dilemma. I want to succeed in work and I really enjoy being involved and attending conferences but ultimately they leave me in a darker place. How do I manage this? Other than holding back and not trying to succeed, I really don’t know.