Adventures into ATDD

When reading into Scrum & LeSS (the scrum-based framework we used), it was clear that we weren’t doing anywhere near as much refinement as was recommended. It was understandable and on the face of it, we didn’t think that we were doing a bad job. However we kept working in spaces that we didn’t understand and mistakes followed.

Within refinement sessions I would bring a “tester hat” to ask meaningful questions and when we had ambiguity, I started using examples, based upon Example Mapping. In fact we tried full on example mapping a few times, although it was viewed as too time consuming. Nonetheless I was using my spare time to read about Behaviour Driven Development, Specification by Example and Acceptance Test Driven Development. Not entirely new, given my interest in exploring these concepts in manual testing (my first post on here!).

Through highlighting wasted effort on incorrectly understood requirements, several deliberate “off the cuff” suggestions and a brown bag, I got myself enough support to have a go at a pilot.

We adapted our refinement to include examples. Sure, we weren’t example mapping but with each AC I had the team clarifying the behaviour with examples. This often led to new ACs and new stories as we got specific about what the behaviour meant. This meant fewer surprises as we got further into development.

We then took those examples and turned them into a mix of layered acceptance tests and a few manual tests. As we were working with several different applications across multiple companies, doing E2E tests seemed unreasonable (and lacked value for the project), so I focused our automation on what mattered most – whether our data manipulation was meeting the required behaviours.

I split the tests into two layers. In the first layer we made it as human readable as possible, describing the behaviour through gherkin in the comments and the steps on how to perform the test in the body.

/// <summary>
/// Scenario: Roast potatoes are yummy
/// Given the user has one or more peeled potatoes
///   And the oven has been preheated
/// When the user roasts the potatoes
/// Then the potatoes are extra yummy
/// </summary>
[Test]
public void RoastPotatoesAreYummy()
{
    UserHasPotatoes();
    UserPeelsPotatoes();
    OvenPreheated(200);
    RoastPotatoes(TimeSpan.Minutes(60));
    RemovePotatoes();
    EatPotatoes();
    VerifyExtraYummy();
}

We then have steps that implement exactly how that happens, controlling the oven. This could have been with Playwright or similar but we just called the appropriate public methods.

public void RoastPotatoes(TimeSpan cookingTime)
{
    oven.Insert(_potatoes);
    oven.Leave(cookingTime);
}

A positive experience

What I liked about this is that it had us thinking about the behaviours more than usual and we kept that going after the experiment. I believe it is still going.

When running a brown bag to share what we’d done, there was real interest in it. Whilst I’d talked about layered acceptance tests in the past, I don’t think the value was understood. However when seeing these tests it was clear what the test was doing (preheating an oven, putting in potatoes and later eating them) and that wasn’t complicated to understand. Then when we’ve got the actual “magic” of controlling the software, the intent of each step is clearer to understand. Sometimes it can be difficult to read code and understand what is the purpose of the step, but this was nice and clear.

Doing it properly

I am very aware that this is not the ideal layered pattern but it was a small compromise and a starting point. I could sell where I wanted to go with these tests whilst demonstrating it in the simplest form. Ideally that ‘Given the user has one or more peeled potatoes‘ is mapped to a method using Cucumber/Reqnroll/SpecFlow then it can have calls for UserHasPotatoes() and UserPeelsPotatoes().

Since then I’ve actually been exploring using reqnroll to create layered acceptance tests that actually use the gherkin and split those into the more traditional Feature, Step Definitions & Drivers. I’ve a super lightweight set of tests at the moment but have more in the works with different technologies.

Disclaimer: If it isn’t obvious, no the feature was not on roast potatoes. I’m just hungry at time of writing the examples. I know that my gherkin and steps aren’t great. There’s no seasoning for a start!