Have you ever heard “I’m not testing the manual”?
Laugh or sigh as applicable first… then I want to ask a question. Why aren’t you testing the manual?
I’ve recently joined a new company and the software that I’m working with has really bad documentation. It is really tricky to use and there’s gaps and lack of clarity in the official documents. Worst still is they are out of date.
Stepping aside from work, recently I wasn’t sure how to make a change to how I received meeting invitations on my personal account. Depending on how they were sent, they were being added to my calendar silently and I nearly missed a couple of events. The first thing I did was explore but couldn’t figure it out, so I checked the help section. Sadly the screenshots were out of date. Trawling threads online solved it but how is that a good user experience?
What is interesting is that testing the manual is actually a great demonstration on why human-led hands on testing is an essential thing that AI or automation alone won’t solve. We need to be engaging with the software as a user who could be new, angry and/or confused. As users ourselves, not just with software but throughout life, we consult the manual/guide/instructions either as a first impression or when things are going badly. At this time it is important that the information given has been tested.
In my earlier days, I used to lean on structured test cases for instructions. When I started wanting to move away from having structured test cases, the argument was “a new tester needs to know how to perform this action”. Very true. However, why doesn’t the user get access to clear guidance? We want to provide documentation to enable ourselves but not our users?
This is why I have started wanting to advocate for having test cases be focused on behaviours (manual or automated) to prove what should happen and the official docs should highlight how this is done. There will of course be techniques that go beyond what a user would do, which is great to call out, but if your test case needs instructions on how to fill out a form, your company is doing something wrong.
So in summary, it is important that we provide our users with access to the same quality of information on using our software as we demand for ourselves. When features are updated and changed, we should be looking at the admin guide or help pages to ensure it is still accurate and when new user guides are published, following them ourselves to make sure they work.