On most software projects that follow Agile principles, it is easy to find agreement that QA practices should be automated whenever possible. In fact, leaders often look to QA automation as a panacea for projects where testing practices are manual and time-intensive to execute. However, when it comes to execution many organizations fail to make the necessary investments to succeed with QA automation.
Why is this? While the value of automated testing is clear for most to see, there are often significant hurdles to overcome: streamlining QA can be seen as increased risk, there are often large gaps between the technical skills needed for manual vs. automated testing, and QA processes could be overlooked as the organization focuses more on core agile tenets like time to market or responsiveness to change. And while these hurdles are significant, there are hidden costs of not automating QA that can be more detrimental to the organization in the long run.
1. Ongoing Operational Costs. If you do not create automated tests, then you are deciding to pay someone to execute the same tests over and over. You are making QA an operations component where the same tasks need to be executed repeatedly by the same individuals. Over the course of time paying someone to run the same tests every time a component changes will often be more expensive effort to automate the test. If a developer had to manually authenticate a username/password during a log in process for a website they (or their boss) would be labeled as wasteful or crazy, however, in QA circles accepting manual processes is often the ‘normal' way of working.
2. Loss of focus. One of Scrum's core values is Focus - Because we focus on only a few things at a time, we work well together and produce excellent work. We deliver valuable items sooner. [1] With manual testing, the QA execution occurs after the development is complete with a significant chunk of time needed to provide feedback to the developer. If there are any defects/confusions it is often too late to succeed in the sprint and QA ends up being ‘squeezed' at the end of the sprint and either passed with limited quality or ‘carried over' to another sprint. With manual testing, it is not uncommon to see Scrum teams testing functionality that was developed the previous sprint. What is the cost of this practice? The cost is the focus of the team as now the developers of the team for the next sprint have a new set of priorities, while also balancing any defects or QA support from the previous Sprint. This leads to diminished quality and output. On the contrary, automating testing allows for instant verification of development and strengthens the team's ability to focus on developing and testing within the same Sprint. QA to Developer feedback times are minimized and the team is given a greater chance to succeed.
3. Dependency on tribal knowledge. Manual testing relies on good documentation to be transferable from one person to the next. Documentation, though, is one of the most easily overlooked outputs of the team and when there is any time crunch it is often the first thing to go. As a result most manual testing relies on institutional or ‘tribal' knowledge of a select group of individuals to fill in the gaps left out by insufficient documentation. When these individuals leave or get pulled onto other projects, risk is introduced to the team as no one is really an ‘expert' on the necessary ways to test the application being developed. Automated testing counters this because the code is the documentation. The automated QA code contains the necessary steps to validate the application, any documentation is supplemental to how to execute the process. When organizations choose to build up a large code base of automated QA code, they are adding to the assets of the organization in a way that ensures value is not lost when someone leaves the team.
The value of QA automation is clear but oftentimes organizations are not prepared to overcome the hurdles to get there. Streamlining QA processes, skills gaps, and general awareness are substantial challenges to overcome. QA teams can start to weigh the costs to automate versus the costs to continue in a manual fashion by paying attention to the hidden costs inherent in manual QA processes. When all of these costs are clearly laid out, feasibility discussions oftentimes change focus from the direct costs to automate QA to the indirect costs remaining manual.