The Quality of Quality Assurance

~ by Randy Wagner

How to Think about QA to Promote Success

Some concepts are perceived in black and white – binary terms. Justice, for example, denotes that a wrong has been righted. What about the quality aspects of justice? Was the punishment severe enough? Was the process fair to both parties along the way? The innocent went free but at what cost? Quality assurance is often viewed in a binary manner as well. We say a functionality passed QA and that is usually the end of it. As long as it doesn’t break in Prod, it’s good enough, right? But is it?

Skip the Shortcuts and Make Time to Ask the Questions Up Front

In nearly all QA activity, the basic metrics are either time or cost. Can we get it done before hard freeze (and sometimes after) or are we doing too much QA? We worry about defect leakage, release deadlines, staffing costs, and tool costs. We discuss the number of test environments we need, and related maintenance, but these discussions rarely result in any change. Most of the time, we are so focused on getting something done quickly that we don’t take time to review the whole process. QA shuffles along until a new manager comes in with different ideas.

I am as guilty of this as anyone. Each week seems busier than the last. Issues crop up that need attention but it’s a bit like refinement sessions. If we’re not taking the time to think things through, ask questions—and be asked questions—then we end up taking more time solving issues that would have been resolved if we’d managed it up front.

Assess the Scope, Nature, and Purpose of the QA Work with Clarity

We need to think of the forest before we get entangled in the trees. Are we doing the right types and amounts of testing at each higher environment in the sequence to Production? Is automated testing enough for regression? Is QA being used to compensate for poor unit testing, limited requirements, or poor release management? How extensive are the test cases and how extensive should they be? Do we have adequate training and mentoring in place? Critically assessing the QA work requires asking about more than just what’s being tested.

Trust the QA Team’s Invaluable Insights and Encourage Them to Speak Up with Confidence

The questions are not just for management. The people doing the work may be deep in the trees, but they often see patterns before they rise to an executive level. Perhaps, instead of just asking status in the weekly team or one-on-one calls, we need to ask questions like the ones above. Are you getting good requirements? Is Dev doing a handover for each story that demonstrates acceptance criteria has been validated before pushing to QA? What would make your testing easier?

I have often found that the best answers come from individuals who ping me after the team call to talk one-on-one about a topic of improvement. Ironically, they are almost always topics that would have been great to discuss in the group setting.

Stay Alert, Trust Your Instincts, and Ask Questions

Nothing in nature or in projects tends to stay in pristine shape forever. Entropy is as powerful a force in QA as it is elsewhere. Projects evolve, people shuffle around, and priorities shift. If we don’t stop and look around, we might miss the good stuff.


Randy Wagner is Director of Quality Assurance for CastleBay Companies. He has 20 years of consulting experience across private and public sectors, Guidewire InsuranceSuite, InsuranceNow, and Duck Creek, with specializations in quality assurance, project management, configuration management, and automation.