Flashlight Tag

~ by Randy Wagner

It’s Not How Bright the Light Is but Where You Point It

Many of us have delightful memories of warm summer evenings finding our friends in the dark with a flashlight, in a nighttime game of tag. QA in a project can often feel like that, with all the suspense, hiding spots, wrong turns, and confirmed suspicions.

There is an old saying that people often look for things where the light is brightest, but it turns out that we, as QA, have the power to point that light in different directions.

The tester is “it” and it is up to her or him to find the hidden defects. Some aren’t well hidden, and others keep us searching late into the night. It is up to us to be thorough in our search.

Limited Visibility Presents Two Key Challenges in QA

The first challenge is that our flashlight is only so bright. Due to ever-limited time and people, we can only do so much testing in a world where possible permutations can run into the millions of dollars. When testing policy, billing, claims, and portal systems, it becomes an awfully big backyard to search.

The second challenge is that we often experience pressure from stream leads or product owners to focus in certain areas that solve an immediate problem but don’t cover everything that should be examined. Especially when closing out a defect, there is rarely time allotted to do proper regression testing.

Key Ways to Overcome Challenges in QA

There are ways around this. While we rarely have the power of the purse to buy a bigger spotlight, good tools and process can give us a boost.

Use Automation and Regression Test Cases when Possible

Obviously, if we have automation as part of the project, we can hit the regression test cases. If the project is using Behavior Driven Development, the defects are, hopefully, being found even before they hit QA. If we’re in a manual mode, however, we still have some tricks up our sleeves.

Be Methodical and Follow the Business Logic

Methodical testing is key. Verify the business acceptance criteria are still successfully executed. Even if we can’t do full regression, the acceptance criteria from the story give a reasonable assurance of success, if they are well-written. Or we can chase the workflow. Step through the business logic beyond the specific functionality so that the test operates in a more realistic context to reveal surprises.

Expand the Depth and Breadth of Testing Data for Increased Visibility

Additionally, the strength of the flashlight beam can be augmented by the depth and breadth of data used in the testing. Varying ages, coverages, makes and models, locations, entities, and other elements provide huge power to spotting defects hidden in the shadows. Add in negative testing, boundary testing, and other vectors where we can beef up the realism of how users will poke at the system. We may only be able to execute so many test cases, but we can vary them enough to increase the visibility.

Listen, then Test

Finally, as with flashlight tag, one of the best tactics is to just listen. When Developers in a daily scrum mention obstacles to finishing stories, or another team upstream makes a change to the business flow, we can bet defects aren’t far away. We need to listen to our colleagues. If something is worth talking about, it’s probably worth testing.

Point the Light into the Shadows

QA may not always be as fun as a kid’s game, but we can certainly draw parallels that can make our work more effective. We rarely control the size of the flashlight, but we do get to point it at the shadows. If we point it at enough shadows, we win the game.

Randy Wagner is Director of Quality Assurance for CastleBay Companies. He has 20 years of consulting experience across private and public sectors, Guidewire InsuranceSuite, InsuranceNow, and Duck Creek, with specializations in quality assurance, configuration management, and automation.