How to Test Smarter: Explore More, Document Less

How to Test Smarter?

Test teams do a great job nowadays, that too under difficult circumstances and compressed schedules. Thanks to budget crunches and concerns about security and usability. This article will help you to test smarter in a better and simple way.

We can help testers to do a better job – a smarter job, by encouraging them to refocus on some of their efforts, by keeping them away from test documentation and instead of using that time to expand exploratory and ad-hoc testing.

The Problem

One of the biggest problems that IT organizations are facing today is the compressed development cycle. No one plans for cycles that are longer for a couple of months, except for huge enterprise initiatives. Managers have to deal with short cycles and get quality software out the door.

Coders have to write software quickly. And the testers have to accommodate themselves to the new and changing functionality on ever-more-complex software without letting the defects to slip through undetected.

The Solution

Something has to be done in order to improve the efficiency of testers. Yes, better SDLC (Software Development Lifecycle) tools can certainly help to improve the productivity of both, the programmer and the tester. And automated test technology is incredibly important to both the agile and traditional teams.

With great development tools, agile processes, continuous integration, and test automation systems, quality assurance has become a serious bottleneck, as the business demands ratchet up the pressure and compress the deadlines.

That means testing should be smarter. Reallocating resources from creating documentation and instead focusing on adding value with Exploratory Testing.

Reducing documentation doesn’t throw quality out the window. Not at all. The Test Scripts are still created and allowed to run. Testing smarter requires the developers and tester’s plan, to build and run essential Unit tests, Functional tests, Acceptance tests, and Security tests.

However, smarter testing means acknowledging the fact that some tests are more important than others, and as such, they should receive more attention as well, including documentation.

Consider, if a traditional agile team is tasked with adding new functionality to a website or mobile application. Early in the sprint, the team would create a test plan and create new test cases or modify the existing ones. And once the coding is done, the team would run those tests and document the execution results along with the defects.

If there are defects, the code would be corrected, and the tests would rerun. In some cases, the defects might require the agile team to reexamine the test cases, as well as the code, and potentially update them as well and rerun tests and repeat it if required.

Creating and updating test cases takes some time and resources as well. So is the process of documenting the test cases and each of the test runs (though automation helps).

Most test documentation adds no value to the business. If the team tests smarter, testers can focus on writing up test runs if and only if defects appear, instead of documenting every test case and test run. If the test run’s results are negative (i.e., no defects), then you must move on.

If the results are positive (i.e., defects appeared), then yes, testers should document the test, including whatever is required to reproduce the defect.

Example

Imagine there are 100 new test cases for a particular sprint. All those 100 test cases must be examined, possibly updated and thoroughly documented.

How to Test Smarter?: Let’s test smart. Say that it’s determined that 10 of those test cases need to be carried forward for future regression testing. Perhaps another 15 tests failed during execution by producing unexpected or undesired results. Then the team needs to document only those 25 key and failed test cases — not all 100 — imagine, how much time is saved.

Use that left out time to improve the quality by encouraging developers, testers and other stakeholders to do more exploratory, ad-hoc type of testing. If the team is fortunate enough to have test-automation tools that can turn ad-hoc tests into reusable test scripts for future regression tests, that’s a bonus then, since exploratory tests can be turned into test-case workflows.

Make no mistake: Before development teams decide to test smarter, and stop documenting certain tests, it is essential to ensure that the testers truly understand the goals of a particular development project or phase, and therefore new tests will not be required for future sprints.

In agile shops, that means knowing the objective of each sprint. Understand what’s new or changing in that sprint and in the backlog. Understand the user stories. Agree which tests are actually required in that particular sprint (and thus it need not be documented) and which tests are required for future Regression Testing and acceptance testing (it should be thoroughly documented).

Ask yourself, “When the end-user receives this sprint’s code, what would he/she be most interested in?” Obviously, you need to test there and document those tests. However, also ask, “What parts of the code would the end-user probably not be thinking about, but where he/she could find problems?”

Those questions will guide developers, testers and other stakeholders towards edge cases and situations that cry out for exploratory and ad-hoc testing.

The team leaders should envision a high-level approach for what should be tested. There will be key scenarios of each sprint that needs to be tested and re-tested because they are highly vulnerable or foundational for future sprints. Once they are identified, then those scenarios can be packaged for future regression testing.

By contrast, code areas that are not of high risk can be tested once, especially if that code is stable and if it won’t affect future feature enhancements. Therefore, no documentation is required.

Conclusion

We are all under pressure to deliver more codes quickly. To accelerate software development without sacrificing quality, test smarter!

Use test automation whenever possible, and continue executing unit tests when a new code is checked into the source-code management system. Document and run regression tests on a critical code, of course, but don’t waste time documenting tests that won’t be needed in the future.

Instead, use your testing resources for exploratory testing. That will improve the quality and thereby accelerate the development lifecycle.

About the Author

Vu Lam is the CEO and founder of QASymphony, a developer of defect capture tools that track user interactions with apps. He was previously with First Consulting Group and was an early pioneer in Vietnam’s offshore IT services industry since 1995. He holds an MS degree in electrical engineering from Purdue University. You can reach him at [email protected]

Let us know your thoughts, questions/comments.