Tools over Skills: Are We Missing the Point?

What does your organization really mean when they say they want to focus on quality?

One of my recent interests has been to understand what an organization really means when they say they want to focus on Quality. Do they mean:

– Better process quality?
– Better tools and frameworks?
– Better people to do testing?

It’s very rare that they mean all the above when they really should mean all the above.

More often than not, the focus really is on test tools and frameworks and not as much on multi-skilled testers who can bring more to the organization than just the technical knowledge of setting up testing tools and frameworks.

Testing Tools over Tester’s Skills

To help identify what an organization really needs, I have listed some common roles related to quality and some pointers on where in an organization they add real value.

Manual testers

These are testers whose primary skills are in manually testing your product or application. Their focus is largely on click and test methods. The manual-test-only approach is best suited in the following cases:

• Application under test (AUT) is very small, lightweight, with a high degree of stability
• Smoke and regression testing new changes with respect to already existing functionality takes less time
• Strong involvement of manual testers in requirements gathering and development processes
• Testers have an insight and influence over what is covered in unit and integration tests written by developers

This basically means that manual testing will be beneficial where the functionality is very small. However, as the application/product complexity increases, manual testing alone will result in some challenges such as:

• More testers will be added to the ever-increasing testing needs of a large project, which in turn, increases organizational costs
• Higher cost and time overhead resulting from repeated testing of already developed functionality
• Increased risk of not testing functionality that has been around for a while with the assumption that new functionality has not broken it
• Decreased motivation and growing lethargy in testers to repeatedly test the same functionality

To address some of these issues, teams start automating test cases with the help of a variety of tools/frameworks.

Test automation engineers

These are testers with a strong focus on testing tools. They have a high interest in test automation using different tools and frameworks. Organizations refer to them by different names: Common titles include “Software Developer in Test”, “Technical QA”, “Test Automation Engineer”, etc. In a lot of cases, these are developers who are hired to write tests. Test automation engineers will add the most value when they work in collaboration with manual testers to ensure:

• Reduced load on manual test efforts through automating new and already existing functionality
• Almost all test cases are covered either through manual or automated tests

As with the manual-test-only approach, having a test automation team focus only on writing tests comes with its own set of challenges:

• Entry barrier of being a developer in order to write functional test automation code
• Maintenance overhead of large number of tests that get created over time
• Concentrated knowledge of test automation within the team
• Lack of focus on overall test strategy

While test automation in collaboration with manual testing is good, having siloed teams approaching two different types of testing will result in testers not being poly-skilled and will increase hiring costs since two different roles will be needed for the same function.

Poly-skilled testers

These are testers who will be able to help with differing testing needs on different teams and whose focus is not on one particular vertical of testing. Their skills lie in their ability to understand the AUT and all (or almost all) factors that influence the quality of that application. They are technical enough to understand development code, capable of setting up tools and frameworks and are able to address the varying quality needs on a team. That means several things:

• Ability to drive robust test practices, which include but is not limited to focusing on various aspects of testing such as performance, usability, and security, the ability to influence the way quality is approached within the team and also across the organization, selection of testing frameworks based on a number of factors (not covered here), test maintenance, defect management, CI/CD practices around functional and integrations tests, and so forth
• Ability to understand an application on multiple levels. This includes the ability to prioritize what gets automated and what gets manually tested based on a variety of factors, such as, business priorities, dynamics of the AUT, effort vs ROI on automating a functionality vs manually testing, and so forth
• Ability to understand metrics and influence changes in the dev/testing approach based on trends analyses

Based on the above broad classifications, I want to revisit my original question: What does your organization really mean when they say they want to focus on quality? If the answer is ‘Let’s get some tools and some developers to write tests in those tools…’, then they are missing the point.

A good quality product is one where testing has happened alongside development. If the organization is looking at testing as an afterthought or as something that needs to be done as only a final gate check, then no set of tools or tests will be able to solve quality issues that will be seen in the product.

Conclusion

In conclusion, what is required is a deeper understanding of the meaning of building quality into the product – and that kind of understanding comes with someone who approaches quality systemically.

Tools and frameworks are mean to achieving testing goals. Poly-skilled testers will use a combination of all three – people, processes, tools, and additionally, their experience – to ensure that quality is not treated as a siloed function but instead, is integrated across the software development lifecycle.

About the author: Roopa Ranganath is an experienced Quality Lead and Agile coach and has significant experience in test automation. She has led several teams on testing (manual, automated and performance) and has a proven track record of reducing the cost to delivery. She is passionate about how organizations approach the quality and have enabled mid and large-size teams to transform their test practices from waterfall to Agile.

Let us know your comments and queries below.