How to Select Correct Test Cases for Automation Testing (and Ultimately Achieve a Positive Automation ROI)

We are currently in the age of Test Automation. Many testing projects are now transitioning their manual test cases to automated ones to enhance productivity and coverage.

One of the crucial steps in initiating Automation Testing is to select the appropriate test cases and determine the return on investment (ROI).

What can you expect from this article?
In this article, I have gathered several important points based on my experience to assist you in choosing the right candidates for automation and determining various factors that can lead to better test results and benefits.

Why Choose Automated Testing?

Automation does not replace or overshadow Manual Testing, but rather complements it. Like Manual Testing, Automation requires a strategy with proper planning, monitoring, and control. When implemented correctly, Automation can become an asset to the team, project, and organization.

There are numerous advantages of Automation; here are a few worth mentioning:

• Efficient in executing routine tasks such as Smoke tests and Regression tests.
• Helpful in preparing Test data.
• Enables the execution of Test cases involving complex business logic.
• Can be utilized for executing cross-platform test cases (e.g., different OS, browsers, etc.).
• Useful in executing test cases that may be challenging to perform manually.
• Applicable when the number of iterations of test case executions is uncertain.

Often, stakeholders perceive test automation as a support tool for Manual Testing. Therefore, it is crucial to understand that automation is the most effective way to enhance the effectiveness, efficiency, and coverage of testing. It not only saves time but also improves accuracy as repetitive tasks executed manually are susceptible to human errors and can be time-consuming.

Candidates for Automation

Important Mistake to Avoid:
One of the most common mistakes made by testers is the failure to select the appropriate test cases for automation.

Don’t simply select any test suite. Thoroughly analyze the test cases and select the candidates for automation based on the most important factor, which is ROI. First, it is crucial to understand and identify ways to achieve a higher and positive ROI.

(ROI – Return On Investment – It is the calculation of benefits in terms of cost-saving, increased efficiency, and quality)

There is no standardized procedure for determining the correct test cases for automation. It depends on the specific application being tested.

Based on my experience, I have outlined a few steps that may provide insights into selecting the test cases and ultimately achieving a positive ROI in automation.

See Also => How to Translate Manual Test Cases into Automation Scripts?

How To Select Correct Test Cases For Automation Testing

Step 1:

Identify the parameters on which you will base your decision to choose a test case for automation.

As of now, I am identifying the following parameters, but you can establish your own parameters based on your application.

• Test case executed with various sets of data.
• Test case executed with different browsers.
• Test case executed with different environments.
• Test case executed with complex business logic.
• Test case executed with different sets of users.
• Test case involving a large amount of data.
• Test case with any dependencies.
• Test case requiring special data.

Step 2:

Divide the application into modules. For each module, analyze and try to identify the test cases that should be automated based on the parameters mentioned earlier. This list may vary from project to project and can be expanded to suit your needs:
Fig 1.0

Y – Yes
N – No

In a similar manner, use this list for all modules to identify the test cases suitable for automation.

Step 3:

Consolidate and group the number of test cases for each module as shown below:

Fig 2.0

Figure 2.0 provides a clear representation of the number of test cases for each module. It aims to quantify the details and provide an estimate for completing the testing manually.

Step 4:

Once you have identified all the specific details, present them in the following manner. We will now calculate the ROI.

Fig 3.0:

We must also consider the following attributes that contribute to determining the ROI:

• Purchasing and licensing cost of the automation tool
• Time required to develop the scripts
• Time required to maintain the scripts
• Time required to manually and automatically analyze the results
• Time and cost of training resources
• Management overhead

Test Automation ROI Calculation Example

ROI is often calculated for a period of 5 years, but this is not compulsory. Based on the factors mentioned earlier, let’s elaborate on the calculation of ROI for 5 years. Keep in mind that you can customize and enhance this calculation according to your specific requirements.

ROI = (Cumulative saving / Automation investment)100

Manual To Automation Testing – What Are The Process Challenges?

I have highlighted the points that I believe are major challenges when automating a test suite.

#1) Automation Need: Every test team has its unique automation needs. We cannot establish a fixed standard, but we can tailor a standard that suits our requirements. This is why automation requires strong support from management and the development team.

#2) Automating the Entire Application: Automating 100% of an application is a daunting task. It is not impossible, but it requires careful planning, monitoring, and time. There are numerous combinations of data, environments, authentication, and authorization attributes that need to be validated, requiring a well-defined automation strategy.

#3) Manual Vs. Automation Mentality: “We typically automate what is important and repetitive, but we prefer to test important functionality manually.” Confusing? It can be! We should establish criteria to determine which test cases are truly important. These criteria can be based on factors such as complex business logic, areas of interest to clients, and high-risk areas.

#4) Deciding on the Framework: Designing a framework is the most critical aspect of automation. I believe more time should be dedicated to developing the framework rather than scripting. When creating the automation plan, prioritize framework design.

Plan the framework design. Identify and create a checklist of the items that will form the framework. With a robust framework, scripting and maintenance become easier.

#5) Team Knowledge: Automation is not solely the responsibility of a few individuals; the entire team should contribute. This not only enhances the skills of the team members but also keeps them motivated. Emphasize building logic rather than solely focusing on learning programming or scripting languages.

#6) Reporting: Every tool has its standard reporting format for test results. Customizing the reporting can be challenging and requires coordination and maintenance, adding to the overall cost.

#7) Trust: We should have confidence in our automation. Although we invest man-hours in building an automation suite, sometimes we don’t trust the test results. Efforts should be channelled into maintaining the scripts. Additionally, the team responsible for manual testing should be involved in the automation process as they have a deeper understanding of the application.

Most of the time, automation is handled by a separate team, which means the actual testing team is unaware of the scripts and ends up running the tests manually because they feel managing the scripts adds to their workload.

See Also => Manual and Automation Testing Challenges.

Conclusion

In many cases, the preferred approach is to automate the Regression suite (here are some challenges in automating the regression suite in an agile environment) as it often contains a large number of test cases. In this case, you can divide the regression suite into smaller subsets and run the appropriate subset based on release requirements.

For instance, if a regression suite contains 1500 test cases, you can divide it into 3 subsets of 500 test cases each and automate them.

Instead of automating the entire suite, you can opt for phased automation. In other words, follow the prototype model for developing the automation suite. Create a structure or framework with a smaller number of test cases initially and start using it, gradually enhancing it by adding more test cases over time.

We should follow the Deming wheel (PDCA cycle) for automation as well. As an ongoing activity, we should focus on building the framework properly, which will result in easier maintenance and implementation of new features.

Automation requires support from the development team and management. Encourage the testing team to actively contribute to Automation Testing as they possess in-depth knowledge of the product.

About the Author: This article was written by Shilpa Chatterjee Roy, who has 8.5 years of experience in the Software Testing field across various domains.

I hope this guide will simplify the ‘Manual to Automation testing’ process. Please feel free to share your experience and thoughts on how you overcome process challenges.

PREV Tutorial | NEXT Tutorial