How Does Test Planning Differ for Manual and Automation Projects?

We all agree that Automation projects are different in nature from Manual testing ones. Although, autonomous Automation projects don’t really exist (or should not exist ideally), both Manual and Automation projects are dealt with differently when being planned.

A mix planned project inevitably gets is executed; this not only affects the current project and casts a shadow on the individual’s capabilities, but can also lead to the loss of confidence in the team for the client/management- affecting further business.  I would rather say that we testers be safe than sorry.

=> Click Here For Complete Test Plan Tutorial Series

A good Dilbert comic about planning:

Before we go any further, I want to establish what this article is NOT going to be about.

#1) This is not an in-depth discussion of Automation frameworks. Different projects use different frameworks depending on the nature of their AUT, architecture, complexities, team’s expertise, etc.

The information regarding the frameworks can be found at the below links:

Test automation frameworks part 1 and part 2.

#2) This is also not about template, format or creation of a Test Plan document. We are going to address the pre-documentation considerations for an automation project, more in the lines of a feasibility analysis.

#3) This also not tools specifically. Every activity in the SDLC takes time, effort, infrastructure- in other words- MONEY.

For a Manual Testing project the cost consuming factors are:

• People
• Tools – Test/Defect management
• Infrastructure – environment
• Time
• Training

For an Automation project, in addition to the above items it needs expenditure for:

• Automation tools
• Add-in for Test Management tool integration
• Add-in to support AUT (like SAP, Oracle, etc)
• Framework set up
• Tool-specific training

Given these circumstances, does the success of an Automation project depend on how well you have written the code, how many reusable components you wrote or in how few lines of code you achieved the desired result?

No.

There is one and the only question that determines the success – “Are you able to generate a better ROI (Return on Investment) in comparison to the Manual route”? – If not immediately, eventually.

If the answer to this question is “NO” then you have planned the Automation project incorrectly.

Normally, a Test Plan has the following sections. We are going to discuss each one of them focusing on Automation specific aspects to consider:

Automation Testing Test Plan Sections

Section #1: Scope

• Choose the Test cases/scenarios that are to be regressed over and over across multiple cycles.
• Sometimes the simplest of Test cases need lots of complicated solutions to be automated. If these are just for a one time use, it obviously does not make sense. Reusability should be your focus.
• Automation Testing does not/cannot perform Exploratory testing.

Section #2: Test strategy

• This section is referred to as the Framework in the Automation world.  Some frameworks are extremely challenging to create and also are effective – but time, effort and competency wise they are demanding.  Always look for a middle ground and do the best you can without jeopardizing overutilization of resources.
• Decide on coding best practices to be used, naming conventions, locations for test assets to be stored, the format of test results , etc. to maintain uniformity and increase productivity.

Section #3: Resources/Roles and Responsibilities

• The first step in this direction is to understand the team’s capabilities and anticipate ahead of the scope of Automation coming into the picture. This will help choose a team that suits both the Automation and Manual testing needs. Also, pick people who have the right attitude – those do not think that Manual testing is beneath their stature.
• Choose a team well versed with AUT, Test Management, Defect Management, and other SDLC activities
• Section #1: Scope

Section #4: Tools

Pick Automation tools based on the following rules:

• Does the company already have licenses for a certain tool, try and see if you can use it
• Look for open-source(but reliable) tools
• Do the team members know the tool already or do we need to bring in someone new? Or train the existing ones?

Section #5: Schedules

• Include time for code-walkthroughs and inspection of the Automation scripts
• Maintain the scripts on a timely basis. If you create a piece of code that you are not going to use for the next 6 months or so, make sure to periodically maintain it to lessen its chances of failure.

Section #6: Environment

• The target environment that your AUT is going to run and the Automation tool that you want to use should be compatible. This is one of the factors to be considered pre-licensing for the tool.
• Also, analyze if the rest of the Management Tools in place and the Automation tool you are trying to bring in are inter-connectible for additional benefit.

Section #7: Deliverables

• Your Test scripts are your deliverables. However, not everyone is automation/programming language savvy. So, plan on creating a “How-to” document that will help the current users and future team members to be able to understand this script even when you are not around.
• Include comments in your script too.

Section #8: Risks

If you are going to propose an automation solution, be sure to choose cost-effective tools and solutions to make sure that the Automation endeavor does not burden the project.

It is important to set the expectation that ROI for an Automation project cannot be positive immediately but can be clearly seen over long periods of time.

Therefore, if you propose automating a system, pick the one that is

• Stable and not too much maintenance
• Has scope for huge regression suites
• Does not have too much of manual intervention or does not depend on a human’s intuition

Section #9: Test data

• Take into consideration the security aspects of the data
• Do not hard code any test data into the scripts. This just leads to too much script maintenance and might induce errors during modification.
• Be very specific. For a Manual test step – ‘enter the first name’, you can say enter any 5 character name. While testing, a tester can type “Swati” or “Seela” or anything else. But for a tool, it can’t make such suppositions. Therefore, provide exact values.

Section #10: Reports/Results

• Script execution results are also technical and might not be easily understood by the rest of the teams. Plan on writing detailed results to notepad or excel sheets as an additional measure.
• Detailed framework documents, review results, defect reports, execution status reports are also expected.

We, as Automation enthusiasts might think that clients/management don’t easily buy the automation proposals.

However, when our ultimate goal is to maximize ROI through Automation, we are in perfect harmony with the management/client’s goals too. This will ensure that we not only get to Automate our project but will be able to do so, with lots of consent, co-operation, and excitement.

Planning and thorough analysis of all the factors listed above can be our ally through this journey. Again, ROI means everything.

This post is written by STH authors team member Swati Seela.

Have queries or things to discuss? Feel free to post in below comments.

=> Visit Here For Complete Test Plan Tutorial Series