Sample Template for Acceptance Test Report with Examples

Part-III: Overview of Acceptance Test Report

Previous Tutorial | NEXT Tutorial

In our previous tutorial on “Acceptance Testing Documentation with Real-Time Scenarios”, we discussed the Acceptance Test plan.

This tutorial will provide an in-depth look at reporting the Acceptance Test Status, Acceptance Test Summary, and Sign-Off.

Included in this tutorial are some generic templates that will enhance your understanding. We will also cover the concept of Acceptance Testing in Agile and Acceptance Test Driven Development.

In short, this tutorial will explain Acceptance test Status Report and Summary report, along with some generic templates for better understanding. It will also provide an overview of Acceptance testing in Agile and test-driven development.

Acceptance Test Status Report

The Acceptance Test Report should always summarize the acceptance tests that have been performed and their results. It should be addressed to all the identified stakeholders who are involved in the Acceptance Testing Phase. The progress should be reported on a day-to-day basis once the execution of Acceptance Tests has started.

Generic Template for Acceptance Test Status Report:

Date: Acceptance Test Status Report Date

Today’s Acceptance Tests Execution details:

• Number of Tests Passed
• Number of Tests Failed
• Number of Tests In-Progress

Acceptance Tests Execution Details till date:

• Total number of Tests
• Number of Tests Passed
• Number of Tests Failed
• Number of Tests In-Progress
• Number of Tests Pending

Defects Details:

• Number of Defects Logged
• Each Defect should have the below details:
  • ID, Summary, Component, Severity
• The total number of defects logged till now (in Acceptance Testing Phase).

This report should be reviewed daily to ensure that the execution is on track and there are no deviations from the planned schedules.

Acceptance Test Summary Report

This report summarizes the status of the entire Acceptance Testing phase. It includes details such as testing activities conducted, references to criteria met, requirement specifications, business rules, execution results, planned schedules, and any deviations.

Generic Template for Acceptance Test Summary Report:

Summary

<Summarize acceptance testing activities including acceptance test design, acceptance test execution, defects, test environment. Also include details like product release, reference to acceptance test plan, entry criteria passed, exit criteria to be met, requirements/business requirement specifications, and business rules.>

Variances

<This helps improve acceptance test planning in future releases.>

Results

<Any reasons for tests not executed should be addressed immediately. This could be due to dependencies, environment issues, etc. The test plan should be reviewed to address such cases.>

Evaluation

<Evaluation should be done for each component under test by analyzing the success rate based on test execution and logged defects. Entry criteria and exit criteria should be evaluated. Deviations should be addressed with reasons and possible action items for future releases.>

Recommendation

<Based on the entire report, provide recommendations for the product to be released or rejected. Consider any required improvements, suggestions from acceptance testers, defects severity, and pass rate when making the recommendation for product launch.>

Efforts

<Provide detailed information about the effort spent on each activity in the phase.>

Sign-off Report

Once the product has passed Acceptance Testing, it will be recommended to go live. Before launching it to production, it must be formally signed-off.

Generic Template for Sign-Off Report:

Product Name, Release Version, Build Number

<Mention the Product Name and its Release Version. Also mention the latest build number that underwent Acceptance Testing.>

Latest Report

<Attach the latest Acceptance Test Report.>

Reviewed On

<Mention the date when the latest Acceptance Test Report was reviewed.>

Reviewed By

<Mention who reviewed the latest Acceptance Test Report.>

Review Comments

<Include all the review comments on the Acceptance Test Report.>

Sign-Off Date

<Mention the date when the Acceptance Testing was signed-off.>

Sign-Off By

<Mention who signed-off the Acceptance Testing for the product.>

Sign-Off Comments

<Provide a statement for the decision to go forward with the product launch.>

In general, major stakeholders should review these reports and agree on the information to be included.

It is important to double-check all the details in the report before sharing it with the stakeholders. Any discrepancies could have a significant impact on business decisions and may result in product failure in the market.

Therefore, reporting should always be handled by specialists or senior members of the team.

Acceptance Testing in Agile

In Agile, the Acceptance Criteria of each User Story is targeted for Acceptance Tests. Acceptance tests are derived from the Acceptance criteria of a user story. Each Acceptance Criteria can have one or more Acceptance Tests to cover the scenario.

Acceptance Tests are usually designed by a QA who is the Subject Matter Expertise in the area. Acceptance Testing in Agile starts much earlier compared to other approaches, usually within the sprints themselves.

It is performed very frequently as each sprint will have new User stories and also enhancements/continuation of previous stories.

Acceptance Testing is performed at two different stages in Agile:

• When the feature is created and in its initial stage – basic.
• When the feature is integrated and stabilized with the other features of the product.

Each user story here has to undergo Acceptance Test and should pass for consideration. Any failures in Acceptance tests should be considered high priority and fixed immediately, which in turn will require re-execution of the acceptance tests.

Story points are assigned to each user story based on the success of Acceptance test results for each acceptance criterion. Acceptance Testing also defines the completion at the User Story level, stating that the Acceptance criteria for the story are met.

Who performs Acceptance Testing in Agile?

Usually, Acceptance Testing in an Agile environment is performed by Product Managers and Subject Matter Experts (including customers and/or beta testers). Sometimes, QA also participates in this activity alongside their regular regression tasks.

Benefits of Acceptance Testing in Agile

Acceptance Testing in Agile has several benefits.

Benefits include:

• Closer collaboration between Product Managers and the team.
• Builds confidence at the User Story level.
• Helps derive more scenarios to cover each Acceptance criterion.
• Increases probability to improve product solutions through Acceptance criteria in User Stories.

Drawbacks

Although there are several benefits, there are also certain drawbacks.

Drawbacks include:

• Not all stories can be considered for Acceptance testing. Only functional stories should be covered, which may result in reduced story-wise coverage.
• Not all Acceptance Criteria can be considered for Acceptance testing. Only functional criteria should be covered, which may result in reduced Acceptance Criteria-wise coverage within User stories.
• Since stakeholders from different backgrounds are involved and story-wise acceptance testing is performed directly, it is quite difficult for everyone to have the same understanding at the individual User story level.
• Due to the shorter release duration compared to other approaches, it is challenging to accommodate Acceptance Testing within sprints.

Acceptance Test Driven Development (ATDD)

This is one of the Agile development practices where the entire team collaboratively discusses each Acceptance criterion of a User Story and builds strong Acceptance Tests around them.

This approach allows different perspectives from each team member to contribute to thinking about each Acceptance criterion, resulting in a greater number of acceptance tests covering more scenarios. Sometimes, ATDD is also called Story Test Driven Development (STDD).

ATDD occurs before development starts. This provides developers with a clear understanding of expectations and how to achieve them. The entire team shares a common understanding of the feature and what is being built.

ATDD describes how the product is built and provides an idea of how the product will function before it is handed over for testing. Hence, it is termed as “Acceptance Test Driven Development”.

Conclusion

The goal of Acceptance Testing, regardless of the approach, is to build customer confidence and satisfaction in the product before it goes live. This can be achieved when there are no or minimal low-severity defects in the product that do not affect any of the functionalities.

In summary:

• Acceptance Tests are Passed.
• Defects are at acceptable levels.
• Flow-wise/Scenario-wise coverage is achieved.
• Product and its solutions are accepted.
• The customer is confident enough in the product.
• All the product documents are updated to match the latest functions.
• Result of the team’s effort.
• Ready for production launch.

Previous Tutorial | NEXT Tutorial

We hope you have gained valuable knowledge from these Acceptance Testing tutorials. Feel free to share your thoughts and ask questions in the comments section below.