Section III: A Glimpse into Acceptance Test Reports
Former Tutorial | Upcoming Tutorial
Recommended IPTV Service Providers
- IPTVGREAT – Rating 4.8/5 ( 600+ Reviews )
- IPTVRESALE – Rating 5/5 ( 200+ Reviews )
- IPTVGANG – Rating 4.7/5 ( 1200+ Reviews )
- IPTVUNLOCK – Rating 5/5 ( 65 Reviews )
- IPTVFOLLOW -Rating 5/5 ( 48 Reviews )
- IPTVTOPS – Rating 5/5 ( 43 Reviews )
In our last tutorial, titled “Real-World Scenarios of Acceptance Testing Documentation,” we delved into the process of Acceptance Test Planning.
This tutorial delves deeper into discussing the compilation and documentation of the Acceptance Test Status, Acceptance Test Summary, and Sign-Off reports.
We have included specific templates in the tutorial to help with your comprehension. Additionally, we will also address the role of Agile and Acceptance Test Driven Development in Acceptance Testing.
In brief, this tutorial will elaborate on the Acceptance Test Status and Summary reports, backed with general templates for a clear comprehension. An insight into Acceptance testing in Agile and test-driven development will also be given.
Topics Covered:
Acceptance Test Status Report
The Acceptance Test Report should summarize carried out acceptance tests along with their outcomes. It should be communicated to all recognized stakeholders participating in the Acceptance Testing Phase. Once the execution of Acceptance Tests starts, the advance status should be covered everyday.
Common Template for Acceptance Test Status Report:
Date: Date for Acceptance Test Status Report
Today’s Insights on Acceptance Tests Execution:
- Total Tests Passed
- Total Tests Failed
- Total Tests in Progress
Detailed Acceptance Tests Execution Report till now:
- Cumulative total of Tests
- Total Tests Passed
- Total Tests Failed
- Total Tests in Progress
- Total Tests Pending
Details of Defects:
- Count of Logged Defects
- Every defect should entail the following details:
- ID, Summary, Component, Severity
- Total number of defects logged till this point (in the Acceptance Testing Phase).
This report should be reviewed each day to ensure that the execution proceeds on the right track and there are no deviations from the planned roadmaps.
Acceptance Test Summary Report
This report compiles the comprehensive status of the complete Acceptance Testing phase. It integrates details like testing activities undertaken, references to criteria matched, requirement specifications met, prevailing business rules, outcomes of execution, planned timeline, and any observed deviations.
Common Template for Acceptance Test Summary Report:
Summary
<Summarize acceptance testing activities like design, execution of acceptance tests, associated defects, test environment. Additional details such as product version, reference to acceptance test plan, entry criteria achieved, exit criteria to be attained, requirements/specifications, and business rules should also be included.>
Variances
<This supports in enhancing acceptance test planning in upcoming versions.>
Results
<Any causes for test non-execution should be resolved promptly. This could be related to dependencies, environment issues, etc. A revisit to the test plan should be made to handle such issues.>
Evaluation
<A thorough assessment should be made for every module being tested by examining the success rate based on test execution and logged defects. The entry and exit criteria should be put through evaluation. The reason behind observed deviations and possible corrective actions for future releases, needs to be noted down.>
Recommendation
<Based on the entire report, outline recommendations to either adopt or reject the product. Contemplate all necessity of improvements, feedback from acceptance testers, the severity of defects, and pass rate when formulating the recommendation for product release.>
Efforts
<Provide comprehensive information about the work invested on every activity in the phase.>
Sign-Off Report
Once the product passes through the Acceptance Testing, it is recommended to be operational. Prior to moving it into production, it must be formally signed-off.
Common Template for Sign-Off Report:
Product ID, Version for Release, Build Sequence
<Provide the name of the Product and its Release Version. Also, include the most recent build number that underwent Acceptance Testing.>
Latest Report
<Attach the most recent Acceptance Test Report.>
Reviewed On
<State the date when the latest Acceptance Test Report was scrutinized.>
Reviewed By
<State who reviewed the latest Acceptance Test Report.>
Review Feedback
<Incorporate all feedback on the Acceptance Test Report.>
Sign-Off Date
<State the date when the Acceptance Testing was signed-off.>
Sign-Off By
<State who gave sign-off for the Acceptance Testing for the product.>
Sign-Off Feedback
<Provide an explanation for the decision to proceed with the product launch.>
Typically, significant stakeholders should check these reports and concur on the details to be presented.
It remains crucial to verify all compiled data in the report before sharing it with the stakeholders. Any inconsistencies might strike a blow on business decisions and may cause product failure in the market.
Therefore, report making should be entrusted to professionals or senior team members.
Acceptance Testing in Agile
In an Agile environment, Acceptance Tests are framed on the basis of the Acceptance Criteria for each User Story. Acceptance tests are derived from the acceptance criteria of a user story. Each Acceptance Criteria might consist of one or more Acceptance Tests to cover the scenario comprehensively.
Acceptance Tests are typically designed by a QA who possesses subject matter expertise in that domain. Compared to other methods, Acceptance Testing in Agile commences earlier, typically during the sprints.
It is carried out quite frequently as each sprint brings new User stories along with enhancements/improvements to previous stories.
Acceptance Testing is conducted during two distinctive stages in Agile:
- When the feature is fabricated and in its elementary stage – basic.
- When the feature is integrated and harmonized with other product features.
Each user story here undergoes an Acceptance Test and should succeed to be eligible. Any failure encountered in Acceptance tests should be considered of high priority and resolved at the earliest, which in turn will need re-execution of the acceptance tests.
Story points are assigned to each user story grounded on the success rate of Acceptance test outcomes for each acceptance criterion. Acceptance Testing also signals the completion at User Story level, confirming that the Acceptance criteria for the story have been met.
Who executes Acceptance Testing in Agile?
In general, Acceptance Testing in an Agile environment is carried out by Product Managers and Subject Matter Experts (including customers and/or beta testers). Sometimes, QA too is part of this activity, in parallel to their regular regression tasks.
Advantages of Acceptance Testing in Agile
Acceptance Testing in an Agile setup brings in several benefits.
Benefits encompass:
- Improved collaboration between Product Managers and the team.
- Fosters confidence at the User Story level.
- Assists in conceiving additional scenarios to envelop each Acceptance criterion.
- Escalates the likelihood to enhance the product’s solutions through Acceptances criteria in User Stories.
Drawbacks
Despite having numerous benefits, there exist certain demerits as well.
Demerits comprise of:
- Not every story can be subjected to Acceptance testing. Only functional stories should be embraced, which might result in diminished story-wise coverage.
- Not every Acceptance Criteria can be subjected to Acceptance testing. Only functional criterion should be embraced, which might result in diminished Acceptance Criteria-wise coverage within User stories.
- Since stakeholders from diverse backgrounds are involved directly in story-wise acceptance testing, it’s quite intricate for everyone to develop the same understanding at the individual User story level.
- Owing to the shorter release timeframe compared to other methodologies, integrating Acceptance Testing within sprints turns out to be challenging.
Acceptance Test Driven Development (ATDD)
ATDD is one among the Agile development practices where the entire team jointly discusses every Acceptance criterion of a User Story and formulates robust Acceptance Tests around them.
This method accommodates different perspectives from each team member to contemplate about each Acceptance criterion, resulting in more in-depth coverages of acceptance tests, thus catering to more scenarios. Sometimes, ATDD is also referred to as Story Test Driven Development (STDD).
ATDD gets underway before development initiates. This gives developers a clear picture of what is expected and how to achieve it. The entire team possesses a common understanding of the feature and how it is being developed.
ATDD outlines how the product is designed and provides a preview of how the product will function before it is assigned for testing. Hence, it is titled as “Acceptance Test Driven Development”.
Conclusion
The prime objective of Acceptance Testing, irrespective of the method, is to instill customer trust and satisfaction in the product prior to its release. This is achievable when there are no or minimal low-severity defects in the product that do not affect any of the functionalities.
To conclude:
- Acceptance Tests are Passed.
- Defects are within an acceptable level.
- Coverage for Flow-wise/Scenario-wise is achieved.
- Product and its solutions are accepted.
- The customer has developed enough confidence in the product.
- All product documents are updated to resonate with the latest functions.
- Outcome of the team’s effort.
- Ready to be launched into production.
Previous Tutorial | Upcoming Tutorial
Our hope is that you have derived valuable insight from these Acceptance Testing tutorials. Please leave your thoughts and queries in the comments section below.