At the onset of my career as a Quality Assurance (QA) specialist, I was employed by a Software-as-a-Service (SaaS) corporation where product deployments were of critical importance, as they could potentially influence the functionality of end users.
As the user base grew, our QA team introduced post-launch auditing to control the risk and minimize the intrusion of deployments on end users.
Recommended IPTV Service Providers
- IPTVGREAT – Rating 4.8/5 ( 600+ Reviews )
- IPTVRESALE – Rating 5/5 ( 200+ Reviews )
- IPTVGANG – Rating 4.7/5 ( 1200+ Reviews )
- IPTVUNLOCK – Rating 5/5 ( 65 Reviews )
- IPTVFOLLOW -Rating 5/5 ( 48 Reviews )
- IPTVTOPS – Rating 5/5 ( 43 Reviews )
This was a fledgling concept to me, arousing numerous doubts and queries:
- What precisely is post-launch auditing?
- If I performed comprehensive auditing during the development period, why is post-launch auditing required?
- Do I need to redo all tests? What exactly does post-launch verification require?
- What actions are necessary when a problem is detected?
I am delighted to declare that I discovered the solutions to all my uncertainties during my early product deployments.
I will now impart that knowledge to you in this piece, which I have chosen to write in a question and answer style, to demonstrate how I found the solutions.
Content Overview:
- What is Post Production Launch Verification?
- What tasks and performances encompass the post-launch verification stage?
- Is it necessary to retest everything?
- How should I devise a post-production launch verification strategy?
- Who is responsible for creating the post-production launch test plan?
- Who approves the post-production launch test plan?
- When should I create the post-production launch verification plan?
- What comes after successfully completing the post-production launch verification?
- What actions are required when I discover a problem?
- What else is crucial to understand about the post-production launch verification process?
- Closing Thoughts
What is Post Production Launch Verification?
Post Production Launch Verification refers to the procedure of assessing a software deployment on the live/production environment once it’s been initiated. It is undertaken to confirm that the deployed features align with the set requirements.
Recommended read => Preparing the Test Environment Effectively Prior to Starting Test Procedures
While examining during the QA stage is vital, post-production launch testing is mandatory for several reasons:
- Data Problems – Fluctuations in data between the production and test conditions might result in corner-case problems that may not be detected during testing.
- Deployment Problems – Manual construction deployment procedures can amplify the risk of deployment problems like omitted configuration settings, DB script evasions, and inaccurate installations of dependencies.
- Unforeseen Affected Areas – Certain scenarios may be overlooked as impact zones, leading to unexpected problems upon deployment.
- Unidentified Impact Areas – In intricate architecture involving various products adhering to a standard construction, even slight alterations could cause functionality issues throughout multiple products.
What tasks and activities encompass the post-launch verification stage?
The post-launch verification stage generally incorporates the following duties and activities:
- Post-production launch verification
- Evaluation result reporting
- Reporting discovered problems in the live/production environment
- Data cleanup following post-launch verification
- Post-launch monitoring (if applicable)
Is it necessary to retest everything?
It is not obligatory to retest everything during post-launch verification. The scope of the tests is dependent on the nature of the deployment and impact analysis.
Intensive testing should be performed during the regular QA phase. Post-launch verification should adhere to a test plan derived from the complete test plan for that specific release.
How should I devise a post-production launch verification strategy?
The creation of a post-production launch verification strategy should echo the test flow followed during the QA cycle.
The approach should include steps to assess both newly introduced and major existing features, corroborate impact areas, and maximize functional coverage.
An effective post-production launch strategy should:
- Include steps for testing new and major existing features
- Corroborate impact areas
- Maximize functional coverage
- Potentially, include any vital bugs discovered during testing
- Potentially, prioritize the test cases
Who is responsible for creating the post-production launch test plan?
The role of creating the post-production launch test plan differs among companies and is reliant on their organizational structure.
For instance, in an QA team structured in the following manner:
In this instance, the QA specialist working on a specific project would devise the initial post-production launch test plan.
Who approves the post-production launch test plan?
The procedure of granting approval for the post-production launch test plan also varies among corporations and is contingent on their organizational structure.
Given the same organizational structure as previously stated, the post-production launch test plan should be assessed and approved by the Test Lead or QA Manager.
When should I create the post-production launch verification plan?
The post-production launch verification plan can be formulated at any time during the software’s development cycle, once the requirements, development scope, and affected areas have been determined and finalized. It is advisable to devise the plan half-way into the sprint to provide ample time for review and approval.
It is best practice to incorporate this test plan with any official QA approval documents before the project proceeds into the deployment and launch phase.
What comes after successfully completing the post-production launch verification?
Once the post-production launch verification is achieved, the following actions should be undertaken:
1) Communication of verification results – Stakeholders should receive the verification results, along with any problems detected during the verification process.
2) Reporting any problems found on the production in the Defect Management tool – This enables the performance of root cause analysis and traceability.
3) Post-launch verification data clean up – Data cleanup must be undertaken after completing the verification process.
For instance, if a deployment affects an e-commerce app, and a test order was created during the process, that test order should be canceled after verification in order to prevent the intrusion on client data.
4) Post-production launch monitoring (if applicable) – Certain launches need constant monitoring in the live/production environment.
For instance, if the page load times have undergone improvements by the team, constant monitoring is required to ensure the efficiency of the improvements. This should be clearly communicated to, and designated to accountable individuals.
What actions are required when I discover a problem?
Any discovered problems should be reported in the Defect Management tool and relayed to stakeholders. If vital issues are discovered in the live/production environment, immediate communication is vital to ascertain if a rollback is required for further examination.
It’s crucial to report all detected issues in the Defect Tracking Tool. It is recommended to categorize these as separate issue types (e.g., Post Production Bug) to differentiate them from bugs found during regular QA cycles. This enables easy filtering if root cause analysis is demanded.
What else is crucial to understand about the post-production launch verification process?
In addition to the actual post-production launch verification procedure, plan, and strategy, consider the following:
- Clearly define the expectations concerning the extent and objective of post-launch verification. Internal and external stakeholders need to be informed that:
- It is not possible to test everything on the live environment.
- It is not feasible to condense days of testing into a few hours that are dedicated to post-launch verification.
- Exercise caution when concluding on the scope of post-production launch testing. There are restrictions to what and how much can be tested on the live environment. Because the site contains actual client data, it needs to be treated with care. Supplemental planning needs to be done for changes involving data transfer, updates, and deletions.
- Perform a root cause analysis for problems detected during post-launch verification to ascertain why the problem was not detected earlier, and learn from previous mistakes. This analysis can help address gaps in implementation and prevent similar issues in the future. This analysis can be executed by the Test Lead or QA Manager with feedback from the project team.
- Consider utilizing server logs for post-launch monitoring. Server logs may unveil events or problems that are not visible to customers but can impact backend processes. Accountability for this monitoring can be assigned to the Development Lead and the DevOps team.
An Example:
Project Synopsis:
A social media app needs changes, particularly for the sign-up process:
- The last name field no longer needs validation.
- A toggle button needs to be incorporated next to the email address field in order to allow users to adjust the privacy settings for their email address on profile.
- Users should be given the choice to choose their own avatars.
- The amount of API calls during the sign-up process needs to be reduced in order to increase the application’s performance.
Post-Production Launch Verification Plan:
S.No. | Description | Expected Result | Status | Comments |
---|---|---|---|---|
1 | Open the homepage of the website | The homepage should successfully load | Pass | |
2 | Click on “Sign up” as a new user | The user should be routed to the registration/sign-up page | Pass | |
3 | Fill in the required fields and click on the “Register” button: – Enter “Lee” as the last name – Toggle the privacy button to “Do not Display” – Select an avatar |
– The user should be routed to their profile page after successful registration. – The phone number of the user should not be displayed. – The avatar chosen by the user should be visible. |
Partial Pass | The avatar is not being displayed correctly and seems like a broken image. Reported in JIRA as BUG-1088. |
4 | Monitoring – Verify whether the performance of the application has improved after this launch | The reduction in API calls during the sign-up process should enhance the performance of the application | Ongoing | The Dev Lead and Dev Ops team are to monitor the application for 24 hours |
5 | Cleaning up post-launch | Delete the test account that was created | Completed |
Closing Thoughts
With most software companies adopting Agile methodologies, the frequency of production launches has risen. While a team adhering to a Waterfall model might make a production launch every one to one-and-a-half months, an Agile team might deploy releases every two to three weeks.
With each production launch, there is a chance of unintentionally affecting the functionality of live users. Executing post-production launch verification immediately post-deployment can impart additional assurance in the deployment, as well as providing a safety net to roll back the launch before any glitches are encountered by live users.
For projects that have a high-impact/risk, the post-production launch verification plan can be organized based on the precedence of the test scenarios. Tests with critical precedence can be executed initially, and the results, as well as any problems, can be communicated to stakeholders. If no critical issues are discovered, post-production launch verification can continue, otherwise a decision is required to either roll back the launch in order to reduce application downtime and impact on live customers.
Further, post-production launch assessments can be automated, with test scripts running on demand as part of regression testing. But extra care is needed when running automated test scripts in the live environment to avoid affecting the data and functionality of live clients.
Post-production launch verification serves as the final defense for software companies. Failure to detect problems during this process can severely damage the reputation of a company. To ensure the reliability of a product, it is paramount to rigorously test changes immediately after deployment.
About the author: Neha B, a Quality Assurance Manager who specializes in directing and managing in-house and offshore QA teams, wrote this enlightening piece.
Share your post-production auditing strategy, advice, and experiences with our audience.