How Do You Decide Which Defects are Acceptable for the Software to Go-live?

At the go-live stage of a software, verifying that all the functions are operational and that the software released to users is dependable and of high-caliber is vital. An inferior or unstable product can lead to financial losses and tarnish a company’s reputation.

The general rule of thumb is that testing should cease once the exit criteria have been achieved, with defects being rectified to a satisfactory degree. However, these guidelines can often be ambiguous. So, how do we arrive at particular decisions regarding defects and how they influence software go-live?

The following aspects should be contemplated:

• What defect proportion is appropriate for software to go-live?
• How do we decide which unresolved defects will be accepted for go-live?
• Which defects carry a higher level of severity compared to others?

To expand your knowledge on this, take a look at this recommended reading about when to conclude testing.

If you’re pondering over these questions, this article seeks to clarify them. Continue reading to delve deeper.

It should be noted that intricate software is never entirely defect-free, and there’s a perpetual decision-making process between resolving defects and crafting functional software. With this thought in mind, how is the degree and category of defects suitable for go-live determined?

Here are some pivotal considerations:

• #1. Functional Defects: When software complies with customer’s stated specifications, it should meet the requirements. Any variations from these requirements are regarded as functional defects. Categorizing functional defects based on severity and priority is fundamental. High-severity and high-priority defects are those that majorly influence the software’s everyday usage and should be rectified before go-live. Some functional defects might be classified as change requests if they weren’t part of the initial requirements but are pivotal to business operations post-go-live. The categorization and prioritization of defects are decided by UAT coordinators jointly with business consumers and analysts, with guidelines provided by the customer on the appropriate percentage of unresolved defects for go-live.
• #2. Performance and Load Defects: Particularly crucial for go-live are performance defects, especially if the software is meant for external users. Slow software can deter users and drive them to consider alternative solutions from rival companies. Considering all performance aspects is vital, encompassing batch processes that could influence total performance. Performance is typically measured by response time and tested using tools like LoadRunner, WebLoad, or Neoload. Contractually documented performance expectations must be proven before go-live. Infrequently used screens or application segments may be postponed for assessment after go-live. Performance testing is usually done after completing functional UAT and attaining the acceptable exit criteria for functional defects.
• #3. Usability Defects: The importance of software usability cannot be overstated. The software should be designed for effortless use by end users, minimized screen navigation, and instinctive features. If there are too many page maneuvers or complexities, users may lose interest in using the software. Usability guidelines are preset prior to software development, and the software must adhere to these standards. Tool restrictions may need to be addressed for optimal usability, and usability consultants may be hired to guarantee a fluid user experience. Like functional and performance defects, usability defects are logged, prioritized, and must meet the exit criteria for go-live.
• #4. Security Defects: Software security is indispensable to protect user data from possible breaches. Dependable software should be designed to thwart unauthorized access and sustain data integrity. Security testing is done during UAT by ethical hackers to discover vulnerabilities that must be rectified before go-live. All security defects must be resolved before go-live. Security also includes login credentials, user roles, and permissions for accessing different application sections and performing specific actions.
• #5. Integration with External Systems: Often when software is deployed at a customer’s site, it needs to be integrated with pre-existing software or external systems. Such integration should be seamless and free of errors, ensuring that all input and output function properly. Compatibility with various platforms may also be needed. Extensive testing must be carried out during system and UAT stages to confirm integration with external systems.
• #6. Reports: Reports produced by the software are critical for verifying data accuracy and reconciliation. Reports must operate as intended and provide a truthful representation of the data within the software. This is particularly significant when data is being migrated from an old system to a new one.
• #7. Data Migration: If the software is replacing an old system, data from the old system must be migrated to the new one. The migrated data should be compatible with the new system and meet the requirements agreed upon during requirement collection. The new system may not have all the old data, but necessary snapshots or transferrable data should be included.

In closing, crafting foolproof exit criteria that ensure a seamless go-live process necessitates a comprehensive understanding of the software, its function, user expectations, and any structural or hardware dependencies. The list above is not exhaustive and may differ based on the specifics of the application. Considering all relevant factors and defining comprehensive exit criteria on a project-by-project basis, taking into account severity, priority, and business requirements is imperative.

Sample Exit Criteria for Go-Live:

The subsequent is merely an example, and actual criteria may change:

• All Priority 1 defects (severity critical and priority 1) need to be resolved.
• 90% of Priority 2 defects (severity high and priority 2) need to be closed, with a logical workaround for the remaining 10% of defects, and a strategy to resolve them.
• Production deployment and a sanity checklist should be prepared.
• A production support team must be assembled and ready to tackle any issues.
• 70% of Priority 3 defects (low priority) must be closed, with a plan ready to deal with the remaining 30%.

Reminder: Severity and priority definitions are proposed during the initial business meetings between the customer and vendor. After logging UAT defects and remedying other problems, UAT coordinators and business sponsors review the remaining unresolved defects. Once all necessary defects for Day-1 go-live have been addressed, the business sponsors determine go-live readiness and proceed with production deployment.

About the writer: This piece was authored by Krishnan Venkatraman, a software testing professional with almost two decades of experience. He has contributed to numerous large and complicated software testing projects.

Feel free to post any questions or comments you may have below.