Introduction to IQ-OQ-PQ:
IQ, OQ, and PQ constitute the 3Q’s of Software Validation Process.
Recommended IPTV Service Providers
- IPTVGREAT – Rating 4.8/5 ( 600+ Reviews )
- IPTVRESALE – Rating 5/5 ( 200+ Reviews )
- IPTVGANG – Rating 4.7/5 ( 1200+ Reviews )
- IPTVUNLOCK – Rating 5/5 ( 65 Reviews )
- IPTVFOLLOW -Rating 5/5 ( 48 Reviews )
- IPTVTOPS – Rating 5/5 ( 43 Reviews )
As testers we all know that the Software Development Team develops the software in-house as per the Software Requirements Specification (SRS), Functional Specification and later the Testing Team verifies the implementation at different levels of testing at various testing environments, from simplest to the high end, which thereby replicates the production environment.
With this approach of SDLC, the Software Development Team generally washes off their hands by handing over the completed software (developed and verified) to the Operations Team. Further, it is Operations Team (generally referred as Ops Team) which takes care of deploying it to a production environment and make it ready to be used by the end users.
Now, here lies the real challenge for the Operations Team to make the software functional on the production environment, because during the software development phases, development and verification has been done in a simulated environment, and quite rarely close to the live environment, only in case of availability of data and configurations of the production environment.
This is where the validation of the software comes into the picture. Once the verification is completed and the software is signed off the by the Program/Product team, Ops Team would carry out a set of activities before accepting the software to be deployed to the production, to prove that the software is behaving as expected, which is nothing but the validation activities.
What You Will Learn:
Verification vs Validation
Here let’s clearly understand the difference between ‘Verification’ and ‘Validation’ activities. ‘Verification’ is to evaluate the software with respect to the given set of requirements and specifications which is done in-house at the Software Development site by the Developers and Testers.
Whereas ‘Validation’ is a set of quality assurance checks which are carried out by the external customers, owners, vendors on the product being delivered to them, to check the suitability before accepting or purchasing the product. Validation activities are mostly carried out at the production site.
Hence, in case of Application Development, it is the Ops Team who is carrying out the Validation activities for the software.
Also read:
Verification vs Validation – What’s the Difference and Why It is Important to Understand It
Phases of Validation Process
Generally, the Validation Process of any product refers to the complete life cycle of a product from the development through use and maintenance. And hence the validation process is broken down into 5 Phases.
5 Phases of Validation Process are:
This 5 phase approach of the Validation process is being followed in many Industries like Manufacturing, Medical, Pharmaceuticals etc. Here validation will be done by the end customer before buying the machinery, equipment or the product.
The constituents of the Validation activities for a software is to prove that ‘the software is ready for consumption by the users’, and to mainly verify the successful installation of the software followed by the functionality and operability.
3Q’s approach: IQ-OQ-PQ
However, in the software context, the 3Q’s approach, IQ-OQ-PQ is being followed as part of Validation and it will be carried out by the Operations team, who are ultimately responsible for deploying the software to the production.
Given below is the Validation Process Flow Diagram:
The template, plan and any other documents which are input to carry out the 3Q’s, will be laid out by the Software Team for their software and it includes the detailed approach, tasks/activities/tests to be conducted as a part of these qualifications along with the test results.
Summary reports will be handed over to Ops Team during the software handover along with the binaries and other deliverables.
At high level,
Overall, the purpose of carrying out IQ, OQ, and PQ is to ensure that the software can be successfully deployed and all the functionalities can be used without any bottlenecks.
Ideally, IQ, OQ, and PQ are the sequential activities, which need to be executed in the order. Unless the installation is done, a functionality of the software cannot be verified and unless the functionality is proven, no point in measuring the performance. Sometimes due to time constraint, PQ can start in parallel to OQ, once the key aspects of OQ are established.
Now, let us understand more about each of these 3 phases in detail.
Installation Qualification (IQ)
Installation qualification also referred as ‘IQ’, is the process of validating if the supplied software (binaries, scripts etc.,) can be successfully installed on the specified environment with the specified configurations, and to verify how these installation steps are recorded in the document called ‘Installation Guide’.
The following items are supplied by the Development Team along with the delivered software package and are used by the Ops Team to carry out IQ.
1) ‘Installation Guide’ document, which documents the installation steps in the selected environments.
2) ‘Configuration Guide’ document to set up the configurable of the software. Sometimes this document becomes a part of the Installation guide document itself.
3) Software package and Installation scripts, preferably automated scripts.
Software Installation Qualification phase is considered to be the most crucial one and typically many issues open up during this phase.
Because:
a) Development environment will not have 100% real-time environment available to verify the installation issues and hence a difference in the environment contributes to several issues.
b) Due to various reasons, there may not be enough collaboration and coordination between the Development and the Operations Team during the initial stages of software development to handle the issues well ahead.
c) There could be some documenting issues while recording the actual installation steps in the document, which may not exactly match at the production environment.
These days, entire software installation procedure will be automated as much as possible via a series of scripts. If there are any issues with the installation, then the automated installation fails due to any miss-match in the configurations and manual intervention to fix those issues are required.
As the Ops team carries out the IQ by strictly following the instructions provided by the Software Team in the Installation guide, it is very important and also the responsibility of the Software Team to ensure that ‘Installation Guide’ is written in such a way that the installation steps match to the real-time environment.
And it is the Testers responsibility to ensure that the ‘Installation’ process is verified in-house along with the document verification for its completeness and to identify any miss-matches with the actual steps to be run on the system against the documented steps in the Installation guide.
The following points need to be kept in mind while writing an Installation Guide and verifying them in-house, in order to minimize the issues during software installation at production.
| SNO | Installation Guide Points |
|---|---|
| 1 | Main and utmost, the ‘Installation Guide’ should be written in a simple and easy to follow language. |
| 2 | Need to ensure that it does not run into long, more than 5 pages. It should be short and neat. |
| 3 | Need to provide the serial numbers for each step of execution in order to track its status. |
| 4 | Automate the steps as much as possible and bundle all of them into a single script. |
| 5 | A standard template should be used to write installation procedure. |
| 6 | Pre-requisites should be clearly mentioned to avoid the miss-match and the steps to verify them needs to be provided. If there is a miss-match, instruction to bring them up to the expected level or to install those packages should be provided. |
| 7 | The typical time taken to install the software should be mentioned in the Installation Guide for the Ops Team to have idea on the approximate timing of installation to plan their activities accordingly. |
| 8 | The services that needs to be brought down during installation, how to bring down, impact of bringing them down needs to be clearly mentioned in the guide. |
| 9 | Providing links to other documents should be avoided and switching from one document to other. Every piece of information needed should be made available in the same document. If additional documents needs to be referred, supply them along with the software package, and in turn they need to be referred in the main documents. |
| 10 | Need to ensure that the name of the script mentioned in the document is same as the one which is packaged along with the binary. |
| 11 | Should ensure that all the scripts referred in the installation Guide document are supplied along with the binary. |
| 12 | Ensure that all the configuration parameters are clearly mentioned in the Installation Guide/Configuration Guide along with the default values and other supported values. |
| 13 | Automated tests to carry out the build verification tests after the software installation is completed should be supplied. They need to be minimum in number and important ones to verify that build is successfully installed. |
| 14 | ‘Smoke Tests’ needs to be supplied to ensure that end to end connectivity of the system is perfect and all the components of the system are talking to each other as expected. |
| 15 | In case of failure of software installation, rollback scripts are supplied along with the package and rollback procedure is clearly written out in the Installation Guide to carry out rollback and restore the system successfully. |
With all the above points to be taken care, it is a best practice to automate the software installation process with minimum human intervention in order to avoid the human errors.
If any issues are found during IQ validation phase, then it will be reported to the Software Team, upon fixing of which, the smoke testing and build verification tests will be carried out to check the success of the software installation.
Hence the IQ phase includes installing the software package followed by conducting the build verification and smoke tests.
So, successful completion of the IQ phase is very important as a successful and right installation of a software ensures that most of the issues related to functionality failures are negated.
Operational Qualification (OQ)
Operational qualification, also called as OQ is the next activity of the software validation process after the successful completion of IQ.
The Operational qualification activity includes the tests to be run in order to verify that the software is operationally fit to be deployed to the consumers. Ideally, the key functionalities of the software are verified as part of this validation process.
An OQ Plan for carrying out the OQ Validation needs to be prepared by the Software Team (Testers) which should cover all the aspects of OQ testing that needs to be carried out, including the details like no. of tests, test schedule, methodology, tools, impact on the service, test execution sequence, method of reporting issues and the SLA’s for fixing them, Defect Triage approach etc.,
The Operational Qualification Tests which are run as part of OQ, are again supplied by the Software Team along with the software deliverables. These Operational Qualification tests are a collection of important tests which are designed based on the ‘Functional Requirements Specification’ document to ensure that the entire software system functions as per the expectation.
This OQ Test Specification Document is prepared by the Test Engineers against the Functional Requirements Specification document. Often this document will be the subset of the System Test Specification document prepared and verified during the system testing phase of the SDLC.
The tests may be altered or updated to suit the Operational Team requirements and the conditions of the site where it will be executed.
Hence additional care should be taken while selecting the tests which are a part of the OQ to ensure that all the key functionalities and the main business workflows are included as a part of this verification.
The following are the tips for Testers while preparing the OQ test specification document.
| Sno | Tips for Testers while preparing the OQ Test Specification Document |
|---|---|
| 1 | Ensure that key functionality tests to prove that software functions as expected are chosen and included and hence the necessary traceability for each of the written test cases are available in the OQ Test Spec document. |
| 2 | Ensure that the tests are neatly written with step by step actions, are self-explanatory and able to be understood by a common man. |
| 3 | Do not refer or avoid usage of any technical terms in the test cases as much as possible, since the user of this document may not know about those terminologies.e that the test data used does not already exists on the system. Provide multiple sets of data, in case if user wants to execute the test cases more than once. |
| 4 | Clearly mention the mandatory and optional pre-requisites for each of the tests. |
| 5 | Include majority of the positive test cases to verify the functionality. |
| 6 | Include very few negative test cases to ensure that software behavior is as expected in case of irrelevant input and the system is able to handle the negative cases successfully. |
| 7 | Test cases related to boundary value need not have to be included, which verifies for extreme values, but use the most common day to day used values as inputs, wherever required. |
| 8 | Mention the configuration values to be set, if needs to be changed from the default values. |
| 9 | Supply the automated test cases to be run, wherever available. Ensure before in hand that those automation scripts can be run on the system where the OQ is being planned. |
| 10 | Ensure that each test cases have the clear ‘Expected’ and ‘Actual’ results as a reference. And add any comments if necessary to explain the actual result. |
| 11 | It is also necessary to include the ‘Acceptance criteria’ for each of the test cases. The Acceptance criteria could be the status of the system after the execution of the test cases. |
| 12 | Supply the ‘Test Data’ to be used for each of the tests accurately. Try to supply most common data from the live. And also few exceptional data, to ensure that system can handle the exceptional cases as well. Ensure that the test data used does not already exists on the system. Provide multiple sets of data, in case if user wants to execute the test cases more than once. |
| 13 | If multiple Operational users are running the tests from different locations in parallel, provide the instruction to carry out testing accordingly with different set of data. |
| 14 | Provide checklists where ever required to ensure that all the configurations, pre-requisites are set as expected before running the tests. |
| 15 | Keep monitoring the logs, when the tests are running, if access is available to the system. |
| 16 | If possible and required provide an execution support to the Operational users during execution of these test cases. |
| 17 | Explain the method of reporting the issues, found during execution. It is better to use the bug tracking tool to track the issues. Monitor each issue carefully and take it to closure as per the agreed SLA’s. |
| 18 | Run ‘Defect Triages’ involving right stake holders to understand the critical and severe issues and provide updates on those issues frequently. |
| 19 | Provide the final ‘OQ Test Execution Summary Report’ template to publish the final results after the completion of the execution. |
So thus prepared OQ Plan and Test Specification should be thoroughly reviewed and signed off by the relevant stakeholders to mainly ensure that either the coverage is not too less or too much and all the key functionalities are covered.
Successful completion of OQ demonstrates that the software will function according to its operational specifications in the selected environment and it is the stage gate in moving the software towards its production and is the signal to go ahead with the next activity of the Validation process which is PQ.
Performance Qualification (PQ)
After ensuring successful IQ, OQ completion the next activity in the Validation process is to ensure if the product/software meets the specified Performance aspects under the expected load consistently without causing any bottleneck in the production environment.
The key aspect of PQ is to ensure that a software, when installed on the expected system, can handle the live load and meet the expected response time and does not crash under the peak loads and stress while handling concurrent users.
Hence PQ is mainly to ensure if the specified performance criteria for a software are achieved over a period of time (maybe a week) on a reliable basis with varying load conditions, as is the pattern in the live. Hence these tests have to be run every day to monitor the software system behavior and hence PQ will take a while to complete till it is ensured that the system is proved for its performance.
Ideally, PQ Validation is carried out post the completion of OQ, where the functionality of the software is ensured and can go ahead with verifying the performance aspect of the product or software. Sometimes due to time constraint, PQ can start in parallel to the OQ, based on the confidence on the percentage of OQ completion.
It is ideal to carry out these performance tests on the live system with the fully loaded system or on conditions similar to live and ensure that there are no bottlenecks on the performance aspects.
The following tests are generally run as part of the Performance Qualification. And the choice of the tests varies from software to software.
#1) Availability Test: To ensure that the software is continuously available without crashing or going down.
#2) Accessibility Test: To ensure that the software is easily accessible from every location with the expected performance speed without any issues.
#3) Load Test: To measure the behavior of the system under the anticipated day to day load and also peak load conditions.
#4) Stress Test: To measure the breakpoint of the system under extreme loading conditions.
#5) Throughput Performance Test: To measure the response time of the system and to measure TPS (transactions per second )
#6) Scalability Testing: System can scale to handle the expected concurrent users.
The Performance Test Scenarios and the corresponding automated scripts are prepared based on the performance related requirements specified in the ‘User Requirements specification’ documents.
As similar to an OQ Plan, a detailed PQ plan which clearly states the testing approach, strategy, plan, and schedule along with tools, should be prepared and run through with the PQ executors.
The performance testing and monitoring tool need to be installed in the environment where the PQ is being carried out to measure and report the performance metrics.
The following are the tips for the testers to enable the Operations Team to carry out the PQ successfully.
| Sno | Tips for the testers to enable the Operations Team |
|---|---|
| 1 | Prepare the key business specific scenarios to carry out the performance testing based on the URS. |
| 2 | Ensure that tests are included to prove that the system meets the expectations of response time, speed, scalability, and stability under various loading conditions. |
| 3 | Ensure that specified load is available or method and tools to generate the required load are clearly mentioned in the respective test cases. |
| 4 | Mention the pre-requisite for each of the scenario clearly, like the pre-load conditions that should exist on the system, number of concurrent users etc., |
| 5 | Mention tools recommended to be used to carry out the performance testing specific to each category of test and for each test. |
| 6 | Ensure that the process to monitor the performance metrics are mentioned clearly. |
| 7 | Guide, support and train the Operations team to carry out the performance testing on the system. |
Upon successful completion of PQ, meeting the performance requirements is very important as any performance related deviations can cause a huge business loss by creating discomfort to the user and the trust on the software to be used will be lost leading to the failure of the software.
In a nutshell, the below table summarizes the IQ-OQ-PQ activities.
| IQ | OQ | PQ | |
|---|---|---|---|
| What | To verify the process of software installation and how the process is documented | To verify the proper functioning of the system | Customers, Owners, Vendors, Operations team |
| Who | Customers, Owners, Vendors, Operations team | Customers, Owners, Vendors, Operations team | Customers, Owners, Vendors, Operations team |
| Where | At owners site, operations team location, live site, prod like environment | At owners site, operations team location, live site, prod like environment | At owners site, operations team location, live site, prod like environment |
| When | When the software is received from the software team, before OQ and PQ. | Before releasing the system for use and after successful IQ completion | Before putting the system to Live and after successful IQ, OQ completion |
The below table explains the various inputs for each of validation phases.
| Type | Input |
|---|---|
| IQ | 1. Design Specification Document
2. Software binaries and other installation scripts 3. Installation Guide Document 4. Configuration guide Document 5. Build Verification and Smoke Test Document |
| OQ | 1. Functional Specification Document
2. OQ Plan Document 3. Operational Qualification Test Document 4. OQ Test Summary Report Template 5. IQ completed successfully |
| PQ | 1. URS (User Requirement Specification) document
2. PQ Plan Document 3. Performance Qualification Test Document 4. PQ Test Summary Report Template 5. IQ and OQ completed successfully |
Conclusion
Even if the product/software has passed all the verification stages and fails to prove any one of IQ-OQ-PQ, the result can be disastrous and will incur a huge cost. Hence successful completion of IQ-OQ-PQ alone is the successful transfer of the product from the development site to the production site.
Overall, the successful completion of IQ-OQ-PQ validation process not only gives the confidence on the software but also gives a peace of mind to the Client, Owner, Software Developers and the Testers.
Running IQ-OQ-PQ also reduces the risk of deploying it to live, without carrying out testing and reduces the cost of failure and mitigates the risk of recall of the products.
So, Guys, Software Developers, and Testers, no celebration after completing development and testing in-house and releasing the software to Ops Team. The celebration is only when it successfully completes IQ-OQ-PQ and the software is live on the targeted system.
Hence the success of a software depends on the successful completion of IQ-OQ-PQ and when the software is live and ready for consumption by the end users.
About the Author: This article is written by STH team member Gayathri Subrahmanyam. She is having more than 2 decades of experience in the field of Software Testing. During her testing career, she has done a lot of TMMI assessments, Test Industrialization works, TCOE setups in addition to handling test deliveries and implemented DevOps practice for a huge engagement. But according to her, learning never stops…
Share your experiences about carrying out the validation process and let us know if you have any queries about this article.