What is Early Testing: Test Early, Test Often BUT How? (A Practical Guide)

What is Early Testing?

Software testing should start early in the Software Development Life Cycle. This helps to capture and eliminate defects in the early stages of SDLC i.e requirement gathering and design phases. An early start to testing helps to reduce the number of defects and ultimately the rework cost in the end.

The various aspects of Early Testing which would help the QA Managers and Leads while developing or devising the Testing Strategy document in SDLC are explained here.

Adoption of Early Test will immensely result in the successful delivery of a Quality Product.

By the end of this tutorial, the Readers, QA Managers, Leads and Testers will have a fair knowledge of the below concepts:

• Why Early Testing in SDLC (Project or a Software Release)?
• Scoping of Early Testing effort
  • What to Test Early?
  • Start and Exit
• Pros and Cons

Let us now explore the nuances in detail!!

Principles of Testing

Figure 1 – Simplified view of Principles of Testing

For a given Software or System or Product release in SDLC, there are various well-defined methodologies or strategies for most of the following Principles of Testing.

• What is Testing?
• Why Testing?
• What to Test?
• How to Test?

However, some of the most lingering questions that many Readers, Testers, Leads, and QA Managers would ask or would like to get more clarity on include (grey area in Figure 1)

• When to start testing in a software release or When should testing start in a project?
• When to start testing and when to stop testing?
• Why testing should start early in SDLC?
• What is an early test in software development?

For easy understanding of the audience, I have clubbed all the ‘grey area’ questions under one umbrella called Early Testing.

Why Testing Early in SDLC?

Let us discuss some events and activities which are a part of testing.

Usually, the Program Management Team assigns a Program Manager (PM) to a given Software Release or a Project. The PM in collaboration with all the stakeholders including Marketing, Development, QA and Support teams comes up with a Release Schedule

In this tutorial, I have chosen Quarterly Release Schedule using the Waterfall model to explain the Early Testing Concepts in detail.

Software Release Testing Schedule

Most of the organizations still follow the traditional Time Based Release (TBR) models where the Software or Product releases are planned for quarterly or half-yearly or yearly delivery.

Predominantly, the Waterfall model is used for executing such Software releases. In some cases for a shorter release cycle, Agile/Scrum model is adopted.

Figure 2 – Typical Quarterly Release Testing Schedule (Not overall Project or Release Schedule)

Impact of Critical or High Severity Defects

Figure 3 – Typical Impact of Critical Defects

Mainly, during the course of Testing, it is expected that

• Critical or high severity defects be identified and logged by Testers.
• Developers will need to fix those defects.
• Subsequently, testers will need to verify the fixes.

Secondly, it is widely acknowledged by many Product and Software Engineering organizations that fixing and verifying high severity or critical bugs at a very large number is

• Time-consuming
• Resource hogging (human + machine)
• Prone to collateral, fixing critical bugs mostly touch a large part of the code including the intersection areas.

Lastly, if a large number of the critical bugs are found during the end of a given release, then one or more of the following negative developments take place.

• High probability of Testing cycle being extended.
• High probability of release deadline being missed.
• A particular feature having a large number of defects may all together need to be pulled out from that particular release.
• Customer commitments being missed.

How about the other Defects?

There are medium and low priority defects that will be identified and logged by the Testers. These also need to be handled appropriately by the Development and the QA Team. Thus, overall it is a voluminous exercise.

There is no Silver Bullet

It is a well-known fact that no amount of Testing can unearth every defect that a Software Product or the System has. Meaning, practically, neither there is an end to testing nor the product is defect free.

However, from the ‘Serviceability’ point of view in a Competitive and Time To Market (TTM) model, there is a need to break the typical mindset to unearth maximum defects early in a Release cycle, especially identification of critical and high severity defects.

Any or all of the above will have a negative impact on the Organization’s business. In this context, adopting ‘Early Testing’ as a separate Test activity will be beneficial for the overall management of SDLC for a given Project or Release.

The Scope of Early Testing Effort

Having understood the objective of Testing Early in the previous section titled ‘Why Early Testing?’, let us now discuss the ‘Scope of Early Test Effort’ in detail.

As we are introducing Testing Early as a new activity to be tracked exclusively during the course of Testing execution, it is recommended to practice the scope of testing effort as explained below

Assumption:

• Entire Project or Software Release schedule is approved and made available to all the stakeholders.
• Overall Test Strategy document is developed, reviewed and approved by all the stakeholders.
• High, Medium, Low priority features to be tested are well documented.
• Test Plans and Test cases for all the Features are developed, reviewed and approved by all the stakeholders.
• All Test Plans and Test Cases are uploaded in a central repository for tracking testing execution.
• All human resources, infrastructure equipment, and tools are available for setting up the test bed(s) and executing Test plans.

What to Test Early?

Figure 4 – Overall approach to the scope of Testing Early

Approach

• Let us take an Example of Release XYZ having 3 High Priority features A, B and C, 10 Medium priority features and 15 Minor (or Low priority) features.
• High Priority features are those which are generating high revenue and/or standards compliance and/or competitor catch-up and/or competitor one-upmanship and all of these.
• High priority features usually involve some complex coding, a large number of new lines of code added.
• A large number of new lines of code may also mean a high probability of intersection areas.
• Usually, High Priority features and/or features that have a large number of new lines of code are best candidates for Testing Early.
• There need not be a separate Test Plan developed for Early Test activity.
• QA Leads or Testers along with the Development Leads or SMEs (Subject Matter Experts) need to discuss and agree upon the Code/Testing coverage for this testing activity.
• Identify appropriate High priority test cases and even some Medium priority test cases if you think it is necessary from each of the feature Test Plans A, B, and C.
• Once the appropriate features and subset of Test cases are identified, make sure they are tracked using the Test tracking tool adopted by the Organization.

Hint: Collaboration is Key! During Early Test activity both the Development and QA teams need to collaborate closely to make sure that the set objectives are achieved with quality results.

Start and Exit on Early Test

It is important that both the Development and the QA Team brainstorm and agree to all the approaches of the entire Early Test activity including the Start and Exit dates so that all are on the same page.

Entry Criteria for Start

• Percentage of Integration testing completion
• Number of open bugs
• No blockers to start Early Test

Activity Phase

• Tracking progress
• No. of code drops during this testing
• Bug fixing approach
• Bug Verification approach
• Record this testing results

Exit Criteria

• Hand-off activities to the Next Phase of Testing (usually Feature Testing).
• Resolution of unresolved bugs found during Early Test.
• Resolution of blockers if any for the next phase of Testing.
• Publish Early testing results.

Pros and Cons

Every new initiative or activity has its own merits and demerits.

Let us explore the Pros and Cons of this testing approach.

Pros

• Ideally suited for the Waterfall model.
• Helps to uncover critical bugs early in the testing cycle.
• Identification of critical bugs early in a release cycle.
• Helps the Development Team to stabilize the Code early.
• Helps to minimize the collateral due to bug fixes.
• Helps the Development Team identify vulnerabilities across intersection areas in detail early in the release cycle.
• Management Team can make appropriate business decisions with due diligence on unresolved critical bugs in that particular Release or a Project.
• Helps to extend test coverage and cycle effectively.
• Helps to distribute Development and Testing resources efficiently and effectively.

Cons

• Not ideally suited for Agile/Scrum model. However such models can adopt Early Test in Sprints with appropriate tweaking.
• There is a chance of reduced Integration Testing by the Development Team.

Conclusion

Customers or end users buy or adopt serviceability product or a system or solution. Validating a software that is running on such system or products for its serviceability is the primary requirement

Key components of Principles of Testing like Why to Test? What is Testing? What to Test? How to Test? are mostly well defined and understood. However, there are some lingering questions that keep propping up in the mind of the Readers, Testers, Leads, and Managers on concepts like Early Testing.

Adoption of Early Testing as an integral activity of the overall Testing Schedule for any given Software Project or a Release immensely benefits the Organization to deliver a robust qualified Product or a System.

Have you ever realized the importance of Early Testing in your career? Feel free to share your thoughts and experiences in the comments section below!!