Ad-hoc Testing: How to Find Defects Without a Formal Testing Process

When it comes to testing, there is a method called ad-hoc testing that does not follow a structured approach. Ad-hoc testing, also known as black box or behavioral testing, is performed without any formal process in place.

Unlike traditional testing, which involves documentation like requirement documents, test plans, and test cases, ad-hoc testing is not typically documented and follows a more intuitive and creative approach.

This type of testing aims to find defects or flaws that may be missed by formal testing methods. Testers perform random tests without a specific use case in mind, with the goal of breaking the system and uncovering any potential issues.

Ad-hoc testing requires skilled and knowledgeable testers who have a deep understanding of the system being tested. It is a comprehensive testing approach that is particularly useful in evaluating the effectiveness of the test coverage.

Let’s Start With an Example of Ad-hoc Testing

Here is an example of how ad-hoc testing can be performed on a UI Wizard:

Imagine you need to create a plan or a template using a UI Wizard. The wizard consists of multiple panes where users input data. As the wizard progresses, there may be certain actions like context pop-ups that are triggered based on user input.

To test this, a tester would typically perform regular tests such as successfully completing the wizard, canceling the wizard midway, editing a created plan, deleting the plan, and checking for appropriate error messages when entering negative values.

In addition to these regular tests, ad-hoc testing can uncover more defects. For example:

• Adding special characters that are not restricted to test if they are handled properly.
• Interacting with pop-ups and trying actions like pressing the backspace button to observe system behavior.

Characteristics of Ad-hoc Testing:

Ad-hoc testing demonstrates a few distinct characteristics:

• Tests are aligned with the objective of finding defects, but they are often drastic and performed with the intention of breaking the system.
• Testers must have a comprehensive understanding of the system being tested. This testing approach aims to identify weaknesses in the testing process.
• Ad-hoc tests are typically performed once, unless a defect is found and needs to be retested.

When Should Ad-hoc Testing Be Performed?

Ad-hoc testing can be valuable in various stages of the software development lifecycle. It is especially beneficial when detailed testing cannot be conducted due to time constraints.

The general recommendation is to perform ad-hoc testing after the execution of the current test bucket is complete. However, experienced testers can determine the ideal time to conduct ad-hoc testing based on their knowledge of the system being tested.

There are also situations where ad-hoc testing may not be necessary:

• If a test case already exists for a known defect, there is no need for ad-hoc testing. The test case needs to be properly documented and retested once the defect is fixed.
• When customers or clients are invited to test a beta version of the software, ad-hoc testing should be avoided.
• For simple UI screens, traditional positive and negative testing should be sufficient to uncover defects.

Types of Ad-hoc Testing

Ad-hoc testing can be categorized into the following types:

#1) Buddy Testing

In buddy testing, a test member and a development member work together on the same module. This allows both parties to gain a broader perspective and understand the design and testing requirements.

#2) Pair Testing

In pair testing, two testers collaborate on a module, sharing the same test setup. This approach promotes brainstorming and collaboration, leading to the discovery of more defects.

#3) Monkey Testing

Monkey testing is performed at the unit testing level. Testers randomly test the system to ensure its ability to handle unexpected scenarios and potential crashes.

Benefits of Ad-hoc Testing

Ad-hoc testing offers several benefits:

• Testers can uncover more defects than with traditional testing methods due to the flexibility of applying innovative and creative testing techniques.
• Ad-hoc testing can be performed at various stages of the software development lifecycle, not just limited to the testing team. Developers can also benefit from performing ad-hoc testing to improve their coding practices.
• Ad-hoc testing can complement other testing methods, potentially reducing the time required for regular testing and improving overall test coverage and product quality.
• Ad-hoc testing does not require extensive documentation, allowing testers to focus on understanding the underlying architecture.
• When time is limited, ad-hoc testing can provide valuable test coverage and quality.

Drawbacks of Ad-hoc Testing

Ad-hoc testing also has some drawbacks:

• Due to the lack of organization and documentation, testers need to rely on memory to recall the details of ad-hoc scenarios, which can be challenging, especially in complex systems.
• If requested to provide information about defects identified during ad-hoc testing, it may be difficult to recreate them in subsequent attempts without proper documentation.
• Effort accountability is also a challenge as ad-hoc testing is not planned or structured, making it difficult to track the time and effort invested.
• Ad-hoc testing requires skilled and knowledgeable testers who can proactively identify potential defect-prone areas.

Best Practices for Effective Ad-hoc Testing

To make ad-hoc testing effective, consider the following best practices:

#1) Identify Defect-Prone Areas

Understand which features are more prone to errors and focus ad-hoc testing efforts on those areas. Analyze the number of defects reported against each feature to prioritize testing.

#2) Build Expertise

Experienced testers can leverage their knowledge and intuition to identify potential defects. Newer testers should use ad-hoc testing as an opportunity to gain expertise on the system being tested.

#3) Create Test Categories

Categorize features based on their visibility and priority. Test the most critical and commonly used features first. Also, consider categorizing tests based on integration between components or modules.

#4) Have a Rough Plan

While ad-hoc testing doesn’t require detailed planning, having a rough plan or jotting down test ideas can help ensure that all tests are executed and nothing is missed.

#5) Use Relevant Tools

Explore tools that can assist in identifying defects that may not be immediately apparent during functional testing. Logs and error messages can provide valuable insights.

#6) Document for Continuous Improvement

Although ad-hoc testing doesn’t mandate extensive documentation, keeping a record of scenarios covered and defects found can help improve overall test cases and product quality.

Conclusion

Ad-hoc testing is a valuable testing technique that allows testers to be creative and innovative. It offers the opportunity to find defects that may be missed by formal testing processes. However, it is important to maximize the strengths of ad-hoc testing while ensuring it adds value to the overall test process and product quality.

About the author: This article is written by Sneha Nadig, a Test Lead with over 7 years of experience in manual and automation testing projects.

Do you perform ad-hoc testing in your projects? What are your suggestions for making ad-hoc testing successful?