How to Find Maximum Valid Defects in Any Application?

Insights from a dedicated testing enthusiast:

Have you ever found yourself in a situation where, after graduating college, you were hired by an organization as a Quality Assurance Analyst or Software Tester instead of a developer?

Have you also been repeatedly advised by so-called “seniors” to pursue a project that will provide you with software development experience rather than testing?

Has it been suggested to you that it’s okay to start your career as a tester, but you should eventually transition to a development role, or else your growth will be limited?

Or have you been led to believe that testing requires very little skill?

If you found yourself nodding to any of the above questions, brace yourself! Because here’s some advice from a person who is passionate about testing and intends to stay in the field for a long time!

What is this article about?

This article not only emphasizes why testing is as important, if not more, than development, but also highlights that discovering defects or bugs requires vision, foresight, talent, and a tremendous amount of skill.

I have also shared a few suggestions and tricks to uncover the weaknesses in software and expose defects.

See Also => More tips for finding bugs in an application

What is Testing?

If we refer to the technical definition, testing is a meticulous investigation process to determine whether a piece of code or a product meets its required specifications for customer use.

Simply put, testing can be broadly defined as a practice to identify if “something is not right” or if “something can be even better”.

Needless to say, this process must be carried out with the objective of discovering errors or, in other words, ensuring that the product or feature meets or exceeds expectations.

The Importance of Testing

When you really think about it – isn’t testing deeply ingrained in our daily lives? Let’s take the common example of shopping. When you, as a consumer, visit an apparel store, you have certain expectations in mind before making a purchase.

The first thing any consumer does is try on the outfit and check if any alterations are needed. Doesn’t this activity, in some way, involve testing or validation? And isn’t it important?

Similarly, in software development, bugs or errors are bound to exist in any feature or product. No matter how perfectly a feature or program is coded, there is always something that can go wrong. This may not necessarily reflect on the developers’ capabilities, but rather on the complexity of the software itself, which can become overwhelming to manage.

Have you ever considered how many implementation or design defects can arise from sheer complexity?

Interestingly, even 50% of development time is spent on testing to ensure that the code functions as intended. Additionally, the cost of fixing a bug or error is much lower in the early stages of the software development life cycle compared to the later stages.

Another crucial aspect to consider is – in a market filled with constantly emerging technologies, with organizations developing numerous competitive products and pitting them against each other, what kind of product do you think customers or consumers would prefer?

It’s safe to say that customers would prefer a reliable, stable, and user-friendly product. This is where testers step in and play an invaluable role in ensuring the quality of the software or product.

Additionally, for high-level product demos to stakeholders or customers, technical representatives usually seek assistance from the test team and heavily rely on their reviews to create customer demos. The technical prowess required to be a tester is evident.

While testing, have you ever questioned why something is done a certain way and not another? If so, you are in a great position to suggest enhancements, provide valuable insights, and even initiate Proof of Concept projects!

What Is A Defect?

A Defect can be described as a failure of a program or feature to meet its expected results or an unsatisfactory condition that prevents the user from achieving their goal.

With the discussion above, as testers or, as I prefer to call it, “test developers,” the primary goal is to expose as many defects as possible, as early as possible. This requires not only a keen eye but also a mindset that can delve deep into the system and identify cases that can lead to failure.

While developers have limited knowledge of the specific feature or program they are working on, test developers have the opportunity to gain comprehensive end-to-end knowledge of the software/product and, in some cases, even delve into its architecture to uncover potential problems.

So who do you think possesses more skill and knowledge?

Best Practices To Maximize The Chances Of finding Valid Bugs

While each individual has their own unique testing style, I would like to share some suggestions and tricks, based on my experience, to ensure good test coverage and maximize the discovery of valid defects.

#1) Research And Self-study

One of the most important aspects is gaining in-depth knowledge of the feature being implemented.

• A comment often made by the development team is that testers lack a complete understanding of the feature, making it challenging for them to differentiate program defects from non-defects.
• As a tester, even if you are not directly involved in technical High-Level Design review meetings, it’s crucial to reach out to the developers and ask for a copy of the High-Level Design, Use Case documents, or any other technical input documentation that can deepen your understanding of the feature.
• Another excellent practice is to have meetings with the developers to discuss the technical aspects of the feature being implemented. Effective communication serves as a valuable means to gain in-depth knowledge not only about the software/product you are involved in but also about customers and related products.
• Additionally, understand how a customer would utilize that functionality. Remember, you act as a bridge between your organization and the customers using the software, so you bear a significant responsibility for understanding the product not only technically but also from the user’s perspective.
• If the feature implements existing technologies, ensure that you understand those underlying technologies.

For example, I had a similar experience when we provided support for a networking concept based on an IEEE standard. I visited the IEEE website and researched various articles related to gain a thorough understanding of the concept.

This approach helped me identify defects that were outside the specified use cases defined by the development team but were valid in terms of the actual functionality specified by IEEE in real-life scenarios.

Of course, it’s apparent that this research helped me uncover more defects than any other team member.

#2) Planning Test Scenarios

Needless to say, research and knowledge are crucial at any stage of testing and bug detection. After the initial stage of self-education, dedicate ample thought to planning the test scenarios.

• It’s an obvious fact that test case planning is a crucial phase in the testing life cycle. The test cases you design and plan directly reflect the quality of the product and the extent of testing conducted. The test cases you outline have a direct impact on the stability of the software that will be delivered to customers.
• After reading a use case document or any other technical functional input document, invest significant thought into designing the test cases. Ensure a good mix of happy path scenarios, error path scenarios, alternate path scenarios, and creative/ad-hoc/out-of-the-box scenarios.
• Be meticulous while describing scenarios and ensure their validity. Conduct multiple reviews with the development team and architects to incorporate their feedback into the test cases. This will ensure that no time is wasted on creating or enhancing invalid test cases.

#3) Investigating Setup Readiness

During most testing cycles, the test team has some time before actual testing begins.

• During this time, it’s good practice to evaluate the readiness of the testing infrastructure. Testers often find themselves in situations where many of their reported defects are related to the setup and get rejected as they cannot be fixed through code changes.
• Testers often find it frustrating when most of their reported defects are rejected due to setup issues, even if they hinder the testing goals. This leads to a lack of test progress and a scarcity of valid defects that can explain the hindrance. In other cases, a test may produce different results each time it is run, making defect replication extremely challenging. Developers often lack the patience and inclination to investigate issues that seem peripheral to the features they are implementing. Reach out for help to stabilize the testing infrastructure as early as possible.

#4) Testing Techniques To Find Defects

Now, let’s dive into the most exciting part of our job: testing and, in turn, defect discovery. We are fortunate individuals who significantly contribute to developers’ code quality improvement.

• Discovering weaknesses: While testing, it’s important to have the mindset to dive deep and identify potential points of failure.
• Happy Path Testing: Firstly, keep an eye on the allotted time for testing a piece of code. Reserve a reasonable portion of time for testing the happy path scenarios. It is likely that you won’t find many bugs in this phase since developers typically ensure effective unit testing for the happy paths.
• Error Path Testing: Next, move on to testing the error path scenarios. Allocate ample time to test the error paths as this is often the most fruitful area. Feel free to be as creative as you like! You are likely to uncover issues here.
• Ad-hoc/Creative Testing: This type of testing is commonly categorized under error path testing. However, it’s good to allocate the last few days of the testing cycle to this activity. I have found several valid defects through ad-hoc testing and negative testing.
• Communicate with Developers Before Filing a Defect: Although testers are not usually expected to directly communicate with developers and can simply report defects, I strongly recommend all testers to communicate with developers to discuss observed issues.

There are two main reasons for this:

a) Most developers prefer to avoid having defects attributed to them due to tight schedules and may reject a defect, asking for more information as a tactic to buy time.

Alternatively, developers may genuinely struggle to understand the steps required to reproduce a scenario that a tester has followed and may return the defect, requesting replication steps, thereby burdening the tester further.

b) Another reason is that we often work with globally distributed teams, and fixes may need to come from developers located in different parts of the world. If the issue is clear to a local developer who is initially assigned the defect, they will ensure that their findings are recorded in the defect and then redirect it to the developer responsible for the fix.

This avoids delays in resolving the defect, as the developer in a different time zone will already have some essential information to initiate the fix, reducing the back-and-forth exchanges between the tester and developer and reducing the defect’s age.

Logs: Providing clear steps for reproducing a problem is not only important for testers; it’s also vital for testers to familiarize themselves with the logs generated during test runs. Including proper logs in a defect report helps in faster resolution. Testers must acquaint themselves with the log information.

Again, communication with developers to understand the log information is crucial. In my experience, developers have always encouraged testers to understand code flow and logging. Once you are familiar with the logs, you can immediately identify the point of failure by analyzing the stack trace. You can also identify additional defects more easily.

Here’s another example that highlights the importance of familiarizing oneself with logs. During our testing cycle, we encountered a scenario where a particular feature appeared to behave correctly according to the specified use cases. However, a field issue arose when customers observed a series of null pointer exceptions in the logs throughout the scenario flow. This was considered a critical escape because it was deemed unacceptable. Resources had to be promptly deployed to develop scripts or tools that could provide direct reports of the null pointer exceptions.

Keep an Eye on Filed Defects: Without being intrusive, it is always beneficial to monitor the defects being filed to avoid duplication and, consequently, rejection. This significantly reduces the number of duplicated defects.

Maintain Run Logs: Although it may seem like an additional task for testers, I believe it is helpful in the long run. Run logs are like diaries that record the day’s work and progress.

Run logs can contain information about the test scenarios run, their behavior under different conditions, the expected output, the actual output, all the steps followed by the tester, and defect information for specific scenarios.

Maintain a neat and understandable record of all this information, and you will have no difficulty reporting status to management or remembering whether a particular scenario was tested or not, even years later. Everything is documented!

Conclusion

The above guidelines can greatly influence new testers in the industry to develop the mindset required to ensure high-quality products. Personally, these practices have immensely helped me become one of the most organized testers in my unit. It is also worth noting that most Subject Matter Experts in any organization are primarily testers.

Enjoy the process of finding bugs and ensuring product quality!

About the Author: This is a guest post by Sneha Nadig, a test lead in a top MNC, with over 7 years of software testing experience.

If you find these tips useful, please let us know in the comments. We welcome more tips and suggestions from our expert readers.