Cognitive Bias in Software Testing: Why Do Testers Miss Bugs?

Cognitive Bias in Software Testing: Have You been Influenced?

The testing world is moving at a very quicker pace with technological advancements in order to ensure “quality at the speed of light”.

“Continuous Integration, Digital transformation, life-cycle automation, shifting quality to the left to minimize costs” etc are some of the magical words that are swinging around. While we speak about these, the underlying question –“Why and how the defect was missed” still continues to be heard and remains unanswered as well. 

And at time it feels as if the most obvious defects have slipped through.

But Why?

Though we all might wish to think that we are very logical, structured and rational, the sad fact is that we are all under the influence of Cognitive Biases which influence our thought processes in our day to day life including our professional work too.

Cognitive Bias – A Brief Description

As per Wikipedia – “A cognitive bias is a systematic pattern of deviation from norm or rationality in judgment. Individuals create their own “subjective social reality” from their perception of the input.

An individual’s construction of social reality, not the objective input, may dictate their behavior in the social world. Thus, cognitive biases may sometimes lead to perceptual distortion, inaccurate judgment, illogical interpretation, or  irrationality. “

Alright… That’s for a good definition… But how does it impact thinking and what it means for the testers in the testing world?

Well !! When the testers approach any testing, they are already influenced by their own biases – framing thoughts and judgments based on what to look for, where there would be potential defects coming up, who is developing it, the entire history of the program etc., and the list goes on.

It is very much imperative for us to know about the different types of biases so that we can be much more aware and indeed think of managing them effectively.

Types of Cognitive Biases in Software Testing to Look For

There are different Cognitive Biases that we need to look for within ourselves and some of them are explained below in detail.

#1) Resemblance Bias

It is very easy for humans to judge a situation based on the resemblance of a similar situation.

For Example, as testers, we often tend to think that web applications will have similar kind of errors while client-server applications would have a similar set of errors.

We, as testers, naturally would be looking out only for those similar errors depending on the nature of the project. Unfortunately, because of this nature, at times we tend to miss out the most obvious ones just because our mind did not allow us to think that.

#2) Congruence Bias

This is the behavior where our mind refuses to think for alternatives.

It means, testers tend to validate only the expected behavior and as a result the negative validations are missed out.

When test cases are being written, we tend to cover all the requirements with their expected behaviors and miss out the negative flows because not all the negative flows are mentioned specifically in the requirements.

They are implicit within the requirement and it is practically not possible to document all the user behaviors as well.

#3) Confirmation Bias

This is a tendency to search for and interpret the information by confirming our beliefs and hypothesis.

Typically, in testing world, am sure that we would have situations where we thought that one specific developer’s code by default would have more defects when compared to the others and hence we would have spent a lot of time on testing the module developed by him.

Being under the influence of these beliefs will tend to increase the risk of missing out defects in the modules developed by others.

#4) The Bandwagon Effect

The bandwagon effect asserts the conduct or beliefs spread among the people.

When a certain number of people believe in something, then it automatically increases the probability of the other person also to believe the same. This happens many a times in our day to day life.

A most common example is when we buy some products. Rather than we independently selecting a product we usually go with the belief of others.

Exactly the same behavior is displayed in the testing world as well. In our peer group, if some people feel that one particular module is defect free, we unknowingly tend to believe the same and the focus we give to that module during validation goes down considerably.

#5) In-Attentional Blindness

This is a behavior whereas testers, we tend to miss the most obvious defects when we are not looking for it.

Imagine a situation, where, you ask a group of people to count the number of persons coming in with a particular dress color, you could observe that people will be so engrossed in counting that and they miss seeing any other important or significant thing around them.

To relate this to the testing world, in an enhancement project for example, where one of the screens were developed new then it is a natural tendency of the testers to focus on the newly developed screen more and miss out the other critical integrations.

#6) Negativity Bias

The negativity bias is a human tendency to give more psychological weightage to bad experiences than a good one.

Where does this come into the testing world? It is very difficult to convince the testers to provide sign off for a build to production since their focus would only be on the defects they have uncovered.

There is no time when they can certify that there are absolutely no defects in the products. This is one of the main reason for which the decision to Go-Live for a product depends on the Product or the business manager though recommendations would be cited by the test managers.

Conclusion

Hope you would have got a better idea of Cognitive bias in Software testing, its influences, and what can be done to eliminate the influence?

And an important fact to realize is that we are blind towards our own biases while we can identify the biases of others (this in itself is a blind spot cognitive bias). However, we could be more conscious and to a large extent, we can give extend more thought on these wherever required.

Do you remember being influenced by Cognitive Bias? Have you noticed Cognitive bias in your peer group? Feel free to express your thoughts in the comments section below!!

About the Author:

This interesting post is written by STH author Geethanjali S – A Certified Scrum Master and PMP Certified having 18+ years of experience in Quality Assurance and Engineering. A Transformational leader having rich experience in leading and delivering QA transformations, Global roll out programs, Mergers & Integration related programs.