Context Driven Testing: 7 Basic Principles with an Example

7 Basic Principles of Context-Driven Testing With an Example:

When I drive to the airport, I usually take the freeway that will get me there in the minimum time and avoids tolls. But that day, I took a longer local road route with a toll. Because I wanted a few extra minutes with my friend on the drive, who traveled a very long distance to spend the weekend with our family. The normal worst choice, in this case, turned out to be the best one.

But, consider this.

What if I was low on gas? 

What if I was low on cash?

I would choose the different option. Why? The context.

[image credit]

When you make decisions based on, the following it is a context-driven decision:

• The people involved
• Circumstances
• Goals
• Available options
• Emotions, etc.

So, what is context-driven testing?

Context Driven Testing is a mindset shift (or School of testing) developed by Cem Kaner, James Bach & Bret Pettichord. Details about it can be found in their famous book: Lessons Learned in Software Testing.

There are 7 basic principles to it. The following are directly picked from their book:

#1) The value of any practice depends on its context.

#2) There are good practices in context, but there are no best practices.

#3) People, working together, are the most important part of any project’s context.

#4) Projects unfold over time in ways that are often not predictable.

#5) The product is a solution. If the problem isn’t solved, the product doesn’t work.

#6) Good software testing is a challenging intellectual process.

#7) Only through judgment and skill, exercised cooperatively throughout the entire project, are we able to do the right things at the right times to effectively test our products.

I am not going to explain each one of them because it is done for us by the experts themselves here.

I am simply going to do a scenario based explanation on my take on Context-Driven Testing.

An Example of Context Driven Testing:

Let us say I am starting a testing project – Project A which includes end-to-end testing for a web based application.

What would be my strategy?

According to the standard processes, this will be the sequence of events:

• Gather requirements and understand the application
• Create a Test plan
• Create Test documentation- Test scenarios, Test cases, Traceability Matrix, etc.
• Have all the documentation reviewed and approved
• Set up QA environment and Test data
• Perform Test execution
• Create bug reports
• Generate and share test execution status reports, etc.

If I ask myself the question, “How did I decide this is what I need to be doing?” My answer would be, best practices, QA standards, Industry guidelines, past experience baselines, etc., right?

I am repeating what I was taught to do or what I have seen others do.

Now, is there something wrong with that? Not at all. This could even work too because there is a certain sense of repeatability and time-testedness to this approach. However, does it pave the way for optimal results?

Doubtful. Why?

Because with every project you are going to deal with different circumstances:

• Documented vs. undocumented requirements
• Closely working vs. geographically distributed teams
• Development and Testing teams belonging to the same company vs. competition
• Sufficient time vs.Tight schedules
• The composition of your team– Newcomers vs. experienced ones. Trained vs. untrained ones.
• Availability of tools- Manual vs. Test management tool usage
• Type of the project- Needs strict adherence to rules (FDA or banking) vs. experimental (like social media)
• The technology of the project. For example: you would not test the web and a windows app the same way
• Requirements of the clients (Some want daily detailed reports, some want just the highlights)
• Process followed (Agile vs. Traditional, scripted vs. exploratory testing)

This list is not exhaustive and you know as well as I do that there are many variables to each project.

Context-driven testing is when you let these circumstances decide your test practices, techniques and even definitions rather than standard, industry-perceived ‘best practices’.

Now, let’s say these are the details I am working with for Project A:

• I am working with a team of 5- 4 newcomers and 1 experienced tester.
• I have no documented requirements.
• My team is in India and the development team is in the US, so we work opposite time zones.
• Client wants a daily detailed status report
• We use a web based bug tracking tool, such as Mantis or Bugzilla and that is all we have.
• I have to do 2 rounds of testing in 10 days with 3 days for test documentation

Here is a rough game plan:

1) Since lots of newcomers are in the team, we need plenty of peer reviews.

2) We also need at least 2 requirement discussion meetings with BA and Dev team. This has to be formal because the teams are located elsewhere and there is little scope for me to go walk up to them with questions.

3) It is an aggressive testing timeline for documentation. The more documentation we write, the more reviews we need, which equates to more time. So, we will have to keep minimum documentation. We are going to document just the main End-to-End TCs and the rest are going to be tested exploratorily.

4) Daily status reports during test execution are going to be created and sent EOD every day.

5) Most testing is exploratory, so advice the time to try to make a brief outline of every test executed. This way we know what is tested and what is not.

6) Defects will be reported real-time into Mantis. Since the team is working in a different time zone, they might have to wait a whole day before they hear from the QA team, in case they need a clarification. Therefore, set up an everyday call at a convenient team, where QA team will demonstrate the recreation of bugs. That way, there will be no need for waiting around or follow up.

And so on.

Once you have an overall strategy, write a basic test plan explaining these points. Now, you are all set to get into a testing project after careful consideration and custom made a strategy for success.

In Summary:

This is Context-Driven testing; making your circumstances (not the standards) the primary inputs and influencers for your test strategy. It urges us to look around and take everything around you into account.

Personally, I am in love with this concept because too often testing practices are perceived as rigid and imitation-based. Someone did it and was successfully, so I am going to do it too. This is the kind of image that scares people from trying and staying in a testing career.

But, there is plenty of scope for creative thinking, analytical skills, and decision making. To know more, read-up on the topic in the links provided above.

Happy context-driven-testing