What is Test Harness and How is it Applicable to us, Testers

I am not a big fan of labels. Here is what I mean by that.

If I have to check few aspects before I determine whether or not QA can be started, I will simply make a list and perform the action. In my opinion, it does not matter if I officially call it a “Test readiness review” operation or not – as long as I am doing what I am supposed to do, I think there is no need to call it a specific name or label.

But I stand corrected. Recently, in my class, I was teaching Agile-scrum model for software development. There was a question ‘how is testing performed in an Agile method?” I was explaining two methods- one is where we try to include it within each sprint and the other is a best practice that I have learnt from the first-hand implementation- which is to lag a QA sprint with respect to the development one.

One of my students asked me if there is a name for the second one and I didn’t because I never placed emphasis on the names themselves.

But at that moment, I felt how important it was to label a process appropriately to make sure that we have a term to refer to the process we are talking about.

Therefore, today we are going to do just that: Learn the process behind the term “Test Harness”.

As I mentioned before in some of my previous articles: a lot can be understood from the literal meaning of the name. So, check your dictionary for what “Harness” means and the big reveal of whether or not, it applies, in this case, is something we will see at the end.

There are two contexts to where Test harness is used:

1. Automation testing
2. Integration Testing

Let’s begins with the first one:

Context #1: Test Harness in Test Automation

In the automation testing world, Test harness refers to the framework and the software systems that contain the test scripts, parameters necessary (in other words, data) to run these scripts, gather test results, compare them (if necessary) and monitor the results.

I am going to try and make this simpler with the help of an example.

Example:

If I was talking about a project that uses HP Quick Test Professional (now UFT) for functional testing, HP ALM is linked to organize and manage all the scripts, runs and results and the data is picked from an MS Access DB – The following would be the test harness for this project:

• The QTP (UFT) software itself
• The scripts and the physical location where they are stored
• The Test sets
• MS Access DB to supply parameters, data or the different conditions that are to be supplied to the test scripts
• HP ALM
• The test results and the comparative monitoring attributes

As you can see, software systems (automation, test management, etc.), data, conditions, results – all of them become an integral part of the Test harness – the only exclusion being the AUT itself.

Context #2: Test Harness in Integration Testing

Now it is time to explore what Test harness means in the context of “Integration Testing”.

Integration testing is to put together two or modules (or units) of code that interact with each other and to check whether or not the combined behaviour is as expected or not.

Ideally, Integration testing of two modules should and would be possible to carry out when both of them are 100% ready, unit tested and good to go.

However, we do not live in a perfect world- which means, one or more modules/units of code that are to be the constituent elements of the integration test might not be available. To solve this situation we have stubs and drivers.

Stud is usually a piece of code that is limited in its function and will substitute or proxy for the actual module of code that needs to take its place.

Example: To further explain this, let me use a scenario

If there is a unit A and Unit B that are to be integrated. Also, that Unit A sends data to Unit B or in other words, Unit A calls Unit B.

Unit A if 100% available and unit B is not, then the developer can write a piece of code that is limited in its capability ( what this means is the Unit B if it has 10 features, only 2 or 3 that are important for integration with A) will be developed and is used for integration. This is called a STUB.

The integration would now be: Unit A->Stub (substituting for B)

On the other hand, if Unit A is 0% available and Unit B is 100% available, the simulation or proxy has to be Unit A here. Therefore when a calling function is replaced by an auxiliary code, then it is called the DRIVER.

The integration, in this case, would be:  DRIVER (substituting for A) -> Unit B

The entire framework: The process of planning, creating and usage of stubs and/or drivers to carry out the integration testing is called the Test Harness.

Note: the above example is limited and the real-time scenario might not be as simple or as straightforward as this. Real-time applications have complex and composite integration points.

In conclusion:

As always, STH believes that the even the most technical of definitions can be derived from the term’s simple, literal meaning.

The dictionary on my smartphone tells me that a “Harness” is (look under the verb context):

“To bring under conditions for effective use; gain control over for a particular end; “

Following this and adapting this to testing:

“A test harness simply is to create the correct framework and use it (and all of its constituent elements) to control the entire activity as to get the most of the situation- whether automation or integration. “

There, we rest our case.

A few more things before we finish:

Q. What are the benefits of a Test Harness?

Now, would you ask what the importance of breath for human life is – it is intrinsic, isn’t it? Similarly, a framework to test effectively is like a given. The benefit, if we have to spell it in so many words- I would say, every testing process has a test harness whether we consciously say that it is “The Test harness” or not. It is like travelling knowing the route, the destination and all the other dynamics of the journey.

Q. What is the difference between test harness and test framework?

I personally think that comparing and contrasting is not often the right approach when understanding related concepts because the lines are often blurry. As an answer to that question, I would say, the Test harness is specific and Test framework is generic. For example, a test harness will include the exact information of the test management tool down to the login IDs to be used. A test framework, on the other hand, will simply say that a test management tool will do the respective activities.

Q. Are there any Test Harness tools?

Test harness includes tools – like automation software, test management software, etc. However, there are no specific tools to implement a test harness. All or any tools can be a part of Test Harness: QTP, JUnit, HP ALM- all of them can be constituent tools of any Test Harness.

About the author: This article is written by STH team member Swati S.

And, always with definitions, there are always differences in opinions. We welcome your opinions and love to hear what you think. Please feel free to leave a comment, questions or suggestion below.