A Simple Guide to Interoperability Testing (with Examples)

Before understanding the technique of “Interoperability Testing”, Lets first understand the term “Interoperability”.

Interoperability is an ability of one system to interact with another system. This interaction is between 2 different systems or 2 different applications all together.

Many a time Interoperability is confused with Integration, compatibility and portability. Well, there are differences between these techniques.

Let me first start by explaining the differences.

Integration – Is a technique when the components of the same system interact with each other. So in testing world, when we do Integration testing, we are actually testing the behavior of the 2 or more, lowest levels of components of the same system.

Compatibility – Is a technique by which 2 or more application interact in the same environment. So in testing world, when we do Compatibility testing; we validate whether 2 or more application or systems behave as expected in the same environment.

The intention here is to check that the two systems perform their expected tasks, without interfering each other working, in the same environment. Like – MS Word and Calculator are 2 different application and they perform their expected behavior independently in the same operating system. So we say that these 2 applications are compatible with each other.

Portability – Is a technique when an application or system behaves as expected when it is moved to another environment. So in Portability testing, we export the application to some other environment and test its behavior. Like, if there is an application which works well in Windows XP, should also work well in Windows 10.

Interoperability – Is a technique how an application interacts with another application. So when we do the Interoperability testing, we check how the data from 1 application is transferred into another application without prior intimation, in a meaningful manner, and further processed to give the accepted output.

This particular paper focuses on Interoperability testing (IOT), so let’s keep our focus on Interoperability. 🙂

Interoperability Testing – A brief introduction

Interoperability = Inter + operable

Inter – means “between ourselves”, “within each other”, “mutual”

Operable – means “capable of performing the given task”

So combining the 2 terms together – Interoperability means 2 (or more) systems, capable of performing their allocated task independently and able to communicate with each other as expected without affecting their individual assigned functionality.

Example #1: Take an example of reserving your flight. Consider you need to travel from New Delhi to New York. Now you don’t have a direct flight. You have to travel from New Delhi to London and then take connecting flight from London to New York. Because you have some time constraints, you reserve your flight from New Delhi to London in “Jet Airways” airways and from London to New York in “Virgin Atlantic”. So that means all your passenger details got traversed from Jet Airways to Virgin Atlantic. So here, Jet Airways and Virgin Atlantic, both are independent application all together and while reserving your flight, your details of booking got exchanged from Jet Airways to Virgin Atlantic in a meaning full manner, without prior intimation.

Example #2: In similar lines, think of the hospital administration system, where the records of patients are exchanged between 1 department to other department. So here department can be linked to an application. Details of the patient get exchanged between 1 application to another application without any prior notice.

So why do we need to do the IOT?

We would need to do the Interoperability testing to ensure that

• The applications in the network perform their expected behavior independently,
• Can exchange information without prior notice
• The information/data is exchanged without interrupting the individual expected behavior
• The data / information which is exchanged does not gets modified or changed

How to do Interoperability testing?

We can follow the Deeming wheel (the PDCA cycle) to carry out the Interoperability testing.

#1) Plan

Planning is the most important phase of determining the strategy of doing almost anything in the software development. Before we actually plan for determining the procedure for doing the IOT, it is imperial that we understand each and every application or system deployed in the network.

We should know for all the applications – its functionality, behavior, input it takes and output it reveals.

I would also recommend that each and every application is fully functionally tested with no defects, before preparing it for the interoperability testing. So when you plan, don’t just think of 1 or 2 application, think of all the application as a single unit. You need to have a birds eye view when planning for this testing technique. Needless to say that – document your plan.

We can use our standard Test Plan document and tailor it a bit as per the requirement to document the planning of IOT. After your test plan is in place, move ahead to derive your test conditions.

The focus of deriving your test condition should not be limited to the individual applications; instead it should be based on the flow of data through all the applications. The conditions should be designed in such a way that, if not all, but most of the applications in the network are traversed.

Once your test conditions are identified, move ahead to design or script (in case you plan to automate) your test cases. You can create a RTM (Requirements Traceability Matrix) to map your test cases with test conditions and your test conditions with acceptance test conditions/requirements.

When you are working on a network, it is again important to plan for the Non Functional testing activities as well. This may not be written or documented anywhere, but it’s mandatory to check for the nonfunctional aspects of the system as a whole. These nonfunctional areas would include performance and security. If required you can create separate plan for Functional testing, performance testing and Security testing; or create a single plan and different document of test conditions for each of these testing types.

#2) Do

Do – is the span of time where you actually do your execution. Do plan your time accordingly to execute the functional and non-functional testing. We follow the testing cycle in this phase of executing the cases, logging the defects, following up with development team to get those resolved, doing the re-test and regression test of the system as a whole, reporting the test results and moving it to closure.

#3) Check

Check – Is the phase where we revisit our test results and try to map those with the RTMs and validate whether all the expected requirements are met and whether all the applications are traversed. We check that the data is traversed and exchanged correctly and smoothly between the applications/systems. We would also need to validate that the data which is traversed does not gets modified.

Also consider to do a retrospective of the entire process of interoperability testing. Identify the areas which worked well, those which did not go well and any action items that need to be taken care of.

#4) Act

Act – Is to act on the retrospective items. The points which were identified as “good practices”, continue executing those and the points which could be worked better, identify the steps to rectify those and act accordingly. Keep 1 thing in mind that the areas or steps which did not work well, should NOT be repeated. After all we should learn from our mistakes and not repeat them.

The 5 ½ Steps:

1. Identify all the applications that are part of the network.
2. Identify their respective functionalities.
3. For each application, identify the Input it takes and the output it returns.
4. Identify those data which would be traversing through all/most of the applications.
5. Identify the expected behavior for each combination of application and date that needs to validated

½  Document it.

Consider the below figure:

Based on the figure, let’s try to replicate the 5 ½ steps:

1. Application 1, Application 2, Application 3 and Application 4 are 4 different systems.
2. Each of these systems has the definite set of functionality which needs to be identified.
3. Input and Outputs of each system need to be identified.
4. In case of Application1, it renders 2 outputs. 1 output forms the input of Application 3 and 1 output forms input of Application 2. The output from Application 2 forms the input to Application 3 and Application 4 and so on.
5. The validity for each of the input and output is checked. The major point to consider here is that the data which is traversing in the form of Input and output does not gets modified AND all the application is covered.

½ This figure in real life may not seem to be this simple. This actually results in more complex structure with n numbers of input and output conditions.

Drawing this kind of figure would give a better picture to identify the data and information which would be traversing through different systems. This would help us to derive the test conditions and cases.

An example:

Let’s consider an example of conducting Interoperability testing for a “Hospital management System”

A hospital consists of the below departments and sub-departments;

Here each department is an application in itself. Each department (application) has its own sub department (modules) and each module has its own units.

So now to consider the scope of IOT, here are few test conditions:

1. A patient who met with a road accident (OPD Department – Accident), need to undergo a leg surgery ( ENT – General Surgery), then has to undergo the physiotherapy (Support department – Physiotherapy) and then gets discharge (Support Department – Closure)
2. A child admitted to critical care (Pediatrics – Critical Care) needs to undergo a surgery (Pediatrics / ENT – General Surgery) and then is discharged (Support Department – Closure/PR)
3. An outside patient consults a general physician (OPD department); takes the prescribed medicines (Support Department – Pharmacy) and walks away.
4. An expecting mother comes for regular checkups (Gynecology department – Mother and child care) , takes the prescribed medication (Support department – Pharmacy) and walks away.
5. A dental patient does the root canal (Dentistry department), takes the prescribed medication (Support department – Pharmacy) and walks away.
6. A patient comes in OPD (general physician) , undergoes a treatment in (Obstetrics & Gynecology department – High Risk Obstetrics ) takes the prescribed medication (Support department – Pharmacy) and is discharged

This way we identify all the test conditions; keeping in mind that most of the department needs to be covered.

We can draw a RTM to show the coverage as:

This way we can identify more test conditions and can draw the RTM to see our exact scope. Also we can determine the depth of our testing efforts based on the RTM.

Like in this example, we see that the “Support Department” is the application which is the exit point for all (most) of the application, hence the testing effort for this particular application is a bit more as compared to other application.

Challenges:

1. Difficult to test all the application with all the permutations and combinations.
2. Applications are developed in different hardware/software combinations and are installed in different environments, so if any of the environment is down, it impacts the testing.
3. Because of the different software’s and environments, determining the testing strategy and executing it is itself a big task.
4. Stimulate the environment for conducting the test, is a big challenge.
5. In case of any defect, doing the Root Cause Analysis is a big challenge.
6. Because the applications are in a network, there would be times when the network is down. Because of this, testing also gets affected.

How can I mitigate these challenges?

1) Try to use the advance testing techniques like :

• OATS (Orthogonal Array testing technique)
• State Transition Diagrams,
• Cause and effect graphs
• Equivalence portioning and Boundary value analysis.

These techniques would help you to identify the interdependency amongst the application and identify the test cases/conditions that would ensure maximum coverage.

2) Try to identify some historical data like – under which circumstances the systems were down, how much time does it takes to be back in action. In that case try to execute those scenarios whose applications are not impacted, or utilize the time to document the scenarios and report results. Moreover whenever you plan or schedule the testing, always consider these historical data as an input for your estimation and plan accordingly.

3) PLAN – Use historical data, past experiences, skill of the team, environmental factors to identify the strategy of the testing. The better your plan, the better would be your execution.

4) Start working on preparing the environment much before than your actual execution starts. Needless to say- plan your steps when you are preparing the environment. Make sure that your environment is all set, ready and up & running when your execution starts.

5) Before starting with the IOT, ensure that the individual applications are fully functionally tested with no defects. Then in case of any defect, you would only need to look for the environmental factors that have resulted into some error.

6) As discussed in point 2, Plan your activity. If it’s a scheduled outage, you should be considering this downtime when you plan your testing.

Interoperability Test on Mobiles:

In Mobiles, we do interoperability test whenever a new App (Mobile Application) is launched. There are many areas that we have to consider when planning for this testing on Mobile devices:

• Types of mobile devices available on market are huge. You would need to list down what all types of devices you would be considering for your testing. You would need to pair a device type along with the OS it supports.
• All the Mobile OS are developed in different programming language. Hence, the app needs to be tested against all the variations of OS.
• Understanding the legal factors and region related contracts.
• Size / resolution of different devices are different.
• Impact on the mobile inbuilt apps also needs to be considered.

So for doing IOT on mobiles, you would need plan and create an RTM just like we did for a computer-based application testing.

The intention, strategy, risks, and execution would be same but the tools and techniques would be different in case of mobiles.

Conclusion:

Interoperability testing is a huge task. This technique requires proper planning which should start parallel when system test planning starts.

There are lots of factors which need to be considered while executing this technique. Keep in mind to have sufficient time for bug fixing and retesting, as this is a huge effort there should be provision for defect follow-ups.

This may happen that you may not attain 100% coverage, but we should be smart enough to select our cases in such a way that most of the applications are covered in a single flow by using good test case writing techniques.

Hope this article was useful to understand Interoperability testing technique. Let us know your queries/comments.