Failure Mode and Effects Analysis (FMEA) – How to Analyze Risks for Better Software Quality & Satisfied Customers!

Failure Mode and Effects Analysis (FMEA) is a method of managing and assessing risks.

When implemented correctly, it can greatly enhance the Quality Assurance processes and ultimately improve software quality. This article aims to introduce you to FMEA as a risk analysis technique that is highly beneficial for enhancing software quality.

Failure Mode And Effects Analysis

FMEA is mainly utilized by upper management or stakeholders. Traditionally, testers have had limited insight into this technique. However, the trend is changing and if testers understand this concept, it can significantly enhance their thought process when it comes to writing test cases. Understanding the stakeholder’s goals, the business, and deriving effective test cases that prioritize risk-prone areas becomes easier utilizing this technique.

• Understanding the goals of testing the application from the stakeholders’ perspective.
• Comprehending the business.
• Deriving high-level test scenarios based on business and management interests.
• Creating effective test cases that provide comprehensive coverage to high-risk areas.
• Prioritizing test cases.
• Deciding what to test and what to defer at any phase.
  

Background

Risk analysis plays a crucial role in Test Management. The question then arises – What is Risk Analysis? And why is it important? To understand this, it is vital to grasp – what is RISK?

See Also => Types of Risks in Software Projects.

Risk, in its literal sense, refers to the possibility of a negative or undesirable outcome or event. If not handled or managed properly, risks can lead to poor quality, dissatisfied customers, and even loss of business.

Risk has two attributes:

• Probability
• Impact

Probability signifies the chances of a particular risk occurring, while impact refers to the extent of the effect of the risk.

What Is Risk Analysis?

Risk Analysis is a mechanism used to thoroughly analyze and study identified potential risks in order to determine their probability and impact. By measuring these attributes, we can identify:

• What to test first?
• What to test more?
• What not to test at this time?

There are several methods of conducting Risk Analysis, broadly classified into two types:

• Informal Techniques: These techniques rely on experience, judgment, and intuition.
• Formal Techniques: These techniques involve the identification and assessment of risk attributes.

Failure Mode And Effects Analysis (FMEA): This is a formal method of conducting Risk Analysis. In the following sections, we will delve deeper into FMEA and provide an example to illustrate its application.

FMEA is a systematic and quantitative tool, presented in the form of a spreadsheet, that assists in analyzing potential failures. To conduct FMEA effectively, it is important to have the right individuals involved, including representatives from all areas of the industry, including customers.

Description

FMEA starts and continues with brainstorming sessions. During these sessions, participants identify all the components, modules, dependencies, and limitations that could fail in a production environment, leading to poor quality, reliability, and potential business loss.

During FMEA, the team not only identifies the extent of the potential loss but also strives to determine the underlying causes of these failures. Measuring FMEA requires three attributes:

• Severity of the Failure (S)
• Priority of the Failure (P)
• Likelihood of the Failure (L)

Each of these attributes is assigned a scale, as shown below:

Severity Scale:

Description	Class	Scale
Loss of data, hardware, or safety issues	Urgent	1
Loss of functionality without a workaround	High	2
Loss of functionality with a workaround	Medium	3
Partial loss of functionality	Low	4
Cosmetic or trivial	None	5

Priority scale:

Description	Class	Scale
Complete loss of system value	Urgent	1
Unacceptable loss of system value	High	2
Possible reduction in the system value	Medium	3
Acceptable reduction of system value	Low	4
A negligible reduction in the system value	None	5

Likelihood scale:

Description	Class	Scale
Certain to affect all users	Urgent	1
Likely to impact some users	Very High	2
Possible impact on some users	High	3
Limited impact to a few users	Low	4
Unimaginable in actual usage	None	5

All three attributes (Severity, Priority, and Likelihood) are individually measured on a scale and then multiplied to calculate the Risk Priority Number (RPN).

i.e. Risk Priority Number (RPN) = SPL

Based on the RPN value, we determine the extent of testing required. A lower RPN indicates a higher risk level.

Let’s examine an example to illustrate this:

Failure Mode Effect Analysis Example

(This example is purely hypothetical and intended for illustrative purposes only. Actual implementation and features may vary)

Let’s consider a banking application with four features:

• Feature 1: Withdraw
• Feature 2: Deposit
• Feature 3: Home Loan
• Feature 4: Fixed Deposits

A Risk Analysis team, consisting of the Bank manager, UAT Test Manager (representing end-users), Technical Architect, Test Architect, Network Administrator, DBA, and a Project Manager, is formed.

After a series of brainstorming sessions, the team identifies the following risks:

• The complex business logic involved in calculating interest rates for home loans.
• The system’s failure when subjected to 200 concurrent users.
• The system’s inability to handle documents larger than 6 MB.

Now, let’s calculate the Severity, Priority, and Likelihood of these identified risks.

Severity:

Feature	Class	Scale
Complex business logic in calculating interest rates for home loans	Very High	2
System’s failure with 200 concurrent users	High	3
System’s inability to handle documents larger than 6 MB	Very High	2

Priority:

Feature	Class	Scale
Complex business logic in calculating interest rates for home loans	Very High	2
System’s failure with 200 concurrent users	High	3
System’s inability to handle documents larger than 6 MB	High	3

Likelihood:

Feature	Class	Scale
Complex business logic in calculating interest rates for home loans	High	3
System’s failure with 200 concurrent users	High	3
The system’s inability to handle documents larger than 6 MB	Low	4

Now let’s combine all these attributes:

Feature	Severity	Priority	Likelihood
Complex business logic in calculating interest rates for home loans	2	2	3
System’s failure with 200 concurrent users	3	3	3
System’s inability to handle documents larger than 6 MB	2	3	4

Now let’s calculate the Risk Priority Number (RPN = Severity * Priority * Likelihood)

Feature	Severity	Priority	Likelihood	RPN
Complex business logic in calculating interest rates for home loans	2	2	3	12
System’s failure with 200 concurrent users	3	3	3	27
System’s inability to handle documents larger than 6 MB	2	3	4	24

The key takeaway here is: The higher the RPN, the lower the risk.

In this example, Feature 1 (Complex business logic in calculating interest rates for home loans) poses the greatest risk, while Feature 2 (System’s failure with 200 concurrent users) has the lowest risk.

How can this analysis aid the development of test cases?

Since Feature 1 presents the highest risk, test cases for this feature should be comprehensive and in-depth. The focus should be on validating complete functionality and the affected modules. Utilize various test case writing techniques (Equivalence Partitioning and BVA, Cause and effect graph, State Transition diagram) to derive the test cases.

The test cases should cover not only functional aspects but also non-functional aspects (Load test, Stress and Volume tests, etc.). Exhaustive testing is required for this particular feature, so the test cases should reflect that. Consider all modules that depend on this critical feature.

Feature 2 poses the LEAST RISK. Therefore, test cases for this feature can focus on major functionality alone. High-level test cases to validate that the feature functions as expected should suffice.

Feature 3 represents a MODERATE RISK. Test cases should cover all major functionality and associated dependencies. Include BVA test cases to validate negative scenarios. The extent of test cases should be between high-risk and low-risk factors. If necessary, include a few non-functional test cases as well.

FMEA And Degree Of Testing

The RPN value helps determine the extent or degree of testing required.

Typically, if:

• RPN is between 1-10, extensive testing is necessary (covering all aspects of the feature/module).
• RPN is between 11-30, balanced testing is recommended (covering all major functionality).
• RPN is between 31-70, opportunity testing is conducted (covering basic functionality).
• RPN is above 70, no testing is conducted or only anomaly reporting is done if time permits.

These ranges or numbers are not fixed and can vary depending on the nature of the project.

Resources: Download FMEA Software and FMEA Template.

Conclusion

Conducting Risk Analysis using FMEA requires time and experience. Desired results can only be achieved with the equal participation of all relevant team members. Though FMEA is a formal technique, it necessitates multiple brainstorming sessions, and it is crucial to document all identified risks.

If implemented correctly, FMEA offers several advantages. It can help identify potential risks and enable the team to plan effective mitigation strategies based on those risks.

About the Author: This article was written by Shilpa Chatterjee Roy, who has been working in the software testing field for the past 8.5 years across various domains.

If you have experience using this technique, feel free to share your insights in the comments below.