CMMI Appraisals – Tester’s Journey as an Appraisal Team Member (ATM)

Evaluation of CMMI

Personal Insight: Serving as an Evaluation Team Participant (ETP) for CMMI offers a priceless experience for any Quality Assurance (QA)/Tester specialist. It grants extensive familiarity with the comprehensive CMMI model. If your company is subject to an evaluation and you can be an ETP, make sure to take this opportunity. This is an achievement you would want to add to your portfolio.

Message to Readers: Evaluations are intricate and can consume a considerable amount of time. I have tried my best to simplify this process in this article. You will obtain a quick overview of CMMI Evaluations through this.

This article is not about the CMMI model itself, but it elucidates what happens during an evaluation. Initially, you may encounter some difficulty understanding (if you are not acquainted with CMMI terminology), but please persist in reading until you encounter an example below.

Also, consider reading => CMM levels for QA Processes

What Is CMMI Evaluation?

An evaluation is an endeavor that helps in identifying the positives and negatives of a company’s processes while verifying their alignment with CMMI best practices.

Looking Back: When an organization opts for a maturity level evaluation (more commonly known as CMMI Evaluation), an authorized agency from the Software Engineering Institute’s (SEI) Partner network is contacted to carry out the evaluation. The authorized agency delegates a consultant to assist the organization in deciphering CMMI, formulating documents, and instituting organizational policies, among other tasks.

This process takes a considerable amount of time, beginning 1 year or 6-7 months (based on the maturity level of the organization’s processes) before the final evaluation. Post the preparatory phase, a Lead Evaluator (LE) is invited to execute the evaluation.

Organizational-Level Preparation

Before the implementation of CMMI, the organization has to:

• Identify personnel to participate in an initial CMMI course
• Formulate a Software Engineering Process Team/Group (SEPT or SEPG) charged with managing software engineering processes (the responsibilities of SEPT/SEPG will be elaborated at the end of this article)
• Create organizational policies
• Employ CMMI advisors

The Journey of an ETP: Upon the completion of the initial CMMI course, you will move onto the training for an Evaluation Team Participant (ETP). The LE provides training on the “SCAMPI Evaluation Method Training” to the ETPs. Following the training, there will be an examination, which you need to pass.

Each ETP will be provided a reference guide termed the Method Definition Document (MDD), which elaborates on SCAMPI (Topics encompassed in the initial CMMI training course are provided at the culmination of this article).

What Is “SCAMPI”, Who Is “LA” And What Does ETP Do?

SCAMPI: Standard CMMI Appraisal Method for Process Improvement.

Lead Evaluator (LE): The Software Engineering Institute (SEI) designates authorized SCAMPI Lead Evaluators (usually referred to as LEs). The LE declares the final findings report and signs the Maturity Level certification. The term “certification” is avoided by the SEI; instead, a company is referred to as having “achieved CMMI Level 5”.

Function of an ETP:
ETPs assist the Lead Evaluators in accumulating and verifying evidence for document reviews, conducting discussions with Functional Area Representatives (FARs), and preparing other essential artifacts (Additional details about ETP responsibilities are mentioned at the end of this article).

A little more about CMMI: Capability Maturity Model Integration is a model for process improvement, developed by the Software Engineering Institute (SEI) at Carnegie Mellon University.

CMMI encompasses an array of “Process Areas” that can be adjusted to fit the culture and behaviors of each specific organization.

CMMI has a total of 22 Process Areas.

[image source]

Every Process Area within CMMI encompasses several “Specific Goals (SGs)” and “Specific Practices (SPs)” that outline the anticipated behaviors within projects and organizations.

Components of the Process Area

SG: A Specific Goal displays the distinguishing features required to meet the Process Area.
GG: Universal Goals are labeled “universal” since the same goal statement pertains to several Process Areas.

Each ETP is accountable for a Process Area assigned by the LE. For example, if there are 22 ETPs, each ETP will be in charge of 1 Process Area across all chosen projects for the Evaluation. Normally, the LE forms compact groups of ETPs, say 10 groups (each group containing two ETPs), where each group presides over multiple Process Areas across all projects.

For Instance,
Suppose we consider the Process Area “Validation”. (Remember that most readers here are Testers/QAs)

Validation (VAL) is an Engineering Process Area at Maturity Level 3.

Objective of this Process Area: The aim of Validation is to demonstrate that a product or its individual parts fulfill their designated use when placed in the intended environment.

This Process Area comprises of two SGs (Specific Goals).

• SG 1 Preparation for Validation
• SG 2 Validation of Product or Product Components

Now let’s see the Specific Practices for SG1.

SG 1 Preparation for Validation

• SP 1.1 Selection of products and product components for validation and the validation methods utilized.
• SP 1.2 Formation and upkeep of the environment required for supporting validation.
• SP 1.3 Formation and upkeep of procedures and standards for validation.

I shall not provide details about all Specific Practices here, but I will use SP1.1 to offer you an understanding of the evaluation process:

• SP 1.1 “Selection of products and product parts for validation and validation methods to be utilized”: The most appropriate evidence for this SP would be the Software Test Plan (STP). The STP encapsulates all the features/components/capabilities that need testing and the method used to test them.

Responsibilities of an ETP

• Initially, the ETP will confirm if the STP is approved.
• The ETP will check for emails of approval or any supporting evidence of approval such as affirmations from those giving the approval in the minutes of the meeting. If minutes are mentioned, the ETP will verify the attendance record to ensure the approver was present for that specific meeting date.
• Does a review plan exist in the Project Management Plan Annexure? Was the review carried out? Who executed the review? When was the review conducted? Was documentation of the points from the review done, and are they closed in the review log? What is the date of closure?
• If the review was executed, a review checklist should exist.
• Is the baseline of the STP integrated? Were all points from the review closed prior to the baseline? What is the date of the baseline?
• Is the method of validation mentioned in the STP?
• What is the reference document used to prepare the STP? Is it the Requirement Capture Document (RCD), Software Requirement Specification (SRS), or another document? If, for example, RCD is the reference document, the ETP will verify the Bi-Directional Traceability Matrix (BTM) for correct mapping from the RCD to the STP. All requirements have to be appropriately encompassed in the STP for validation.
• Are there guidelines present for formulating the STP? Does the organization possess a standard STP template? What’s the version number of the STP? (Creating guideline and template falls under a different Process Area, but I have incorporated this point for reference.)

It’s crucial to verify the dates! Any document becomes a baseline after a review, so the baseline date should be after the review date. Similarly, the dates mentioned in the checklist, the date of the review, and the closure date of the review points should be correctly ordered in the respective documents.

You should now have a more thorough understanding of the responsibilities of an ETP. Reciprocal verification of all the documents prepared throughout the Software Development Life Cycle, ensuring their dependencies on each other, together with conducting reciprocal verification covers the central responsibilities of an ETP. Besides this, the ETP engages in interviews with members of the FAR. I will expand on this subject in upcoming articles.

A Few More Illustrations:

These domain-independent examples should offer additional insight into the evaluation process.

As an example: When assessing the minutes of a weekly status meeting conducted on a specific date, the ETP makes sure of the verification of two points:

1. Discussion of risks during the team meeting.
2. Adding a new team member and the project manager’s assurance of subsequent training.

#1) Risk: Risks fall under the Risk Management (RSKM) Process Area.

The ETP will check if the risk is mentioned in the Risk Log. If it’s there, a Risk Mitigation Plan or Contingency Plan should exist. If a Mitigation/Contingency Plan exists, there may be action items related to the plan.

If there’s an action item, monitoring might be required. Monitoring is a task under the Project Monitoring and Control (PMC) Process Area. The ETP should understand what needs to be verified and where to perform the verification. This example demonstrates the interconnection of Process Areas and the multiple artifacts the ETP needs to evaluate to ensure proper adherence to the practices.

#2) Training:
All activities regarding training come under the Organizational Training (OT) Process Area.

In this circumstance, the project manager sends a request for training to the HR department. HR will identify an internal/external trainer for the individual and conduct the training. This is a general scenario, but what else happens in OT?

Let’s dive deeper into how OT functions. To illustrate: Presume the organization has allocated a certain sum for training in this fiscal year. HR will determine the needs for training through data collected from managers and personnel, prioritize them, and also take into consideration the prospective needs of the organization.

For instance, if the organization decides to foray into communication projects along with software development, HR will either hire new resources/managers/team leaders or identify current individuals within the organization who can serve in this new field.

Areas of Verification for an ETP

• Determination of training needs by HR
• Identification of people who would participate in the training
• Selection of trainers
• Notification to the manager and the participant to schedule their calendars and subsequent confirmation in advance
• Date of requests to the Admin and IT team for infrastructure (room for training, laptop, software, food, transportation, etc.)
• Is there an attendance sheet in place?
• Was feedback taken from the participant post-training?
• Did HR modify the Organization Skill Matrix post-training?
• If any training is pending from the previous year, planning is done to conduct it in the current year and documentation of reasons for not executing the training in the previous year is done. etc.

Please remember that this does is not an exhaustive summarization of activities regarding Organizational Training; there are additional activities as well. These are instances recollected from memory.

• Duties of SEPT/SEPG (Software Engineering Process Team/Group):
  • Scrutinize organizational processes
  • Create standards for processes
  • Formulate stages of SDLC and provide details
  • Formulate procedures for meetings
  • Develop procedures for estimation
  • Create procedures for planning
  • Create Quality Assurance procedures
  • Create procedures for Configuration Management

• The Initial Training Course for CMMI covers the following topics:
  • Introduction to the model
  • Improvement of processes based on the model
  • Overview of the components of CMMI
  • Institutionalization of Processes across the Organization
  • Process areas of the CMMI model
  • Structure for continuous and staged representations

<

ul>

Responsibilities of an ETP:

<

ul>

Each ETP is responsible for comprehending the CMMI model, understanding the organizational context, and applying the SCAMPI methodology to evaluate implementation.

All ETPs participate in

Related

Best Iptv Service Provider 2024 With 40k+ Channels And 150k+ VOD . Hurry Up