There are many checks that an organization must implement to ensure proper compliance with legal regulations or alignment with market standards or business objectives. Often, the results are reported only at the top management of the internal process that carried them out without being fully shared with other business processes. The reason is likely due to the absence of a common interest and the idea that it adds a cost without adding value.
Typically, an enterprise already has a great deal of control data collected for a variety of reasons. The data are useful, collected for legitimate reasons, analyzed in detail and, if necessary, results in the issuance of risk treatment plans that are consequently implemented for corrective action. At this point, if everything seems to have been done correctly, then the cycle of checks begins again. This is the expectation of the enterprise, but asking some different questions can create a new scenario. Could those collected, processed and stored data also be used in other business contexts? And if aggregated differently, do they acquire new meanings?
One would think that the control systems applied on strictly vertical processes (pyramidal, autonomous, with a very specific purpose) still have room for improvement if they are able to identify common areas of control (with different depths of analysis) and share results between internal processes. This would lead to more rational data collection, committing resources without wasting time on repeating activities, and the possibility of obtaining new information from the shared database at zero cost. Even if vertical, the processes must not be parallel independent flows but must collaborate by sharing activities and data whenever possible.
It becomes of interest to identify a common point among all. A system that is a collector for all processes and is suitable for collaborating with everyone. The ideal candidate is the Capability Maturity Model1 (CMM) (figure 1), which can be applied to all relevant controls. This model makes it possible to create interactions with all processes due to its nature as a method for assessing the formal degree of adherence to thresholds of increasing aptitude to implement the rules and activities assigned. It evaluates the ability of an enterprise to deal with situations, perform actions and satisfy requirements and, therefore, is sufficiently general to be applicable to any process.
The CMM is a qualitative method based on the measurement of the progress of predefined levels of increasing maturity. Pure numerical methods are only apparently excluded but later integrated. Each qualitative threshold is always combined with normalized numerical values between zero and one and weights to focus more on one threshold over another during consolidation or to allow the extraction of a numerical degree for the metric considered.
The maturity model is not a substitute for a numerical performance evaluation method but evaluates the ability to provide an estimate of the correctness of the results of the quantitative metric used. It gives an opinion on the ability of the numerical method to provide a quality assessment and works in a complementary way to obtain further information. Therefore, it adds value to the numerical methods adopted.
Activity-Based Maturity Model
Operationally, the maturity model is represented in its simplest form by a set of activities and their assessments. Maturity is a concept that lends itself to designing business development scenarios from various perspectives as a consequence of how it is interpreted. It expresses only a metric of the ability to complete an activity; however, with more data, an estimate can be made to whether it has the ability to achieve the assigned goal. The higher the maturity, the lower the uncertainty in achieving the objectives. This is also the way of defining risk as reported by the International Organization for Standardization (ISO) definitions “effect of uncertainty on objectives” and “combination of the probability of an event and its consequence.”2, 3 Maturity can be related to risk, but not only risk.
Consider the opposite of capability, which is the inability meter or (unwanted) ability to create flaws in a management system. It becomes easy to recalculate it as a vulnerability metric and immediately have an overview of the issues to be addressed to create a practical operational improvement plan.
Another way of looking at the CMM is its placement. It is the connecting link between the operational world divided between the various processes and the world of controls (figure 2).4, 5, 6, 7 This link is dynamically built according to the logic of collaboration between processes (with Agile techniques) to create value through a rational redistribution of tasks.
How to Create the CMM
Creating the CMM starts with activities. The activity is an operating rule defined by the enterprise, in competition with many other rules, to meet business objectives. The set of assessments of all internal enterprise activities is also seen as a way of checking the correct performance of the business. Therefore, “activities” is interchangeable with similar words such as “controls,” “actions” and “countermeasures.” This dualism between actions that take place and controls on the action performed forms a link between the world of operations and that of controls.
The set of all these activities constitutes the checklist of the maturity model. The identification of which controls to include in the model is a simple iterative process. The first step is the identification of all the rules with which the business must comply in relation to its mission (i.e., the set of main laws that affect the business, the clauses imposed by customers, the market standards adopted, the objectives chosen, the code of conduct, internal policies and procedures).
The following iterative steps serve to refine the set of controls. After each assessment, depending on the emergence of a need for greater detail or a different orientation of the business, the individual controls can be reviewed with the introduction of new ones, with the doubling of existing ones, or with the modification of the meaning or definitive cancellation.
The dominant logic is that of Agile8 processes as summarized by the ITIL 4 Guiding Principles.9 There can be no fear in changing the model. Adaptability is an integral part of every step of the iterative process. Each iteration tries to improve the previous one in order to have a maturity model that is increasingly aligned to the context of the business.
No one should expect that everything will be in place for the start, but rather one should start with what exists and then incrementally optimize the model with the resources available at the moment. Accelerating the process means committing more resources; the speed of the iteration depends on the resources that the enterprise decides to commit. The frequency of monitoring is determined by maximizing the benefit in the balance between cost and data quality. This decision is made based on the results of the risk analysis.
Assessment of the CMM
The CMM cannot be comprehensively assessed by maturity alone. Other information must be considered for an exhaustive view of all risk factors. Maturity is the basis of assessment, but it must be integrated with the knowledge of the ability to improve it or not. In addition to the ability to carry out an activity, it is necessary to consider the possible impact in case the expected result is not reached, the probability of the worst event occurring and the progress of a corrective plan.
The simplest assessment methodology to adopt to collect information is the qualitative one, evaluating experts on the subject. These subject matter experts estimate the correct level of implementation of the activity and other information surrounding it. The qualitative assessment is always complemented by numeric attributes (hidden from the user) to allow for easy consolidation according to each desired level of aggregation.
Maturity is assessed through a grid of possible predefined levels of implementation (figure 3). They are used to declare the completion of the activity. Each level is equipped with numeric attributes to allow for quick consolidation. Among these, at least the numerical score and a value to weigh the level of risk connected must be entered. The elements inserted, and the numerical attributes, are subject to periodic changes to better focus the attention of the results obtained on specific events.
It is easier to request the impact as a degree of expected loss compared to the intrinsic value of the entity under evaluation (figure 4). It is a fairly simple way to collect information, as it follows how the operational person involved in the assessment thinks. The expert of the specific topic under consideration is certainly familiar with the topic, its value and the consequences in the event of a negative event. This guarantees the high quality of the data collected.
The possibility of the negative event happening is measured on five levels (figure 5). Increasing the levels does not benefit the quality of the result. However, it is an approximate assessment based on individual sensitivity and experience. More is not necessary because they are predictions of the future that will inherently never produce certain results. The error is amortized in the consolidation by the presence of assessors with different propensities for risk who tend to converge toward a balance point between optimists and pessimists.
The progress of the corrective activities (figure 6) records the level of implementation of the planned actions. Conceptually, it is the current state of the implementation plan of the countermeasures identified to bring the risk level below the risk acceptance threshold. The end of the plan coincides with the acceptable state of the risk as established in the risk analysis, while this assessment gives an indication of the ability to achieve the desired state.
How the CMM Is Fed
The indicators that form the maturity model are the results of the metric that determines the level of maturity of the activities related to the business objectives. They can also be called key maturity indicators (KMIs) to clearly identify these elements that evaluate specifically the maturity from all other enterprise indicators.
A KMI is generally fed manually through a self-assessment by the owner of the activity itself. Then there are additional degrees of verification to certify the accuracy of the assessment. The first level is that of the central managers of the process where the activity is located. An additional level is implemented by onsite review visits through interviews and observation. The top level is an internal audit10 that carries out the verification to the utmost detail.
There are at least two further possibilities to include assessments in the maturity model. One is derived from corporate monitoring and reporting systems on process performance, and the other is derived from the catalog of business risk managed by the enterprise risk management (ERM)11 process. The first can be fully automated as it is managed by a data processing system, but the second depends on the tool used for risk management (i.e., semi-automatic).
The reporting systems and business process monitoring systems are rich in metrics used to numerically determine the performance of specific actions or elements. The main indicators are called key performance indicators (KPIs) and are generally produced automatically within dedicated and different systems. The method used to link a KPI to a KMI is to paraphrase the sentence that establishes the performance indicator in order to derive a control statement.
For example, in the quality process there could be a KPI that states “defect in the finished product...,” which could be associated with a KMI that states “the finished product...complies with customer requirements.” A conversion rule is used to map KPI states to KMI states. Probably refresh rates between KPIs and KMIs are not the same, but it is not a problem. Simply, at each iteration, the best information becomes available at that moment. Using robotic process automation (RPA) technology, the work is carried out with absolute accuracy, with low cost and in the desired time.
Business risk catalog reflects the set of fears around the enterprise being uncertain it will achieve its goals. Risk is assessed with different metrics (financial risk is generally numerical while operational risk is qualitative) and often with nonautomatic processes. The main results of the analysis are called key risk indicators (KRIs), and many are linked to specific activities. Transferring the risk assessment into a maturity assessment of a control (or vice versa) is always possible because it is a one-to-one correspondence.
For example, one risk included in business continuity could be a KRI that states “risk of non-effectiveness of fire prevention measures,” which could be associated with a KMI that states “fire prevention measures are aligned with business objectives.” The state of the KRI (figure 7) and its action plan are mapped to a state of the KMI according to a predetermined correspondence relationship during import in the maturity model. It will not, therefore, be a completely automatic import process if the data source is a simple spreadsheet, but the conversion and loading process will still be helped by special scripts.
The risk levels have a numerical value between zero and one. The risk level can be calculated either through a risk matrix or by simply multiplying the three factors assessed, namely maturity, impact and probability. The matrix allows maximum freedom in the transformation, but it is still a matrix that needs to be maintained over time. Multiplication is certainly the simplest solution but tends to increase the density of numerical values to near zero, requiring a nonlinear transformation to facilitate the graphic display of the points. The result is a uniform spacing of the risk labels on the surface of the graphs with an amplification of the values near zero and a compression toward the value one (figure 8).
Who Manages the CMM?
It is necessary to determine who in the enterprise should manage the maturity model. The obvious choice is to resort to an existing process by reviewing the responsibilities and relationships with other processes. In relation to the risk governance scheme12 (figure 9), the enterprise risk monitoring process is the most suitable choice. This process is already in contact with operations and business processes and collects operational risk factors. It also produces reporting that can be extended to meet the needs of other departments such as internal audit.
Source: ISACA®, CRISC™ Review Manual 6th Edition, USA, 2015
The enterprise risk monitoring process needs to be improved with regard to formal relations with all departments that implement controls in the enterprise. A careful division of tasks is necessary, with a focus on increasing the collaboration between the processes involved in internal controls (figure 10). Areas of collaboration between processes are, for example, common controls or with different focus or sharing of their own sub-processes with processes with greater skills or resources. Increasing collaboration brings out synergies, increases the quality of work and consequently also increases the confidence in the results of internal processes.
For example, consider activities of the quality or information security process that adhere to ISO standards and that have been identified as relevant controls in the CMM. Then, the audit of these controls can be done jointly by internal inspectors of the quality or information security process teams as experts on the subject and with the onsite supervision of the risk monitoring officer who guarantees independence of judgment. Using the composition of the audit team as described in ISO 1901113 as a comparison, there is reasonable justification for this way of proceeding.
Quality and information security process auditors are not true auditors if they act alone (they belong to the audited process), but they become so if their work is subject to independent check. They are hired as experts under the coordination of the risk monitoring officer (lead auditor). This guarantees the objectivity of the audit as it is independent of the processes and oriented only to the analysis and control activity. Therefore, it guarantees the verification work, supervising it through two audit techniques: the interview and observation. In this way, the entire inspection is certified and internal audit can include it in its audit register. This synergy increases the audit perimeter without dedicating additional staff.
THE APPROPRIATE CONTROL AREAS THAT CAN BE MANAGED IN THIS WAY MUST BE IDENTIFIED, LEAVING THE INTERVENTIONS THAT REQUIRE A DEDICATED AND SPECIALIZED EFFORT…TO THE INTERNAL AUDIT PROCESS.
The described method lightens the internal audit process; however, success depends on the enterprise’s organizational structure. This method requires the presence of control personnel, internal to the processes being verified, to collaborate with risk monitoring. In addition, the appropriate control areas that can be managed in this way must be identified, leaving the interventions that require a dedicated and specialized effort, such as accounting audits, to the internal audit process. With the right available resources, it is possible to cover all the events that help achieve business objectives.
Where to Act to Effectively Represent the Outcomes
After linking the operational world and the control world with the CMM, it is recommended to review the methods for presenting the results of the risk analysis. It is useful to take advantage of the added value of the additional information derived from the assessment of maturity. It should be noted that a natural parallelism is created between vulnerability and risk and inversely with maturity. The vulnerability of a control is a more intuitive metric in correlating the severity of the assessment with the operation that causes it rather than doing it through pure knowledge of the risk.
A scatter bubble chart can be used to effectively summarize the phenomena of risk together with the maturity of the controls.14 The chart should be symmetrical on the ordinates to distinguish internal factors from external ones because they reasonably have responsibility for taking charge of the problem on two different organizational levels. Internal factors are generally operational and can be treated locally with accountability and risk treatment; external factors are organizationally attributed to central management and, therefore, cannot be treated locally.
Each risk should have all its controls distributed on the chart, each with a double evaluation, the maturity reached by the control itself and the risk of not compensating for any weaknesses found. Reading the chart is quite simple because there are only two pieces of information that need to be identified, namely whether the risk should be treated and in what order it should be treated. It is not necessary to determine numerical value but only to identify the relative position in the chart and the color/shape of the bubble to get information on the severity of the risk.
The position provides the first answer. If the control or risk bubble is far from the admissibility level of the risk, it must always be treated until it is acceptable.15 The second answer is linked to the color of the bubble, which represents the progress of the risk treatment corrective action plan. Higher risk bubbles (distance) with indefinite risk action plans (color/shape) come first over risk with effective remediation plans. Further information is obtained by expanding the bubble and viewing the detail. Because the graphic is digital, the user can drill down and focus on the data. It is best to always work on an active chart and never on paper.
Corrective actions are the responsibility of the control owners. Therefore, having identified the risk to be treated, the risk manager’s activity will be to contact the control owner, identified by the type of control and the organizational diagram of the assessed entity, and agree on changes to the risk plan to fall within the permitted parameters. If an organizational model based on the CMM is adopted, the ERM process can focus only on the risk that requires a specific numerical assessment. All other risk can be derived automatically through a qualitative assessment based on the aggregation of operational vulnerabilities.
Conclusion
By converging all internal control processes with the CMM, favoring an effective collaboration policy between processes to eliminate any redundancies, and acting continuously to simplify the overall organization of the operational chain of command and implementation control, it is possible to increase the value of the control, both for the evident synergies and for the expected increased quality of information due to the continuous iterative correction inherent in the control process itself.
A single point of collection for all relevant controls for an enterprise is its real strength. The increased visibility of many different indicators and the ease of being able to use them in analysis or in the creation of alarms of failure to implement process objectives helps internal control by improving its capacity and effectiveness in implementing corrective actions. Having a lot of information is helpful, but having the ability to use it easily is the real advantage.
When considering time, since the maturity assessment will presumably follow the frequency of the fastest process, all processes will have the information they need always updated in time. Costs are expected to decrease due to the reduced redundancy between processes, which also allows for greater productivity. The quality of the information improves because risk is intrinsically linked to controls (through the CMM) and the controls are assessed directly by the control owners, who represent the best source of information.
A SINGLE POINT OF COLLECTION FOR ALL RELEVANT CONTROLS FOR AN ENTERPRISE IS ITS REAL STRENGTH.
Therefore, iterative processes, collaboration between internal processes, simplification at each organizational review and having a single point of convergence of internal control should also guarantee over time the effectiveness of the monitoring of the risk management model through the assessment of the maturity of controls and the internal audit activity extended to all risk factors identified and not just financial ones.
Endnotes
1 CMMI Institute, http://cmmiinstitute.com/
2 International Organization for Standardization (ISO), ISO Guide 73:2009 Risk management—Vocabulary, Switzerland, 2016, http://www.iso.org/standard/44651.html
3 International Organization for Standardization (ISO), ISO 31000 Risk Management, Switzerland, 2018, http://www.iso.org/iso-31000-risk-management.html
4 Sbriz, L.; “Enterprise Risk Monitoring Methodology, Part 1,” ISACA® Journal, vol. 2, 2019, congou.everwoodsite.com/archives
5 Sbriz, L.; “Enterprise Risk Monitoring Methodology, Part 2,” ISACA Journal, vol. 2, 2019, congou.everwoodsite.com/archives
6 Sbriz, L.; “Enterprise Risk Monitoring Methodology, Part 3,” ISACA Journal, vol. 6, 2019, congou.everwoodsite.com/archives
7 Sbriz, L.; “Enterprise Risk Monitoring Methodology, Part 4,” ISACA Journal, vol. 3, 2020, congou.everwoodsite.com/archives
8 Manifesto for Agile Software Development, “Principles Behind the Agile Manifesto,” http://agilemanifesto.org/principles.html
9 Symeonides, M.; “The Seven Guiding Principles of ITIL 4,” Axios Blog, 2 January 2020, http://info.axiossystems.com/blog/the-7-guiding-principles-of-itil-4-0
10 Institute of Internal Auditors North America, Standards and Guidance—International Professional Practices Framework (IPPF), USA, July 2015, http://na.theiia.org/standards-guidance/
11 Committee of Sponsoring Organizations of the Treadway Commission (COSO), Enterprise Risk Management—Integrated Framework: Executive Summary, USA, September 2004, http://egrove.olemiss.edu/cgi/viewcontent.cgi?article=1037&context=aicpa_assoc
12 ISACA®, CRISC Review Manual 6th Edition, 2015, http://congou.everwoodsite.com/resources
13 International Organization for Standardization (ISO), ISO 19011:2018, Guidelines for auditing management systems, Switzerland, 2018, http://www.iso.org/standard/70017.html
14 Op cit, Sbriz, 2020
15 Op cit ISO 31000 Risk Management
Luigi Sbriz, CRISC, CISM, CDPSE, ISO/IEC 27001 LA, ITIL v4, UNI 11697:2017 DPO
Has been the risk monitoring manager at a multinational automotive company for more than five years. Previously, he was responsible for information and communication operations and resources for the Asia-Pacific region (China, Japan and Malaysia), and before that was the worldwide information security officer for more than seven years. For internal risk monitoring, he developed an original methodology merging an operational risk analysis with a consequent risk assessment driven by the maturity level of the controls. He also designed a cybermonitoring tool and an integrated system for risk monitoring, maturity model and internal audit. Sbriz was a consultant for business intelligence systems for several years. He can be contacted at http://it.linkedin.com/in/luigisbriz or at http://sbriz.tel.