AMP Toolbox
Qualitative Risk Analysis (consequence X likelihood)
Note: description of this tool is adapted from the FAO –EAF tool box.
Steps
Overview
Risk assessment basically involves the calculation of the magnitude of potential consequences (levels of impacts) and the likelihood (levels of probability) of these consequences to occur.
Risk = Consequence x Likelihood; where: (i) Likelihood is the Probability of occurrence of an impact that affects the environment; and, (ii) Consequence is the Environmental impact if an event occurs.
The C × L matrix method therefore combines the scores from the qualitative or semi-quantitative ratings of consequence (levels of impact) and the likelihood (levels of probability) that a specific consequence will occur (not just any consequence) to generate a risk score and risk rating. Essentially, the higher the probability of a "worse" effect occurring, the greater the level of risk.
This C x L risk assessment process involves selecting the most appropriate combination of consequence and likelihood levels that fit the situation for a particular objective based upon the information available and the collective knowledge of the group (including stakeholders, academics, managers, industry, researchers and technical staff) involved in the assessment process.
Using the Risk Matrix below, if the assessment group concludes that the most appropriate combination for the assessment of the Risk of a particular objective is that it is possible that a major consequence could occur, this is a Major Consequence (3) and a Possible Likelihood (3). These two scores are multiplied to generate a High Risk (9) which is an unacceptable level of risk. Therefore increased management actions would be needed to achieve the objective.
.jpg)
Figure 1: Risk analysis (Likelihood X Consequence) matrix with different levels of consequences or impacts at the top and the levels of probability or frequency on the left. The different coloured cells in the middle of the diagram represent the different Risk score (in brackets) and the Risk categories (No-Risk in blue; Low-Risk in green; Moderate-Risk in orange and High-Risk in red). Additionally, the management response and the reporting requirements are addressed for each Risk category. More information on the above matrix can be found below under Source of Information.
Additionally, determining the acceptable level of impact is a very important part of the risk assessment (and management) process because it defines how the process operates. This should be changed to suit local circumstances, given that a level of impact may be acceptable in one circumstance but not in others. Moreover, the description of what level of impact is ascribed to what level of consequence can also vary. So, for example, the same level of impact could be considered a moderate consequence for one objective but a high consequence for another.
Tips
Facilitator:
An experienced facilitator is required to make this system work efficiently. The facilitator needs to understand the basis of risk assessment, how this method operates and must be aware of how the descriptions in each of the tables are defined to assist the group to make good decisions about the most appropriate C × L combinations.
Method:
It is important to have some level of quantitative information and/or a good level of qualitative information available to do the assessment, as well as to understand well the consequences in order to structure the different levels of impacts efficiently.
It is an adaptable approach, since it enables the adaptation of levels of impacts and the likelihood, as well as the acceptability associated with a specific objective in a specific situation. However, the higher the levels of categories are, the greater the difficulty in choosing the levels of consequences (unless a very high degree of knowledge exists) and the acceptable level of impact.
Ensure that the levels and descriptions for each table are sufficiently unambiguous – especially the maximum level of impact that is considered acceptable.
When defining levels of consequence, it is important not to use language that is associated with uncertainty, as this will cause confusion between the specification of consequence and likelihood. Consequence statements should be worded as propositions that can have a formal likelihood associated with them.
The combination of consequence and likelihood chosen should be based on the risk of something happening within a defined time period – not the risk of it happening at any point in the future. A convenient time frame to use is the timeframe of the management plan, which is usually around five years.
It is vital to ensure that when choosing the combination of consequence and likelihood that the selected likelihood score relates to the likelihood of a particular consequence level actually occurring, NOT just the likelihood of the activity/event/management occurring. This is an extremely common error to make.
It is NOT necessary to have full certainty about issues to rate risk, nor does uncertainty automatically generate a high risk. The level of uncertainty is only a component of the risk calculation process. Risk assessment is therefore making the MOST informed decision you can that includes uncertainty.
Not assessing the risk for an issue because there is a lack of information essentially means that the current actions are rated as being acceptable.
Participants:
It is essential to have a training session with all participants before they begin the formal assessment process if they are to participate effectively.
Ensure that participants actually read out loud the FULL descriptions of both consequence and likelihood together when they propose a suitable combination – not just the category levels – as it is common for people to unconsciously reinterpret the levels based on their biases towards what outcome they want.
The discussion process to assign risk levels needs to be undertaken using a language that is very familiar to participants. The process can be confusing enough without adding language inconsistencies. Therefore all the supporting material needs to be in the language that will be used.
A large discrepancy in scores between individuals often reflects that they are really assessing different issues, have different ideas of acceptability or have different knowledge bases. Ensure that participants are using the descriptions for the levels, and not creating their own interpretation of what the levels should be.
If a large group is participating in the risk assessment workshop, it can be more efficient to have the final risk score combinations chosen by a smaller "expert" panel (this panel can include non-technical people). The broader audience can provide their input during an open discussion phase and comment on the written outcomes. Detailed reporting of discussion can help with disputes over selected risk scores that may occur subsequent to the risk assessment workshop.
Pedigree
The usefulness of a risk assessment of marine activities depends on the method(s) used and the purpose of the results. Each of the methods comes with advantages and disadvantages. A number of factors influence the most appropriate method to be used in a given circumstance, including the type of activities, the study area, potential gaps in data and/or models and the uncertainty issue of risk assessment. Depending on the characteristics of the problem under review and the availability and form of data required, the analyst needs to decide upon the use of a qualitative or a quantitative approach. Although the bulk of the effort in developing methods of risk analysis has been addressed by quantitative methods since these are the most correct and practical ones, critical aspects of risk frequently require qualitative evaluation. Qualitative risk analysis may use “expert” opinion to estimate probability (or frequency) and consequence (or impacts) often through linguistic expressions. This subjective approach may be sufficient to assess the risk of a system, depending on the decisions to be made and available resources. Formal processes for eliciting expert opinion have been developed to provide consistency in qualitative information gathering (e.g. the Delphi technique). Concerning qualitative uncertainty estimates, one has to rely on subjective estimates of uncertainty.
Synergy
It is important to have previously well-defined objectives as well as the factors that influence the achievement of those objectives.
In addition, this analysis could be useful to launch a quantitative approach in those situations where a high risk or a priority has been identified.
Usage
Medium
Cost
Low - moderate. The costs are relatively low if there is a leader with specific skills on the issue and some experience in risk assessment who runs the process. The major costs are associated with the meeting costs. If all participants are local, then these costs can be very low.
Capacity
Moderate. The method has a moderate level of complexity and will usually require the workshop group to have some period of instruction before they can participate effectively. The main capacity required for this method is to have a leader who has sufficient knowledge of the specific issue or area (e.g. eutrophication, biodiversity, fisheries, etc.); and experience in risk assessment and specifically the application of this C × L method so that the group can be guided appropriately. It is likely that there will be many people within an area who are skilled at this type of risk assessment methodology but who may not have applied it to the specific issue or area in question. One option could be to use such a professional in combination with a scientist or expert in the issue or area in question to co-facilitate.
Background requirements
Low - moderate. One of the advantages of the qualitative risk assessments is that it can be used in situations where quantitative data are scarce or when only qualitative data are available. However, a minimal amount and quality of data may be available in order to provide a defensible specification of consequence and likelihood.
Participation
Moderate - high. Although the method can be applied by only one person, it is commonly applied by a group or in open workshops. The profile of the participants will change depending on the situation. In the case of highly technical issues the main input will usually be based on expert opinion. When more general objectives are assessed, a wider set of opinions will be included.
Time range
Short. The process can be completed quickly. All the risks associated with a particular objective can be calculated within a day or two during a single comprehensive workshop, given that the consequences are well-known and identified. Time should also be allowed to document the outcomes of the meeting and include summaries of the supporting information that was used to determine the risk scores.
Source of information
Steps
Step 1- Set the scene
Key Activity Scenario analysis (including risk assessment)
Purpose
To perform a risk analysis to assess the risk of not achieving the goal set previously, as well as to identify and prioritise those issues where some necessary management actions could be required. In other words, it is the practice of assessing the impact of uncertainty on achieving objectives, organising information and contributing to the decision-making process.
The risk assessment is a useful management tool which will:
Key Activity Scenario analysis (including risk assessment)
Purpose
To perform a risk analysis to assess the risk of not achieving the goal set previously, as well as to identify and prioritise those issues where some necessary management actions could be required. In other words, it is the practice of assessing the impact of uncertainty on achieving objectives, organising information and contributing to the decision-making process.
The risk assessment is a useful management tool which will:
- Highlight the greatest risks needed to support allocation decisions for limited resources
- Allow management agencies to ask "what if" questions regarding the consequences of various potential management actions
- Facilitate explicit identification of environmental values of concern
- Identify critical knowledge gaps, thereby helping to prioritise future research
Overview
Risk assessment basically involves the calculation of the magnitude of potential consequences (levels of impacts) and the likelihood (levels of probability) of these consequences to occur.
Risk = Consequence x Likelihood; where: (i) Likelihood is the Probability of occurrence of an impact that affects the environment; and, (ii) Consequence is the Environmental impact if an event occurs.
The C × L matrix method therefore combines the scores from the qualitative or semi-quantitative ratings of consequence (levels of impact) and the likelihood (levels of probability) that a specific consequence will occur (not just any consequence) to generate a risk score and risk rating. Essentially, the higher the probability of a "worse" effect occurring, the greater the level of risk.
This C x L risk assessment process involves selecting the most appropriate combination of consequence and likelihood levels that fit the situation for a particular objective based upon the information available and the collective knowledge of the group (including stakeholders, academics, managers, industry, researchers and technical staff) involved in the assessment process.
Using the Risk Matrix below, if the assessment group concludes that the most appropriate combination for the assessment of the Risk of a particular objective is that it is possible that a major consequence could occur, this is a Major Consequence (3) and a Possible Likelihood (3). These two scores are multiplied to generate a High Risk (9) which is an unacceptable level of risk. Therefore increased management actions would be needed to achieve the objective.
Figure 1: Risk analysis (Likelihood X Consequence) matrix with different levels of consequences or impacts at the top and the levels of probability or frequency on the left. The different coloured cells in the middle of the diagram represent the different Risk score (in brackets) and the Risk categories (No-Risk in blue; Low-Risk in green; Moderate-Risk in orange and High-Risk in red). Additionally, the management response and the reporting requirements are addressed for each Risk category. More information on the above matrix can be found below under Source of Information.
Additionally, determining the acceptable level of impact is a very important part of the risk assessment (and management) process because it defines how the process operates. This should be changed to suit local circumstances, given that a level of impact may be acceptable in one circumstance but not in others. Moreover, the description of what level of impact is ascribed to what level of consequence can also vary. So, for example, the same level of impact could be considered a moderate consequence for one objective but a high consequence for another.
Tips
Facilitator:
An experienced facilitator is required to make this system work efficiently. The facilitator needs to understand the basis of risk assessment, how this method operates and must be aware of how the descriptions in each of the tables are defined to assist the group to make good decisions about the most appropriate C × L combinations.
Method:
It is important to have some level of quantitative information and/or a good level of qualitative information available to do the assessment, as well as to understand well the consequences in order to structure the different levels of impacts efficiently.
It is an adaptable approach, since it enables the adaptation of levels of impacts and the likelihood, as well as the acceptability associated with a specific objective in a specific situation. However, the higher the levels of categories are, the greater the difficulty in choosing the levels of consequences (unless a very high degree of knowledge exists) and the acceptable level of impact.
Ensure that the levels and descriptions for each table are sufficiently unambiguous – especially the maximum level of impact that is considered acceptable.
When defining levels of consequence, it is important not to use language that is associated with uncertainty, as this will cause confusion between the specification of consequence and likelihood. Consequence statements should be worded as propositions that can have a formal likelihood associated with them.
The combination of consequence and likelihood chosen should be based on the risk of something happening within a defined time period – not the risk of it happening at any point in the future. A convenient time frame to use is the timeframe of the management plan, which is usually around five years.
It is vital to ensure that when choosing the combination of consequence and likelihood that the selected likelihood score relates to the likelihood of a particular consequence level actually occurring, NOT just the likelihood of the activity/event/management occurring. This is an extremely common error to make.
It is NOT necessary to have full certainty about issues to rate risk, nor does uncertainty automatically generate a high risk. The level of uncertainty is only a component of the risk calculation process. Risk assessment is therefore making the MOST informed decision you can that includes uncertainty.
Not assessing the risk for an issue because there is a lack of information essentially means that the current actions are rated as being acceptable.
Participants:
It is essential to have a training session with all participants before they begin the formal assessment process if they are to participate effectively.
Ensure that participants actually read out loud the FULL descriptions of both consequence and likelihood together when they propose a suitable combination – not just the category levels – as it is common for people to unconsciously reinterpret the levels based on their biases towards what outcome they want.
The discussion process to assign risk levels needs to be undertaken using a language that is very familiar to participants. The process can be confusing enough without adding language inconsistencies. Therefore all the supporting material needs to be in the language that will be used.
A large discrepancy in scores between individuals often reflects that they are really assessing different issues, have different ideas of acceptability or have different knowledge bases. Ensure that participants are using the descriptions for the levels, and not creating their own interpretation of what the levels should be.
If a large group is participating in the risk assessment workshop, it can be more efficient to have the final risk score combinations chosen by a smaller "expert" panel (this panel can include non-technical people). The broader audience can provide their input during an open discussion phase and comment on the written outcomes. Detailed reporting of discussion can help with disputes over selected risk scores that may occur subsequent to the risk assessment workshop.
Pedigree
The usefulness of a risk assessment of marine activities depends on the method(s) used and the purpose of the results. Each of the methods comes with advantages and disadvantages. A number of factors influence the most appropriate method to be used in a given circumstance, including the type of activities, the study area, potential gaps in data and/or models and the uncertainty issue of risk assessment. Depending on the characteristics of the problem under review and the availability and form of data required, the analyst needs to decide upon the use of a qualitative or a quantitative approach. Although the bulk of the effort in developing methods of risk analysis has been addressed by quantitative methods since these are the most correct and practical ones, critical aspects of risk frequently require qualitative evaluation. Qualitative risk analysis may use “expert” opinion to estimate probability (or frequency) and consequence (or impacts) often through linguistic expressions. This subjective approach may be sufficient to assess the risk of a system, depending on the decisions to be made and available resources. Formal processes for eliciting expert opinion have been developed to provide consistency in qualitative information gathering (e.g. the Delphi technique). Concerning qualitative uncertainty estimates, one has to rely on subjective estimates of uncertainty.
Synergy
It is important to have previously well-defined objectives as well as the factors that influence the achievement of those objectives.
In addition, this analysis could be useful to launch a quantitative approach in those situations where a high risk or a priority has been identified.
Usage
Medium
Cost
Low - moderate. The costs are relatively low if there is a leader with specific skills on the issue and some experience in risk assessment who runs the process. The major costs are associated with the meeting costs. If all participants are local, then these costs can be very low.
Capacity
Moderate. The method has a moderate level of complexity and will usually require the workshop group to have some period of instruction before they can participate effectively. The main capacity required for this method is to have a leader who has sufficient knowledge of the specific issue or area (e.g. eutrophication, biodiversity, fisheries, etc.); and experience in risk assessment and specifically the application of this C × L method so that the group can be guided appropriately. It is likely that there will be many people within an area who are skilled at this type of risk assessment methodology but who may not have applied it to the specific issue or area in question. One option could be to use such a professional in combination with a scientist or expert in the issue or area in question to co-facilitate.
Background requirements
Low - moderate. One of the advantages of the qualitative risk assessments is that it can be used in situations where quantitative data are scarce or when only qualitative data are available. However, a minimal amount and quality of data may be available in order to provide a defensible specification of consequence and likelihood.
Participation
Moderate - high. Although the method can be applied by only one person, it is commonly applied by a group or in open workshops. The profile of the participants will change depending on the situation. In the case of highly technical issues the main input will usually be based on expert opinion. When more general objectives are assessed, a wider set of opinions will be included.
Time range
Short. The process can be completed quickly. All the risks associated with a particular objective can be calculated within a day or two during a single comprehensive workshop, given that the consequences are well-known and identified. Time should also be allowed to document the outcomes of the meeting and include summaries of the supporting information that was used to determine the risk scores.
Source of information
- EAF Planning and Implementation Tools: Qualitative Risk Analysis (consequence X likelihood).
- Fletcher WJ 2005, ‘The application of qualitative risk assessment methodology to prioritize
issues for fisheries management’, ICES Journal of Marine Science 62: 1576−1587. - Gaylard S. (2009). A risk assessment of threats to water quality in Gulf St Vincent. Environment Protection Authority, Adelaide South Australia. ISBN 978-1-921125-90-X.
- VLAAMS INSTITUUT VOOR DE ZEE PLATFORM VOOR MARIEN ONDERZOEK. Environmental risk assessment of marine activities.
- Breen P., Robinson L.A., Rogers S.I., Knights A.M., et al., (2012), An environmental assessment of risk in achieving good environmental status to support regional prioritisation of Management in Europe. Marine Policy, 36, 1033-1043.