AMP Toolbox
Evaluate the ongoing policy
Introduction
Sources of unpredicted changes potentially affecting policies are multiple: gaps in scientific knowledge, changing environmental conditions, patterns of human use or even political priorities.
Evaluation and adjustments are key aspects of adaptive policies. Evaluation examines the outcome of measures against what was expected, and should be designed to ensure that the lessons learned are fed back into the decision-making process. Appropriate reporting of adaptive policy evaluation is thus important.
Adaptive policies may be designed to include automatic adjustments. Some of the inherent variability on socio-economic and ecologic conditions under which a policy must operate can be anticipated, and monitoring can help triggering important policy adjustments to keep the policy functioning well. Regarding how conditions can be anticipated or not, that is if the system can be well understood or not, adjustments can be fully automatic, semi-automatic or after review. Evaluation is required in these latter cases.
Some EU environmental policies, such as the MSFD or the WFD require periodic revisions (six-year cycles) of the full policy. This obviously contributes to the adaptive character of these policies but these periodic revisions require going through the complete policy cycle (Step 1. Setting the stage and so on) and are outside the scope of what is considered here as policy adjustments. However it is advisable to evaluate these policies regularly before the end of their cycles.
Key questions
Sources of unpredicted changes potentially affecting policies are multiple: gaps in scientific knowledge, changing environmental conditions, patterns of human use or even political priorities.
Evaluation and adjustments are key aspects of adaptive policies. Evaluation examines the outcome of measures against what was expected, and should be designed to ensure that the lessons learned are fed back into the decision-making process. Appropriate reporting of adaptive policy evaluation is thus important.
Adaptive policies may be designed to include automatic adjustments. Some of the inherent variability on socio-economic and ecologic conditions under which a policy must operate can be anticipated, and monitoring can help triggering important policy adjustments to keep the policy functioning well. Regarding how conditions can be anticipated or not, that is if the system can be well understood or not, adjustments can be fully automatic, semi-automatic or after review. Evaluation is required in these latter cases.
Some EU environmental policies, such as the MSFD or the WFD require periodic revisions (six-year cycles) of the full policy. This obviously contributes to the adaptive character of these policies but these periodic revisions require going through the complete policy cycle (Step 1. Setting the stage and so on) and are outside the scope of what is considered here as policy adjustments. However it is advisable to evaluate these policies regularly before the end of their cycles.
Key questions
- Who should make the evaluation?
- What aspects of the policy need to be evaluated, gaps in target achievement, other aspects?
- When is the evaluation of the policy or measures performed?
- How should the evaluation be? How should the evaluation be reported?
Key actions
- Mechanisms (who, what, when, how [including budget]) for evaluation and reporting schemes should be provided in the policy itself or, if appropriate, at the level of the measures composing the policy.
Tools & methods
The technical methodologies used for “appraisal ” (Steps 2 & 4) and evaluation are similar. Each one should identify and measure, where possible, both the direct and indirect benefits of the policy, programme or project. The main difference is that evaluation tends to be based on actual data, and appraisal on forecasts and projections. Tools usually required are those used for stakeholder involvement, policy evaluation and those for project management, aiming to support policy makers to regularly check the consistency and performance of policies.
Specific tools are:
- Analytic Hierarchy Process in ArcGIS
- DPSWR framework
- EcoPath
- Fisheries Library in R
- Habitat Priority Planner
- MarineMap
- MARXAN
- PANDA
- SimLab
- Stakeholder Meetings
- Stakeholder Workshops
- PERSEUS presentation materials
- Questionnaires
- Focused conversations
- Facilitation – online descriptions
- Conflict management, negotiation and consensus building
- Consensus workshop method
- Brainstorming
- SWOT (strengths, weaknesses, opportunities and threat)s analysis
- Communication templates and tools
- Social and economic assessment methods
- Quantitative stock assessment methods
- Asset/objective – impact/threat matrix
- Conceptual and qualitative modelling
- Community based or participatory monitoring and evaluation
- ASSETS
- AMBI
- AQUATOX