1 of 22

����������������COORDINATION Department of Environmental Sciences and Engineering, New University of Lisbon PORTUGAL��PARTICIPANTS�-- Department of Environmental Studies, University of the Aegean GREECE�-- Institute for the Protection and the Security of Citizens. EC J.R.C. – Ispra ITALY�-- The Macaulay Institute. University of Aberdeen UNITED KINGDOM�-- Economic and Social Institute. Free University of Amsterdam THE NETHERLANDS�-- Departament of Human Geography. University of Seville SPAIN��END-USERS�1) Institute of Water – INAG PORTUGAL�2) NA.ET.KY. Regional Development Agency of Cyclades GREECE�3) General Secretary for Water. Regional Goverment of Andalucia SPAIN�������

ADVISOR Contract EVK1-CT-2000-00074

ENERGY, ENVIRONMENT AND SUSTAINABLE DEVELOPMENT RTD PROGRAMME

 

2 of 22

The project aimed to produce a set of guidelines to river basin authorities and related EU agencies for the execution of integrated evaluation of projects.

  • IA links scientific processes to the social and institutional context of decision-making.

  • IA objects are characterised by uncertainties, values in dispute, high stakes, conflicts and frecuently urgent decisions.

  • IA involves a broad range of scientific and lay practioners who are connected to the users of the assessment product.

3 of 22

4 of 22

At the first stage, the project was built on the ex-post analysis of the strengths and weaknesses of river basin project evaluation processes in the EU.

The 5 case studies at this stage were:

ADVISOR Contract EVK1-CT-2000-00074

ENERGY, ENVIRONMENT AND SUSTAINABLE DEVELOPMENT RTD PROGRAMME

 

5 of 22

6 of 22

Evaluation Practices: concluding remarks (1)

 

Characteristics of the decision making process (1)

  1. Accuracy: all studies report high scientific uncertainties and inconclusive nature of scientific evidence.
  2. Completeness: all case studies seem to be bound by great omissions of information.
  3. Sources of information: very little if any info coming from social research.
  4. Not enough verification and colleague consensus about sources. Reliability of information is not consensual.
  5. Low degree of acceptance by the non-scientific communities.

7 of 22

Evaluation Practices: concluding remarks (2)

 

Characteristics of the decision making process (2)

f) Assessment is not really considered as an independent , even less previous, stage in the design of the projects.

g) Evaluation constitutes a basically informal and iterative process, sometimes diffuse, throughout the entire project life, even throughout the post-decisional stage.

h) The implicit strategy, strongly incorporated into the project design, makes it difficult if not impossible to discuss global alternatives.

i) The complexity of ecological and social processes tends to be simplified by the starting diagnosis, which is basically incomplete.

8 of 22

Evaluation Practices: concluding remarks (3)

 

Implications for the development of evaluation theory

The assessment processes, both formal and informal, are dominated by the existing hegemony (dominant stakeholder nets and perspectives), resulting in a narrow and incomplete problem diagnosis and in proposals which ignore high levels of scientific uncertainty.

The complexity and inherently political nature of the decision making process, demands that the full range of values and beliefs is considered in a context where social and technical incommensurability is recognised.

This requires the development of assessment tools and methodologies that go beyond conventional scientific assessment, capturing different perspectives and accommodating uncertainty in both natural and social systems.

9 of 22

SOCIAL MULTI-CRITERIA EVALUATION�Water management alternatives for improving urban supply: Costa del Sol Occidental (Málaga, Spain)

10 of 22

Case study: Costa del Sol

11 of 22

12 of 22

13 of 22

SOCIAL MULTI-CRITERIA EVALUATION APPLIED IN THE COSTA DEL SOL CASE STUDY

INSTITUTIONAL ANALYSIS

  • Examination of normative context
  • Analysis of national and local media
  • Study of economic and political processes
  • Participant observation
  • Identification of actors

WORKING WITH ACTORS (I): INTERVIEWS AND QUESTIONNAIRES

  • Problem definition
  • Identification of alternatives
  • Identification of criteria
  • Suggestion of actors to add

MULTICRITERIA ANALYSIS

  • Impact matrix (alternatives/criteria): technical information and opinion polls
  • Equity matrix (alternatives/actors)
  • Dendrogram of coalitions

WORKING WITH ACTORS (II):

FOCUS GROUP

  • Presentation and discussion of results
  • Diagnosis corroboration
  • Revision of alternatives and criteria

INTERPRETATION

OF RESULTS

14 of 22

SOCIAL MULTI-CRITERIA EVALUATION APPLIED IN THE COSTA DEL SOL CASE STUDY

INSTITUTIONAL ANALYSIS

  • Examination of normative context
  • Analysis of national and local media

+ presence in local press

  • Study of economic and political processes
  • Participant observation:
    • Foro de Málaga
    • Agenda 21 for Málaga province
  • Identification of actors

ADVISOR in

La Opinión de Málaga

6 July, 2003

15 of 22

SOCIAL MULTI-CRITERIA EVALUATION APPLIED IN THE COSTA DEL SOL CASE STUDY

INSTITUTIONAL ANALYSIS

  • Examination of normative context
  • Analysis of national and local media
  • Study of economic and political processes
  • Participant observation:
    • Foro de Málaga
    • Agenda 21 for Málaga province
  • Identification of actors

WORKING WITH ACTORS (I): INTERVIEWS AND QUESTIONNAIRES

  • Problem definition
  • Identification of alternatives
  • Identification of criteria
  • Suggestion of actors to add

MAIN PROBLEMS

Ineffective river basin administration.

Lack of forward planning and land-use management.

Lack of coordination among the authorities responsible for water management and planning.

Weak information and participation mechanisms.

From shortage of water

to

resource mismanagement

Definition of

the problem

16 of 22

SOCIAL MULTI-CRITERIA EVALUATION APPLIED IN THE COSTA DEL SOL CASE STUDY

INSTITUTIONAL ANALYSIS

  • Examination of normative context
  • Analysis of national and local media
  • Study of economic and political processes
  • Participant observation:
    • Foro de Málaga
    • Agenda 21 for Málaga province
  • Identification of actors

WORKING WITH ACTORS (I): INTERVIEWS AND QUESTIONNAIRES

  • Problem definition
  • Identification of alternatives
  • Identification of criteria
  • Suggestion of actors to add

Alternatives:

Heightening of the La Concepción dam

Use of desalinated water

Reuse of waste water

Modernisation of irrigation systems

Rationalised use of groundwater

Improved efficiency and water savings in the urban water supply

Territorial policies to control urban development

Non-intervention: maintenance of status quo

Criteria:

Implementation costs

Operating costs

Effect on employment

Effect on economic activity

Impact on the ecological status of water systems

Impact on other ecosystems

Visual impact on the landscape

Degree of institutional difficulty

Degree of social acceptance

Equitable distribution of costs and benefits

Time required to fulfil the established objective

Identification of Alternatives

Actions aimed at increasing available resources /

Measures to improve demand management

Criteria

Economic

Environmental

Social

Institutional

17 of 22

SOCIAL MULTI-CRITERIA EVALUATION APPLIED IN THE COSTA DEL SOL CASE STUDY

INSTITUTIONAL ANALYSIS

  • Examination of normative context
  • Analysis of national and local media
  • Study of economic and political processes
  • Participant observation:
    • Foro de Málaga
    • Agenda 21 for Málaga province
  • Identification of actors

WORKING WITH ACTORS (I): INTERVIEWS AND QUESTIONNAIRES

  • Problem definition
  • Identification of alternatives
  • Identification of criteria
  • Suggestion of actors to add

Alternatives:

Heightening of the La Concepción dam

Use of desalinated water

Reuse of waste water

Modernisation of irrigation systems

Rationalised use of groundwater

Improved efficiency and water savings in the urban water supply

Territorial policies to control urban development

Non-intervention: maintenance of status quo

Criteria:

Implementation costs

Operating costs

Effect on employment

Effect on economic activity

Impact on the ecological status of water systems

Impact on other ecosystems

Visual impact on the landscape

Degree of institutional difficulty

Degree of social acceptance

Equitable distribution of costs and benefits

Time required to fulfil the established objective

Opinion Polls

Instituto de Estudios Sociales de Andalucía

Ecobarómetro de Andalucía

425 polls in:

City of Malaga

Costa del Sol

Rest of the province

18 of 22

  • Multi-criteria evaluation as an interface between scientific and technical issues and the social context to address specific societal issues.
  • NAIADE (Novel Approach to Imprecise Assessment and Decision Environments) has been chosen.
  • NAIADE deals with information affected by different degrees of uncertainty
  • Criteria values can be expressed in the form of: crisp, stochastic, fuzzy numbers and linguistic expressions
  • It generates a ranking of alternatives

SOCIAL MULTI-CRITERIA ANALYSIS

To explore alternatives with appropriate monetary and non-monetary criteria

19 of 22

SOCIAL MULTI-CRITERIA EVALUATION APPLIED IN THE COSTA DEL SOL CASE STUDY

MULTICRITERIA ANALYSIS

  • Impact matrix (alternatives/criteria): technical information and opinion polls
  • Equity matrix (alternatives/actors)
  • Dendrogram of coalitions

Institutional Analysis + Working with actors (I) =

Impact

Matrix

Equity

Matrix

In both cases support to increasing efficiency and reutilization of residual water.

In both cases lack of support to no intervention and hydraulic works.

Dendrogram of coalitions

Split between the administrations responsible for water management (Regional Water Department/ Confederación Hidrográfica del Sur / Costa del Sol public water supply and sewerage company).

20 of 22

SOCIAL MULTI-CRITERIA EVALUATION APPLIED IN THE COSTA DEL SOL CASE STUDY

WORKING WITH ACTORS (II):

FOCUS GROUP

  • Presentation and discussion of results
  • Diagnosis corroboration
  • Revision of alternatives and criteria

Institutional Analysis + Working with actors (I)

+ Multicriteria Analysis =

Main results:

  1. Stakeholders suggested the inclusion of new alternatives.
  2. Following an open debate, one of these alternatives (reforestation) figured among those receiving most support from the participants.

Focus Group

December 2003

21 of 22

The main results of the exercise can be synthesised as follows:

    • Improved the quality and effectiveness of the information interchange and the reflection process

    • it contributed to clarifying the nature of conflicts and creating the conditions necessary to construct a dialogue process

    • it opened the social debate on water resource allocation and mapped all the implications and interests involved

    • it stimulated the formulation of innovative diagnosis and alternatives

22 of 22

FINAL CONCLUSIONS

Deliberative decision processes will be successful only if it

really matters. Outcomes of deliberative processes should be

clearly embedded in real decision-making.

More than one tools and types of representation/participation

should be used together to overcome the limitations of any

given representation form.

Deliberative decision-making is not proposed as an alternative

but as a complement to legitimate representative democracy

structures.

Evaluation is closely dependent on the social and

institutional surroundings and on the balance of power

among actors, habitually affected by an increasing

polarization.

Still, reciprocally deliberative processes can build up an

Impetus for changes in the broader social context.