�MONITORIMI DHE VLERËSIMI PËR POLITIKAT E SHTETIT TË SË DREJTËS

Webinar 5

Sistemi i Informacionit të MEL - Mbledhja e të Dhënave

IMPLEMENTATION OF MEL SYSTEM – MEL INFORMATION SYSTEM (MIS)

• MIS serves to Collecting, Managing, Analysing, and Synthesizing Data to Reach Evaluative Conclusions;
• Consideration is now given to determining how this initial planning for data collection is put into operation while also addressing important technical concerns.
• MIS ensures that high quality data are first obtained (data collection), appropriately managed and stored (data management and analysis), and appropriately integrated (data synthesis) to support sound evaluative judgments and conclusions. ”

MIS for MEL

• a degree of integration with organizational MIS functions is needed, this may only relate to monitoring functions. Evaluation functions and their products, such as lessons learned and evaluative conclusions, are often poorly integrated.”
• “An important principle in designing approaches to handling data associated with monitoring and evaluation is to commence planning and implementation at an early stage.”
• Concerns with validity and reliability are particularly prominent in discussions of the quality of monitoring and evaluation data”

MIS for MEL

• “The notion of validity is connected with that of measurement and is particularly associated with quantitative methods. ”
• “Qualitative researchers may substitute the related term credibility instead of validity, reflecting their concern with the nature of engagement between the evaluator and the subject and how data are interpreted.
• Data reliability refers to whether the data collected are consistent over time and the results are repeatable.
• Reliability is considered as a precondition for data validity (McDavid et al., 2013).”

MIS for MEL

Data Collection Plan may include the following items:

​

• Fully developed data collection tools (e.g., survey instrument)
• Sampling approaches adopted for the different types of data collection tools, including sampling frames and sampling method;
• Protocols for managing any risks associated with data collection;
• Ethical approval for implementation of evaluation activities where this is required and has been obtained”;

Data Collection Phase

Planing and Designing Data Collection Tools

• Primary Data – data that M&E practitioners collect themselves for the process of monitoring and evaluation using instruments such as interviews, focus groups, surveys or stories and discussion;
• Secondary – pre-existing data such as census, survey data, or administrative data
• When choosing sources of data, it is helpful to start with a desk review to better assess what type of data is needed for each indicator of M&E plan;

Type of measurements

• Nominal scales consist of assigning unranked categories that represent more of quality than quantity. Any values that may be assigned to categories only represent a descriptive category (they have no value in terms of magnitude). A nominal scale is used for classification/grouping purposes;

​

• Ordinal variables: allow the measured items to be ranked in terms of the order,

​

• Interval variables: Interval variables provide more flexibility in terms of measurement as they not only allow measured items to be ranked but also help quantify the sizes of differences between them;

​

• Ratio variables: Ratio variables are measured by scale;

Sampling

• When planning for data collection and thinking of the type of data that will be collected, it is important to assess the target audience (or population) and a representative sample and sampling strategy ;
• Sampling applies to both qualitative and quantitative monitoring/evaluation methods.
• random sampling (also referred to as probability sampling) is often applied when primarily quantitative data collection tools are used for monitoring/evaluation purposes;
• non-random sampling (also referred to as non-probability or purposeful sampling) tends to be applied to monitoring/evaluation that relies upon qualitative data;
• Proper sampling can reduce the chances of introducing bias in the data - Bias is any process at any stage in the design, planning, implementation, analysis and reporting of data produces results or conclusions that differ systematically from the truth

Random Sample Size

• Depends on the confidence level of the data – how accurate we expect the results to be against the true values (or the population value);
• A 95% level of confidence means that there are 5% chance that results will be not accurate, if we were to repeat the same measurements 100 times, 95 times our results would be accurate and 5 times would be wrong
• Sampling error/margin of error is the estimate of error that arises when data is gathered on a sample rather than on the entire population;
• https://www.surveymonkey.com/mp/sample-size-calculator/

Non-Random Sampling

• For non-random/purposeful sampling, an indication of whether an adequate sample has been reached or not is called data saturation;
• The following questions can help determine how many people to include in the sample, achieving both data saturation and credibility:
• Should all population segments be included in the sample?
• Should people with diverse perspectives be included in the sample?
• Should the findings be triangulated?

Sampling Methods and their application

Tools for Data collection for monitoring

• Surveys

​

​

• In-depth interviews

​

​

• Focus Groups

Other tools for qualitative data

• A case study is a qualitative data collection method that is used to examine real-life situations and if the findings of the case can illustrate aspects of the intervention being monitored and/or evaluated;
• Observation is a research technique that M&E practitioners can use to better understand participants’ behaviour and the physical setting in which a project, program or policy is being implemented;
• Other methods include:
• Brainstorming
• Role Play
• SWOT Analyses
• Photographs and Videos

Tools and methods for evaluation

• Impact evaluation aims to identify if there is evidence of intervention impact;

​

• This is done by evaluating the situation before (without) and after(with) the intervention;

​

• Experimental Method

​

​

​

• Quasi-Experimental Method

FALEMINDERIT PËR VËMENDJEN!

Këto webinare janë pjesë përbëse e Moduleve të Integrurara për MEL të Politikave të Shtetit të së Drejtës, përgatitur nga Lëvizja Europiane në Shqipëri në kuadër të projektit “Ndërtimi i Partneritetit mbi Çështjet Themelore: Fuqizimi i OSHC-ve për procesin e anëtarësimit në BE” financuar nga Bashkimi Evropian. �