EVALUATION CHALLENGES
In the process of conducting a programme evaluation, SDS Consulting usually looks out for constraints that exist every evaluation environment. When it comes to performing evaluations in the real world of research, mistakes are made, details go awry, and challenges can seem insurmountable. So, in this two-part series, we will discuss six key challenges in evaluation and solutions to those challenges (in Part Two), which we hope will guide you toward more effective evaluation.
Challenge 1: Poor Planning
Failing to plan is planning to fail, and the same is true when conducting an evaluation. Poor planning can lead to underestimating the time required to conduct an evaluation, inadequate resources including personnel and money, and a lack of a clear path to attain intended outcomes. Poor planning can also result in evaluation process gaps which negatively impact the quality assurance of the evaluation.
Challenge 2: Ineffective Approaches
An evaluation must design, ab initio, correct approaches that will lead to desired evaluation outcomes including, using correct data collection methods, understanding various data types and sources to use, understanding evaluation scope and objectives, and putting in place the right evaluation team with requisite skills.
Challenge 3: Key Evaluation Questions
It is important to have the right questions to ask that utilize the OECD-DAC criteria of relevance, efficiency, effectiveness, and impact of the programme. Asking the wrong questions can affect results expected. Questions that are unclear, that are not specific to the programme, that use too much jargon, that do not consider the audience, that are biased in any way, and that do not have a clear and understandable method for participants to respond will affect the evaluation process.
Challenge 4: Data Collected
Data collection enhances the quality of an evaluation. Since data is mostly dependent on the key questions asked, wrong questions in the data collection tools lead to wrong data. In addition, raw data requires to be properly cleaned, categorized, and organized into a format that is palatable, understandable, and usable for the evaluator to write an evaluation report.
Challenge 5: Participation of key stakeholders
There are various instances that may affect the effective participation of key stakeholders in the evaluation. Participants may have concerns about the confidentiality of information mostly if they are revealing personal information. Participants may perceive anxiety due to the location of the interview, uneasiness, or lack of rapport with the researcher, or stressors in their daily life. Participants with low literacy levels can misunderstand questions in surveys or interviews. Participants may be unable to participate because of their tight schedules.
Challenge 6: COVID-19 pandemic
The pandemic has changed the way evaluations are conducted because of various challenges that COVID-19 has presented. To keep up with guidelines of social distancing and minimized travel, meetings and interviews with the stakeholders have to a great extent moved to online platforms. This has presented issues of internet connectivity with stakeholders unable to attend KIIs and FGDs because of poor internet connectivity. The pandemic has further isolated some categories of stakeholders because their participation in an evaluation is pegged on a one-on-one interaction with evaluators, which is majorly a challenge. The use of virtual and online platforms for meetings also mean that a lot of body language and facial communication could be missed during the interviews.
Compiled by
Wanjugu Mathenge
Project Associate
SDS Conaulting
