A. INTRODUCTION
SDS Consulting is experienced in capturing the impact of programme activities during and after implementation through programme evaluations. We work with clients to use the information gathered to improve decision-making, organizational learning, accountability, impact, and revising programme implementation.
B. WHAT IS A PROGRAMME EVALUATION?
A Programme evaluation is a systematic and objective assessment of an ongoing or completed programme. Evaluations are designed to guide implementers and enable evidence-informed decision making on the ongoing or an ending programme and in addition, serves as an accountability tool for donors. Evaluations can be done midway of a programme implementation (mid-term evaluations) and at the end of a programme implementation process (end-term evaluations).
1. Mid-Term Evaluations – Midterm evaluations (MTEs) aim to assess the continued relevance of an intervention and the progress made towards achieving its planned objectives. They provide an opportunity to make modifications to ensure the achievement of these objectives within the lifetime of the programme. In addition, MTEs provide an opportunity to ascertain the intervention is still coherent with the client’s strategic objectives; is relevant and useful to the key stakeholders and is being conducted in an efficient manner.
2. End-Term Evaluations – Following completion of the programme, end-term evaluations examine its immediate outcomes or long-term impact and summarize its overall performance against the client’s strategic objectives, programme outcomes and objectives. End-term evaluations also provide important findings and recommendations which point out the positive and the negative aspects of the activities conducted.
C. THE OECD/DAC CRITERIA
SDS Consulting uses the OECD/DAC criteria which was established by the Development Assistance Committee (DAC) from the Organization for Economic Cooperation and Development (OECD). This criterion has shifted the focus of programme evaluations away from solely assessing program outputs or use of funds to assess impact of an intervention to the key stakeholders. SDS is aware that the criteria is based on six general principles : –
• All programme implementors should have an evaluation policy.
• Evaluations should be impartial and independent.
• Evaluation results should be widely disseminated.
• Evaluation should be used; therefore, feedback to decision-makers is essential.
• Donor and recipient agencies should be partners/cooperate with the evaluation to strengthen recipient agencies and reduce administrative burden.
• Evaluation should be part of the aid planning from the start.
We use the five criteria to evaluate programme interventions – relevance, effectiveness, efficiency, impact, and sustainability. Below is a table that clearly defines the criteria in detail .
1. Relevance – The extent to which the objectives of the intervention are consistent with beneficiary requirements, country needs, global priorities and partner and donor policies.
Typical Evaluation Questions
• Was a needs analysis carried out at the beginning of the programme reflecting the various needs of different stakeholders? Are these needs still relevant?
• Have the stakeholders taken ownership of the programme concept and approach since the design phase?
• How does the programme align with and support the Clients strategies?
• How well does the programme complement and link to activities of other donors at local level?
2. Effectiveness – The extent to which the programme’s immediate objectives were achieved, or are expected to be achieved, considering their relative importance.
Typical Evaluation Questions
• Is the programme making sufficient progress towards its planned objectives?
• Will the programme be likely to achieve its planned objectives upon completion?
• Have the quantity and quality of the outputs produced so far been satisfactory?
• Do the benefits accrue equally to men and women?
• How have stakeholders been involved in programme implementation?
• How effective has the programme been in establishing national ownership?
• Is programme management and implementation participatory and is this participation contributing towards achievement of the programme objectives?
• Has the programme been appropriately responsive to the needs of the beneficiaries and changing partner priorities?
• Has the programme approach produced demonstrated successes? In which areas (geographic, sectoral, issue) does the programme have the greatest achievements?
• What have been the constraining factors and why? How can they be overcome?
3. Efficiency – A measure of how economically resources/inputs (funds, expertise, time, etc.) are converted into results.
Typical Evaluation Questions
• Have resources (funds, human resources, time, expertise, etc.) been allocated strategically to achieve outcomes? Have resources been used efficiently?
• Have activities supporting the strategy been cost-effective? In general, do the results achieved justify the costs?
• Could the same results be attained with fewer resources?
• Have programme funds and activities been delivered in a timely manner?
4. Impact – The strategic orientation of the programme towards making a significant contribution to broader, long-term, sustainable development changes.
Typical Evaluation Questions
• Can observed changes (in attitudes, capacities, institutions, etc.) be causally linked to the programme’s interventions?
• In how far is the programme making a significant contribution to broader and longer-term development impact? Or how likely is it that it will eventually make one?
• Is the programme strategy and programme management steering towards impact?
• Can any unintended or unexpected positive or negative effects be observed because of the programme’s interventions? If so, how has the programme strategy been adjusted?
• Have positive effects been integrated into the programme strategy? Has the strategy been adjusted to minimize negative effects? Should there be a second phase of the programme to consolidate achievements?
5. Sustainability – The likelihood that the results of the programme are durable and can be maintained or even scaled up and replicated by programme partners after major assistance has been completed.
Typical Evaluation Questions
• How effective and realistic is the exit strategy of the programme? Is the programme gradually being handed over to the national partners?
• Once external funding ends will national institutions and implementing partners be likely to continue the programme or carry forward its results?
• Are national partners willing and committed to continue with the programme?
• How effectively has the programme built national ownership? Are national partners able to continue with the programme? How effectively has the programme built the necessary capacity of people and institutions?
• Has the programme successfully built or strengthened an enabling environment (laws, policies, people’s attitudes, etc.)?
• Are the programme results, achievements and benefits likely to be durable?
• What would support their replication and scaling up?
Compiled by,
Wanjugu Mathenge,
Project Associate,
SDS Consulting.
