END OF PROGRAM EVALUATION OF THE AGRA SOIL HEALTH PROGRAM

TERMS OF REFERENCE FOR THE END OF PROGRAM EVALUATION OF THE AGRA SOIL HEALTH PROGRAMThe Alliance for a Green Revolution in Africa (AGRA, www.agra.org) is a not-for-profit organization working with African governments, other donors, NGOs, the private sector, and African farmers to significantly and sustainably improve the productivity and incomes of resource poor smallholder farmers in Africa. AGRA aims to ensure that smallholders have what they need to succeed: good seeds and healthy soils; access to markets, information, financing, storage and transport; and policies that provide them with comprehensive support. Through developing Africa's high-potential breadbasket areas, while also boosting farm productivity across more challenging environments, AGRA works to transform smallholder agriculture into a highly productive, efficient, sustainable and competitive system, while protecting the environment.AGRA’s vision is of a food secure and prosperous Africa achieved through rapid, sustainable agricultural growth based on smallholder farmers who are primarily engaged in the production of staple food crops in Africa. The main goals of AGRA by 2020 are to: i) Reduce food insecurity by 50 percent in at least 20 countries ii) Double the incomes of 20 million smallholder families and iii) Put at least 15 countries on track for attaining and sustaining a uniquely African Green Revolution.AGRA’s strategy operates across four programmatic areas. The four programmatic areas which work in an integrated manner across all three portfolios are the: Soil Health Program (SHP), Program for Africa’s Seed Systems (PASS), Market Access and Policy and Partnerships. Other areas such as water, farmer based organizations, access to finance, extension, and gender have been incorporated into AGRA’s implementation strategy.The SHP is one of the four core Programs of AGRA, and was established and largely funded by the Bill and Melinda Gates Foundation (BMGF). The mission of the SHP is to increase incomes, improve food security and reduce household poverty by promoting the use and adoption of integrated soil fertility management (ISFM) practices among smallholder farmers and creating an enabling environment for farmers to adopt the practices in an efficient, equitable and sustainable manner across sub Saharan Africa.Established in August 2008, the SHP had the following medium-term objectives that were expected to be achieved by 2015: (i) To create physical and financial access to appropriate fertilizers for about 4.1m smallholder farmers in an efficient, equitable and sustainable manner; (ii) To improve access to locally appropriate ISFM knowledge, agronomic practices and technology packages, for around 4.1 m smallholder farmers in an efficient, equitable and sustainable manner; (iii) To influence a national policy environment for investment in fertilizer and ISFM.SHP is implemented through four thematic sub-programs that focus on each of the program objectives highlighted above. These include: (i) ISFM Technology scale-out; (ii) Extension Support Function (iii) Fertilizer supply and policy, and (iv) Training and EducationDuring the period September 2010 and June 2011, the SHP underwent a ‘Strategy Refresh’ exercise. The process entailed a comprehensive review of the SHP overall strategy and business plan. The aim was to assess whether, the program targets, available resources and the program’s underlying assumptions were still valid, given the internal and external changes soon after the program inception. The key output from the strategy refresh was a “Four-year Medium Term Strategy and Plan (MTP; 2011-2014), which was more focused and had realistic targets as opposed to the initial business plan. The revised strategy prioritized and focused SHP investments to six countries, notably; Ghana, Mali, Mozambique, Tanzania, Nigeria and Ethiopia (P1+2 countries), and specifically in the Breadbasket regions within P1 countries. The Strategy also prioritized integrated interventions targeting barriers across the entire agricultural value-chain, including access to inputs via innovative financing mechanisms.Over a period of five years, the program has invested, to-date in 126 grantee projects, a portfolio amounting to over $119 million in 13 countries. By February, 2014, this constituted 96% of the $124m available for SHP grant making. The grants focus on projects that increase the knowledge and enhance the use of appropriate agronomic practices, to improve the quality of soil and sustain smallholder food production in Sub-Saharan Africa. By end of February, 2014, about 13 grantee projects had ended; 93 were ongoing, while 24 were in the approval process.As noted above, the first phase of the SHP will come to an end on November, 2015. Grant-making is expected to end by November, 2014.In view of the above background, AGRA intends to hire the services of a competent firm (‘the consultant’) to undertake an end-of-program evaluation of the AGRA Soil Health Program.2.0 Objectives of the end-of-program evaluationThe main objective of this end-of-program evaluation is to assess the overall program performance in relation to its objectives, and to provide input into the new strategic directions or implementation designs/strategies of another phase. The evaluation will assess what has worked, what did not work so well, key lessons learned and what should be emphasized or adjusted, in the second phase. The evaluation will essentially cover the followings three components of the Soil Health investment portfolio:(i) Programmatic impacts – what outputs have been delivered, what changes (outcomes, both intended and unintended) have occurred among the beneficiaries, relevance of program interventions, return on investments and the sustainability of the achievement to-date;(ii) Program delivery mechanisms – the approach adopted by the program to achieve the results highlighted above, e.g., models/best practices employed by the program; e.g. Agro-dealer network; “going beyond demos”; partnerships developed (e.g., Country Soil Health Consortia); fertilizer business; legume seed production; micro-dosing technology, cropping systems e.tc. How successful these models have been, lessons learned and whether these results are likely to be sustainable.(iii) Program management for results- This would look at the internal program management processes to assess their adequacy, efficiency and effectiveness. These will include the program structure; staffing; grant-making processes; quality of grants funded; level of integration and alignment with other AGRA programs; and how the program has utilized the expertize of the Soil Health Technical Advisory Committee.Specifically, the end of program evaluation shall:(i) Determine the relevance or appropriateness of the Program theory of change, design, strategies, management structure and delivery mechanisms and determine whether or not they are adequate to realize the intended program objectives;(ii) Critically assess to what extent, if any, the program achieved its intended objectives; outputs, and whether these are beginning to yield into outcomes, both intended and unintended;(iii) Ascertain to what extent the program was effective and efficient in achieving its intended objectives;(iv) Assess whether the program has been cost effective in achieving the desired outcomes and the likelihood that those results will be sustained over the medium to long term;(v) Identify to what extent SHP integrates and aligns with other AGRA programs, the quality of partnerships built, if any, with national and international researchers; soil scientists and other stakeholders in supporting program delivery and whether these are sustainable over time;(vi) Assess the efficacy of the different models utilized by the program, for instance, in scaling-up ISFM practices, financing mechanisms for farmers to access inputs (particularly improved legume seeds and fertilizers), and the produce marketing support initiatives;(vii) Identify challenges and document lessons learned and best practices that will inform future strategic program decisions. The evaluation will provide recommendations on practical strategic actions that need to be taken to improve future implementation and ensure sustainability of outcomes and institutionalization of key lessons learnt. Suggestions on how to build stronger governance systems, gender integration, and how future program initiatives should feed into ongoing and planned national-level agricultural strategies to enhance synergy, complementarity and value addition will be expected.The SHP end of program evaluation will assess the program performance since its inception in August, 2008 to August, 2014. Given that the program has over time attracted a large number of grantees, a statistically representative sample of SHP grants in the target countries will be selected for the assessment. The evaluation shall entail an in-depth review of the selected grants to determine whether SHP has made progress towards achieving its stated goal and objectives.Thematically, the end of program evaluation will focus on the following areas:3.1 Quality and relevance of program design· Assess major changes in program context, threats and opportunities which may have occurred since the inception of the SHP, and which may have had implications on the program performance.· Review program risks and test the underlying hypotheses and assess how these were managed.· Assess program strategies and how they responds to emerging risks and national priorities· Assess the extent to which program outputs, outcomes and objectives were achieved at all levels;· Positive and negative, direct and indirect, planned and unplanned results of the program;· Effectiveness of program, sub-program and project activities;· Assess the strategic importance of the program achievements and how it interfaces with the next phase of the program tenure;· Document challenges, lessons learned and best practices in program, sub-program and project implementation3.3 Efficiency in resource utilization· Assess the use of resources against budgets;· Conduct a value - for - money analysis of program results to assess the quality of results/deliverables against the cost of achieving these results in comparable contexts;· Assess potential and options for future sustainability, scalability and replicability of SHP achievements.· Assess management issues such as the overall organizational structure of SHP, staff configuration, execution modalities, and quality of grant making and management approach, arrangements for implementing grants, capacity gaps, utilization of the expertize of the SHP Technical Advisory Committee, use of technical assistance / consultants and how they affected program results.· What portfolio investments e.g. ISFM scale-out, combination of one or more technologies etc., offer lessons to learn? How can the model be sharpened?·What has been learned and what can be built upon in the next phase of the program?· How did the lessons learned during the Program Strategy Refresh help to improve program investments and management?While a detailed survey design and methodology to achieve the objectives of the end of program evaluation shall be defined by the consultant, and approved by AGRA, the following is broadly suggested as a guide. The Evaluation methodology will be a combination of qualitative and quantitative approaches for data collection and analysis. Essentially, this will entail: Desk review of key program documents and program implementation progress reports from grantees, program officers and other partners. Such documents shall include but not limited to the background program documents; grant proposals; progress reports; projects’ rapid assessment reports, and other key documents related to the program. Besides the review of relevant literature related to the assignment, the consultants shall also undertake field data collection using structured interviews with key informants, notably; farmer organizations, agro-dealer networks, project administrative and technical personnel, program partners/and donors, AGRA program staff and management; and targeted Focus Group Discussions (FGDs), as appropriate.To enrich the qualitative data obtained through the above qualitative methods, the Consultants shall also administer questionnaires to a representative sample size of farmer households, agro-dealers, and other relevant stakeholders. To enhance likelihood of achieving the evaluation objectives stated above, the selection of grants shall be done from a sampling frame of SHP-funded projects which shall be clustered into: projects that have ended; projects half-way into implementation; and projects that are in the early stages of implementation. Furthermore, the consultants shall work closely with the SHP and the M&E staff to ensure effective coverage of the projects, countries, ecological zones and cropping systems. Each of the components of the evaluation highlighted above shall adopt its appropriate methodology that best offers higher chances to obtain sufficient data to address its research questions. It is expected that the consultants will develop and institute an in-field quality control mechanisms to ensure robust and valid data is collected and analyzed for reporting.The consultant (s) shall be expected to prepare and submit to the client a set of key reports in the course of undertaking this assignment. These reports shall be reviewed and accepted by the client before payment is approved. The following have been identified as key reports to be submitted:An Inception Report – This shall be prepared and submitted within two weeks after the signing of the contract. The consultant will prepare this after reviewing key technical documents and after discussion with the client. The inception report shall focus on: the understanding of the Terms of Reference and scope, the relevant methodology to be adopted, the evaluation design and key questions, and, the work-plan for the assignment. The inception report shall be reviewed by both the M&E and the program’s team within 5 days after submission, and shall have to be approved before proceeding to the next phase.5.2 Progress brief - While there is no formal progress report required during the assignment implementation, between inception and Draft report submission, the consultant (s) shall be expected to regularly (bi-weekly) share with the client, key emerging issues and trends to avoid surprises or misconceptions by either party.5.3 Draft Report – This shall be prepared and submitted to the client towards the end of the assignment. The draft report shall require feedback in form of comments, questions and inputs from the client. In addition, the consultant (s) will be required to present the Draft Report to a wider AGRA audience for validation.5.4 Final Report – This shall be no more than 40 pages (excluding annexes), and submitted to the client on, or before the expiry of the assignment contract. Any valid extension may be mutually agreed between the Consultant (s) and the Client, provided it carries no extra cost to the latter. The following will also be expected from the Consultant:· A master copy of the final report suitable for reproduction, and four copies, in full-color and bound, as well as soft copies.· Submission of the final report, after incorporating the comments/inputs on the presented draft report. The final report shall include actionable recommendations· All data-sets (in SPSS/Stata) and questionnaires used during the assignment shall be a property of AGRA, and shall be the responsibility of the consultant to carefully deliver them to AGRA.To ensure consistency, enhance utilization and avoid any surprises, the firm (consultant) shall in the course of the evaluation, be required to regularly provide an update on the evaluation progress. Reporting shall be directly to the M&E Director, and in close working relationship with the SHP Directors and Program Officers.The evaluation is expected to be completed within a 90-day period, effective from the date a contract is signed.Access to project documents, progress and evaluation reports, program databases, contacts for grantees, financial records and other program related files, depending on the consultant’s requirements will be availed. Access to the Program Directors and AGRA staff shall be guaranteed upon request.9.0 Criteria for selection of ConsultantThe consultant will be selected through an open and competitive process and will be based on their proven experience, qualifications and ability to deliver a high quality product in a timely and efficient manner. Particular qualifications and experience of the lead consultant team members include:· PhD in Agricultural Sciences (Soils, Agronomy, Natural resources etc.), Social Sciences, Economics or related field, and with at least 10 years of relevant experience (leader);· 10 years’ experience in similar survey fieldwork (data collection, validation, entry and analysis) in sub-Saharan Africa;· Recent experience with results-based management evaluation methodologies;· Experience in leading teams in field (training, field logistics, human relations, teamwork)· Demonstrable ability and experience in working with communities and the team’s capacity to undertake the study in different countries;· Excellent writing skills, with publication record in one discipline related to the assignment· Demonstrable analytical skills;· Excellent knowledge of English. A working knowledge in French will be an added advantage.A detailed elaboration of issues to be addressed/covered;A description of the evaluation plan (see annex 1) including details of the proposed methodology, sampling, study design; analysis and reporting, and milestones for the evaluation and a timetable of activities.Detailed budgetDescription of the pay schedule for the reviewPast performance summaries (at least three brief descriptions of past or current contracting mechanisms for assignments similar in size, scope and complexity to this tender) and list of references that demonstrate performance in conducting similar evaluationsAt least one previous relevant report and list of previous reportsCVs conforming to the qualifications listed above for persons to manage and conduct the evaluation. Only CVs strictly relevant to the assignment shall be accepted.Supporting documents includingmandatory institutional documents such as incorporation papers and most recent financial statements.Interested parties are requested to send their detailed technical & financial proposals (ensure the two are separate documents) to: mande@agra.org by April 21st2014.Detailed TOR for the evaluation is available on the AGRA website and additional requests for background information by interested parties can be sent to the same e-mail address –mande@agra.org.