Evaluation of the IMF's Role in Poverty Reduction Strategy Papers and the Poverty Reduction and Growth Facility
David Goldsbrough, Isabelle Mateos y Lago, Martin Kaufman, Daouda Sembene, Tsidi Tsikata, Steve Mugerwa, Alex Segura-Ubiergo, and Jeff Chelsky
In 1999, the IMF and the World Bank adopted a new frame work for supporting economic reform in low-income member countries to achieve the objectives of poverty reduction and economic growth. The frame work consists of two key elements: country-authored Poverty Reduction Strategy Papers, drawing on broad-based consultations with key stake holder groups; and a vehicle for the provision of IMF concessional lending, the Poverty Reduction andGrowth Facility. This evaluation takes stock of progress to date and attempts to identify short comings that may require course corrections in the design and implementation of the initiative.
toggleEvaluation of the IMF's Role in Poverty Reduction Strategy Papers and the Poverty Reduction and Growth Facility
Summary of Major Findings, Lessons, and Recommendations
Chapter 1 Introduction: Origin and Objectives of the PRSP and the PRGF and Scope of the Evaluation
Chapter 2 Implementation of the PRS Approach in Areas Relevant to the IMF Mandate
Chapter 3 Joint Staff Assessments
Chapter 4 PRGF-Supported Program Design
Chapter 5 The IMF's Way of Doing Business
Chapter 6 Is the New Approach Making a Difference?
Annex 1 Survey of PRSP Stakeholders
Annex 2 Results from Survey of IMF Staff1
Annex 3 PRSP Sourcebook: Does It Provide Adequate Space for Policy Flexibility, and Is It Based on Strong Empirical Evidence?
Annex 4 Assessment of the Adaptation of the IMF's Internal Policy Process
Annex 5 Modalities of Participatory Processes: How Broad-Based Has Participation Been?
Annex 6 Qualitative Appraisal of JointStaff Assessments
Annex 7 Determinants of Targeted and Actual Fiscal Adjustment
Annex 8 Composition of Targeted Changes in Expenditures and External Financing in IMF-Supported Programs
Annex 9 Changes in Domestic Policy Processes and Institutions: Evidence from Case Studies
toggleStatement by the Managing DirectorIMF Staff ResponseIEO Comments on ManagementStaff ResponsesSumming Up by the Acting Chair
Statement by the Managing Director on the Independent Evaluation Office Report on the Evaluation of Poverty Reduction Strategy Papers and the Poverty Reduction and Growth Facility
Staff Response to the Independent Evaluation Office Report on the Evaluation of Poverty Reduction Strategy Papers and the Poverty Reduction and Growth Facility
Independent Evaluation Office Comments on Management/Staff Responses to the Evaluation of Poverty Reduction Strategy Papers and the Poverty Reduction and Growth Facility
The Acting Chair's Summing Up Evaluation Report of Poverty Reduction Strategy Papers and the Poverty Reduction and Growth Facility by the Independent Evaluation Office
Aligning the structure of incentives with intermediate objectives
Too much focus on documents and on BWI-driven procedures
Insufficient scope for treating different countries differently
Insufficient benchmarks to monitor progress toward the intermediate objectives of improved domestic policy processes, and consequent lack of clarity about the criteria for assessments and financing decisions
Asymmetry of commitments, with too little on what the BWIs are expected to deliver
The role of the IMF
Lack of clarity about what the IMF should be delivering in some areas
Insufficient recognition of the changes the PRS approach implies for the IMF's "way of doing business"
Aligning incentives and objectives
Clarifying the IMF's role and improving its effectiveness
Some Reflections on the IMF's Longer-Term Role in Low-Income Countries
The Poverty Reduction Strategy (PRS) approach consists of a series of process innovations designed to encourage broader-based participation in the development of a country-owned, long-term strategy for growth and poverty reduction that could also be a framework for coordinating donor support. It was accompanied by a transformation of the IMF's concessional lending facility into the Poverty Reduction and Growth Facility (PRGF). It is too early to evaluate the success of the new approach in achieving its longer-term objectives, especially the extent of reduction of poverty; progress in this dimension will become evident only over a longer period of time. The evaluation has, therefore, focused on intermediate stage outcomes, that is, the quality of the broader-based policy formulation process, the nature of the strategy and policy framework that has evolved, the interaction between this framework and the PRGF, and the effectiveness of the IMF's role. We summarize here our major findings as well as the lessons to be drawn from them and make a number of recommendations for the future. The final section reflects on some implications of the evaluation for the longer-term role of the IMF in low-income countries.
The broad picture that emerges from our study is that the PRS approach has the potential to encourage the development of a country-owned and credible long-term strategy for growth and poverty reduction, which could provide an effective framework for coordinating the efforts of donors and international financial institutions (IFIs), including the IMF. However, actual achievements thus far fall considerably short of potential. This is partly because it is unrealistic to expect quick gains given the initial conditions from which the process started in most low-income countries. But there were also shortcomings in the design of the initiative that have reduced its effectiveness, including a lack of clarity about the role that the IMF should play.
Participation in the formulation of Poverty Reduction Strategy Papers (PRSPs) was generally more broadly based than in previous approaches, and most stakeholders involved in the process viewed this as a significant improvement. However, the participatory processes were typically not designed to strengthen existing domestic institutional processes for policy formulation and accountability (e.g., through parliament). In a few cases, institutional arrangements to sustain the process are beginning to develop around the budgetary cycle.
The PRS process has had limited impact in generating meaningful discussions, outside the narrow official circle, of alternative policy options with respect to the macroeconomic framework and macro-relevant structural reforms. This reflects in part the absence of any mechanism to ensure that key issues were aired and the broader debate well-informed. Lack of clarity about the role of the IMF in this area contributed to this outcome. In the relatively few cases where a broader debate did occur, there was a positive impact on policy outcomes.
Results in terms of ownership are mixed. The approach has often generated relatively strong owner-ship in a narrow circle of official stakeholders responsible for driving the process, but much less among other domestic stakeholders. The perception that the approach is overly influenced by procedural requirements of the Bretton Woods institutions (BWIs) is widespread.
In terms of diagnostics, the approach has generally contributed to a significant improvement in understanding the multidimensional nature of poverty, which has implications for designing poverty reduction strategies. However, the approach has been much less effective in identifying constraints to accelerating growth and making it more pro-poor. The approach has so far not contributed significantly to understanding the linkages between growth, poverty incidence, and macroeconomic policies at the individual country level. These issues present analytical challenges that are not necessarily resolved through participation alone.
Strategies outlined in PRSPs generally constitute an improvement over previous development strategies, in the sense of providing greater poverty focus, a longer-term perspective, and some results orientation. However, most PRSPs fall short of providing a strategic road map for policymaking, especially in the area of macroeconomic and related structural policies. The focus of most PRSPs is on the composition of public expenditures, especially social sector spending, with much less emphasis on other aspects of a broader strategy to encourage poverty-reducing growth. Even in the area of public expenditure, the operational value of PRSPs is often limited, because of the still rudimentary nature of most costing and prioritization. In many cases, PRSPs also avoid addressing key strategic choices involving "controversial" structural reforms. These weaknesses imply that in most cases PRSPs do not yet provide a policy framework in which PRGF-supported programs can be anchored.
Except in a few countries where the process is beginning to be embedded in domestic institutions, there is limited feedback from initial implementation to policy design. This is particularly problematic in the area of macroeconomic policy where the original PRSP was often overtaken by events or proved unrealistic, and there was little in the PRSP to guide choices on key strategic trade-offs involved in recalibrating macroeconomic targets.
Capacity constraints have been a severe impediment to progress in the implementation of the PRS approach. There has been insufficient attention to developing a systematic plan of action to strengthen capacity, including in the IMF's areas of primary competence. Budgetary processes are weak, and the linkages between the PRSP, medium-term expenditure frameworks, and budgets are generally poor. In particular, public expenditure management (PEM) systems are generally too weak to allow the PRSP to play a central role in implementing expenditure priorities or modifying them on the basis of feedback on actual costs and outcomes. Strengthening PEM has been recognized as central to the success of the initiative and is one area where systematic monitoring by the BWIs on the basis of commonly agreed benchmarks is being undertaken (at least for the Heavily Indebted Poor Countries (HIPCs)). The results suggest moderate progress. In other areas, however, capacity limitations, which constrain policy design and implementation, have largely not been addressed systematically, including through BWI contributions.
On balance, joint staff asessments (JSAs) do not perform adequately the many tasks expected of them. The clarity, candor, and comprehensiveness of the assessment are uneven, with scope for improvement even in "good practice" cases. This partly reflects a built-in bias to reach a "yes or no" signal—which is always "yes" in practice, encapsulated in standardized language. Other factors limiting the usefulness of JSAs include the lack of explicit benchmarks in most areas on which to base the assessment. Their main contribution has been in giving feedback to the authorities on weaknesses in the PRSP, but JSAs are virtually unknown outside the narrow official circle and consequently have no impact on the broader policy debate. They do not incorporate systematic inputs from development partners and, in practice, have played a limited role in informing lending decisions, including those of the BWIs.
The effectiveness of the IMF's contribution has varied considerably across different components of the initiative and across countries—with marked differences between "good" and "average" practice. Its overall contribution has fallen well short of the (admittedly very ambitious) goals it set for itself in the original policy documents:
IMF staff typically did not participate actively to inform the policy debate among domestic stake-holders during the PRS formulation process and to ensure that key macro-relevant issues were aired. This is because IMF staff generally interpreted the emphasis on country ownership as implying that involvement on its part should be limited.
IMF contributions to developing a better under-standing of country-specific micro-macro linkages have also been fairly limited. The process has led to much greater awareness within the IMF of the need for ex ante poverty and social impact analysis (PSIA), and this is evident in internal IMF processes, but it has not yet translated into a mainstreaming of such analysis in program design.
On the positive side, there are signs that the "policy space" in the macroeconomic area has widened—in the sense of greater openness on the part of the IMF to considering alternative country-driven policies—at least in countries where macroeconomic stabilization is no longer a pressing issue.
Success in embedding the PRGF in the overall strategy for growth and poverty reduction has been limited in most cases—partly reflecting shortcomings in the strategies themselves. Nevertheless, program design under the PRGF has incorporated greater fiscal flexibility to accommodate aid flows, and there is no evidence of generalized "aid pessimism" or a systematic "disinflation" bias. Expenditures designated as poverty reducing have increased markedly since 1999, although there are questions about how "pro-poor" some of this spending is. IMF structural conditionality has been streamlined, but we have not been able to reach a definitive conclusion on what has happened to aggregate IMF–World Bank conditionality, which is not monitored by the two institutions. There were only minor improvements in program implementation under the PRGF.
In terms of outcomes, only tentative messages emerge at this stage.
BWI measures of the quality of policies and institutions suggest that PRS countries generally started out in a better position than non-PRS low-income countries, but did not improve at a faster pace.
Short-term growth for PRS/PRGF countries is only marginally higher than in the earlier period. However, these countries seem to have weathered the worsening of the external environment in 2000–02 better than other low-income countries, which experienced a decline in growth.
Evidence on poverty-related outcomes, drawn from the parallel evaluation by the World Bank's Operations Evaluation Department (OED), is still too limited to draw definitive conclusions. The most notable improvements concern various input- and output-related measures (e.g., number of teachers, school enrollment, vaccination rates), but outcomes such as maternal and infant mortality rates have generally not improved.
To summarize, in cases where the PRS approach has been integrated into domestic institutional structures, there have been important improvements in domestic policy processes. In most cases, however, while the PRS approach has generally helped make progress in the right direction, achievements are more tentative. Where the PRS approach has been implemented as an externally imposed, ad hoc exercise, it has acted more as a distraction from strengthening the domestic policy framework.
These broad findings suggest two sets of lessons that illustrate where the approach, and the IMF's role therein, need strengthening. The first set of lessons relates to the structure of incentives that has been generated by the PRS approach, which are not well aligned with the intermediate objectives of the approach. The second set of lessons relates to the IMF's role.
A focus on improving fundamental domestic policy processes in countries is more likely to yield longer-term gains, in terms of an improved policy environment for effective use of development aid, than would a traditional focus on particular policy measures. Whether this refocusing is achieved in practice depends in large part on the underlying incentives faced by countries under the initiative. Unfortunately, actual incentives under the PRS approach do not focus sufficiently on improvements in domestic policy processes and institutions. There are four specific aspects to this problem.
The delicacy of attempting, from outside, to en-courage political systems to move in the direction of greater openness, accountability, and participatory policymaking has led to too much emphasis on procedural components. In practice, therefore, what countries perceive as necessary to receive concessional financing and debt relief is to produce specific documents following various procedures, including for participation. At its best, the approach can still lead to important gains in domestic policy processes, and it has done so in some cases. At its worst, however, it risks being a distraction from the task of making improvements in existing domestic procedures for policy formulation and implementation.
The philosophy underlying the original initiative was to allow for considerable country variation in approach. As implemented, however, there has been insufficient scope for treating countries differently. The value added of each of the principles underlying the PRS approach varies across countries depending upon circumstances. Some countries had no experience of strategic planning in the area of development and stood to benefit enormously from the exercise of putting together an integrated poverty reduction and growth strategy that is results oriented and provides a long-term perspective. Others had a strong planning tradition but limited experience with broad-based participatory policy formulation; in these cases introducing a participatory element was potentially beneficial. Some countries had both elements, but donors (including the BWIs) were not always willing to treat the country's own strategy and domestic processes as a basis for their partnership with the authorities. These differences in starting conditions illustrate the need for the PRS approach to be implemented in a manner that capitalizes on existing strengths and focuses efforts on "the missing ingredients" rather than starting anew in all areas. Otherwise, not only can ongoing efforts to improve existing domestic processes be disrupted, but weak domestic capacities may be used inefficiently.
The PRS approach was largely defined in terms of broad principles related to processes and inputs. More specific milestones to monitor progress vis-à-vis intermediate objectives, including for strengthening domestic processes according to the broad PRS principles, were expected to be developed at the country level. This has not happened and benchmarks for measuring substantive improvements in policy processes are lacking, both at the country level and initiative-wide. Moreover, these improvements are largely not assessed in JSAs, which generally focus on the more procedural aspects of the participatory process as well as on the specific policy content of the PRSP. Ambiguities about intermediate objectives have also contributed to diverging expectations of different stakeholders and insufficient accountability.
More fundamentally, the PRS approach as designed and implemented does not sufficiently address how trade-offs between domestic ownership and BWI assessments of policies will be handled. Decisions on such trade-offs are still taken, but in a manner that (i) is not transparent, with countries left to divine what is "acceptable" by looking at various guidelines and through exchanges with BWI staff; (ii) focuses too much on procedural compliance; and (iii) often leaves country authorities and other domestic stakeholders feeling that the BWIs control the process partly through unwritten rules of the game.
PRSs that substantially meet the goals set out in the original policy papers require considerable technical and analytical capacities, not to mention financial support. However, in the current setting, nothing ensures that the capacity-strengthening priorities of countries will be addressed, and the links with lending decisions can be ambiguous. While both the IMF and the World Bank are broadly committed to "aligning" their assistance programs to the PRSP, it is not clear in practice how much countries have to gain by treating the PRSP as an effective strategic road map, rather than as a procedural formality.
The bold expectations set out in the initial policy documents require greater changes in the IMF's way of doing business than have occurred so far and are probably overpromising what the IMF can deliver with existing resources. There are several reasons why the IMF's effectiveness under the new approach has not matched the original expectations.
The original policy papers may have over-promised what the IMF could reasonably be expected to deliver, given prevailing resource constraints and its comparative advantage. One example concerns the determination of a medium-term external resources envelope that strikes the right balance between needs, sustainability, and realism as well as the associated efforts to mobilize the requisite donor support. Clearly, these are vital components of the overall strategy, but it is not clear that the IMF is well suited to deliver them.
There is also ambiguity about what donors require in terms of a "signaling role" from the IMF, especially when macroeconomic stabilization has been achieved. This is at the heart of the issue of long-term program involvement, to which we return in the section "Some Reflections on the IMF's Longer-Term Role in Low-Income Countries."
Weaknesses in the PRSPs inevitably impede PRGF-supported programs from adapting fully to their expected role. In some countries, this will be a transitional issue as strategies are strengthened and become better integrated into domestic institutions. However, this will happen slowly in many cases, and how the IMF role should adapt in such "difficult" cases remains unclear.
Moreover, some country authorities are reluctant to accept a more active role for the IMF, especially with regard to the broader policy debate.
The implications of the PRS approach for the IMF "way of doing business" have not yet been fully acknowledged or acted upon. The approach implies a very different way of organizing IMF inputs based on: a country-driven strategy that sets priorities within a long-term timeframe; emphasizing contributions to informing a broader policy debate rather than traditional program negotiations; and operating within a "partnership framework" that recognizes explicitly that IMF contributions are only one part of a broader picture. All of these factors mean that the IMF program and surveillance roles in PRSP countries will be qualitatively different from those in other countries. But the consequences have largely not been spelled out, resulting in an attempt to address a very different set of challenges with an essentially unchanged institutional approach.
For example, it is clear from our discussions with staff that many, while recognizing the value of the PRS approach, do not see it as implying fundamental changes in the way the IMF would contribute to a broad-based policy debate on the macroeconomic aspects of countries' strategies. However, the evaluation indicates that, for the PRS approach to have a meaningful impact in the IMF's areas of responsibility, encouraging a well-informed debate is critical.
Moreover, the BWIs have not used the PRS approach sufficiently as a mechanism for identifying priorities on what they should deliver and for coordinating key inputs from other partners, drawing on the countries' own priorities. For the IMF, this has meant that scheduling and integrating inputs from the World Bank into program design remains difficult.
Based on these lessons, we make six broad recommendations. The first three concern the design and implementation of the PRS/PRGF approach. The other three are directed at clarifying expectations about the IMF's role and improving its effectiveness. Each recommendation outlines a broad direction of change rather than specifying a detailed blueprint. This approach seems preferable since there are a number of ways the recommended changes could be implemented that will involve, inter alia, choices on resource allocation that are best left to IMF management and the Executive Board. We give examples of possible steps to illustrate what we have in mind, but other approaches to implementing the recommendations are also possible.
The rationale underlying the first three recommendations is to bring the incentives built into the design of the initiative more in line with its intermediate objectives. The approach we favor is one built around (i) greater country-driven flexibility (e.g., to allow countries to choose their own path toward improving domestic policy processes); (ii) transparency about those country-specific choices, so as to allow other stakeholders to express their views; (iii) monitoring on the basis of explicit benchmarks and strengthened ex post accountability; and (iv) a clearer framework in which donor support is decided, including through candid and graduated BWI assessments of countries' strategies.1
Recommendation 1. Introduce greater flexibility in the implementation of the PRS approach to fit better the needs of countries at different stages of the process and with different capacities and political and administrative systems.
The PRS approach would have greater value added if it was made more flexible in practice to fit different country situations. Countries need to be put even more firmly in the driver's seat by determining themselves:
(1) How the policy formulation, implementation, and monitoring processes will be conducted and built up over time, and with what rules of the game (e.g., for opening up the policy process to previously excluded groups or strengthening budgetary processes). Progress would be monitored against an explicit set of country-determined intermediate benchmarks (see Recommendation 2).2
(2) What the output of these processes will be in terms of documents (e.g., PRSP Progress Reports) and on what periodicity they will be prepared, relying as much as possible on domestic institutional arrangements and reporting vehicles. IMF process requirements (e.g., linking reviews under the PRGF to completion of specific PRSP documents) should be minimized and oriented around domestic processes (so that they do not conflict with domestic timetables and/or duplicate domestic instruments).
Recommendation 2. Shift the emphasis of the initiative from the production of documents to the development of sound domestic policy formulation and implementation processes.
This broad recommendation would involve the following elements:
(1) Build in greater results orientation. Countries should be encouraged to establish—with help from BWIs where needed—substantive criteria for judging progress toward key intermediate objectives such as developing (i) an operational road map that provides strategic guidance for setting priorities and resolving trade-offs; and (ii) effective institutional arrangements for formulating, implementing, monitoring, and updating this road map, with a firm link to budget processes.3 The choice of the criteria/benchmarks judged to be most important would likely vary by country, but in some areas broad guidance could be developed by BWI staff or others, upon which countries could draw as a starting point.4
The existing PEM benchmarking exercise is an example of what we have in mind: it combines country-specific action plans for improving domestic processes; clear benchmarks for measuring progress, established with country consultation; and periodic ex post assessments that provide inputs for country-specific and initiative-wide assessments (see Chapter 4, section on "Strengthening Public Expenditure Management"). Given the central importance of good PEM systems to setting and implementing priorities, this is a tool that all PRSP countries—not just the HIPCs—should be encouraged to adopt. However, the types of benchmarks chosen by each country will depend on the improvements that their PRS process identifies as a high priority.
(2) Shift the emphasis of the incentives structure faced by countries from procedural aspects and production of documents to achieving substantive changes in domestic processes and policies objectively measured as described in (1) above. The new set of incentives would include:
Transparency. Countries should present their intentions and objectives, along with the benchmarks selected to monitor progress in a manner open to public scrutiny.
Accountability. IMF (and World Bank) staff would be responsible for providing clear and candid assessments of the progress made by each country in implementing the PRS approach, both in relation to the goals set by the country itself and against initiative-wide benchmarks (see Recommendation 3).
BWI support. IMF (and World Bank) staff would help countries identify key constraints in making progress toward PRS objectives and support efforts to ameliorate them. This would include not only domestic capacity constraints but also obstacles stemming from policies in advanced economies and other trading partners.
Selectivity in lending. Realistically, access to donors' resources will remain the key incentive for countries under the PRS approach. Ideally, donor decisions on the volume of resources provided should be linked to the progress countries are making under the approach. To facilitate this, IMF assessments in its area of expertise need to provide as clear and candid a signal as possible (see Recommendation 3). The criteria guiding the IMF's own lending decisions under the PRS approach could also be improved in this regard (see the section "Some Reflections on the IMF's Longer-Term Role in Low-Income Countries").
Recommendation 3. Clarify the purpose of the JSA and redefine the vehicle accordingly.
An objective assessment of a country's efforts toward poverty reduction and growth, reflecting the joint perspective of the staff of the IMF and World Bank, can be of value to many audiences: (i) to the authorities, as feedback on how to improve the effectiveness of their efforts; (ii) to BWI Executive Boards, as input into their lending decisions; (iii) to other donors, as a signal of BWI views on the quality of the poverty reduction and growth framework of the country; and (iv) to the public, for purposes of monitoring and accountability. In our view, therefore, the JSA is a useful concept whose potential has not been realized and we recommend making the following changes:
(1) JSAs should focus on the adequacy of domestic policy choices and the quality of domestic processes—including monitoring and implementation arrangements, budget decision making, and public expenditure management systems—as well as actual progress toward intermediate objectives, and less on the quality of the PRSP as a document.
(2) To foster clear and candid assessments, we propose: (i) making explicit the criteria and benchmarks used by staff to form their judgments (see also Recommendation 2); (ii) reporting on the views of third parties (especially local stakeholders and donors) when available, and discussing differences of view;5 and (iii) eliminating the need for JSAs to reach a binary (yes or no) conclusion as to the adequacy of the PRS as a basis for BWI concessional lending. JSAs should aim to provide a graduated assessment of the strength of the PRS and related processes as well as of the quality of policies.
(3) The effectiveness of the above recommendations could be enhanced if JSAs were produced on an independent schedule (e.g., once a year), rather than being linked to a specific PRSP document—which inevitably creates a focus on the document rather than the process, and creates another procedural requirement for the country. This would have the added benefit that BWI monitoring of PRS implementation would continue regardless of PRGF-related developments.
(4) The JSAs would be more effective if, in addition to flagging weaknesses in the PRS, they indicated clearly what are the main obstacles to over-come (including those not under the control of country authorities); what the IMF proposes to do to help address them in its areas of responsibility; and what needs remain unaddressed, especially in the area of capacity building.
There are a number of ways these key functions could be carried out and an instrument like the JSA is only one possibility. For example, an alternative would be to rely on other instruments within the IMF (e.g., the surveillance process) and the World Bank (the Country Assistance Strategy (CAS)).6 However, we see merits in maintaining a joint instrument, since it is more consistent with the partnership dimension of the PRS approach.7 Indeed, looking forward, the assessment instrument may need to adapt further in this direction. For example, if general budgetary support by bilateral donors to a number of PRSP countries continues to expand, a joint "partner" assessment—built around the common performance assessment frameworks (PAFs) that have been developed for several countries (e.g., Tanzania and Mozambique), but expanded to incorporate explicit IMF inputs—might be a logical development in the future.
The practical implications of the adoption of the PRS approach for IMF operations go well beyond the formulation and negotiation of IMF-supported programs and have not been fully recognized. In order to orient our recommendations in this area, we note that two types of decisions need to be taken with respect to the IMF's role:
Decisions concerning the "architecture" of the IMF's role in low-income countries. This involves the nature of the IMF's signaling function as well as the role of IMF conditionality and financing in the longer-term PRS framework. These decisions go beyond the scope of the present evaluation and we do not make specific recommendations. However, we discuss in the section "Some Reflections on the IMF's Longer- Term Role in Low-Income Countries" a number of principles that should be taken into account.
Within the present "architecture," decisions concerning how much the IMF should be directly involved in helping countries implement the PRS approach and changes in the IMF "way of doing business" that need to be made, irrespective of choices on the scope of the IMF role, to improve the effectiveness of the institution's contribution. These issues are addressed in Recommendations 4–6.
Recommendation 4. Clarify what the PRS approach implies for the IMF's own operations and strengthen the implementation of the agreed role.
The thrust of this recommendation is to clarify what is expected of the IMF under the PRS approach and to outline changes that would improve its effectiveness in meeting these expectations.
(1) IMF engagement in the PRS process
More emphasis should be given to IMF activities that help to better inform broad-based policy discussions in its areas of competence. Guidelines to staff need to be clarified so as to encourage more active inputs to such discussions, including analyzing alternative policy options and trade-offs. In exchanges with the evaluation team, some IMF staff expressed concern that such activities go beyond the Fund's mandate and can potentially conflict with domestic ownership. However, domestic ownership should not be interpreted to mean lack of transparency in what the IMF would expect on macroeconomic policy and the reasons why it takes this view. One possible approach might be for the IMF staff to prepare a short note on key macroeconomic issues that need to be addressed in the broader growth and poverty reduction strategy, which the authorities could make public if they wished.
Rather than establish uniform "standards" for the IMF's role, expectations should be tailored to country-specific circumstances, including the government's wishes. These country-specific "rules of the game" should be made public—perhaps in the form of an aide-mémoire attached to the JSA or the Article IV staff report—and could describe how IMF staff are expected to participate in the broader policy debate, including what supporting analytical work they will undertake. This is also likely to require a combination of: (i) more "stand alone" missions, set apart from program negotiations, and (ii) enhancing the role of resident representatives.
There should be more systematic explorations of country-specific macro-micro linkages—both through the IMF's own analysis and research activities and by drawing more systematically on the work of others.
Article IV surveillance reports should be used systematically as a vehicle to convey the IMF's own thinking and analysis on key issues, and the staff should routinely seek the views of the authorities and others on the types of issues that should be addressed in surveillance.
Assess systematically, as part of broader IMF surveillance activities, obstacles to the achievement of PRSP objectives originating in trade and subsidy policies of main trading partners (drawing, to the maximum extent possible, on inputs from other agencies). The Global Monitoring Report (World Bank and IMF, 2004a) has already made important steps in this direction.
(2) PRGF-related activities
The rationale for IMF policy recommendations and program design should be subjected to broader scrutiny and debate. Possible steps in this direction could include (i) facilitating wider dissemination and discussion of IMF analysis that forms the basis of its policy advice (including, where appropriate, technical assistance reports); and (ii) encouraging greater openness to "independent/external voices" as inputs into program design, when agreeable to the authorities.8
Clarify the approach to be taken by the IMF in those cases where the PRS approach has added some value but has not yet produced an operational road map or the necessary institutional framework for implementation. Even in these cases, there may be significant scope for opening up the policy space and incorporating evidence on macro-micro linkages (including PSIA) more systematically into program design, even though the priorities for such activities will not be driven by a coherent country-driven strategy.
Clarify what the BWIs are trying to achieve jointly through the streamlining of conditionality and how this fits with stronger domestic ownership; establish a system for the monitoring of aggregate Bank-Fund conditionality at the country level.
(3) Streamline IMF documentation and Board scrutiny of PRS documents
Exactly how this streamlining should be done will depend on what role the JSA plays (see Recommendation 3), but there seem to be two possible approaches: (i) make the JSA the central assessment instrument, with operational meaning—in which case "comfort" letters and detailed analysis of the PRS strategy in other staff reports (Article IV or PRGF-related) would be unnecessary; or (ii) incorporate the functions of the JSA into surveillance and/or program documents. In any event, IMF Board discussion of the PRSP itself seems unnecessary (al-though Directors should continue to receive PRSPs for information). At the initiative-wide level, separate annual reports on Progress in Implementing the PRSP Approach and the Global Monitoring Report on the Millennium Development Goals (MDGs) appear duplicative and could be merged.
Recommendation 5. Strengthen prioritization and accountability on what the IMF itself is supposed to deliver within the broader partnership framework, built around the priorities emerging from the PRS process, and ensure that resources match commitments.
The IMF should tailor its involvement more closely to country needs, taking into account the contributions from other partners. Because these needs vary widely and because resource constraints on the IMF will remain tight, a more systematic approach is needed to (i) setting priorities on what the IMF itself will deliver; and (ii) ensuring appropriate coordination of key inputs from other stakeholders. Prioritization and coordination could be improved as follows:
(1) Generate, as part of the PRS process, specific priority actions for the IMF to assist the country concerned to reach its objectives, including for analytical inputs and long-term capacity building. If the IMF is not well-suited to produce a particular input (e.g., some PSIA), but this input is judged critical for program design, a clear identification—stemming from the PRS priorities—of who has agreed to do what and by when would strengthen incentives. Such key deliverables would be agreed with the government and made public. These priority actions would then be the basis for evaluations of IMF performance, both at the country level and in aggregate.
(2) The IMF's own budget decisions on allocation of administrative resources should be geared to these priorities. Linking realistic resource allocation decisions to a more explicit set of priorities should help improve decision making and make clearer to all what the IMF has committed to, and what it has not. However, the evaluation suggests that the ambitious original expectations of the IMF's role in the PRS approach would require more resources than have been provided so far.
(3) Experiment with broader "external reviews" of the PRS/PRGF process, monitoring in particular the performance of donors and international financial institutions (IFIs) in providing support, and not just performance of the national authorities. Tanzania's Independent Monitoring Group (IMG) is one possible example of such an approach.9
Recommendation 6. The IMF should encourage a strengthening of the framework for establishing the external resources envelope as part of the PRS approach.
The present "architecture" of the PRS approach does not provide a clear framework for helping countries and donors decide what is an appropriate medium-term resource envelope in which the macro-economic strategy should be derived. The evaluation has not produced sufficient evidence for us to recommend a particular "right" approach on this issue. However, we would not favor a greatly expanded role for the IMF that would risk taking it beyond its comparative advantage. Elements of a solution would include:
(1) The country itself, not the IMF or World Bank, should eventually play the central role in elaborating macroeconomic frameworks and catalyzing donor support. The pace at which this transition can be made will depend on specific capacity constraints in each country, but country leadership seems essential to "owning" the process.
(2) The IMF role would be to provide debt and macroeconomic sustainability assessments and judgments on the policy framework, but it would not be responsible for the "normative" judgment on appropriate aid levels over the medium term.
(3) The tension between "ambition" and "realism" in determining the external resource envelope can perhaps be handled by presentation of alternative projections (consistent with assumptions of stronger policy reforms as well as additional external financing). The IMF should provide increased analytical support for such approaches when requested. But the choice to prepare alternative projections should remain with the country and not be a uniform requirement.
(4) Improving aid predictability is a wider problem that the IMF cannot resolve on its own. Consistent with the PRS approach, donors should be making and implementing firmer medium-term commitments, linked where possible to country budget processes with clear triggers for financing decisions. However, this is only feasible if the PRS process generates an effective operational road map. The challenge is how to reconcile PRSP countries' concern for aid predictability with donors' concern for aid effectiveness, which implies some performance-based selectivity. For the IMF, this requires finding a way—perhaps through a strengthened JSA—to provide signals to the donor community on macroeconomic performance that are sufficiently calibrated, and take account of the longer-term framework of donor involvement, to be a useful input into selectivity decisions without providing excessive "on-off" signals for financing. We return to this issue below.
Many of the issues raised in this evaluation have to be addressed in the context of the larger issue of the IMF's longer-term role in low-income countries. A discussion on this subject has already begun.10 It is beyond the scope of this evaluation to make detailed recommendations on the strategic policy choices involved, but our investigation has highlighted a number of unresolved issues—some of which have not yet been aired sufficiently in the IMF's own papers on the subject—that will require the Executive Board to take several critical decisions. We summarize here the key decisions that will be needed and suggest a number of principles that should be borne in mind as they are taken.
The key decisions are:
(1) What should be the signaling role of the IMF in the longer term and how should it be provided?
The revealed preferences of major development partners and of many low-income countries suggest that if substantial donor support is to continue, an IMF signal of the quality of macroeconomic policy will be required. This certainly requires an IMF signal that is credible and clear, but it may or may not require financing and conditionality. Decisions on the signaling role should also distinguish between (i) the need for judgments on the overall macroeconomic framework and strategy (for which a JSA-type vehicle may be needed), and (ii) monitoring of shorter-term performance where needed (a role traditionally performed by IMF conditionality; see (3) below).
(2) What should be the IMF's financing role in low-income countries and what should be the criteria for "exit" from such financing?
The existing criteria for such financing are based on the concept of "protracted balance of payments need," which is so vague as to be difficult to distinguish from development financing in practice. Assessments of a protracted balance of payments need are expected to consider not just the country's over-all balance of payments position at the time of the request for financing, but also "past, present and prospective external performance [indicators], such as export and import performance, terms of trade development, access to capital markets, cost of debt service, foreign reserves [and] medium term financing gaps that the [country] might experience in its growth oriented adjustment effort." A present balance of payments need is not required. These criteria do not provide effective guidance for when the IMF should shift its support from traditional lending arrangements to other forms of engagement and in practice have justified continued financing.
In contrast, there has not been enough emphasis on the implications of the PRS process itself for an "exit" strategy from IMF lending. Unbundling the various nonlending functions that the IMF is expected to perform under the PRS approach (see Recommendation 4) makes clear that the IMF can remain very substantially involved in the PRS process without providing baseline financing over long periods.11
(3) What will be the nature of IMF conditionality in the longer term as the PRS approach takes root?
This will require decisions on what is the rationale of IMF conditionality in a longer-term, partner-ship-oriented framework where everyone recognizes that donors are "in for the long haul" and there is a strong emphasis on country ownership.12 One possible route would be to restructure IMF conditionality to fit within a broader "partnership" approach to monitoring and assessing progress in implementing the PRS. For example, the IMF could join the collective approaches to monitoring PRS progress that are evolving between those donors (including the World Bank) providing direct budgetary support (e.g., the PAFs for Tanzania and Mozambique). However, even within such a framework, the IMF (and its partners in the PRS approach) might come to the conclusion that there are grounds to maintain a separate quantitative monitoring of the short-term evolution of key macroeconomic policy variables—given that providing short-term feedback on key budget constraints and macroeconomic sustainability is a central IMF contribution. But such monitoring need not necessarily be tied to IMF lending, nor should continuation of other flows be mechanically interrupted on the grounds of nonperformance of short-term conditionality.13
(4) What should be the IMF's role in strengthening the framework for establishing the external resources envelope that countries may count on?
As noted in Recommendation 6, the evaluation evidence suggests no "right" answer to this question—in part because the IMF has not, until recently, encouraged much experimentation with different approaches. However, elaboration of such a framework is not something the IMF can do on its own; all development partners will need to be involved. Part of the agreed framework should be understandings on the implications of IMF signaling on short-term macroeconomic performance for other financing flows.
We recommend that the following principles help guide these decisions:
The approach to the IMF's role in the architecture of the PRS should build upon the specific comparative advantage of the IMF. These include analysis and policy advice on macroeconomic and related structural policy issues, which are preconditions for sustained pro-poor growth. They should incorporate a focus on management of budget constraints and longer-term sustainability issues; technical assistance and capacity building in these areas; and a relatively short-term financing role to help cushion against adverse shocks that might push the longer-term growth and poverty-reduction strategy off track.
The IMF's contribution in low-income countries will be only one part of a broader, partnership-based framework, and its modalities should be designed with this framework in mind. In other words, strong coordination elements rather than "stand alone" approaches need to be built in from the outset, whether it be with respect to PSIA, conditionality, or decisions on overall external financing.
The choice of appropriate instruments to provide signaling should be based on the needs of low-income countries and their development partners. However, the credibility of different signaling devices will depend partly upon the "architecture" chosen. At present, programs associated with IMF lending arrangements are so central to the IMF's role in low-income countries that nonprogram signaling tends to have less credibility. This could change if a different structure, with new signaling and financing instruments, were adopted.
The modalities of the IMF's role should be mindful of the incentives created and should be designed to encourage countries to press ahead with their policy and process changes under the PRS approach. To give an example, one possible way of integrating the IMF's relatively short-term financing into the PRS approach that might be consistent with such a principle would be for the PRGF not to provide financing when there is no immediate balance of payments need (nor to design somewhat arbitrary phaseout schedules), but to incorporate provisions for greater contingent ease of access to financing when needed because of adverse exogenous shocks—provided that the JSA (or its successor) has given a sufficiently positive signal on the country's progress in implementing the PRS.
Decisions on the intensity and length of IMF involvement in PRS countries and the way the IMF organizes its operations and allocate its staff resources to these countries need to be made in an integrated manner, so as to ensure consistency between commitments and available resources.
To summarize, there can be no doubt on whether the IMF has a longer-term role to play in low-income countries. It clearly does, if only because macroeconomic stability is widely accepted as a pre-condition for sustainable pro-poor growth. However, it is important that the IMF's role is kept within the institution's comparative advantage. The challenge is to translate that basic approach into two sets of choices: first, on how to "embed" the IMF's contribution most effectively into a broader partnership framework and, second, what should be the intensity and scope of the IMF's involvement in that broader framework.
An alternative approach to modifying incentives would be to establish challenging universal minimum standards in a number of areas (e.g., for participatory processes or ex ante PSIA). See, for example, Oxfam (2004). We do not favor such an approach. Even though driven by the best intentions, it would result in the imposition of even more procedural requirements on PRS countries. Moreover, uniform standards do not allow for the diversity of countries' situations or for necessary priority setting.
To give a concrete example to illustrate what we have in mind, a country might propose to broaden participation and monitoring through strengthening the oversight functions and capacities of parliament and at the local government level, with formulation and implementation of its medium-term strategy built around the domestic budgetary process. In which case, the BWIs should accept such an approach by not pressing for other possible procedural components and by accepting for their own internal purposes PRS-related documents oriented around these domestic processes. The country would establish benchmarks for monitoring progress in reaching the objectives it has set for itself, but the BWIs would provide their own frank assessments of the country's strategy and the policy processes by which it was derived.
This recommendation is similar to the call for an operational framework made by staff of the European Commission (EC) and World Bank in their "Joint Note on Supporting the PRSP Process in Africa" (EC and World Bank, 2003).
For example, benchmarks on various issues being developed by the New Partnership for Africa's Development (NEPAD) might be drawn upon by African countries. In other cases, UN agencies (or even international nongovernmental organizations (I-NGOs)) might make suggestions or countries may choose benchmarks tailor-made to their own priorities and circumstances (e.g., for participatory monitoring of local expenditures). The important point is that the countries themselves should choose what are the priorities for monitoring progress and not be overburdened with a multiplicity of requirements
We are not suggesting that the JSA should become a "negotiated" document with all of these stakeholders.
The CAS details the partnership between the Bank and each borrowing country. Usually covering a three-year period, it defines the level and composition of assistance to be provided based on needs, portfolio performance, and overall policy performance.
Moreover, unlike the JSAs, the alternative vehicles are not automatically made public.
The Tanzania case is a good example and illustrates that a change in the "way of doing business" toward strengthening the capacity of countries to engage in discussion on such policy advice requires primarily a change in attitude and organizational approach. We are not suggesting that the IMF has to provide the resources for any additional outside inputs, but bilateral donors may wish to do so. Indeed, in the Tanzania case the additional analysis was financed by a bilateral donor.
The IMG was appointed jointly by the Government of Tanzania and donors in 2002 to examine the factors that undermined the impact of aid and its contribution to supporting Tanzania's PRS. In particular, the IMG would monitor the policy dialogue, financial management and governance issues, predictability of aid flows, and the nature and use of technical assistance.
Since the summer of 2003, several discussions were held in the IMF's Executive Board on various aspects of the role of the Fund in low-income countries. This discussion is still ongoing. The staff papers on which the Board discussions were based, along with the preliminary conclusions reached by the Board, are available on the IMF's website (www.imf.org). The key papers are as follows (starting with the most recent): "The Fund's Support of Low-Income Member Countries: Considerations on Instruments and Financing" (February 24, 2004); "Fund Assistance for Countries Facing Exogenous Shocks" (August 8, 2003); "Role of the Fund in Low-Income Member Countries Over the Medium-Term—Issues Paper for Discussion" (July 22, 2003).
Present policies call for access to PRGF resources to decline with successive three-year arrangements. However, policies that contemplate defining the appropriate levels of access for, say, the fourth and fifth rounds of three-year PRGF arrangements (i.e., 12 or 15 years into the initiative) raise very uncomfortable questions. It is hard to see any justification for such protracted, albeit low-level, financial involvement except on the grounds that only traditional programs carry sufficient credibility as signaling devices (either with the country or donors).
The earlier IEO evaluation of the prolonged use of IMF resources suggests that traditional approaches to conditionality may lose their effectiveness in such situations, especially with regard to the types of reforms that take time and cannot easily be broken down into specific short-term conditions. (See IEO, 2002, Chapter 5, especially paras. 35–37).
This concern was raised by a number of I-NGOs in their inputs to this evaluation. The Mozambique case study suggests a possible approach. Delays in completing reviews under the PRGF-supported programs flagged emerging concerns to donors but did not trigger an immediate interruption of financing by those donors providing direct budgetary support. Instead, active participation by IMF staff in a donors' forum served as a mechanism for signaling what progress was being made in resolving the outstanding issues. An agreement was eventually reached and the program review completed, without any interruption in these broader financing flows.