By Toby McIntosh
The existence of strong access to information policies at international financial institutions yielded better project outcomes, according to an academic study.
In particular, project outcomes were better when access to information (ATI) policies included an independent appeals process.
When ATI policies “are accompanied by independent appeals processes, we observe a strong and positive relationship with performance,” according to the academic researchers, who evaluated 20,000 aid projects in 183 countries financed by 12 donor agencies, both multilateral and bilateral.
The paper published in the American Journal of Political Science was written by Dan Honig of University College London, Ranjit Lall of the London School of Economics and Political Science, and Bradley C. Parks of William and Mary. See the authors’ final version and online appendix here; replication archive here.
Separately, another new academic study looked at transparency in the development aid agencies of 37 donor countries, ranking them in a “Citizen Aid Transparency Dataset.” (More about this one below.)
Follow EYE at @tobyjmcintosh. Sign up in the right column to receive new articles.
Why Appeals Mechanisms Matter
Having an appeals process as part of an ATI policy improves project outcomes through two distinct mechanisms, according to Honig, Lall and Parks.
“The first is a project correction effect whereby an appeal concerning a given project leads to performance-enhancing modifications to this same project, whether by generating new information about its design or implementation (if the appeal is successful) or by prompting officials to pay greater attention to potential performance threats (if the appeal is unsuccessful),” they wrote.
“The second is a shadow of the future effect whereby officials strengthen project design and implementation in anticipation of future appeals that could reveal performance problems,” according to the authors. They add that “even ATI requests and appeals that concern completed projects (and are ultimately denied) can increase bureaucrats’ awareness that they are being monitored.”
The project correction effects “may be important,” according to the authors, who noted that “anecdotal evidence indicates that appeals can markedly alter the behavior of officials involved with concerned projects.”
They expanded: “However, we expect project improvements to occur primarily through shadow of the future effects. As a proportion of total projects, the number of appeals cases tends to be small. Additionally, most appeals are submitted after the concerned project has concluded—that is, when new information cannot be used to remedy project problems.”
In a related finding, they observed, “The performance dividends associated with well-enforced ATI policies increase when recipient countries have higher levels of civic engagement and press freedom, conditions under which citizens are more likely to make use of these policies and to pressure authorities to improve project outcomes.”
“In contrast,” they said, performance dividends “decline when recipients have domestic ATI regimes and a greater capacity to control corruption and to maintain the rule of law, conditions under which project problems are less common and alternative mechanisms for exercising project oversight are available.”
The study involved 12 agencies, only three of which lack appeal mechanisms, but significantly the large dataset covered time periods before the date when the World Bank added an appeal process to its policy.
They found that the presence of an ATI policy alone is associated with an average increase in our six-point performance scale of 0.02, whereas the presence of an ATI policy with an appeals mechanism is associated with an average increase of 0.29, which is equivalent to moving from the median point on the scale to the 60th percentile. “A pretty large jump, author Lall told EYE.
Researchers Create `Citizen Aid Transparency Dataset’
The “Citizen Aid Transparency Database” created by two university researchers provides a comparison of transparency at development aid agencies of 37 donor countries (212 agencies in all).
A copy of the article, How transparent are aid agencies to their citizens? Introducing the Citizen Aid Transparency Dataset, was provided to EYE by Bernhard Reinsberg of the University of Glasgow who co-authored the study with Haley Swedlund of Radboud University. It will be published in the Journal of International Development.
The authors credited 10 student interns who evaluated the 212 agencies using a system that has 120 indicators, thus creating the large database. The goal is to evaluate the information available to citizens via the agencies’ website.
The European Union scored the highest, followed by Great Britain and Austria.
The report evaluates performance in 19 categories, and when “relevant and possible” the scoring covers “sub-indicators.”
The 19 categories are divided into two groups, one on “governance,” and the other called “agency practice.”
In the governance category, countries are rated on: aid strategy, aid report, aid management, organigram, administrative costs, staff numbers, staff salaries, staff vacancies, transparency policies and contact information.
Under agency practice, the questions cover: total aid, ODA/GNI quota, ODA split recipients, sectors channels, aid quality, project aid, and aid evaluations.
Asked by EYE if the 19 categories count equally in the rankings, Reinsberg replied: “On your question — factor analysis is more complicated than that: it actively finds the correlations in the data, meaning those variables that hang more strongly together will get more weight in the index. So it is much more sophisticated (data driven) than just 1/19 equal weighting.”
Authors’ Conclusions
“Analysis of the data suggests a number of important findings,” the authors wrote. “First, the aid agencies that perform the best on other indexes, notably the Aid Transparency Index (ATI), do not necessarily perform the best on our indicators, they said. “This suggests that our index is capturing a different dimension of transparency than existing indexes, and that transparency towards international audiences is not inherently correlated with transparency toward domestic audiences (as measured via websites).”
Second, the concluded, “[o]ur analysis suggests that within the same donor country, the transparency of different aid agencies can vary considerably, suggesting that all aid agencies do not face the same pressure to be transparent about aid giving.”
Third, they report “important quality differences with respect to the type of data provided.” They add, “The most transparent agencies not only make information available, but also provide information in an easily accessible manner.”
