Evaluation and counter bureaucracy

Following the thread from my blog Monday….

World Bank evaluation expert Aaditya Mattoo warned the Aid for Trade meeting against the tyranny of measurement in evaluation. That we should not avoid programme evaluation simply because we can not perfectly measure the impact of activities.

Former USAID chief Andrew Natsios draws a similar conclusion in his much quoted paper ‘The Clash of the Counter-bureaucracy and Development’ and highlighted today from Oxfam blogger Duncan Green.

Natsios refers to the tension in the development world between the compliance side of aid programs (the counter bureaucracy) and the technical program side and that this “imbalance threatens program integrity”.

He continues that the “counter bureaucracy ignores a central principle of development theory—that those development programs that are most precisely and easily measured are the least transformational, and those programs that are most transformational are the least measurable.”

What makes this report fun to read is not just the lively debate in the main document but the inclusion at the beginning of an ironic memo from the Duke of Wellington complaining about England’s own version of the counter bureaucracy during the time of the Napoleonic Wars.

“Gentlemen,

Whilst marching from Portugal to a position which commands the approach to Madrid and the French forces, my officers have been diligently complying with your requests which have been sent by His Majesty’s ship from London to Lisbon and thence by dispatch to our headquarters.

We have enumerated our saddles, bridles, tents and tent poles, and all manner of sundry items for which His Majesty’s Government holds me accountable. I have dispatched reports on the character, wit and spleen of every officer. Each item and every farthing has been accounted for with two regrettable exceptions for which I beg your indulgence.

Unfortunately the sum of one shilling and ninepence remains unaccounted for in one infantry battalion’s petty cash and there has been a hideous confusion as to the number of jars of raspberry jam issued to one cavalry regiment during a sandstorm in western Spain. This reprehensible carelessness may be related to the pressure of circumstance, since we are at war with France, a fact which may come as a bit of a surprise to you gentlemen in Whitehall.

This brings me to my present purpose, which is to request elucidation of my instructions from His Majesty’s Government so that I may better understand why I am dragging an army over these barren plains. I construe that perforce it must be one of two alternative duties, as given below. I shall pursue either with the best of my ability, but I cannot do both:

1.) To train an army of uniformed British clerks in Spain for the benefit of the accountants and copy-boys in London or, perchance…
2.) To see to it the forces of Napoleon are driven out of Spain.
Your most obedient servant,
Wellington”

Natsios summarizes the problem with the current compliance system as:

• Excessive focus on compliance requirements to the exclusion of other work, such as program implementation, with enormous opportunity costs
• Perverse incentives against program innovation, risk taking, and funding for new partners and approaches to development
• The Obsessive Measurement Disorder for judging programs that limits funding for the most transformational development sectors
• The focus on the short term over the long term
• The subtle but insidious redefinition of development to de-emphasize good development practice, policy reform, institution building, and sustainability.

Natsios ends, “Let me conclude with one simple question asked in a different form by the Duke of Wellington. Do Washington policy makers wish USAID, PEPFAR, and the MCC to implement serious development programs or comply with the demands of the Regulatory Lords of Washington? They cannot do both.”