Goals, targets, indicators: Diplomacy by numbers?
By Bill Orme*
In the third week of June, diplomats reconvened in New York to redact and refine the draft text of the declaration to be announced next September to frame a new development strategy and officially launch the already informally agreed 17 Sustainable Development Goals (SDGs) . Sincere efforts to strengthen stated commitments to human-rights principles and official transparency and accountability will be met with counterproposals for more caveats and ambiguity in its few compliance requirements.
Illustrative of the perils in collective General Assembly editing was the contretemps in May over the term “vulnerable groups” in an outline for an orchestrated public dialogue at the September summit. Who, some wondered aloud, are these “groups,” exactly? Are they defined by race, religion, age, income, gender, geography? Or all or none of those categories? And who defines them, and for what purpose?
A proposal by Egypt to change the locution to “people in vulnerable situations” was debated for two hours, until the Assembly adjourned without agreement on the terminology. (An eventual compromise also crept into the draft declaration: “People in vulnerable situations and marginalized groups.”)
But the real post-2015 debate is focused on the indicators that will factually define and monitor the 17 goals and 169 targets for the next 15 years. The indicators will not be selected until next March, six months after the SDGs are formally adopted.
This daunting task has been entrusted to a misleadingly named Inter-Agency and Experts Group (IAEG) on SDG indicators, a 28-country subset of the 193 national statistical offices represented on the UN Statistical Commission. The IAEG, led by co-chairs from Italy and the Philippines, held its inaugural session June 1-2. It didn’t go well.
Over the weekend before that meeting, as delegates traveled to New York, the UN Statistics Division — a small office of UN data specialists who serve as the staff secretariat for the member states’ UN Statistical Commission — sent them an unexpected list of what it called “priority” indicators for the 169 SDGs targets, one for each.
The Statistics Division’s previous proposal, submitted in March, had 304 provisional indicators of widely varying quality. Though fewer, proportionately, than the 48 indicators originally assigned to the 18 targets of the eight MDGs, 304 was still six times as many in absolute terms. That had deepened concerns in senior UN circles that the post-2015 compendium of goals, targets and indicators was becoming too overstuffed and complex to measure or communicate, much less actually achieve.
While the 17 goals and 169 targets were now politically untouchable, the indicators had not yet been selected, and were seen by some as the last opportunity to insert greater clarity and concision into the SDGs. The UN’s parallel advisory Sustainable Development Solutions Network recommended trimming the indicators further, to an even one hundred. Many of the network’s proposals, as in the Statistics Division’s priority list, were designed to serve two or more targets at once.
The short list of 169 indicators had been hurriedly assembled just days before circulation, with input from UN agencies that had provided previous indicator recommendations. The participating IAEG statisticians from member states complained that few of them had even seen the new list before their UN meeting got underway. And some who had read it complained crankily but correctly that it did not incorporate recommendations submitted by member states in response to the previous indicators list, relying instead on UN agency proposals.
More critically, some member states argued that this one-indicator-per-target proposal violated General Assembly instructions on the letter and spirit of the draft SDGs.
Many targets had been deliberately drafted with several related challenges, such as reducing air, water and soil pollution (SDG 3.9); banning both child marriage and female genital mutilation (SDG 5.3); and ensuring public access to information while protecting broader “fundamental freedoms” (SDG 16.10). Choosing which items to prioritize — clean air over clean water? — would leave some targets stripped of essential and previously agreed elements, some delegates charged.
More than a few countries viewed the single-indicator proposal as a stealth attempt at SDGs reverse-engineering by UN technocrats intent on streamlining the General Assembly’s proposal. The Assembly had previously decreed that indicators “must not undermine or re-interpret the targets,” but should correspond directly to the language of each, neither adding new or “contentious” elements nor subtracting through omission any agreed-on provisions.
In many cases, the collapsing and cross-referencing of indicators is statistically defensible. Yet if indicators become less numerous than the targets, and some target components are left statistically orphaned, those indicators themselves become the “real” targets. Indeed, they become the real SDGs, which are ultimately just thematic clusters of targets and indicators. If progress toward a target is not factually monitored, that target becomes little more than rhetoric.
Diplomats and statisticians from many member states were not happy with the June 1-2 proceedings and said so. Under whose authority had the missing indicators been excised?, some asked. Were the UN staff specialists in attendance “observers,” as the expert group’s mandate stated, or were they the prime movers, as meeting optics and documents sometimes suggested?
Equally displeased were some of the specialized UN agencies and civil society groups that had pressed for specific development objectives in the SDGs and their corresponding but now-deleted indicators.Brazil, a driving force behind the post-2015 negotiations, archly declared that it “was not comfortable with the [IAEG] governance process so far.” Egypt’s ambassador, speaking for the Arab states, questioned the Group’s authority to proceed with its indicator-review plans, prompting a mid-peroration outburst from a Swedish delegate demanding to know when the diplomat would be concluding his remarks so that the statisticians could get back to their “important technical work.”
The Group then struggled to reach agreement on such practical matters as to how (or if) to divide up their indicator-evaluation tasks among thematic working groups in the months ahead. They decided to split into two, one on “interlinkages” and the other on an overall “statistical framework” — still far from dealing directly with the conceptual and methodological strengths and weaknesses of the proposed indicators themselves.
Stefan Schweinfest, chief of the UN Statistics Division, conceded at the close of the IAEG’s inaugural session that “it has not been an easy meeting,” a wry bit of German understatement.
The next IAEG meeting, however, may not be any easier.
It is not taking place until October, and the group hopes to deliver a full set of recommended indicators by December. That is not much time for such a complex undertaking. Most of the preparatory work must be done by those who are paid by the UN to do so — the small corps of Statistics Division professionals — with 28 national statistical agencies looking over their shoulders. This is a new, difficult task for the Statistics Division, which is run by data experts, not diplomats or policymakers.
UN communications people are still grappling with how to explain and to sell this new post-2015 agenda to the world at large, a challenge unhelpfully illustrated by a working title with a built-in Dec. 31 expiration date. Now they must also explain how world leaders will know exactly what they are approving in September, when precise definitions and monitoring plans for the 17 goals and 169 targets will not be agreed until the following year.
That concern could prompt premature UN circulation of incomplete and, in some cases, ill-conceived indicators before the September summit. Pressure to see such a document will build during the culminating two weeks of pre-summit negotiations in New York in July. That’s understandable. Without indicators, it’s hard to evaluate the targets, and without a clear grasp of the targets, it’s impossible to meet the goals. But with indicators that are imprecise or insufficient or too narrowly mathematical, it will never be known if the goals are really being achieved.
Earlier this year, the General Assembly delegated to the UN Statistical Commission the job of choosing SDG indicators that are relevant, methodologically sound, global in scope and comparable across countries, limited in number and “easy to communicate and access.”
The Commission in turn assigned that task to its newly minted 28-member Inter-Agency Expert Group. But as the Group’s difficult birth this month showed, the hard work of fulfilling that General Assembly mandate has barely begun.
* The author is a strategic communications consultant who worked for the United Nations over the past decade as a leader of global advocacy campaigns and efforts to support independent journalism in emerging democracies. Orme is a former director of the Committee to Protect Journalists and a correspondent for The New York Times, The Los Angeles Times and The Economist. In addition, he was a spokesman and external communications chief for the UN Development Program and an adviser on media projects in Africa for the Gates Foundation and the UN Peacebuilding Office. See his complete analysis on the indicators debate here.