More attention and money now go into reforming the humanitarian system than ever before. But we still have a long way to go before the perceptions and experience of affected people – those we intend to serve – become integral to the way aid programs are designed and managed. The problem is that humanitarian agencies are committed to strengthening accountability to affected people, but are still missing practical tools to deliver on their commitment.
Ground Truth takes accountability to affected people to the next level by offering humanitarian agencies a practical way to both listen to the people supposed to benefit from aid, and to integrate what they learn into the design and implementation of their programs. Our focus is on helping agencies to continually track how they are meeting the needs of affected people, and to improve performance while operations are on-going. Ground Truth is thus different from many other approaches that assess the quality and effectiveness of humanitarian programs based on voluntary standards and mostly after their completion.
Ground Truth’s signature approach is to use short, frequent surveys of the perceptions of crisis-affected people as a means to help decision-makers adjust humanitarian programmes – a new performance management tool. Our methodology combines traditional social science models of participation with techniques adapted from the customer satisfaction industry. We ask a small number of questions, but ask them frequently, to get a “signal” from affected people and track progress over time. Read more about our approach to survey design and data collection here.
Ground Truth fills a gap in the existing accountability architecture by focusing on how affected people themselves experience humanitarian aid. By providing a regular ‘signal’ from affected people, and encouraging transparent and horizontal communication and more equitable relations between aid providers and the intended beneficiaries of aid, Ground Truth enables other quality and accountability initiatives to get greater traction. In turn, Ground Truth benefits greatly from their skills and experience, for instance in the areas of outward communication and community mobilization.
Ground Truth offers operational agencies independently collected, continuous feedback from affected populations that will tell them where they are getting it right and where they are not. Our people work with operational agencies to analyze the data, ‘close the loop’ with respondents, and integrate the feedback into their operations, thereby building a performance management approach rooted in the experience of affected populations.
Ground Truth supports three major donor priorities: better coordination (by providing a single source of community feedback around an emergency response), improved effectiveness (by providing a tool for performance management), and enhanced accountability to affected people (by asking them to provide feedback on humanitarian programs and supporting agencies to respond). Ground Truth contributes to these goals by providing reliable metrics that allow donors to compare performance across different humanitarian operations and over time. It also helps them make sense of what’s happening on the ground, in the ‘fog’ of humanitarian operations.
The goal is to create continuous streams of feedback. What works best in any specific humanitarian operation will be worked out on a case-by-case basis. In an emergency, we may collect data as frequently as every fortnight. In protracted situations, the rhythm is less frequent. Key in all programs is ensuring field staff have enough time between rounds to internalize and act on the information. This in turn avoids the danger of 'survey fatigue', when respondents feel that answering questions offers them limited or no benefits.
We start the process of formulating questions with a careful enquiry into the context and the ‘theory of change’ of the program or intervention. We do this in close collaboration with the managers of the humanitarian program as well as the people whose feedback we seek. We engage the latter through focus group discussions and other dialogic processes. From these two streams of intelligence, we strive for:
Our focus is on getting reliable evidence of performance across four key and interrelated dimensions of humanitarian performance:
We try to create a blend of questions across these categories to get the best picture of the respondents’ experience of and attitudes toward the program and organizations running it. The relative utility of each category varies for each case, depending on the nature of the program (notably whether it is an emergency or more protracted situation), the context, and the theory of change.
Ground Truth uses a range of tools for data collection and aggregation, cross checking the reliability of data collected in different ways (from face-to-face to cell phones) and employing technologies that match the operational environment.
We are developing strong partnerships with a range of local civil society organizations, telecoms providers and communications agencies.
We avoid selection bias by employing local data collectors that are independent from the organisations we evaluate, whenever it is possible to do so. Our data collection strategies are informed by the contexts we are working in, and we approximate random sampling as well as possible given the situation on the ground. We ensure representativeness by comparing the demographics of our sample data to the demographics of the full population of interest, when such data is available. As we ask few questions, participation rates are high, which increases the representativeness of samples. As a norm we work in challenging environments where clients lack any systematic information about beneficiary attitudes prior to our involvement. It is our firm belief that, in these contexts, imperfect data is better than no data and we take data quality into account when sharing our insights and giving reccomendations.
We avoid survey fatigue by asking few questions and making sure that the cost-to-benefit ratio for survey respondents is heavily weighted on the benefit side. We encourage the agencies (through the incentives that are central to the Ground Truth design) to close the loop by responding to the concerns raised by affected populations, thereby demonstrating the value in providing feedback. If agencies do not do so, this will be reflected in future feedback, both in response rates and ultimately in ratings.
They complement one another. The Ground Truth data can be triangulated with the data agencies themselves collect through their regular M&E. Indeed, it can help in directing agency attention to issues that they may want to probe in greater depth through M&E. Thus, Ground Truth data supplements agency generated data and, hopefully, enhances results on the ground.
There are strong linkages with DRR and preparedness. In the longer term, we will work with national authorities, national civil society organizations and humanitarian agencies to include training modules on the Ground Truth survey methodology in their disaster preparedness curricula. This will ensure there is a cadre of people ready and able to get into the field from the start of a disaster response. Ground Truth's survey methodology will thus be hard-wired into disaster preparedness and response at the national level.
As the methodology gains traction, Ground Truth will assume stewardship of the evolving methodology and, to avoid the biases inherent in self-collected data, become an independent auditor of data quality. Meanwhile, we envisage local independent civil society groups emerging to collect affected populations data on behalf of the humanitarian agencies.
In addition to its stewardship role, Ground Truth will host a repository of survey designs and data. All humanitarian organizations can access this database as they design their surveys. They can also use it to share and compare their results. As more organizations enter data elicited using our standard questions, the more they learn from each other’s experience. Moreover, Ground Truth is already providing trainings on the use of its methodology.
Ground Truth receives generous contributions to its core budget from the Swiss Agency for Cooperation and Development (SDC) and the IKEA Foundation. Specific programs receive funding from a range of donors including DFID and ECHO. Ground Truth sees its role – stewarding the methodology, verifying data collection quality, and managing the data bank – as providing a public good for the rest of the humanitarian system and will continue to require support on a modest scale from the donor community to cover its core costs and operations.