Citizen engagement – time for a reality check
Written by Jethro Pettit, Institute of Development Studies
People living in poverty have gradually been re-cast, in the eyes of development agencies, from passive beneficiaries of aid to rights-claiming citizens who should hold their governments to account. Citizens and states must be supported to perform their roles more effectively – with citizens developing capacities to ‘engage’ and state actors becoming more responsive. In reality, existing power relations often prevent the less powerful from being active citizens. Our recent evaluation of the efforts of Swedish civil society organisations (CSOs) and their partners in Nicaragua, Pakistan and Uganda illustrates the challenges posed by, what we describe as, citizens’ "rational passivity”. Using the Reality Check Approach, we lived with the same poor and marginalised families for 3-5 days, twice, over two-year period.
Peopling policy processes? Methodological populism in the Bangladesh health and education sectors
Written by David Lewis, London School of Economics & Political Science, UK
Policy makers are frequently characterized as being out of touch with the communities they serve. But closing the “gap” between policy makers and people is not straightforward, since distancing effects are produced by a combination of geography, politics and knowledge. This article analyses the case of an experimental initiative in Bangladesh known as the “reality check” that attempted to influence policy makers in the health and education sectors by providing them with people-centered data gathered at community level. The case is analyzed as an example of “methodological populism” that combined participatory and ethnographic approaches, and as one that challenged current managerialist cultures of what can be considered as acceptable evidence for policy. The case highlights tensions between participation, populism and policy that are potentially productive but constrained by three sets of factors: (i) contestations over the status of “popular knowledge”, (ii) the need for critical “policy spaces” within policy processes in which policy makers can engage with such knowledge, and (iii) the “disruptive temporalities” within policy processes that tend to inhibit learning. Drawing on the “guarded hopefulness” of meta-modernist theory, the paper concludes that if more attention can be paid to such issues, initiatives informed by methodological populism such as the reality check could be further built upon in ways that may contribute to the humanization or “peopling” of policy processes.
Policy makers are frequently characterized as being out of touch with the communities they serve. But closing the “gap” between policy makers and people is not straightforward, since distancing effects are produced by a combination of geography, politics and knowledge. This article analyses the case of an experimental initiative in Bangladesh known as the “reality check” that attempted to influence policy makers in the health and education sectors by providing them with people-centered data gathered at community level. The case is analyzed as an example of “methodological populism” that combined participatory and ethnographic approaches, and as one that challenged current managerialist cultures of what can be considered as acceptable evidence for policy. The case highlights tensions between participation, populism and policy that are potentially productive but constrained by three sets of factors: (i) contestations over the status of “popular knowledge”, (ii) the need for critical “policy spaces” within policy processes in which policy makers can engage with such knowledge, and (iii) the “disruptive temporalities” within policy processes that tend to inhibit learning. Drawing on the “guarded hopefulness” of meta-modernist theory, the paper concludes that if more attention can be paid to such issues, initiatives informed by methodological populism such as the reality check could be further built upon in ways that may contribute to the humanization or “peopling” of policy processes.
Ethics, intimacy and distance in longitudinal, qualitative research: Experiences from Reality Check Bangladesh
Written by Malin Arvidson, Lund University
Abstract: This article contributes to debates about ethical and methodological dilemmas experienced in international development studies. It departs from a research experience based on a longitudinal study, the Reality Check Approach, that puts intimacy, immersion and consensus at its core. These concepts signify an ethically motivated approach that aims to ‘give voice’ to people living in poverty. They also describe an ideal research relationship assumed as the basis for good quality data. The article examines the difficulties encountered when faced with ambiguous meanings in people’s responses, and shortcomings of the approach. These include the combining of ethical and instrumental motivations in the research framework and ambivalent roles and conflicting ethics, highlighted in the conflictual notions of ‘giving someone space to talk’ and ‘making someone talk’.
Abstract: This article contributes to debates about ethical and methodological dilemmas experienced in international development studies. It departs from a research experience based on a longitudinal study, the Reality Check Approach, that puts intimacy, immersion and consensus at its core. These concepts signify an ethically motivated approach that aims to ‘give voice’ to people living in poverty. They also describe an ideal research relationship assumed as the basis for good quality data. The article examines the difficulties encountered when faced with ambiguous meanings in people’s responses, and shortcomings of the approach. These include the combining of ethical and instrumental motivations in the research framework and ambivalent roles and conflicting ethics, highlighted in the conflictual notions of ‘giving someone space to talk’ and ‘making someone talk’.
Can We Know Better? Reflections for DevelopmentWritten by Robert Chambers
This book is intended for all who are committed to human wellbeing and who want to make our world fairer, safer and more fulfilling for everyone, especially those who are ‘last’. It argues that to do better we need to know better. It provides evidence that what we believe we know in international development is often distorted or unbalanced by errors, myths, biases and blind spots. Undue weight has been attached to standardised methodologies such as randomized control trials, systematic reviews, and competitive bidding: these are shown to have huge transaction costs which are rarely if ever recognized in their enormity. Robert Chambers contrasts a Newtonian paradigm in which the world is seen and understood as controllable with a paradigm of complexity which recognizes that the real world of social processes and power relations is messy and unpredictable. To confront the challenges of complex and emergent realities requires a revolutionary new professionalism. This is underpinned by a new combination of canons of rigour expressed through eclectic methodological pluralism and participatory approaches which reverse and transform power relations. Promising developments include rapid innovations in participatory ICTs, participatory statistics, and the Reality Check Approach with its up-to-date and rigorously grounded insights. Fundamental to the new professionalism, in every country and context, are reflexivity, facilitation, groundtruthing, and personal mindsets, behaviour, attitudes, empathy and love. Can We Know Better? is essential reading for researchers and students of development, for policy makers and evaluators, and for all those working towards the better world of the Sustainable Development Goals. |
Monitoring and Evaluation in Health and Social Development Interpretive and Ethnographic PerspectivesEdited by Stephen Bell and Peter Aggleton
The Reality Check Approach has proven to be a powerful companion to quantitative research in the design and evaluation of programs. In Chapter 12 of the new publication Monitoring and Evaluation in Health and Social Development: Interpretive and Ethnographic Perspectives, RCA practitioner Dee Jupp discusses examples from Ghana and Nepal where RCA studies added invaluable insights to quantitative research which may have otherwise gone unnoticed or inadequately interpreted. In Nepal for example, a RCA study preceded the evaluation of a DfID-supported Rural Access Programme to help develop a theory of change constructed from people’s perspectives and to enhance the design of household surveys. Whereas the initial project theory of change believed that greater access would decrease outward migration, the RCA study showed the exact opposite - people perceived that outward migration would increase. To assist with the households survey, the RCA study noted that many people were using private providers for healthcare, which had not been initially included as a response option in the surveys. In Ghana, RCA is being used in a mixed method process as part of the Millennium Village Project. The RCA study helped substantiate baseline survey data which indicated that food expenditure was increasing at the same level as increases in total expenditure. Typically, food expenditure decreases as a proportion of total expenditure as expenditure increases, so the survey team was ready to dismiss this as a measurement error prior to listening to the RCA experience in the field. It also helped to explain why the incidence of malaria remained high despite nearly 90 percent of households surveyed as having at least one mosquito net. Basically, the nets were unused for most of the year because it was too hot to sleep with them and many people did not actually understand the connection between using the nets and malaria prevention. The Reality Check Approach was also employed as part of a multiple methods evaluation looking at 40 years of development interventions in East Nepal. While existing government statistics still had many areas of the Koshi hills area of East Nepal classified as poor, the RCA study found that this was incorrect, and that significant change and development had occurred in this area. The RCA study uncovered that the introduction of cardamom cultivation through people’s own efforts rather than external development programmes was, in fact, the main driver that had turned this region from a cash poor area to relatively cash rich. |
Methodological Considerations in Evaluating Long-Term Systems Change
Written by Andrew Koleros, Dee Jupp, Sean Kirwan, Meeta S. Pradhan, Pushkar K. Pradhan, David Seddon, and Ansu Tumbahangfe
Published by American Journal of Evaluation 1-17 aje.sagepub.com
Published by American Journal of Evaluation 1-17 aje.sagepub.com
RCA participated in a multiple methods evaluation study to look at the long-term impact of development interventions in the Koshi Hills region of eastern Nepal. Evaluations typically assess the linear linkages between specific interventions and intended program outcomes, but this is likely to attribute any changes to a particular program(s) while obscuring the wider social and economic factors contributing to changes. This evaluation hoped to limit the normative assumptions that come from a project focus by using multiple methods and by focusing on a much longer period of time not tied to any individual project. With the help of insights gathered from the RCA study in the Koshi Hills region, the evaluation narrative was based around the perspective of the people in that area (their Theory of Change). This helped to limit normative assumptions while adding rigour and explanation to the findings. The RCA study also helped uncover a key driver of significant change in the Koshi Hills region which wasn’t reflected in existing data, namely the introduction of cardamom cultivation. This was a particularly important discovery because the cardamom was introduced to the region through people’s own efforts rather than through external development programmes.
In this article in the American Journal of Evaluation, the evaluation team, including RCA practitioners Dee Jupp and Ansu Tumbahangfe, discuss the methods used in this unique long-term evaluation along with the lessons learned and implications for future studies.
RCA participated in a multiple methods evaluation study to look at the long-term impact of development interventions in the Koshi Hills region of eastern Nepal. Evaluations typically assess the linear linkages between specific interventions and intended program outcomes, but this is likely to attribute any changes to a particular program(s) while obscuring the wider social and economic factors contributing to changes. This evaluation hoped to limit the normative assumptions that come from a project focus by using multiple methods and by focusing on a much longer period of time not tied to any individual project. With the help of insights gathered from the RCA study in the Koshi Hills region, the evaluation narrative was based around the perspective of the people in that area (their Theory of Change). This helped to limit normative assumptions while adding rigour and explanation to the findings. The RCA study also helped uncover a key driver of significant change in the Koshi Hills region which wasn’t reflected in existing data, namely the introduction of cardamom cultivation. This was a particularly important discovery because the cardamom was introduced to the region through people’s own efforts rather than through external development programmes.
In this article in the American Journal of Evaluation, the evaluation team, including RCA practitioners Dee Jupp and Ansu Tumbahangfe, discuss the methods used in this unique long-term evaluation along with the lessons learned and implications for future studies.