When it comes to implementing a project, we are often struck with a question – will the outcomes be effective enough? Even though a project is developed keeping in mind the needs assessment, problem statement and research design focusing on specific objectives and goals, it is the unpredictability of societal processes, phenomena, and empirical findings that can shape the very outcome of a well-designed project. Therefore, to implement a robust program or initiative, various tools and methods of research can be employed. One such broad criteria of practice is outcome assessment, which refers to all those tools, techniques and methods that examine the project outcomes and its efficacy. These cross-cutting methods and tools can not only streamline a project and research but also help in familiarizing the system stakeholders with targeted outcomes of the project.
There are the various effective outcome assessment tools that can strengthen initiatives and research projects –
- RAPID Outcome Assessment (ROA): Developed as Research and Policy Development program by the Oversees Development Institute (ODI), RAPID Outcome Assessment method utilizes learning methodology to approach and outline the contribution of a particular project’s activity on a change in policy or the policy environment. Flexible and visual in nature, this tool can be used along with other evaluation tools and methods.
The process unfolds in three crucial stages. The first stage is the preparation stage which involves reviewing documents and engaging in informal conversations to draft an initial understanding of the project’s history and intended outcomes. The second stage is a workshop where stakeholders collaboratively identify the primary policy change processes. The final stage includes follow-up activities. This allows researchers to refine the narratives of change, pinpoint key policy actors and events and assess their contributions to the changes, although minor adjustments may be necessary along the way.
Figure 1: Three Stages of Outcome Mapping, Earl et al., /IDRC by ResearchtoAction.org
- Outcome Mapping: In a project, there can be many facets of results and outcomes, some of these aspects are evident, but some are concealed but waiting to be discovered. Outcome mapping unearth those overlooked gaps in research initiatives and projects. But how? In the planning stage, outcome mapping assesses the stakeholders that a particular project or initiative is targeting, the impact that project intends to create and strategic efforts to attain it. For a project that is ongoing, outcome mapping can be useful in designing and collecting details on the results of the change process in the project. This method deploys 12 steps under the categories of system mapping, which means creating contextually relevant image of system actors, their functions, roles, relationships, and perspectives. Intentional design, which refers to clarifying the objective, aim, vision and challenges of a project. Outcome and performance monitoring provides a structure for the on-going monitoring of a project’s actions and the partners’ progress for the achievement of outcomes and evaluation planning which requires processes, tools for designing an evaluation plan. The figure below represents the 12 steps of outcome mapping provided by Research to Action.
- Logic Model: A tool like logic model can measure a project outcome and enhance evaluation practice by developing the project’s narrative description and a story of origin. In fact, it is also useful in facilitating program planning and outlining implementation strategy. However, there are many ways in which one can design a template for a logic model, but overarchingly, and according to Better Evaluation, a logic model consists of a problem statement, goal, rationale and assumption, resources and related activities, outputs, and outcomes.
Therefore, logic models help with weaving a thread of interconnectedness between resources, activities, and outcomes of a project, simplifies communication by summarizing the progress of a project and enables consensus amongst various stakeholders involved in the project.
- Outcome Harvesting: According to evaluation practitioners Ricardo Wilson-Grau and Heather Britt, “outcome harvesting is a method that enables evaluators, grant makers, and managers to identify, formulate, verify, and make sense of outcomes. The method was inspired by the definition of outcome as a change in the behavior, relationships, actions, activities, policies, or practices of an individual, group, community, organization, or institution. Using Outcome Harvesting, the evaluator or harvester gleans information from reports, personal interviews, and other sources to document how a given program or initiative has contributed to outcomes. These outcomes can be positive or negative, intended or unintended, but the connection between the initiative and the outcomes should be verifiable.”
The process involved in outcome harvesting includes designing the harvest identifies the primary users of the harvest, as well as the intended uses of the same. These users, or their agents, the second step involves reviewing documentation and drafting outcome descriptions, the third step focuses on mapping out social agents and actors who have made a significant contribution in the outcomes, the harvesters approach them to collect data on how they have impacted those outcomes, in step four, harvesters pick and choose the gathered data to ensure accuracy, the fifth step is to analyze and interpret the data collected, and in the sixth and final step, harvesters suggest issues for discussions based on the findings.
- Goal Attainment Scale (GAS): Goal Attainment Scaling (GAS) method measures the outcome data from diverse contexts set out on a 5-point scale of -2 to +2. This method enables positioning of data onto a quantitative measurement scale, which is aimed at clarifying the reliable identification and qualitative goal impact measurement. This is, in fact, particularly valuable since it would strike a balance between operational and strategic calculations from the first to the highest management levels of an organization.
- Beneficiary Assessment: This approach places beneficiary at the center and analyze the value of a project or initiative based on lived experience and the perception of the person impacted. According to World Bank, this information-gathering approach focuses on evaluating the value of an activity through the lens of its primary users, emphasizing their perceptions and experiences. It adopts a qualitative methodology, aiming to derive deeper insights by integrating shared experiences with observational data. Central to this approach is the recognition and prioritization of the viewpoints of others, ensuring that their perspectives and lived realities are at the heart of the assessment. By doing so, it fosters a more inclusive and nuanced understanding of the activity’s impact, moving beyond surface-level metrics to capture the richness of individual and collective experiences.
- Theory of Change Framework: One of the most used tools, United Nations Development Assistance Framework (UNDAF) describes theory of change as “a method that explains how a given intervention, or set of interventions, is expected to lead to specific development change, drawing on a causal analysis based on available evidence. The theory of change helps to identify solutions to effectively address the causes of problems that hinder progress and guide decisions on which approach should be taken.” Moreover, the theory of change serves as a conceptual framework for monitoring, evaluation, or integrated monitoring and evaluation systems. It is useful for consolidating current evidence about a program, clarifying points of consensus or contention regarding how the program operates, and identifying gaps in evidence. This framework can be utilized for individual evaluations, cluster evaluations of projects within a program, or for synthesizing insights from various evaluations and research initiatives.
- Social Impact Assessment Framework: United Nations Environmental Program (UNEP) defines social impact assessment as a process of research, planning and the management of social change or outcomes arising from policies, initiatives, projects and various strategic decisions in society. It’s an umbrella framework which includes the evaluation of all impacts on people, communities and how they co-exist and interact with various structures and institutions like economy, polity, culture, and social surroundings. Social impact assessments are crucial for planning interventions because they assist in identifying stakeholders, promoting participation and analyzing the local context to evaluate potential impacts. Furthermore, collect baseline data, anticipate stakeholder reactions and inform decision-making concerning alternatives, site selection and mitigation strategies.
- Randomized Controlled Trails: As described by United Nations Children’s Fund (UNICEF) randomized controlled trial (RCT) is a tool to practice impact evaluation where the population impacted by the program or intervention is selected at random from the eligible population, and a control group is also chosen at random from the same eligible population. It assesses the extent to which specific, planned impacts are being achieved. In an RCT, the program or policy is viewed as an ‘intervention’ in which the elements of the program/policy are evaluated. RCT offers a very dynamic response to questions of causality, enabling evaluation practitioners and implementers to understand the outcomes of an initiative.
Most Significant Change Technique: The Most Significant Change (MSC) technique is a tool to collect, review and select stories about the significant changes in the lives of the people impacted by a project. This approach engages a diverse set of stakeholders at different levels of an organization discussing their experiences and then choosing the stories that they consider most significant. This entire process propels ongoing dialogue and learning about programs and what needs to be improved to effectively meet the needs of the community.
