In my last several posts, I described our Embedded Research & Evaluation (ER&E) process.
Our ER&E Process: A Step-By-Step Guide, continued
Steps 1 and 2 describe how to identify an Embedded Research and Evaluation (ER&E) project opportunity, and the importance of setting up the project team with the program staff leading the effort. Steps 3, 4, and 5 discuss how to ensure researchable issues and methods align with program theory - what the program is designed to do and how it intends to do it. The steps are the same for ER&E and traditional evaluation; the difference is in the timing of the evaluation effort. Steps 6 and 7 discuss defining metrics and determining data and information needs, along with deciding how metrics, data, and information will be obtained and tracked.
ER&E occurs alongside of, and in conjunction with, program delivery. In addition to helping program staff assess program operations in real time (the evaluation component), ER&E ensures that key data and information needed to inform future program direction (design, interventions, delivery channels, marketing, etc.) are collected throughout the program offering.
Steps 8, 9, and 10 cover mapping the evaluation and research methods/activities to the research objectives and questions, assigning task owners, and setting up a schedule and check-in meetings.
Step 8. Map the evaluation and research methods/activities - and data and information collected - to the research objectives and questions. Mapping methods/activities to objectives ensures that the evaluation methods/activities they support are defined. I like to set up a matrix -- research objectives and questions as the matrix rows and methods/activities as matrix columns. Within the matrix cells, I provide brief information pertinent to the activity, such as the number of interviews or the type of analysis. This matrix also serves as a summary of the research and evaluation plan.
As the activities are completed, map data and information to the research objectives and questions in a similar manner. This helps ensure methods/activities provide what is needed, and it can highlight emerging trends or findings as activities progress.
Step 9. Assign task owners and set the schedule. Task owners are responsible for tracking progress on agreed metrics and tasks, and reporting to the team during check-in and ad hoc calls or meetings. I believe that combining the research and evaluation project schedule with the assigned task owners (along with those required for support or buy-in) keeps the project and staff on schedule - visible accountability.
Step 10. Schedule a regular check-in meeting. Regularly scheduled meetings ensure that the activities continue to support the research objectives – and that rogue activities do not develop. Other meetings can occur in addition to regularly scheduled calls. This is particularly important if a new pressing issue arises.
In the Next Issue
Next up is adapting the research and evaluation project as we learn from our efforts. A core principal of ER&E is that it is flexible and responsive in real- and near-real-time to changing conditions and findings.