Implementation research, conducted in conjunction with program delivery, highlights the effects of program adjustments and outcomes. Implementation research can overlap with other research and evaluation, employing the same methods and activities. This is not a problem, but rather an opportunity to leverage research and evaluation dollars across multiple objectives. This is particularly important – implementation research does NOT need to cost significantly more. In fact, when we coordinate implementation research with both program delivery and independent evaluation, all stakeholders can gain valuable insight for a nominal increase in expenditures.
The online article posted on BJM, Implementation research: what it is and how to do it, is a good primer. I like how it lays out the types of implementation research objectives along with pertinent research questions, methods, and data collection approaches as the partially captured table below illustrates. The article focuses on health and social program interventions, but just replace “health” with “energy program” to easily see the applicability to our industry.
The “Explore” objective is a concise way to describe a primary objective of embedded research and evaluation (ER&E). As discussed in earlier BIG Why blog posts, an ER&E effort is focused on a specific program challenge or question. It allows for experimentation with attributes being studied and with evaluation approaches. Learnings from ER&E inform the program in real time, facilitating much quicker changes to program design and corrective action when issues are identified. And, future evaluation is adapted based on what is learned – and what isn’t – as the ER&E effort progresses. In a nutshell, it is exploration.
In the Next Issue
The other implementation research objectives discussed in the BJM article are: describe, influence, with adequacy, with plausibility, with probability, explain, and predict. In the next few issues, I will discuss the implementation research objectives and how each can support a more insightful research and evaluation effort for energy conservation programs.
About This Blog
We are on the brink of an evaluation renaissance. Smart grids, smart meters, smart buildings, and smart data are prominent themes in the industry lexicon. Smarter evaluation and research must follow. To explore this evaluation renaissance, I am looking both inside and outside the evaluation community in a search for fresh ideas, new methods, and novel twists on old methods. I am looking to others for their thoughts and experiences for advancing the evaluation and research practice.
So, please…stay tuned, engage, and always, always question. Let’s get smarter together.