Implementation Research, Good $ Sense

 

Implementation research, conducted in conjunction with program delivery, highlights the effects of program adjustments and outcomes. Implementation research can overlap with other research and evaluation, employing the same methods and activities. This is not a problem, but rather an opportunity to leverage research and evaluation dollars across multiple objectives. This is particularly important – implementation research does NOT need to cost significantly more. In fact, when we coordinate implementation research with both program delivery and independent evaluation, all stakeholders can gain valuable insight for a nominal increase in expenditures.

The online article posted on BJM, Implementation research: what it is and how to do it, is a good primer. I like how it lays out the types of implementation research objectives along with pertinent research questions, methods, and data collection approaches as the partially captured table below illustrates. The article focuses on health and social program interventions, but just replace “health” with  “energy program” to easily see the applicability to our industry.

18-12-04implementation research objectives

The “Explore” objective is a concise way to describe a primary objective of embedded research and evaluation (ER&E). As discussed in earlier BIG Why blog posts, an ER&E effort is focused on a specific program challenge or question. It allows for experimentation with attributes being studied and with evaluation approaches. Learnings from ER&E inform the program in real time, facilitating much quicker changes to program design and corrective action when issues are identified. And, future evaluation is adapted based on what is learned – and what isn’t – as the ER&E effort progresses. In a nutshell, it is exploration.

 

In the Next Issue

The other implementation research objectives discussed in the BJM article are: describe, influence, with adequacy, with plausibility, with probability, explain, and predict. In the next few issues, I will discuss the implementation research objectives and how each can support a more insightful research and evaluation effort for energy conservation programs.

About This Blog

We are on the brink of an evaluation renaissance. Smart grids, smart meters, smart buildings, and smart data are prominent themes in the industry lexicon. Smarter evaluation and research must follow. To explore this evaluation renaissance, I am looking both inside and outside the evaluation community in a search for fresh ideas, new methods, and novel twists on old methods. I am looking to others for their thoughts and experiences for advancing the evaluation and research practice.

So, please…stay tuned, engage, and always, always question. Let’s get smarter together.

Picture of Teresa Lutz

Teresa Lutz

Earlier in my career, I worked for a utility supporting the design and delivery of energy conservation programs through evaluation and research. At that time, I did not love the evaluation process or the evaluation community. The value of evaluation was a tough sell to my coworkers, and I agreed the evaluation process and results could be better. We wanted more timely feedback, recommendations we could implement, and insight beyond what we already knew. As a consultant, I hold those experiences close. I avoid doing ‘evaluation for evaluation’s sake’. I am fixated on figuring out the Big WHY of what we do, what works and what doesn’t. It is through knowing this that we can improve and prosper in this industry.

More posts by Teresa Lutz