The Big Why of Evaluation

Embedded Research & Evaluation - Pivots Are Good

Written by Teresa Lutz | April 24, 2018 at 3:46 PM

Embedded research and evaluation (ER&E) is brilliant on many fronts, one of which is the ability to pivot research quickly based upon what we are finding from current ER&E efforts. And, although speed of discovery and execution is an objective with ER&E, sometimes slowing down is necessary. This new phase of research dictates a slower pace as we work with our utility clients to develop and hone the research and the methods.


Embedded Research and Evaluation In Practice: Likeliness To Convert

As part of this blog series, I am writing about our ER&E process to explore how we can get more from commercial and industrial audit programs. The initial phase of our ER&E project evaluates how we can drive more energy savings projects through our ASHRAE Level II audits. Through this process we found that determining a customer’s “likeliness to convert” an audit into a project might better direct program recruitment and engagement strategies and lead to more cost effective processes.

Customer Recruitment and Engagement

For the energy audit program in this particular ER&E study, the customer recruitment role is owned by utility staff. Although this limits our interactions on the front-end, our staff does engage with the customer during the:

  • Energy audit scheduling call
  • Energy audit
  • Post-audit report delivery
  • Post-report delivery follow-up

Each of these engagement points provides an opportunity for us to gather data and information that we can use to inform recruitment efforts and to improve the engagement process. Specifically, we want to know:

  • Customer motivation for doing an ASHRAE Level II energy audit
  • Customer pain points that we can address as we go through the audit
  • Project approval process and timing
  • Measures installed, or to be installed, as a result of this, or a prior, energy audit

Customer Motivation

Customer motivation might seem easy enough to assess. A typical uninspiring response to “Why did you decide to schedule an energy audit” is, “We had one a few years ago and are eligible for another one now.” Or, “My manager told me to schedule it.” Or, “It’s free, so why not see what we can learn.”

Sometimes, though, we get a more meaningful response like, “We participated in the audit program a few years back and as a result, we replaced several drives on one of our production lines with more efficient units. This not only saved energy, but we increased production with less downtime and reduced our O&M costs for that line. We want to see what other opportunities there are to save energy and improve processes.” Our next step is to see if these different types of responses are an indicator of the likeliness to act upon audit report recommendations. Are less thoughtful responses an indication of less interest in pursuing projects? Or, are there other, more critical indicators? If there are more critical indicators, what are they, and what’s the best way to get that information?

In the Next Issue

In the next few posts, I will share more information about our “likeliness to convert” embedded research as it unfolds.