RLW Analytics' innovative approaches to the evaluation of DSM and energy service programs have made the firm a leader in both the residential and commercial/industrial markets. Our cross-functional team of program design experts and research design professionals are continually adapting new solutions to each client's unique set of challenges. In response to these challenges, RLW Analytics has blended engineering and statistical methodologies in a way that has shaped and defined state of the art evaluation design for utility DSM programs.

Impact Evaluation

RLW Analytics evaluation innovations include the Model-Based Statistical Sampling (Analyze-IT) for sample design and analysis. The advancement of this theory provides the basis for developing efficient sampling plans with targeted precisions in program evaluation, thereby optimizing evaluation budgets and ensuring high levels of statistical precision. This fundamental evaluation tool has been utilized in hundreds of evaluations nationwide and has set a high standard for evaluation sampling techniques.

RLW has also led a fundamental movement towards the use of rigorous engineering assessments following a stratified sampling framework in program evaluations. Often times, the most rigorous data collection and analysis strategies, e.g., metering supported engineering studies or DOE 2 engineering modeling, is embedded within a more general data collection activity, e.g., telephone surveys. This evaluation framework has been termed the Engineering Calibration Approach (ECA). ECA is an innovative approach to develop measurement based, independent program impacts associated with energy conservation measure installations. ECA provides the analytical structure for the integration of several tiers of primary and secondary data collection through use of a stratified ratio estimation framework. Specifically, ECA has successfully overcome the challenge of containing costs associated with primary data collection, without compromising measurement accuracy or the informational content of the final analysis.

Process Evaluation

Process evaluations are directed at determining how select elements of program implementation can be modified to increase program acceptance or improve operational efficiency. RLW Analytics' approach to process evaluations is to explicitly design them to provide feedback for improving program design and delivery. Many data collection techniques are often used in the performance of a process evaluation, resulting in a need for an analytical process that examines all data available as a whole and explores the greater truths they contain. RLW believes that the success of a process evaluation rests upon the synthesis of information in a way that leads to a better understanding of the program and of the market in which it operates. RLW has expertise in many data collection techniques associated with the performance of a process evaluation, including the following:

• Review of program records, materials, tracking systems and operation manuals.
• Personal interviews, with utility staff, contractor staff, and collaborative parties.
• On-site inspections of program participants and non-participants.
• Focus groups, of customers or contractor personnel ranging from 8-10 individuals in size.
• Telephone surveys of program participants, non-participants and other market actors

Market Transformation Evaluation

In some regions of the country, regulators are shifting focus from the classical DSM impact study to explore the behavior of energy consumers, the functioning of energy-related product markets, and ultimately market transformation. Market transformation can be defined as when a program or initiative induces a lasting change in the structure of an energy product or service market or the behavior of market participants that results in greater adoption and penetration of energy-efficient technologies and practices. As a recognized leader in this area of energy research, RLW Analytics has conducted market effects and baseline assessment studies and presented papers on its research and evaluation methodologies. Our consultants can work with your team to develop a sound market transformation research program and to produce a defensible market effects measurement plan or assessment that is customized to meet your information needs.