RESEARCH STRATEGY: ELECTRONIC THERMOSTATS FORRESIDENTIAL APPLICATIONS WITH KNOWN MEASURECHARACTERISTICSPROPOSAL DATE: JUNE, 2016
INTRODUCTION
This document describes research needed to support RTF-Proven savings values for electronic line-voltage thermostats in applications where baseline thermostat, heating system, and building type are known. This measure applies to single-family and multi-family homes with either zonal-electric or zonal-electric plus DHP heat. A separate Research Strategy describes activities needed to estimate proven-worthy mixes of measure identifiers for cases where building type, heating system, or baseline thermostat are unknown.Research is needed because the RTF currently does not have sufficient data to reliably estimate average energy savings for this measure. Previous studies1 for Eugene Water and Electric Board (EWEB), Hydro Quebec, Northeast Utilities, and others provide significant insight into the measure, but these studies all have limited precision and/or applicability to our region.
RESEARCH OBJECTIVES
To obtain proven-worthy savings estimates for this measure, the RTF recommends a study based on pre/post data for a sample of zonal-electric-resistance buildings that have undergone whole-building thermostat retrofits plus a comparison sample. Here, “whole building retrofit” refers to cases where all, or nearly all, existing thermostats are replaced with electronic thermostats.The primary research objective is to estimate the average savings due to a whole-building retrofit, as a percent of baseline heating energy, with roughly 20% relative precision at the 90% confidence level. In addition to the whole-building savings rate for the full retrofit, researchers should also capture the number of thermostats replaced in each building in order to estimate
(Robinson et al, 2002) Robinson, D., H. Reichmuth, and A. West. Comfort STAT Program Evaluation Final Report. Prepared for Eugene Water and Electric Board, 2002 (Michaud et al, 2009) Michaud, N., L. Megdal, P. Baillargeon, C. Acocella. Billing Analysis & Environment that “Re-Sets” Savings for Programmable Thermostats in New Homes. Prepared for Hydro Quebec. Energy Program Evaluation Conference proceedings, Portland, 2009. (Johnson et al, 2000) Johnson, R., D. Bhagani, and S. Carlson. Measured Impact of Mechanical Thermostat Replacement. Prepared for Northeast Utilities. ACEEE proceedings, 2000.
the average savings per thermostat. The RTF expects to use SEEM models based on RBSA data to estimate base-case heating energy for different heating zones and to estimate unit energy savings (UES) by multiplying base-case heating energy by the percent savings for a wholebuilding retrofit, and dividing by the typical number of thermostats. For homes with zonal + DHP heat, the RTF expects to estimate the UES by multiplying the zonal electric UES by SEEM’s estimate of the portion of heating energy delivered by zonal-electric units.Research planners should assess whether average savings per thermostat is likely to vary significantly between single-family and multi-family homes; if building type is expected to have a large effect on unit savings, the study design should that into account.The following issues are closely related to this measure’s savings but are not objectives for this Research Strategy:
- Ductless heat pumps. Although the presence of a DHP is expected to have a strong effect on this measure’s savings, the RTF does not recommend attempting to empirically estimate this effect because it would be difficult to obtain proven-worth precision and because the DHP + zonal measure application is not expected to be cost effective.
- Baseline heating energy. The RTF does not currently need additional primary research on baseline heating energy for single-family homes with zonal-electric heat. Research described in the Research Strategy for Weatherization for Electric Resistance Multifamily Homes is expected to lead to baseline heating energy estimates that are sufficiently reliable for this measure.
- Market mix of bi-metal versus electronic thermostats. This is not required for proven worthy savings estimates at this time because recent research suggests very low market penetration of electronic thermostats. Market mix will need to be updated if new construction measure applications take off.
DATA COLLECTION AND ANALYSIS
This section sketches out a candidate data collection and analysis approach as a means of demonstrating one potential path forward and estimating the likely size of the research lift. Prior to fielding a study based on this or any other approach, research planners should independently assess the likelihood of meeting the research goals. The RTF recognizes that there may be alternative approaches that meet the research objectives defined above.The candidate research design is a pre/post heating energy analysis with a comparison group. Based on prior studies, the RTF’s a priori measure savings estimate is 5% of baseline heating energy. This is a small effect relative to the random variability in energy consumption that one typically observes from one site to the next or across time periods within a single site. Because of this, researchers must carefully consider how different analysis approaches are likely to handle different aspects of variability:
- Variability in actual savings. Actual measure savings depends on the condition of existing thermostats and occupant responses to temperature swings.
- Variability in actual baseline consumption. Typical consumption levels can vary wildly from one site to the next. (This variability is is not critical in the pre/post study design.)
- Random consumption variability. Random deviations from the consumption trend that prevails at a site during a given time period. These lead to uncertainty in regression fits
This candidate approach assumes day-level AMI data will be available for the study sample. By increasing the number of observations at each site, the AMI data is expected to decrease uncertainty in site-specific regression fits. If AMI data is not available, the RTF recommends either collecting interval data at the service panel or increasing the pre-period billing data from one year to two years and increasing the sample sizes (see Sample Design, below).2
Data collection
The treatment-group sample unit is a single building which has undergone a whole-building thermostat retrofit. The following data is collected for each building included in the treatment sample:
- Whole-house consumption data for at least 1 year pre- and post-retrofit, at one-day intervals;
- Site location (city or zip code), number and type of thermostats replaced (plus total number of thermostats present if different), display functionality of replacement thermostats (manual, digital, programmable), heating system type (should be electric zonal), and (for multi-family buildings) number of dwelling units.
- An indication of whether the building underwent program retrofits likely to have a significant effect on heating energy during the study period.For the comparison group, only consumption data, location, and other program participation are needed.
Day-level data should be cleaned to account for anomalous readings such as apparent vacation periods. In addition, the RTF recommends limiting the analysis sample to buildings that did not experience significant program interventions related to heating energy (weatherization, DHP installation, etc.) and limiting the comparison sample to sites that have heating signatures consistent with electric resistance heat. Most importantly, researchers should use data filters to minimize likely sources of bias, and data cleaning decisions should be carefully documented.
To ensure that day-level data will be available for pre-period analysis in this or other studies, AMI-equipped utilities should consider maintaining day-level consumption data repositories that routinely go back at least one year.
Analysis
Researchers believe this measure achieves savings by narrowing the thermostat’s dead band, thus enabling occupants to live comfortably at a lower average indoor temperature. Effectively, the measure alters the Delta-T between a home’s exterior and interior, but it does not affect the home’s UA. Because of this, the candidate approach is to use a two-stage analysis that fits site-specific change-point models in the first stage, and where each site’s model is specified to use a slope parameter that does not change between the pre and post periods. Each site’s2 It is not clear to the RTF that this measure’s savings can be detected with reasonable accuracy from any practical study using monthly billing data. Research planners should consider this question carefully before fielding a billing analysis study of this measure. savings estimate is then calculated based on the off-set between the site’s pre- and post retrofit heating energy trends.3
The second analysis stage estimates the average percent savings using a ratio estimator
Sample Design
Using the standard sample mean calculations and assuming a single-family coefficient of variation (CV) equal to 3.0, the sample target for 20% precision with 90% confidence is 608 usable single-family homes. This CV is based on figures published in (Johnson et al, 2000), which used 2-3 months of pre/post end-use metering. This sample estimate assumes that one year of day-level AMI data will achieve site-level precision similar 2-3 months of end use metering. Using month-level data, the EWEB study (Robinson et al, 2002) reports figures that imply roughly twice the CV found by Johnson et al. For month-level data the RTF recommends increasing the sample size by a factor of four (or accepting poorer precision).4 This recommendation is specific to the present research question and should not be generalized to other contexts.
Since the average savings is expected to be small relative to potential external effects, researchers are encouraged to include a comparison group in their study design. A comparison group can mitigate the risk of savings estimates being confounded by macroeconomic or other “tidal” population changes. For instance, (Robinson et al, 2002) detected a change in the comparison group’s NAC that was of a similar order of magnitude as the change in the treatment group NAC. To minimize the uncertainty incurred in taking the difference-indifferences, researchers should use the largest comparison group sample that is practical.
The RTF does not have a solid empirical basis for estimating the MF sample size needed to achieve the 20% precision target. The RTF recommends a sample of approximately 150 whole building retrofits of multi-family homes. This is based on a qualitative assumption that many household-level differences will smooth out at the building level. Although some savings factors vary at the household level, others (e.g., floor-plan and thermostat model) vary at the building level. As a result, the RTF recommends an MF sample size that is somewhat larger than the number of buildings needed to cover 608 individual dwelling units.
ESTIMATED COST RANGE
The RTF estimates the research described above would cost between $100,000 and $250,000. This estimate is driven by the data coordination, management, and analysis costs. It does not include any program costs.
3 Change-point models may be better than VBDD for studying this measure because the savings mechanism is a reduction in interior air temperature. This reduces the interior exterior ∆T, which VBDD may partially account for by selecting a lower heating degree base. Thus, VBDD may capture the basic effect in some combination of heating degree-bases and regression parameters. In a change-point model the change in interior temperature can be represented by parameters that are directly fit by the model, provided the pre/post models are restricted to share a common slope. See (Johnson et al, 2000).4 Based on 2-3 months of pre/post end-use metering, (Johnson et al, 2000) derived results that suggest an effective CV of 3.0 for estimated site-specific savings. This is about half the CV suggested by figures published (Robinson et al, 2002) which estimated 380 kWh in average NAC savings, and a NAC savings standard deviation of 2400 kWh. Research planners should carefully scrutinize these CV assumptions.
(< $25k) ($25k-$100k) ($100k-$250k) ($250k-$500k) ($500k-$1MM) ($1MM-$2MM) (> $2MM)
Regional Technical Forum Research Strategy Electronic Thermostat For Residential Applications With Known Measure Characteristics – Regional Technical Forum Research Strategy Electronic Thermostat For Residential Applications With Known Measure Characteristics –
[xyz-ips snippet=”download-snippet”]