Evaluation of the 2013–2014 Programmable and Smart Thermostat

Evaluation of the 2013–2014 Programmable and Smart Thermostat

Evaluation of the 2013–2014

Programmable and Smart

Thermostat Program

January 29, 2015

Prepared for:

Vectren Corporation

One Vectren Square

211 N.W. Riverside Drive

Evansville, Indiana 47708

This page left blank.

Prepared by:

Carlyn Aarish

Matei Perussi

Andrew Rietz

Dave Korn

Cadmus: Energy Services Division

Table of Contents

Executive Summary ....................................................................................................................................... 2

Introduction .................................................................................................................................................. 4

Evaluation Objectives and Methods ............................................................................................................. 6

Methods ................................................................................................................................................. 6

On-site Data Collection .................................................................................................................... 7

Pre-installation Survey ..................................................................................................................... 7

Space Temperature and Air Conditioner Run-time Metering ......................................................... 8

Pre-/Post- Billing Analysis ................................................................................................................ 9

Post-installation Survey ................................................................................................................. 16

Results ......................................................................................................................................................... 17

Response Rates..................................................................................................................................... 17

Energy Savings ...................................................................................................................................... 17

Results of Gas Billing Analysis: Model-Specific Evaluated Savings (Average Participant) ............. 17

Results of Electric Billing Analysis: Model-Specific Evaluated Savings (Average Participant) ....... 18

Benchmarking ................................................................................................................................ 20

Participant Temperature Settings and Behavior .................................................................................. 21

Pre-Installation Period ................................................................................................................... 21

Post-Installation Period .................................................................................................................. 23

Participant Demographics and Satisfaction Ratings ............................................................................. 36

Demographics ................................................................................................................................ 36

Satisfaction with Thermostat ......................................................................................................... 39

Conclusions ................................................................................................................................................. 41

Gas Savings..................................................................................................................................... 41

Electric Savings ............................................................................................................................... 41

Participant Satisfaction .................................................................................................................. 41

Interpreting Results ....................................................................................................................... 41

Appendix A: Pre-Installation Survey ........................................................................................................... 43

Appendix B: Air Conditioner/Heat Pump Data Collection .......................................................................... 49

Appendix C: Post-Installation Customer Surveys ........................................................................................ 54

1

Executive Summary

In 2013-2014, the Vectren Corporation (Vectren), a natural gas and electric provider, offered a thermostat program to residential customers who used manual thermostats in their homes. CLEAResult, the program administrator, worked with their subcontractor, Water and Energy Solutions, Inc. (WES) to install 300 Nest and 300 programmable thermostats in the homes of randomly selected Vectren natural gas and electric (i.e., dual-fuel) customers who previously underwent a home energy assessment

(through the Energizing Indiana Program). In addition to the new thermostats, customers received training on proper operation of their new thermostats.

WES installed the thermostats between October 14, 2013, and January 24, 2014. Figure 1 shows a map

of the thermostat installation locations by thermostat type.

Figure 1. Map of Completed Thermostat Installations for Vectren Thermostat Program

Vectren hired Cadmus to evaluate the program and determine the energy savings from the Nest thermostat over the baseline (manual thermostats) and conventional programmable thermostats.

Specifically, the objectives of the evaluation are to:

1. Evaluate the amount (therms) and percentage of gas saved on heating; and

2. Evaluate the amount (kWh) and percentage of electricity saved on cooling.

Cadmus assessed energy savings using pre- and post-installation billing data. Table 1 shows the evaluated gas savings as a percentage of heating gas usage, and Table 2 shows the evaluated electric

savings as a percentage of cooling electric usage.

2

Table 1. Nest and Programmable Thermostat Gas Savings as Percentage of Heating Gas Usage

Thermostat

Group

Group

Participant

Sample

Size

197

Pre Usage

(therms)

548

Savings

(therms)

55

Savings

(%)

10.0%

Range of

Savings

(therms)

Range of

Savings

(%)

47 to 63 8 to 11%

Nest

Control 2,611 575 -14

-2.5%

-12 to -17 -2 to -3%

Adjusted Gross 197 548 69

12.5%

60 to 77 11 to 14%

Programmable

Participant

Control

184

2,611

602

575

15

-14

2.5%

-2.5%

8 to 22 1 to 4%

-12 to -17 -2 to -3%

Adjusted Gross 184 602 30

5.0%

22 to 37 4 to 6%

Table 2. Nest and Programmable Thermostat Electric Savings as Percentage of Cooling Electric Usage

Thermostat

Group

Group

Sample

Size

Pre Usage

(kWh)

Savings

(kWh)

Savings

(%)

Range of

Savings

(kWh)

Range of

Savings

(%)

Nest

Participant

Control

191

2,714

3,080

3,001

357

-70

11.6%

206 to 508 7 to 17%

-2.3%

-18 to -122 -1 to -4%

Programmable

Adjusted Gross

Participant

Control

Adjusted Gross

191

205

2,714

205

3,080

2,537

3,001

2,537

429

273

-70

332

13.9%

270 to 589 9 to 19%

10.8%

131 to 415 5 to 16%

-2.3%

-18 to -122 -1 to -4%

13.1%

181 to 483 7 to 19%

Participants with the Nest thermostat reduced their heating gas consumption by approximately 12.5%, compared to only 5.0% for participants with a programmable thermostat. The Nest saved more gas than the programmable thermostat by keeping the average home temperature approximately 0.2 degrees lower than the homes with a programmable thermostat in the heating season, and an average of 0.7 degrees lower during the daytime on weekdays, when homes are commonly unoccupied. We assume temperature reductions in Nest homes are attributable to its Auto-Away feature, which automatically sets back the temperature when it senses no one is home.

Participants in the Nest and programmable thermostat groups reduced cooling electric consumption by approximately the same amount (13.9% and 13.1%, respectively). Despite nearly the same percentage savings, Nest participants had a slightly higher average air conditioner run time (1.8%) compared to programmable thermostat participants (1.2%). The baseline cooling electric usage in the Nest participant group was 21% higher than the baseline for the programmable thermostat group, so we would expect the air conditioner run time for Nest participants to be higher. We assume the higher baseline usage in the Nest participant group is attributable to the Nest participant homes having higher occupancy (and thus higher cooling loads) compared to the programmable thermostat homes (see occupancy data in Demographics section).

3

Introduction

In 1995, the U.S. Environmental Protection Agency (EPA) began promoting programmable thermostats with the ENERGY STAR

®

label. Utility companies started offering rebate programs based on claims that programmable thermostats could save 10% to 30% of residential heating and cooling energy if users programmed setbacks when the home was unoccupied or occupants were sleeping.

1

However, evaluations of these programs showed low realization rates and many studies found that only about half of users actually programmed their thermostats due to the poor user interface designs and complicated settings.

Two conditions can decrease or eliminate savings benefits from programmable thermostats. They are:

1. Some users with manual thermostats already use temperature setbacks regularly, essentially duplicating the operation of a programmable thermostat.

2. Not all users program their programmable thermostats. Some users set the thermostats at a constant temperature setpoint. Several studies have shown that consumers find programmable thermostats difficult to operate, so they often do not program the thermostat at all.

2

One study found that only 47% of programmable thermostats are actually programmed in an energy saving manner.

3

In a 2013 study, Cadmus observed both conditions (Table 3). Study participants responded to surveys

about their thermostat behavior. The portion of thermostats set to regular, scheduled setpoints does not differ much by technology, but programmable thermostats are left at a constant setpoint more often, possibly because of the difficulty of programming.

Table 3. Programmable and Manual Thermostat Behavior Patterns from 2013 Cadmus Study*

Behavior

Regular Scheduled Setpoints

Constant Setpoint

Manual With Changing Setpoints

*Totals may not sum due to rounding.

Manual Thermostats

48%

36%

16%

Programmable Thermostats

56%

14%

29%

1

U.S. Environmental Protection Agency. Summary of Research Findings from the Programmable Thermostat

Market. Memo to Manufacturers on Programmable Thermostat Specification Review. Washington, D.C. 2003.

Available online: https://www.energystar.gov/ia/partners/prod_development/revisions/downloads/thermostats/Summary.pdf

2 Nevius, M., and Pigg, S. “Programmable Thermostats That Go Berserk: Taking a Social Perspective on Space

Heating in Wisconsin.” Proceedings of the 2000 ACEEE Summer Study on Energy Efficiency in Buildings, 8.233-

238.244, 2000.

3

Meier, A., et al. (Lawrence Berkeley National Laboratory and University of California Davis). “How People

Actually Use Thermostats.” Presented at American Council for an Energy Efficient Economy proceedings,

Pacific Grove, California, August 15-20, 2010.

4

Based in part on the findings of programmable thermostat program evaluations, the EPA suspended

ENERGY STAR

®

labeling of programmable thermostats in 2009. Since then, the nation’s top thermostat manufacturers have released a new generation of Wi-Fi-enabled, smart thermostats designed with more user-friendly programming in addition to wireless control options.

In 2013-2014, Vectren, administered a thermostat program to evaluate the impact of a smart thermostat, the Nest Learning Thermostat (Nest), on energy usage compared to baseline (manual) and programmable thermostats.

The utilities chose to evaluate the Nest because of its unique features. Nest’s Auto-Away feature applies proprietary algorithms to occupancy data to determine when the home is unoccupied and activate temperature setbacks. The Auto-Schedule feature learns users’ behaviors based on how they set the thermostat and automatically programs a setback schedule. In addition, users can control the Nest remotely using a smartphone, tablet, or computer, and publishes a monthly energy report via e-mail.

The thermostat also has features useful to utility programs and evaluators: continuous communication to back-end databases of setpoints, space temperatures, and HVAC run times, among other data. The ability to monitor thermostats via the Internet also allows utilities to offer lower cost demand response programs.

The Vectren program enrolled 600 dual-fuel (gas and electric) customers with manual thermostats.

4

Customers were randomly selected from a database of customers who had received a home energy audit. These customers were assigned to two treatment groups—half received a Nest thermostat and half received a standard programmable thermostat.

Participants receiving the Nest were required to have Internet in their home so that they could use the

Wi-Fi features. The utilities chose the Honeywell TH211 to represent a conventional programmable

thermostat in this evaluation. Figure 2 shows the Honeywell TH211 and Nest thermostat installed in

participant homes.

Figure 2. Programmable (left) and Nest (right) Thermostats Installed in Program Participant Homes

4

A small percentage of participants had programmable thermostats that they operated manually

5

Evaluation Objectives and Methods

The objective of the program was to evaluate the amount (therms) and percentage of gas saved on heating and the amount (kWh) and percentage of electricity saved on cooling using a Nest compared to conventional manual and programmable thermostats.

Cadmus evaluated energy savings for three groups of customers identified as having a manual thermostat in home energy audit data.

5

1. 300 households received a Nest thermostat;

2. 300 households received a standard programmable thermostat; and

3. A control group of 3,845 households continued to use a manual thermostat (did not have a new thermostat installed as part of the study).

We compared energy savings from the Nest and programmable thermostats using a pre-/postinstallation billing analysis of participants’ energy consumption. We used the control group to determine adjusted gross savings from the Nest and programmable thermostats.

To support the energy billing analysis, we installed indoor temperature loggers and air conditioner run time loggers in half the participant homes. We used the indoor temperature data to determine average indoor temperature by hour and by day of week and categorized the patterns of use. We used the air conditioner run time data to determine average air conditioner run time by hour and day of week. We also conducted pre- and post-installation surveys to assess participant behavior and determine any changes over the study period that might eliminate the participant from the analysis.

Methods

Cadmus assessed energy savings and participant behavior using a combination of billing data, metered

data, and customer survey data. Table 4 summarizes the evaluation activities completed to collect and

analyze these data.

5

A small percentage of participants had programmable thermostats that were unprogrammed and operated as manual thermostats.

6

Table 4. Vectren Thermostat Program Evaluation Activities

Activity

Group 1:

Nest

Thermostats

Group 2:

Programmable

Thermostats

On-site data collection

Pre-installation survey

Metering ambient household space temperature

Y

Y

Y

Metering air conditioner run time

Pre- and post-installation billing analysis

Y

Y

Post-installation Survey Y

* This group allowed Cadmus to establish a base case for the billing analysis.

Y

Y

Y

Y

Y

Y

Group 3:

Control*

N

Y

N

N

N

N

On-site Data Collection

Water and Energy Solutions, Inc. (WES) completed thermostat installations in 600 Vectren dual-fuel customer homes between October 14, 2013 and January 24, 2014, providing half the homes with a Nest

thermostat and half with a standard programmable thermostat. Figure 3 shows a map of the thermostat

installation locations by thermostat type. WES followed the protocols outlined in Appendix B.

Figure 3. Map of Completed Thermostat Installations for Vectren Thermostat Program

Pre-installation Survey

At the time of installation, WES used an iPad to survey customers about how they used their old

thermostat and to collect demographic information. The survey is attached as Appendix A.

7

Space Temperature and Air Conditioner Run-time Metering

Cadmus collected space temperatures and air conditioner run times from approximately half the Nest and programmable thermostat homes. At the time of the thermostat installation, WES technicians installed an Onset UX100-003 logger next to each participant’s thermostat to record the space temperature every five minutes. WES also installed an Onset UX90-004 logger on each participant’s air conditioner condenser to record air conditioner run time.

WES installed indoor temperature meters and air conditioner run-time meters in 300 (50%) of the homes:

6

Half were installed in Nest homes and half were installed in programmable thermostat homes.

Analysis of Participant Behavior

To understand how programmable thermostat participants actually used their thermostats, we assessed space temperature data for each participant who returned a temperature logger. We noted if the participant established a programmed schedule of setbacks or used the programmable thermostat as if

it were a manual thermostat. Figure 4 and Figure 5 show example temperature data for two

participants, one in each of the two behavior categories.

Figure 4. Temperature Data for a Participant with Irregular Behavior

6

WES collected indoor temperature data so that Cadmus could review and categorize the behavior of participants, and collected air conditioner run-time data so that Cadmus could investigate any anomalous findings in the billing analysis.

8

Figure 5. Temperature Data for a Participant with Programmed Setpoints

Pre-/Post- Billing Analysis

Cadmus provided Vectren with names and addresses for the 600 program participants and 3,845 nonparticipants (control group) sampled from Energizing Indiana Home Energy Audit data. Vectren

provided the data fields outlined in Table 5 for each customer’s gas and electric bills September 2012

through September 2014.

Field

Table 5. Requested Billing Data Fields

Definition

Provided by Cadmus

Customer name

Service street address

Service city

Service zip code

Customer’s First and Last Names

Street Address

City

Zip Code

Provided by Vectren

Billing Account Number Customer’s Billing Account Number

Premise/Location

Number

Billing Days

Location Account Number (tied to the premise)

Number of Billing Days in Each Month

Monthly Usage (kwh or therms) for Each Month Usage

Read Date

Meter Read Code

Account Status

Date of Meter Reads in Each Month

Meter Read Code (indicates whether the meter reading was estimated or true)

Indicates Active, Inactive, or Closed

9

We evaluated gas savings attributable to the program by conducting a billing analysis, following these steps:

1. Matched thermostat installation dates and customer information to the billing data;

2. Used participant zip codes to map to the nearest weather station;

3. Obtained daily average temperature weather data from September 2012 through September

2014 for seven National Oceanic and Atmospheric Administration weather stations, representing all participant zip codes;

4. Used daily temperatures to determine base 45-85 heating degree days (HDDs) and cooling degree days (CDDs) for each weather station; also mapped the typical meteorological year 3

(TMY3) normal heating and cooling degree days by zip code for each home ;

7

and

5. Matched billing data periods with the CDDs and HDDs from associated stations.

Pre- and Post-installation Period Definitions

WES installed thermostats for Vectren customers between October 2013 and late January 2014.

For participants, Cadmus defined the pre-installation period as before the installation of the new thermostat, and the post-installation period as after the installation of a new thermostat. For the control group (nonparticipants), Cadmus based the control group pre- and post-installation periods on the average installation dates of the participants. We used the average participant installation date of

November 16, 2013.

Using the billing data from September 2012 through September 2014, Cadmus paired the pre- and postinstallation months to ensure that we compared the same months before and after thermostat installation.

8

Gas Billing Analysis Model

Cadmus estimated savings from each customer using a PRInceton Scorekeeping Method (PRISM) specification using pre- and post-installation billing data for each customer in the Nest group, programmable thermostat group, and control group. These models provided weather-normalized pre- and post-installation annual usage for each participant and nonparticipant.

Through this regression model approach, we obtained estimates of energy savings for each group and each customer. For each participant and control home, Cadmus estimated heating-only PRISM models in both the pre- and post-installation periods to weather-normalize raw billing data. Each model allows the heating reference temperature to range from 45 degrees to 85 degrees.

7

Cadmus used the PRISM models to select the best base temperature for each home.

8

In order to obtain the most reliable estimate of pre-period normalized usage, Cadmus estimated a model using all 12 pre-installation period months.

10

The PRISM model specification we used is:

𝐴𝐷𝐢 𝑖𝑑

= 𝛼 𝑖

+ 𝛽

1

𝐻𝐷𝐷 𝑖𝑑

+ πœ€ 𝑖𝑑

Where for each customer ‘i’ and month ‘t’:

ADC it



β

ο₯ i

1

HDD

it

it

= The average daily gas consumption in the pre- or post-installation program period

= The participant intercept representing the average daily base load

= The model space heating slope

= The base 45-85 average daily HDDs for the specific location

= The error term

From the above model, Cadmus computed weather-normalized annual consumption (NAC) for each heating reference temperature as follows:

𝑁𝐴𝐢 𝑖

= 𝛼 𝑖

∗ 365 + 𝛽

1

𝐿𝑅𝐻𝐷𝐷 𝑖𝑑

+ πœ€ 𝑖𝑑

Where:

NAC i



i

= The normalized annual consumption

= An intercept representing the average daily base load for each participant



i

* 365 = The annual base load consumption (non-weather sensitive)

β

1

= The heating slope (usage per HDD from the model above)

LRHDD it

= Annual, long-term HDDs of a typical month year (TMY3) in the 1991–

2005 series from the National Oceanic and Atmospheric Administration, for Evansville, Indiana

β

1

LRHDD it

= The weather-normalized, weather-sensitive annual heating usage, also known as HEATNAC

ο₯

it

= The error term

Cadmus screened and removed accounts that yielded negative heating NACs from the analysis. From the various models with correct signs on all of the parameters, we chose the best model of each participant’s pre- and post-installation periods based on that with the highest R-squared value.

11

Gas Data Screening

Cadmus screened and removed the following gas customers from the analysis:

ο‚·

Customers with less than seven pre-installation paired months or less than seven postinstallation paired months;

ο‚·

Customers that yielded total NACs less than 200 therms;

ο‚·

Customers that yielded negative heating NACs;

ο‚·

Customer bills that contained outliers, vacancies, or equipment changes; and

ο‚·

Customers whose post-installation survey responses indicated vacancies, changes in occupants, or equipment changes

Table 6, Table 7, and Table 8 present the gas attrition levels for the Nest, programmable thermostat,

and control group customers from the screening criteria above, respectively. For participants, the attrition was primarily due to insufficient paired billing data, removal of outliers, and surveys indicating changes, while the control group attrition was primarily due to insufficient paired billing data.

Table 6. Nest Thermostat Gas Account Attrition

Sample Screen

Remaining

Participants Percentage

300 100%

Dropped from Sample

Number Percentage

0 0%

Original Nest sample

Insufficient pre- and/or post-period data (less than seven pre-period and six post-period months)

246 82% 54

PRISM screens

Removal of outliers*

Surveys Indicate Changes

240

206

197

80%

69%

66%

6

34

9

Final Nest Analysis Sample 197 66% 103

* This entailed an account-level inspection of pre- and post-period usage data to assess vacancies, equipment changes, and other anomalies.

18%

2%

11%

3%

34%

Table 7. Programmable Thermostat Gas Account Attrition

Sample Screen

Remaining

Participants Percentage

300 100%

Dropped from Sample

Number

0

Percentage

0%

Original programmable thermostat sample

Insufficient pre- and/or post-period data

(less than seven pre-period and six postperiod months)

265 88% 35 12%

PRISM screens

261 87% 4 1%

Removal of outliers*

202 67% 59 20%

Surveys Indicate Changes 184 61% 18

Final Programmable Thermostat Analysis

Sample 184 61% 116

* This entailed an account-level inspection of pre- and post-period usage data to assess vacancies, equipment changes, and other anomalies.

6%

39%

12

Table 8. Control Group Thermostat Gas Account Attrition

Sample Screen

Remaining

Participants Percentage

3845 100%

Dropped from Sample

Number

0

Percentage

0%

Original Nonparticipant Sample

Insufficient pre- and/or post-period data (less than seven pre-period and six post-period months)

2851 74% 994 26%

PRISM screens

2800 73% 51 1%

Removal of outliers*

2611 68% 189 5%

Surveys Indicate Changes

Final Control Group Analysis Sample

2611

2611

68%

68%

0

1234

0%

32%

* This entailed an account-level inspection of pre- and post-period usage data to assess vacancies, equipment changes, and other anomalies.

After screening, the final gas analysis sample included 197 Nest thermostat participants (66%), 184 programmable thermostat participants (61%), and 2,611 control group customers (68%).

Electric Billing Analysis Model

Cadmus estimated savings from each customer using a PRInceton Scorekeeping Method (PRISM) specification using pre- and post-installation billing data for each customer in the Nest group, programmable thermostat group, and control group. These models provided weather-normalized pre- and post-installation annual usage for each participant and nonparticipant.

Through this regression model approach, we obtained estimates of energy savings for each group and each customer. For each participant and control home, we estimated heating-only PRISM models in both the pre- and post-installation periods to weather-normalize raw billing data. Each model allows the heating reference temperature to range from 45 degrees to 85 degrees and the cooling reference temperature to range from the heating reference temperature to 85 degrees.

The PRISM model specification we used is:

𝐴𝐷𝐢 𝑖𝑑

= 𝛼 𝑖

+ 𝛽

1

𝐻𝐷𝐷 𝑖𝑑

+ 𝛽

2

𝐢𝐷𝐷 𝑖𝑑

+ πœ€ 𝑖𝑑

Where for each customer ‘i’ and month ‘t’:

ADC it



β

i

1

HDD

it

β

2

HDD

it

ο₯ it

= The average daily electric consumption in the pre- or post-installation program period

= The participant intercept representing the average daily base load

= The model space heating slope

= The base 45-85 average daily HDDs for the specific location

= The model space cooling slope

= The base 45-85 average daily CDDs for the specific location

= The error term

13

From the above model, Cadmus computed weather-normalized annual consumption (NAC) for each heating and cooling reference temperature as follows:

𝑁𝐴𝐢 𝑖

= 𝛼 𝑖

∗ 365 + 𝛽

1

𝐿𝑅𝐻𝐷𝐷 𝑖𝑑

+ 𝛽

2

𝐿𝑅𝐢𝐷𝐷 𝑖𝑑

+ πœ€ 𝑖𝑑

Where:

NAC i

= The normalized annual consumption



i

= An intercept representing the average daily base load for each participant



i

* 365 = The annual base load consumption (non-weather sensitive)

β

1

= The heating slope (usage per HDD from the model above)

LRHDD it

= Annual, long-term HDDs of a typical month year (TMY3) in the 1991–

2005 series from the National Oceanic and Atmospheric Administration, for Evansville, Indiana

β

1

LRHDD it

= The weather-normalized, weather-sensitive annual heating usage, also known as HEATNAC

β

2

= The cooling slope (usage per CDD from the model above)

LRCDD it

= Annual, long-term CDDs of a typical month year (TMY3) in the 1991–

2005 series from the National Oceanic and Atmospheric Administration, for Evansville, Indiana

β

2

LRCDD it

= The weather-normalized, weather-sensitive annual cooling usage, also known as COOLNAC

ο₯

it

= The error term

We screened and removed from the analysis any accounts that yielded negative cooling NACs and negative base load. If a model heating slope was negative, we estimated a cooling-only PRISM model.

From the various models with correct signs on all of the parameters, we chose the best model of each participant’s pre- and post-installation periods based on the one with the highest R-squared value.

Electric Data Screening

Cadmus screened and removed the following electric customers from the analysis:

ο‚·

Customers with less than seven pre-installation paired months or less than seven postinstallation paired months;

ο‚·

Customers that yielded cooling NACs less than 100 kWh;

ο‚·

Customers that yielded negative base load NACs;

ο‚·

Customer bills that contained outliers, vacancies, or equipment changes; and

ο‚·

Customers whose post-installation survey responses indicated vacancies, changes in occupants, or equipment changes.

14

Table 9, Table 10, and Table 11 present the electric attrition levels for the Nest, programmable

thermostat, and control group customers from the screening criteria above, respectively. For participants, the attrition was primarily due to insufficient paired billing data, removal of outliers, and survey data indicating changes, while the control group attrition was primarily due to insufficient paired billing data.

Table 9. Nest Thermostat Electric Account Attrition

Sample Screen

Remaining

Participants

300

Percentage

100%

Dropped from Sample

Number

0

Percentage

0%

Original Nest sample

Insufficient pre- and/or post-period data

(less than seven pre-period and seven postperiod months)

247 82% 53

PRISM screens

Removal of outliers*

Surveys Indicate Changes

245

210

191

82%

70%

64%

2

35

19

Final Nest Analysis Sample 191 64% 109

* This entailed an account-level inspection of pre- and post-period usage data to assess vacancies, equipment changes, and other anomalies.

18%

1%

12%

6%

36%

Table 10. Programmable Thermostat Electric Account Attrition

Sample Screen

Remaining

Participants Percentage

300 100%

Dropped from Sample

Number Percentage

0 0%

Original programmable thermostat sample

Insufficient pre- and/or post-period data

(less than seven pre-period and seven postperiod months)

PRISM screens

275

269

92%

90%

25

6

Removal of outliers*

236 79% 33

Surveys Indicate Changes

Final Programmable Thermostat Analysis

Sample

205 68% 31

205 68% 95

* This entailed an account-level inspection of pre- and post-period usage data to assess vacancies, equipment changes, and other anomalies.

8%

2%

11%

10%

32%

15

Table 11. Control Group Thermostat Electric Account Attrition

Sample Screen

Remaining

Participants Percentage

3845 100%

Dropped from Sample

Number

0

Percentage

0%

Original nonparticipant sample

Insufficient pre- and/or post-period data (less than seven pre-period and seven post-period months)

3039 79% 806 21%

PRISM screens

2971 77% 68 2%

Removal of outliers*

2714 71% 257 7%

Surveys Indicate Changes

Final Control Group Analysis Sample

2714

2714

71%

71%

0

1131

0%

29%

* This entailed an account-level inspection of pre- and post-period usage data to assess vacancies, equipment changes, and other anomalies.

After screening, the final gas analysis sample included 191 Nest thermostat participants (64%), 205 programmable thermostat participants (68%), and 2,714 control group customers (71%).

Model-Specific Evaluated Savings (Average Participant)

Since the control group pre-installation period usage was not identical to the participant pre-installation usage, Cadmus used a percentage of pre-installation usage approach to obtain adjusted gross participant savings (via the following formula):

𝐴𝑑𝑗. πΊπ‘Ÿπ‘œπ‘ π‘  π‘†π‘Žπ‘£π‘–π‘›π‘”π‘  = π‘ƒπ‘Žπ‘Ÿπ‘‘π‘–π‘π‘–π‘π‘Žπ‘›π‘‘ π‘ƒπ‘Ÿπ‘’ π‘ˆπ‘ π‘Žπ‘”π‘’ (

πΆβ„Žπ‘Žπ‘›π‘”π‘’ 𝐼𝑛 π‘ƒπ‘Žπ‘Ÿπ‘‘π‘–π‘π‘–π‘π‘Žπ‘›π‘‘ π‘ˆπ‘ π‘Žπ‘”π‘’

π‘ƒπ‘Žπ‘Ÿπ‘‘π‘–π‘π‘–π‘π‘Žπ‘›π‘‘ π‘ƒπ‘Ÿπ‘’ π‘ˆπ‘ π‘Žπ‘”π‘’

πΆβ„Žπ‘Žπ‘›π‘”π‘’ 𝑖𝑛 πΆπ‘œπ‘›π‘‘π‘Ÿπ‘œπ‘™ πΊπ‘Ÿπ‘œπ‘’π‘ π‘ˆπ‘ π‘Žπ‘”π‘’

πΆπ‘œπ‘›π‘‘π‘Ÿπ‘œπ‘™ πΊπ‘Ÿπ‘œπ‘’π‘ π‘ƒπ‘Ÿπ‘’ π‘ˆπ‘ π‘Žπ‘”π‘’

)

Through this process, we obtained the percentage reduction of energy use in both the participant groups and the control group (specifically, we determined savings as a percentage of Pre-NAC,

PREHEATNAC, or PRECOOLNAC).

9

Then, we calculated the percentage reduction as the change in participant usage minus the change in control group usage. Multiplying this adjusted gross percentage reduction by the participant pre-installation period usage, we obtained the adjusted gross participant savings, effectively accounting for any differences in pre-installation period heating usage between participants and the control group.

Post-installation Survey

In July 2014, Cadmus distributed a post-installation survey by mail to collect information on participants’ behaviors and satisfaction with their new thermostat. This survey screened out any customers who added equipment, changed equipment, or showed prolonged vacancies. The survey is attached as

Appendix C.

9

For gas savings, this method was applied both in terms of total usage (NAC) and total heating usage

(HEATNAC). For electric savings, this method was applied in terms of cooling usage (COOLNAC).

16

Results

Response Rates

In July 2014, Cadmus mailed customer surveys to all 600 Vectren program participants. We also sent instructions to the 300 participants who received loggers on how to remove and mail back their loggers.

Table 12 shows the logger and survey return rates as of November 7, 2014.

Table 12. Logger and Survey Return Rates (as of November 7, 2014)

Returned Item

Temperature Logger

Count Response Rate

80%*

Motor Run-time Logger

Surveys

239

192

332

64%*

55%**

*Return rate as percentage of participants who received loggers (300 participants)

**Return rate as percentage of participants who received surveys (all 600 participants)

Of the participants who received loggers, 80% returned the temperature loggers and 64% returned the run time loggers. These response rates are lower than expected and may be due to the length of the study period. Because the loggers were in place over six months, participants may not have felt as responsible for returning them as they might in a shorter study. To increase response rates, we mailed a letter to participants in September, reminding them to return the loggers. After participants received the letters, we called them to see if they received the letter and offered to explain how to remove the loggers.

Cadmus received mail-in surveys back from 55% of participants. This response rate is higher than expected. Mail-in surveys typically have response rates of 10-15%.

Energy Savings

Results of Gas Billing Analysis: Model-Specific Evaluated Savings (Average Participant)

Table 13 shows the participant and control group changes in gas usage by thermostat type.

17

Table 13. Gas Savings as Percentage of Total Gas Usage

Thermostat

Group

Group

Sample

Size

Pre Usage

(therms)

Savings

(therms)

Savings

(%)

Range of

Savings

(therms)

47 to 63

Range of

Savings

(%)

6 to 8%

Participant 197 744 55 7.4%

Nest

Control 2,611 766 -14

-1.9%

-12 to -17 -2%

Adjusted Gross 197 744 69

9.3%

60 to 77 8 to 10%

Programmable

Participant

Control

184

2,611

778

766

15

-14

1.9%

-1.9%

8 to 22

-12 to -17

1 to 3%

-2%

Adjusted Gross 184 778 30

3.9%

22 to 37 3 to 5%

The control group increased its gas usage by approximately 2%, which might be normal year on year change. Cadmus applied the adjusted gross savings formula to determine the difference in these percentage savings. For participants, the Nest thermostats achieved adjusted average gross savings of

69 therms, with a pre-installation period usage of 744 therms. This represents a 9.3% reduction of preperiod usage. The programmable thermostats achieved adjusted gross savings of 30 therms, with a preinstallation period usage of 778 therms. This represents a 3.9% reduction of pre-installation period usage.

Cadmus also evaluated gas savings as a percentage of pre-period heating usage (Table 14).

Thermostat

Group

Nest

Programmable

Table 14. Gas Savings as Percentage of Heating Gas Usage

Pre Range of

Group

Sampl e Size

Heating

Usage

(therms)

Savings

(therms)

Savings

(%)

Savings

(therms)

Participant

Control

Adjusted Gross

Participant

Control

Adjusted Gross

197

2,611

197

184

2,611

184

548

575

548

602

575

602

55

-14

69

15

-14

30

10.0%

-2.5%

12.5%

2.5%

-2.5%

5.0%

Range of

Savings

(%)

47 to 63 8 to 11%

-12 to -17 -2 to -3%

60 to 77

11 to 14%

8 to 22 1 to 4%

-12 to -17 -2 to -3%

22 to 38

4 to 6%

The Nest thermostats saved 12.5% of heating gas usage and the programmable thermostats saved 5.0% of heating gas usage.

Results of Electric Billing Analysis: Model-Specific Evaluated Savings (Average Participant)

Table 15 shows the participant and control group changes in electric usage by thermostat type.

18

Thermostat

Group

Nest

Programmable

Table 15. Electric Savings as Percentage of Total Electric Usage

Group

Participant

Control

Adjusted Gross

Participant

Control

Adjusted Gross

Sample

Size

191

2,714

191

205

2,714

205

Pre Usage

(kWh)

10,730

10,606

10,730

9,020

10,606

9,020

Savings

(kWh)

357

-70

429

273

-70

332

Savings

(%)

Range of

Savings

(kWh)

Range of

Savings

(%)

3.3%

206 to 508

-0.7%

-18 to -122

2 to 5%

-1 to 0%

4.0%

270 to 589

3.7%

3 to 5%

3.0%

131 to 415 1 to 5%

-0.7%

-18 to -122 -1 to 0 %

181 to 483 2 to 5%

The control group increased its electric usage by approximately 1%, which might be normal year on year change. Cadmus applied the adjusted gross savings formula to determine the difference in these percentage savings. For participants, the Nest thermostats achieved adjusted average gross savings of

429 kWh, with a pre-period usage of 10,730 kWh. This represents a 4.0% reduction of pre-installation period usage. The programmable thermostats achieved adjusted gross savings of 332 kWh, with a preinstallation period usage of 9,020 kWh. This represents a 3.7% reduction of pre-installation period usage.

Cadmus also evaluated gas savings as a percentage of pre-installation period cooling usage (Table 16).

Thermostat

Group

Nest

Programmable

Table 16. Electric Savings as Percentage of Cooling Electric Usage

Range of

Group

Sample

Size

Pre Usage

(kWh)

Savings

(kWh)

Savings

(%)

Savings

(kWh)

Participant

Control

Adjusted Gross

Participant

Control

191

2,714

191

205

2,714

3,080

3,001

3,080

2,537

3,001

357

-70

429

273

-70

Range of

Savings

(%)

11.6%

206 to 508 7 to 17%

-2.3%

-18 to -122 -1 to -4%

13.9%

270 to 589 9 to 19%

10.8%

131 to 415 5 to 16%

-2.3%

-18 to -122 -1 to -4 %

Adjusted Gross 205 2,537 332

13.1%

181 to 483 7 to 19%

The control group increased cooling electric usage by approximately 2%, which might be normal year on year change. Cadmus applied the adjusted gross savings formula to determine the difference in these percentage savings. For participants, the Nest thermostats achieved adjusted average gross savings of

429 kWh, with a pre-installation period cooling electric usage of 3,080 kWh. This represents a 13.9% reduction in pre-installation period cooling electric usage. The programmable thermostats achieved adjusted gross savings of 332 kWh, with a pre-installation period cooling electric usage of 2,537 kWh.

This represents a 13.1% reduction in pre-installation period cooling electric usage.

19

Benchmarking

Table 17 shows a comparison of the gas savings results of this evaluation compared with those from other Cadmus thermostat evaluations using

pre/post billing analysis methods.

Date Location

Table 17. Summary of Cadmus Thermostat Gas Savings Study Results*

T-stat Type Original

Sample

Size

Evaluated

Sample

Size

Sample

Attrition

Rate

Original

Control

Group

Evaluated

Control

Group

Control

Group

Attrition

Rate

Savings per

Participant

(Therms)

Precision at 90%

Confidence

July 2011 Indiana Programmable 68 61 10% N/A N/A N/A 37 ±21%

July 2011 Indiana Programmable

July 2011 Indiana Programmable

September

2012

Massachusetts Ecobee Wi-Fi

July 2013

New

Hampshire

Venstar

ColorTouch

T5800

September

2014

Indiana Nest

September

2014

Indiana Programmable

283

371

86

29

300

300

20

255

334

43

23

197

184

10%

10%

50%

21%

N/A

N/A

N/A

N/A

34% 3,845 2,611

39% 3,845 2,611

N/A

N/A

N/A

N/A

N/A 43

N/A 35

N/A 86 (11%)

±21%

±21%

±31%

N/A 69 (8%) ±20%

32% 69 (9.3%) ±12%

32% 30 (3.9%) ±26%

Participant Temperature Settings and Behavior

Cadmus used participant survey responses and space temperature logger data to understand how participants set their thermostats.

Pre-Installation Period

This section describes the results of Cadmus’ temperature setting analysis and participant behavior analysis during the pre-installation period.

Temperature Settings

Cadmus used participant responses from the pre-installation customer surveys to assess heating and cooling setpoints by hour and by day of the week in the pre-installation period. These setpoints were

reported by participants; we did not verify or measure these numbers. Figure 6 shows the weekday and

weekend heating setpoints reported by participants. Figure 7 shows the weekday and weekend cooling

setpoints reported by participants.

Figure 6. Self-reported Pre-installation Heating Setpoints Using Manual Thermostat

Weekdays vs. Weekend (Weekday n=517; Weekend n=515)

21

Figure 7. Self-reported Pre-installation Cooling Setpoints Using Manual Thermostat

Weekdays vs. Weekend (Weekday n=516; Weekend n=516)

Five hundred seventeen program participants (86%) reported their baseline weekday heating setpoints and 515 (86%) reported their baseline weekend heating setpoints. From 8:00 a.m. to 11:00 a.m. and

8:00 p.m. to 9:00 p.m. on weekdays and weekends, the average reported setpoint is approximately 0.4 degrees lower than other times of day, indicating a possibly popular time for participants to use setbacks. During all hours of the day, the average reported setpoint is approximately 69.2 degrees.

A total of 516 program participants (86%) reported their baseline weekday and weekend cooling setpoints. On weekdays and weekends, the reported cooling season temperature settings were within

0.03 degrees for each hour of the day. For both weekdays and weekends, the average reported setpoint was 72.7 degrees. Based on participant responses, there is no period of the day or week where there is a significant setback.

Participant Behavior

Cadmus assessed the baseline behaviors of the participants based on their survey responses (Table 18).

22

Table 18. Self-reported Pre-installation Behavior Using Manual Thermostat*

Behavior Count Percentage

424 81% I manually change the thermostat settings using a regular daily schedule

I manually change the thermostat settings using no set schedule (depending on weather and/or home activity)

I use a single setpoint throughout each season (winter, spring, summer, fall)

75

22

14%

4%

Total 521 100%

*Totals may not sum due to rounding

A total of 521 (87%) of program participants reported how they controlled their manual thermostats prior to participating in the program. The majority of participants (95%) reported manually changing their temperature settings. Eighty-five percent of these participants (81% of total) reported manually changing the thermostat settings using a regular daily schedule. Fifteen percent (14% of total) reported manually changing the thermostat settings using no set schedule. The remaining participants (4%) reported using a single setpoint.

Cadmus compared these survey responses to research we completed with another client in 2013. The

results are summarized in Table 19.

Table 19. Comparison of Self-reported Behavior between Vectren Study and 2013 Cadmus Study*

Behavior

Manual Thermostats (2013

Cadmus Study)

Manual Thermostats

(2013-2014 Vectren Nest

Evaluation)

Regular Scheduled Setpoints

Manual With Changing Setpoints

48%

36%

81%

14%

Constant Setpoint 16%

4%

The behavior of Vectren program participants differs greatly from the behavior of the participants in the

2013 study. Vectren program participants control their thermostats with a regular schedule much more frequently and use changing setpoints or a single setpoint much less frequently. These results suggest

Vectren program participants may already practice regular setbacks and might not have as large a potential for energy savings as the population in the 2013 study.

Post-Installation Period

This section describes the results of Cadmus’ space temperature, air conditioner run time, and participant behavior analysis for the post-installation period.

Temperature Setting in Heating Season

Two hundred thirty-nine Vectren program participants (80%) returned their temperature loggers as of

November 7, 2014. Cadmus used logger data to evaluate the average heating season home temperatures by hour and by day of the week for the programmable thermostat and Nest treatment

23

groups. Figure 8 and Figure 9 show average indoor temperature in the heating season for programmable

thermostats and Nest, respectively,

Figure 8. Average Hourly Metered Indoor Temperature

During Heating Season for Programmable Thermostats (n=239)

Figure 9. Average Hourly Metered Indoor Temperature

During Heating Season for Nest Thermostats (n=239)

Programmable thermostat users have similar indoor temperatures for weekdays and weekends, while

Nest users appear to have a slight reduction in temperature from 9:00 a.m. to 6:00 p.m. on weekdays.

During this period, the temperature in Nest homes is on average 0.7 degrees cooler on weekdays than

24

on weekends. Because this is a common time period for homes to be unoccupied, we assume this is attributable to either the Nest’s Auto-Away feature, which automatically triggers a setback when it senses the home is unoccupied, or its Auto-Schedule feature, which uses data on how participants manually set their thermostat to automatically program a schedule of setbacks.

Figure 10 and Figure 11 show a comparison of programmable thermostat and Nest participant indoor

temperatures on weekdays and weekends, respectively.

Figure 10. Average Hourly Metered Indoor Temperature on Weekdays During Heating Season (n=239)

During weekdays, homes with programmable thermostats had lower nighttime temperatures compared to homes with Nest thermostats. Between the hours of 12:00 a.m. and 7:00 a.m., the average hourly temperature was 0.2 degrees lower than homes with a Nest thermostat. During the daytime, however, the homes with Nest thermostats had lower indoor temperatures compared to programmable thermostat homes. Between 9:00 a.m. and 10:00 p.m., indoor temperature was 0.5 degrees cooler in

Nest homes than in programmable thermostat homes, on average. The difference in average hourly temperature ranges from 0.2 degrees to 0.9 degrees during this period. These data suggest the Nest thermostat used the Auto-Away feature or Auto-Schedule feature to implement setbacks during daytime hours when many participants were away from home.

25

Figure 11. Average Hourly Metered Indoor Temperature on Weekends During Heating Season (n=239)

Like on weekdays, the programmable thermostat homes had slightly lower indoor temperatures at night. Similar to weekdays, the Nest homes had lower indoor temperatures compared to programmable thermostat homes during the day; however, the difference in indoor temperature during the day was not as large on weekends as it was on weekdays. Between the hours of 9:00am and 10:00pm on weekends, the average hourly indoor temperature ranges from 0.2 to 0.4 degrees cooler in Nest homes than in programmable thermostat homes. On average, the temperature was 0.3 degrees cooler in Nest homes during this period.

Figure 12 and Figure 13 compare the metered weekday and weekend temperature settings,

respectively, of programmable thermostat and Nest thermostat participants to the baseline setpoints they reported using with their manual thermostats.

26

Figure 12. Self-reported Setpoints for Manual Thermostats Compared to Metered Indoor

Temperatures for Programmable and Nest Thermostats (Weekdays)

Figure 13. Self-reported Setpoints for Manual Thermostats Compared to Metered Indoor

Temperatures for Programmable and Nest Thermostats (Weekends)

27

Compared to participants’ self-reported baseline heating setpoints, participants with Nest and programmable thermostats had lower indoor temperatures during the heating season, with the Nest participants having the lowest daytime temperatures and the programmable thermostat participants having the lowest nighttime temperatures. Homes with Nest thermostats had the biggest difference in indoor temperature compared to programmable thermostat homes between the hours of 10:00 AM and

6:00 PM on weekdays, when the average hourly temperature was 0.7 degrees lower than homes with a programmable thermostat. We assume this is attributable either to Nest’s Auto-Away feature, which automatically triggers a setback when it senses the home is unoccupied, or its Auto-Schedule feature, which uses data on how participants manually set their thermostat to automatically program a schedule of setbacks. Homes with programmable thermostats had the lowest indoor temperatures between the hours of 12:00 AM and 7:00 AM on weekdays and weekends, when the average hourly temperature was

0.2 degrees lower than homes with a Nest thermostat. On average, the homes with Nest thermostats had indoor temperatures 0.2 degrees lower than the homes with the programmable thermostats.

Cooling Season Temperature Settings

Cadmus used the indoor temperature logger data to evaluate the average indoor temperatures in the cooling season by hour and by day of the week for the programmable thermostat and Nest treatment

groups. Figure 14 and Figure 15 show average indoor temperatures for programmable and Nest

thermostats, respectively.

Figure 14. Average Hourly Metered Indoor Temperature

During Cooling Season for Programmable Thermostats (n=239)

28

Figure 15. Average Hourly Metered Indoor Temperature

During Cooling Season for Nest Thermostats (n=239)

Both programmable thermostat and Nest thermostat users have slightly cooler indoor temperatures on weekends compared to weekdays. On weekends, the indoor temperature in programmable thermostat homes is on average 0.5 degrees cooler compared to weekdays. For Nest homes, the average indoor temperature is 0.4 degrees cooler on weekends compared to weekdays. The metered indoor temperature data show that home indoor temperatures peak at approximately 5:00 p.m. or 6:00 p.m. and continue to drop until 7:00 a.m. Because air conditioner run time also drops during this period (see

Air Conditioner Run Time Analysis section), we assume the drop in indoor temperature is primarily attributable to a drop in outdoor temperature at night.

Figure 16 and Figure 17 show a comparison of programmable thermostat and Nest participant indoor

temperatures on weekdays and weekends, respectively.

29

Figure 16. Average Hourly Metered Indoor Temperature on Weekdays During Cooling Season (n=239)

Figure 17. Average Hourly Metered Indoor Temperature on Weekends During Cooling Season (n=239)

During weekdays and weekends, homes with the Nest thermostat had lower indoor temperatures than homes with programmable thermostats. The indoor temperature in Nest homes were 1.3 degrees cooler than programmable thermostat homes on weekdays and 1.2 degrees cooler than programmable thermostat homes on weekends. Indoor temperature data for both participant groups show the same profile of temperature peaks and drops, with the Nest homes consistently approximately one degree cooler than the programmable thermostat homes.

30

Figure 18 and Figure 19 compare the metered weekday and weekend indoor temperatures, respectively,

of programmable thermostat and Nest participants to the baseline behavior they reported using with their manual thermostats.

Figure 18. Self-reported Setpoints for Manual Thermostats Compared to Metered Indoor

Temperatures for Programmable and Nest Thermostats (Weekdays)

31

Figure 19. Self-reported Setpoints for Manual Thermostats Compared to Metered Indoor

Temperatures for Programmable and Nest Thermostats (Weekends)

Compared to participants’ self-reported baseline cooling setpoints, participants with Nest and programmable thermostats had higher indoor temperatures during the cooling season, with the programmable thermostat participants having the highest indoor temperatures. For both participant groups, the highest indoor temperatures occurred between the hours of 7:00 a.m. and 5:00 p.m. or 6:00 p.m. This is also the period when air conditioner run time was highest (see Air Conditioner Run Time

Analysis section), so we assume indoor temperatures begin to drop at 5:00 p.m. or 6:00 p.m. due to a drop in outdoor temperature at night. On average, the homes with programmable thermostats had indoor temperatures 1.3 degrees warmer than the homes with the Nest thermostats.

Air Conditioner Run Time in Cooling Season

Cadmus used participant air conditioner run time logger data to understand how participants used their

air conditioners. Figure 20 and Figure 21 show the average hourly metered air conditioner run time on

weekdays and weekends, respectively.

32

Figure 20. Average Hourly Metered Air Conditioner Run Time

During Cooling Season on Weekdays (n=192)

Figure 21. Average Hourly Metered Air Conditioner Run Time

During Cooling Season on Weekends (n=192)

On weekdays and weekends, homes with the Nest thermostat had slightly higher air conditioner run times compared to programmable thermostat homes (1.8% compared to 1.3% on weekdays and 1.8% compared to 1.5% on weekends). The overall average run time was 1.8% in Nest homes and 1.2% in programmable thermostat homes. The slightly higher run times in Nest homes is expected because the

Nest participant group had a 21% higher pre-installation cooling electric usage. We assume the higher pre-installation usage in the Nest participant group is attributable to the Nest participant homes having

33

higher occupancy (and thus higher cooling loads) compared to programmable thermostat homes (see occupancy data in Demographics section).

Participant Behavior

Participant behavior is a primary driving factor for achieving energy savings with thermostats. To assess participant behavior among programmable thermostat users, Cadmus evaluated how participants operated their thermostats using their survey data and metered indoor temperature data. To assess behavior among Nest participants, we evaluated participant engagement with the thermostat by looking at Wi-Fi connectivity.

Programmable Thermostat Operation

Cadmus categorized the programmable thermostat participants’ post-installation behaviors based on

their survey responses and space temperature data. Table 20 shows programmable thermostat

participant behavior based on their survey responses.

Table 20. Self-reported Programmable Thermostat Participant Behavior (Based on Survey Responses)

Participant Behavior

Manual (Baseline)

(n=521)

Programmable

(n=176)

I manually change the thermostat settings

I use a single setpoint

I rely on my thermostat to change

96%

4%

N/A*

46%

32%

22%

Total 100% 100%

*Manual thermostat users cannot rely on their thermostat to change because they cannot program schedules.

Compared to baseline (pre-installation) case, a significantly higher percentage of programmable thermostat participants reported using a single setpoint (32% compared to 4%). Based on participant responses, programmable thermostats converted approximately one-fifth of participants from manually adjusting their thermostat to programming their thermostats.

Table 21 shows participants’ categorized behavior based on temperature data compared to their survey

responses for programmable thermostat users.

Table 21. Programmable Thermostat Behavior (Based on Metered Temperature Data)

Assumed Thermostat Setting

Survey Responses

(n=176)

Temperature Logger Data

(n=125)

Rely on Thermostat Program

Override Thermostat Program

Cannot Determine

22%

78%

N/A

37%

51%

12%

Total 100% 100%

When comparing the results of the temperature data analysis and survey responses, programmable thermostat participants appear to rely on their thermostat program more than is reported. This may be

34

because some participants manually adjust their thermostat with regular setbacks, making their temperature setting profile appear like a programmed schedule. However, only 37% of participants appear to have relied on their thermostat program by the end of the study period.

Nest Participant Engagement

Cadmus also assessed Nest participants’ engagement with their thermostat using data provided by Nest

Labs. Table 22 shows the level of customer engagement of program participants with Nest thermostats

compared to the general (nonparticipant) population of Nest users in Indiana who ordered a thermostat from nest.com.

Table 22. Customer Engagement of Program Nest Population Compared to Indiana Nest Population*

Population of Nest

Users

Nests

Shipped

Nests

Connected**

Nests

Connected

(%)

Nests

Registered***

Nests

Registered

(% of

Connected)

Nests

Registered

(% of Total)

Program

Indiana****

300

N/A

249

N/A

83.0%

95.3%

185

N/A

74.3%

90.0%

61.7%

85.8%

*Data provided by Nest Labs

**Connected thermostats include all Nests that were ever connected to the internet

***Registered Nests include all Nests that were "paired" to a structure, which occurs when the customer sets up an account so they can use the app, web account, etc.

**** The Indiana Nest population “connected” rate is based on Indiana orders from nest.com. The “registered” rate is based on all Indiana connected devices.

Program participants with the Nest thermostat were less likely to connect their thermostat to the internet and register their Nest compared to the general population of Nest users in Indiana who ordered a thermostat from nest.com. Eighty-three percent of program participants connected their Nest thermostat to the internet, whereas 95% of Nest users in Indiana connected their thermostats to the internet. Readers should note that we would expect users who use the internet to order a thermostat from nest.com to be more likely to connect their thermostat to the internet.

Although the Nest’s Auto-Schedule and Auto-Away feature work without an internet connection, there are several features participants cannot use without an internet connection: the Nest’s HVAC control algorithms cannot receive the latest updates, participants cannot control their thermostat remotely using a smartphone, tablet or computer, and participants cannot receive the monthly e-mailed energy reports. Because participants who did not connect their Nest could not use these features, the program population might have less potential for energy savings than the general population of Indiana residents who purchased a Nest thermostat outside of the program. Readers should note, however, that Cadmus did not evaluate the impact of algorithm updates, remote control, or monthly energy reports on participant energy use.

35

Of the participants who did connect their thermostats to the internet, 74% of program participants registered their thermostats compared to 90% of users in Indiana with internet-connected Nests. The lower percentage of registered Nests among program participants might indicate that program participants were slightly less engaged with their thermostats than the general population of Nest users in Indiana. The reason for this disparity in engagement might be because the program was designed to offer the Nest for free; customers were not necessarily motivated to engage with their Nest on their own. The lower level of engagement in registering the thermostat could be an indicator of less engagement in using Nest’s features (such as the remote control and energy reports) and could consequently be an indicator of slightly lower potential for energy savings compared to registered thermostats. However, readers should note that our analysis of the indoor temperature profiles indicate the Auto-Away and Auto-Schedule features are the key cause of savings with Nest and these features work even if the thermostat is not connected or registered. Still, a program designed to offer incented thermostats, rather than free thermostats, could attract customers who are more likely to be engaged with their thermostat and consequently might increase energy savings potential slightly.

Participant Demographics and Satisfaction Ratings

Cadmus used participant surveys to collect demographic and satisfaction ratings from program participants. This section assesses the differences in demographics between the programmable and

Nest thermostat groups and how these might have caused the observed differences in energy savings and indoor temperatures. Evaluated demographics include participant age, occupancy, household income, and home age.

Demographics

Cadmus used the pre-installation survey to assess the demographics of the participant population.

Participant Age

Of the 583 participants who responded to the pre-installation survey, 338 (58%) provided their age.

Figure 22 shows the ages of participants as reported in the participant surveys.

36

Figure 22. Age of Participant Population by Participant Group

(n progrrammable

=179, n nest

=159, n total

=338)

Of the participants who reported their age, 46% are over 55 years of age, with 28% over the age of 65.

Based on the survey responses, the programmable thermostat group had more than three times the participants over age 65 compared to the Nest thermostat group (42% compared to 13%). Participants over the age of 65 are more likely to be retired and home on weekdays. Assuming this is true for the sample, the potential for energy savings from weekday daytime setbacks is lower in homes with participants over age 65 compared to under age 65. The loss of potential for weekday daytime savings for this demographic is greater in homes with the Nest than programmable thermostat because Nest’s

Auto-Away and Auto-Schedule features have the largest impact on savings during this period (as shown in temperature data analysis). In addition, assuming participants over age 65 are less likely to use smartphone, tablet, and computer technologies, this demographic is less likely to control Nest remotely and view monthly energy report e-mails.

Occupancy

In the pre-installation survey, we asked participants to provide the number of home occupants for each hour of the day on weekdays and weekends. Of the 583 participants who responded to the survey, 500

(86%) reported their home occupancy. Based on survey responses, there was no significant difference in occupancy during daytime versus nighttime, so we averaged the reported occupancy for each hour. The average number of occupants for any given hour on weekdays and weekends are shown by participant

group in Figure 23.

37

Figure 23. Household Occupancy by Participant Group

(n progrrammable

=

249

, n nest

=

251

)

The average number of occupants for any given hour was higher in Nest homes compared to programmable thermostat homes. On weekdays, Nest thermostat homes reported having an average of

3.4 occupants, whereas programmable thermostat homes reported having an average of 2.3 occupants.

On weekends, Nest thermostat homes reported having an average of 3.2 occupants, whereas programmable thermostat homes reported having an average of 2.1 occupants. The higher occupancy in

Nest thermostat homes could explain why the baseline cooling loads were 11% higher per square foot in

Nest homes compared to programmable thermostat homes (2.0 kWh/sqft compared to 1.8 kWh/sqft) and why the air conditioner run times were higher in Nest homes compared to programmable thermostat homes (35% compared to 25%).

Household Income

Of the 583 participants who responded to the survey, 42 (7%) reported their income. Income levels by

participant group are shown in Figure 24.

38

Figure 24. Reported Household Income by Participant Group

(n progrrammable

=23, n nest

=19, n total

=42)

Based on survey responses, the household incomes in the Nest participant group were higher compared to the programmable thermostat participant group. In the Nest participant group, 48% reported household incomes $50,000 or greater, compared to 38% of the participants in the programmable thermostat group. When interpreting these results, readers should note that only 7% of program participants reported their household income, and 21% more Nest participants reported their income compared to programmable thermostat participants (23 Nest participants compared to 19 programmable thermostat participants).

Home Age

Of the 583 participants who responded to the survey, 84% (492) reported their home age. We received similar response rates from both participant groups: 83% (249) Nest participants and 81% (243) of programmable thermostat participants reported home age. The average year of home construction in both groups was approximately the same (1962 in Nest homes and 1961 in programmable thermostat homes).

Satisfaction with Thermostat

Cadmus used the post-installation survey to assess participants’ satisfaction with their thermostats.

Figure 25 show participant satisfaction with the programmable thermostat and the Nest thermostat,

respectively, as reported in their customer surveys.

39

Figure 25. Satisfaction with Thermostat by Participant Group

(n progrrammable

=174, n nest

=142; n total

=316)

Although a similar percentage of participants in each group were satisfied with their thermostat (90% of programmable thermostat users and 94% of Nest users reported being with “satisfied” or “very satisfied”), more participants with the Nest thermostat reported being “very satisfied” (62% of Nest participants compared to 37% of programmable thermostat participants). Participants with a standard programmable thermostat were more likely than participants with a Nest thermostat to be “very dissatisfied” with their thermostat. Four percent of survey respondents with a programmable thermostat and 1% of survey respondents with a Nest thermostat reported being “very dissatisfied.”

Most of the respondents with a programmable thermostat who reported being “dissatisfied” or “very dissatisfied” cited the thermostat was difficult to use or program. Not enough Nest participants provided responses to identify any common reasons for being dissatisfied.

40

Conclusions

Gas Savings

Overall, participants with the Nest thermostat reduced their heating gas consumption by approximately

12.5%, compared to only 5.0% for those who used a standard programmable thermostat. Our findings indicate the gas savings are higher in the Nest thermostat homes due to a reduction in indoor temperature during the daytime on weekdays. On weekdays between 9:00 a.m. and 10:00 p.m., the temperatures in Nest homes was an average of 0.7 degrees lower than homes with a programmable thermostat. Because this is a common time for homes to be unoccupied, we assume the reduction in temperature during this period is attributable either to Nest’s Auto-Away feature, which automatically triggers a setback when it senses the home is unoccupied, or its Auto-Schedule feature, which uses a data on how participants manually set their thermostat to automatically program a schedule of setbacks.

The Auto-Away feature has an especially significant impact on participants who frequently override their thermostat setbacks by automatically reinstating setbacks when they go away. (Note that programmable thermostat cannot reinstate an overridden setback until the next setback period.) Based on our analysis of thermostat operation, 51-78% of programmable thermostat users override their programmed schedule. As a result, the Nest has greater potential than the programmable thermostat to capture savings during the daytime on weekdays, when many participants might leave home without turning down their thermostats.

Electric Savings

Participants in the Nest and programmable thermostat groups reduced cooling electric consumption by approximately the same amount (13.9% and 13.1%, respectively). Despite nearly the same percentage of savings, Nest participants had a slightly higher average air conditioner run time (1.8%) compared to programmable thermostat participants (1.2%). The baseline cooling electric usage in the Nest participant group was 21% higher than the baseline for the programmable thermostat group, so we would expect the air conditioner run time for Nest participants to be higher. We assume the higher preinstallation usage in the Nest participant group is attributable to the Nest participant homes having higher occupancy (and thus higher cooling loads) compared to programmable thermostat homes.

Participant Satisfaction

Participants with a Nest thermostat were more likely to report being satisfied with their thermostat than participants with a programmable thermostat. Of participants who responded to a customer survey,

90% of programmable thermostat users and 94% of Nest thermostat users reported that they were

“satisfied” or “very satisfied” with their thermostat.

Interpreting Results

When interpreting the results of this study, readers should take the following considerations into account.

41

Program Design

Depending on the design of future thermostat programs, this program might not represent an appropriate comparison of the Nest and programmable thermostat savings. This program design included professional installation of the Nest and programmable thermostats. Without a professional installer, a major advantage of the Nest thermostat is that it is designed to be easy for the user to adopt a schedule of setbacks. With the Auto-Schedule feature, the Nest automatically programs a schedule of setbacks using data on how participants manually set their thermostat. Alternatively, a standard programmable thermostat must be programmed by the user. As described in the introduction, and as shown in our analysis of thermostat operation, many users discontinue (or never start) using a programmed schedule. If future program designs do not include professional installation of the thermostats, then the Nest might yield more savings in comparison to the programmable thermostat than this study indicates.

Another characteristic of this program design that might slightly reduce the energy savings potential compared to other program designs is the offering of the thermostats for free. Because the thermostats were free, customers were not necessarily motivated to engage with their programmable or smart thermostat on their own. A program designed to offer incented thermostats, rather than free thermostats, could attract customers who are more likely to be engaged with their thermostat and consequently slightly increase energy savings potential.

Persistence of Savings

When interpreting the results of this study, readers should note that this evaluation only assessed the energy savings impact of Nest and programmable thermostats in the first year after the thermostat installation; the energy savings impact might change over time. Savings from a standard programmable thermostat might degrade over time if users override their schedules. Based on our analysis of thermostat operation, 51-78% of programmable thermostat users override their programmed schedule.

In contrast, savings from the Nest thermostat have the potential to increase over time due to the Auto-

Schedule feature learning over time and automatically scheduling setbacks, and due to automatic algorithm updates for thermostats connected to internet.

Energy Savings Potential

When comparing the energy savings potential between the Nest and programmable thermostats, readers should note that because the Nest is connected to the internet, users have the potential to participate in additional energy efficiency utility programs that programmable thermostat users cannot.

For example, two programs Nest offers to utility partners are the Rush Hour Rewards program and

Seasonal Savings program. The Rush Hour Rewards program is a demand response program that pays participants for allowing the Nest thermostat to automatically adjust their temperature settings before and during peak demand hours to reduce demand. The Seasonal Savings program tunes-up participants’ setback schedules at the beginning of each winter and summer season in an effort to ensure users maintain energy-efficient schedules. Readers should note that Cadmus has not evaluated the energy savings impact of any of Nest’s utility programs.

42

Appendix A: Pre-Installation Survey

Program Explanation

Thank you for participating in the Vectren thermostat study. The information gathered from this survey will help us evaluate your thermostat technology.

Estimated Time: 10-15 minutes

Program Awareness

1. What motivated you to participate in this study? Please check all that apply.

β–‘

Keep up with latest technology and trends

β–‘

Saving money on my energy bills

β–‘

Saving energy

β–‘

Having a thermostat that gives me more control over room temperature

β–‘

Getting a free thermostat

β–‘

Wanting to replace a broken thermostat

β–‘

Wanting to replace a poorly working thermostat

β–‘

Participating in another Vectren program (if yes, please specify below)

Vectren program (if applicable): _________________________

General Thermostat Settings

2. Which of the following best describes how you use your current thermostat?

β–‘

I manually change the settings using a regular schedule

β–‘

I manually change the settings using no regular schedule (depending on weather and/or home activity)

β–‘

I use a single setpoint throughout each season (winter, spring, summer, fall)

β–‘

Other: _____________________________________________

3. How do you plan to use your new thermostat?

β–‘

I plan to program my thermostat with different temperatures for different times of day

β–‘

I plan to let my thermostat learn my schedule and program itself (Nest participants only)

β–‘

Other: _____________________________________________

43

4. In general, how do you decide what temperature to set your thermostat to? Please check all that apply.

β–‘

Based on comfort

β–‘

Based on trying to keep my utility bill low

5. Please select any supplemental heating you use:

β–‘

Electric space heater

β–‘

Gas fireplace

β–‘

Wood burning stove/fireplace

β–‘

Other supplemental heating (if applicable):___________________

β–‘

N/A

Heating Season Settings

6. How do you typically set your thermostat on a weekday during the heating season?

6am-

8am

8am-

10am

10am-

12pm

12pm-

2pm

2pm-

4pm

4pm-

6pm

6pm-

8pm

8pm-

10pm

10pm-

12am

7. How do you typically set your thermostat on a weekend during the heating season?

6am-

8am

8am-

10am

10am-

12pm

12pm-

2pm

2pm-

4pm

4pm-

6pm

6pm-

8pm

8pm-

10pm

10pm-

12am

8. How do you typically set you thermostat when you are away for an extended period of time, such as for vacation, during the heating season?

β–‘

Temperature: _____

β–‘

I turn my thermostat off

12am-

6am

12am-

6am

44

β–‘

I do not adjust my thermostat when away for an extended time

Cooling Season

9. How do you typically set your thermostat on a weekday during the cooling season?

6am-

8am

8am-

10am

10am-

12pm

12pm-

2pm

2pm-

4pm

4pm-

6pm

6pm-

8pm

8pm-

10pm

10pm-

12am

10. How do you typically set your thermostat on a weekend during the cooling season?

12am-

6am

6am-

8am

8am-

10am

10am-

12pm

12pm-

2pm

2pm-

4pm

4pm-

6pm

6pm-

8pm

8pm-

10pm

10pm-

12am

11. How do you typically set your thermostat when you are away for an extended period of time, such as for vacation, during the cooling season?

12am-

6am

β–‘

Temp: _____

β–‘

Off

β–‘

I do not adjust my thermostat when away during the daytime

12. I plan to continue using the same weekday, weekend, and away thermostat settings with my new thermostat.

β–‘

True

β–‘

False

If you selected False, please describe how you plan to change your thermostat settings.

Demographics

13. How informed are you about all the ways you can save energy in your home?

β–‘

Very informed

45

β–‘

Somewhat informed

β–‘

Neither informed nor uniformed

β–‘

Somewhat uninformed

β–‘

Very uninformed

14. Do you own or rent your home?

β–‘

Own

β–‘

Rent

15. What is the approximate age of your home?

β–‘

_____ years

β–‘

Don’t know

16. How many people typically occupy your home during weekdays?

Temp

Adults

Teenagers

Children

Infants

6am-

8am

8am-

10am

10am-

12pm

12pm-

2pm

2pm-

4pm

4pm-

6pm

6pm-

8pm

8pm-

10pm

10pm-

12am

12am-

6am

17. How many people typically occupy your home during weekends?

Temp

Adults

Teenagers

Children

Infants

6am-

8am

8am-

10am

10am-

12pm

12pm-

2pm

2pm-

4pm

4pm-

6pm

6pm-

8pm

8pm-

10pm

10pm-

12am

12am-

6am

18. Which of the following best describes your total annual household income before taxes?

β–‘

Less than $15,000

β–‘

$15,000 to less than $25,000

β–‘

$25,000 to less than $35,000

β–‘

$35,000 to less than $50,000

46

β–‘

$50,000 to less than $75,000

β–‘

$75,000 to less than $100,000

β–‘

$100,000 to less than $150,000

β–‘

$150,000 or more

β–‘

I prefer not to answer this question

19. Which of the following best describes your age?

β–‘

Less than 18 years old

β–‘

18-24 years old

β–‘

25-34 years old

β–‘

35-44 years old

β–‘

45-54 years old

β–‘

55-64 years old

β–‘

65 years or older

20. Gender

β–‘

Male

β–‘

Female

Customer Satisfaction

Please select a rating to indicate your satisfaction with the following:

21. The contractor was knowledgeable about the Nest Thermostat Program.

1—Strongly Disagree 2—Disagree 3—Neutral 4—Agree 5—Strongly Agree

22. The contractor conducted himself/herself in a professional manner.

1—Strongly Disagree 2—Disagree 3—Neutral 4—Agree 5—Strongly Agree

23. I was satisfied with the time it took for my thermostat to be installed.

1—Strongly Disagree 2—Disagree 3—Neutral 4—Agree 5—Strongly Agree

24. My overall experience in the Vectren program was positive.

1—Strongly Disagree 2—Disagree 3—Neutral 4—Agree 5—Strongly Agree

47

25. Additional Comments:

Vectren Smart Thermostat Release Information

As a participant in VECTREN’s Smart Thermostat Program and upon completion of installation, you will be required to provide an electronic signature on the Customer Agreement of Terms & Conditions form. By signing the form, you agree to the terms and conditions detailed below. The terms and conditions are as follows:

ο‚·

Vectren reserves the right to alter or discontinue the Smart Thermostat Program and all other

Vectren rebate offers at any time without notice.

ο‚·

Programmable thermostats are limited and are available on a first-come, first-served basis.

ο‚·

Vectren does not guarantee that energy efficiency measures installed, or services provided through this program, will result in energy and cost savings.

ο‚·

Vectren reserves the right to deny or limit any request for services.

ο‚·

No warranties on product or service installations are provided by Vectren. The program provider, WES, warrants installation services and all products for defects in workmanship or materials for one (1) year following installation. Home owners should call (866) 611-5404 for service.

ο‚·

Vectren and the Program Administrator, CLEAResult, disclaim any and all liability, loss or damage, and make no guarantees related to participation in the Smart Thermostat Program, including liability arising out of the use or installation of the equipment, sharing of any energy usage and billing data with third parties, and any taxes that may be imposed as a result of participation in the program.

ο‚·

Participant agrees and consents to Vectren sharing participant’s energy usage and billing data collected during the data collection period with other third parties. Participant agrees to waive any and all liability arising out of Vectren sharing participant’s energy usage and billing data with other third parties.

Please Sign Below to Accept

_______________________

Email

__________________________

Click to Accept and Complete Survey

[The website then notified customer if they had missed any questions in the survey. If complete, the site provides a timestamp of when the survey was completed.]

48

Appendix B: Air Conditioner/Heat Pump Data Collection

This appendix outlines the types of air conditioner and heat pump data Cadmus collected to analyze savings for the Nest Thermostat Program.

Condenser

Cadmus collected the following information on the program participant condensers:

ο‚·

Information collected:

 Type (air conditioner or heat pump)

 Make

 Model number

 Serial number

 Refrigerant type (e.g., R-410A or R-22)

 Year or age (as available)

 Efficiency rating as available (SEER, EER, HSPF (for heat pump only), COP (for heat pump only)

ο‚·

Photographs (such as those shown in Figure 26) taken of:

 Condenser

 Nameplate (must be legible)

Figure 26. (Left to right): Standard 2.5-Ton Carrier Air Conditioner,

Standard 2.5-Ton Carrier Heat Pump, Nameplate of Heat Pump

Evaporator

Cadmus collected the following data of the program participant and evaporators:

ο‚·

Information collected:

 Make

 Model number

49

ο‚·

 Serial number

 Metering device (e.g., fixed orifice or TXV)

Photographs (such as that shown to the right) taken of nameplate

Air Handler

Cadmus collected the following information and photographs of the program participant air handlers:

ο‚·

Information collected:

 Make

 Model number

 Serial number

 Fan motor type (PSC or ECM)

ο‚·

Photographs (such as those shown above) taken of fan motor (where accessible)

Survey Collection

Cadmus field technicians had participant customers fill out a program survey while they were on-site installing equipment. The homeowner would fill out a survey, which was contained on an iPad tablet and took about 10 minutes.

Thermostat and HVAC Meter Installation

Data Collection

Cadmus collected heating and cooling system make and model information, as well as thermostat type and homeowner-preferred setpoints for each season. We recorded a description of the thermostat’s scheduled program (where applicable).

Types of Loggers

Cadmus installed the following types of loggers:

ο‚·

Thermostat temperature and humidity (Onset UX100-003 Temp/RH Logger, shown below)

ο‚·

Motor on/off (Onset UX90-004 State Logger, shown below)

50

Installation Procedure – Thermostat Logger

Cadmus’ installation procedure for thermostat loggers was to calibrate and launch them prior to arriving at the home. To install, Cadmus placed the thermostat temperature logger on or near each thermostat

in the home. Figure 27 shows proper placement of a thermostat logger.

Figure 27. Thermostat Logger Installed Near Programmable (left) and Nest (right) Thermostats

If Cadmus could not place the logger on top of the thermostat, we used 3M double-sided adhesive to attach it to the thermostat. We avoided using adhesive on any walls, as removal can be difficult. If

Cadmus could not place the logger on or attach it to the thermostat, or if the homeowner preferred to have it out of sight, Cadmus asked the homeowner to suggest a location that is representative of the indoor temperature controlled by the thermostat.

In order to ensure accurate data collection, Cadmus did not place any loggers in the following areas:

ο‚·

Drawers or closet

ο‚·

Near lights

51

ο‚·

Near windows

ο‚·

Near doors

ο‚·

In or near the kitchen

ο‚·

Near auxiliary heat sources, such as a unitary electric heater or fireplace, with the exception of rooms that are heated by secondary sources and do not contain the primary heating thermostat

ο‚·

In or near bathrooms

ο‚·

Near any type of electric load that generates heat (such as a TV or computer)

Cadmus recorded the following data during site visits, in addition to taking a photograph of the logger:

ο‚·

Thermostat location

ο‚·

Logger type and serial number

ο‚·

Site identification number

Installation Procedure – Air Conditioner Logger

Cadmus installed run-time loggers to record the precise time the air conditioner condensers turned on and off. The run-time data loggers recorded motor on and off conditions by sensing an alternating current magnetic field. These motor loggers are not normally weatherproof, so Cadmus placed them in weatherproof heat-sealed plastic bags. We calibrated each logger’s sensitivity (set to maximum sensitivity) and launched it prior to arriving at the home. Cadmus installed these loggers either on top of

the condenser (Figure 28) or on the conduit to the condenser (Figure 29).

Figure 28. Motor Logger on Condenser

52

Figure 29. Motor Logger on Electric Conduit to Condenser

Cadmus verified proper placement of each motor logger by noting the logger response when the motor

was running. Figure 30 shows the LED icons the logger displayed to show when the motor was on or off.

Figure 30. Logger Display

In addition to taking a photograph of each air conditioner logger, Cadmus recorded the following information during the site visit:

ο‚·

Condenser location

ο‚·

Logger type and serial number

ο‚·

Site identification number

53

Appendix C: Post-Installation Customer Surveys

Cadmus mailed post-installation customer surveys to participants on July 17, 2014. We created one version for participants who received loggers, which included instruction on removing and returning the loggers, and one version for participants who did not receive loggers. Blank copies of each survey version are below.

54

PLEASE PROVIDE YOUR FEEDBACK – PLEASE RESPOND BY JULY 25, 2014

Instructions

1. Please fill in each bubble completely.

2. Please return by July 25, 2014.

3. To return, place in included pre-paid bubble mailer with temperature sensor(s) and air conditioner logger and leave in any

USPS mailbox.

4. If your mailbox has a signal flag, you can leave package in your mailbox and raise the flag to signal pick-up.

1. What type of thermostat do you have?

ο‚‘

Honeywell

ο‚‘

Nest

2. How do you control your thermostat?

ο‚‘

I manually adjust the temperature as needed

ο‚‘

I use a single temperature setting

ο‚‘

I rely on my thermostat to change the temperature at different times of day

ο‚‘

I use a mobile app to adjust the temperature as needed (Nest owners only)

3. Did the number of occupants in your home increase or decrease since your thermostat was installed?

ο‚‘

Yes, increased (# of additional occupants:____)

ο‚‘

Yes, decreased (# of fewer occupants: ____)

ο‚‘

No

4. Since your thermostat was installed, were any new appliances or equipment installed in your home that require additional natural gas usage?

ο‚‘

Yes (Items:____________________________)

ο‚‘

No

5. Since your thermostat was installed, were any new appliances or equipment installed in your home that require additional electricity usage?

ο‚‘

Yes (Items: ____________________________)

ο‚‘

No

6. Were you away from your home during the 2013-

2014 heating season (winter months)?

ο‚‘

Yes (approximate # of days: ____)

ο‚‘

No

7. If you answered “Yes”, were you away more, less, or about the same as the previous winter (2012-2013)?

ο‚‘

More

ο‚‘

Less

ο‚‘

About the same

8. Were you away from your home during the 2014 cooling season (summer months)?

ο‚‘

Yes (approximate # of days: ____)

ο‚‘

No

9. If you answered “Yes”, were you away more, less, or about the same as the previous summer (2013)?

ο‚‘

More

ο‚‘

Less

ο‚‘

About the same

10. Other than weather, were there any other changes that occurred since your thermostat was installed that would cause your energy usage to be higher or lower than the previous year?

ο‚‘

Yes, higher

ο‚‘

Yes, lower

ο‚‘

No

ο‚‘

If Yes, describe: _________________________

______________________________________

11. How satisfied are you with your current thermostat?

ο‚‘

Very satisfied

ο‚‘

Satisfied

ο‚‘

Dissatisfied

ο‚‘

Very dissatisfied

12. If you answered “Dissatisfied” or “Very dissatisfied”, please describe why: _________________________

__________________________________________

13. How satisfied are you with the Vectren thermostat program?

ο‚‘

Very satisfied

ο‚‘

Somewhat satisfied

ο‚‘

Not too satisfied

ο‚‘

Not at all satisfied

14. If you answered “Dissatisfied” or “Very dissatisfied”, please describe why: _________________________

__________________________________________

15. Please provide us with any feedback about the program: __________________________________

__________________________________________

Thank You!

Your feedback will help to improve our programs.

To be entered in a drawing to win a

$250 gift card

please enter your contact information below.

*

Name: ____________________________________

Address: __________________________________

Phone: ___________________________________

If you have questions, please contact the Cadmus

Group at 617-673-7139.

*

Only one entry per person will be entered into the drawing. Respondents must be 18 years of age or older to complete this survey. The survey may not be forwarded to another person. The survey will remain available until Friday, August 1 st

at 12 p.m. Central Standard Time. The random drawing will be conducted by September 1 st

from among all eligible entries received. One participant will be selected to win a

$250 Visa gift card. The gift card will be delivered via US mail to the winning participant within three weeks of the drawing date. Employees of Vectren Corporation, its affiliates and subsidiaries, their immediate family members and/or those living in the same household as such employees are not eligible to win. This contest is void where prohibited.

PLEASE PROVIDE YOUR FEEDBACK – PLEASE RESPOND BY JULY 25, 2014

Instructions

1. Please fill in each bubble completely.

2. Please return by July 25, 2014.

3. To return, fold survey in thirds, seal with included sticker, and leave in any USPS mailbox. Postage is already paid.

4. If your mailbox has a signal flag, you can leave envelope in your mailbox and raise the flag to signal pick-up.

1. What type of thermostat do you have?

ο‚‘

Honeywell

ο‚‘

Nest

2. How do you control your thermostat?

ο‚‘

I manually adjust the temperature as needed

ο‚‘

I use a single temperature setting

ο‚‘

I rely on my thermostat to change the temperature at different times of day

ο‚‘

I use a mobile app to adjust the temperature as needed (Nest owners only)

3. Did the number of occupants in your home increase or decrease since your thermostat was installed?

ο‚‘

Yes, increased (# of additional occupants:____)

ο‚‘

Yes, decreased (# of fewer occupants: ____)

ο‚‘

No

4. Since your thermostat was installed, were any new appliances or equipment installed in your home that require additional natural gas usage?

ο‚‘

Yes (Items:____________________________)

ο‚‘

No

5. Since your thermostat was installed, were any new appliances or equipment installed in your home that require additional electricity usage?

ο‚‘

Yes (Items: ____________________________)

ο‚‘

No

6. Were you away from your home during the 2013-

2014 heating season (winter months)?

ο‚‘

Yes (approximate # of days: ____)

ο‚‘

No

7. If you answered “Yes”, were you away more, less, or about the same as the previous winter (2012-2013)?

ο‚‘

More

ο‚‘

Less

ο‚‘

About the same

8. Were you away from your home during the 2014 cooling season (summer months)?

ο‚‘

Yes (approximate # of days: ____)

ο‚‘

No

9. If you answered “Yes”, were you away more, less, or about the same as the previous summer (2013)?

ο‚‘

More

ο‚‘

Less

ο‚‘

About the same

10. Other than weather, were there any other changes that occurred since your thermostat was installed that would cause your energy usage to be higher or lower than the previous year?

ο‚‘

Yes, higher

ο‚‘

Yes, lower

ο‚‘

No

ο‚‘

If Yes, describe: _________________________

______________________________________

11. How satisfied are you with your current thermostat?

ο‚‘

Very satisfied

ο‚‘

Satisfied

ο‚‘

Dissatisfied

ο‚‘

Very dissatisfied

12. If you answered “Dissatisfied” or “Very dissatisfied”, please describe why: _________________________

__________________________________________

13. How satisfied are you with the Vectren thermostat program?

ο‚‘

Very satisfied

ο‚‘

Somewhat satisfied

ο‚‘

Not too satisfied

ο‚‘

Not at all satisfied

14. If you answered “Dissatisfied” or “Very dissatisfied”, please describe why: _________________________

__________________________________________

15. Please provide us with any feedback about the program: __________________________________

__________________________________________

Thank You!

Your feedback will help to improve our programs.

To be entered in a drawing to win a

$250 gift card

please enter your contact information below.

*

Name: ____________________________________

Address: __________________________________

Phone: ____________________________________

If you have questions, please contact the Cadmus

Group at 617-673-7139.

*

Only one entry per person will be entered into the drawing. Respondents must be 18 years of age or older to complete this survey. The survey may not be forwarded to another person. The survey will remain available until Friday, August 1 st

at 12 p.m. Central Standard Time. The random drawing will be conducted by September 1 st

from among all eligible entries received. One participant will be selected to win a

$250 Visa gift card. The gift card will be delivered via US mail to the winning participant within three weeks of the drawing date. Employees of Vectren Corporation, its affiliates and subsidiaries, their immediate family members and/or those living in the same household as such employees are not eligible to win. This contest is void where prohibited.

Was this manual useful for you? yes no
Thank you for your participation!

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Download PDF

advertisement