Question:
You are given the full text of a scientific research paper.

    Summarize the study’s **policy implications**—how the research findings can inform policy or decision-making. Your response should:
    - Interpret what the results suggest for governments, regulators, or institutions
    - Highlight potential actions, strategies, or decisions informed by the study
    - Use policy-appropriate language without speculation beyond the study’s conclusions
    - Be clear, practical, and informative

    Write a short paragraph for the “Policy Implications” section of a policy brief.

    Scientific Research Paper:
    Articles
https://doi.org/10.1038/s41560-020-0589-6
1Marron Institute of Urban Management and Center for Urban Science & Progress, New York University, New York, NY, USA. 2Guarini Center on
Environmental, Energy and Land Use Law, NYU School of Law, New York, NY, USA. 3Department of Civil and Urban Engineering and Center for Urban
Science & Progress, New York University, New York, NY, USA. ✉e-mail: ckontokosta@nyu.edu
E
nergy use in buildings accounts for a significant proportion
of urban greenhouse gas (GHG) emissions, particularly in
high-density cities1. For example, New York City’s (NYC’s)
most recent carbon inventory estimates that building energy use
is responsible for approximately 67% of citywide emissions2. Given
the substantial contribution of the built environment to global GHG
emissions, city policymakers have made increasing building energy
efficiency a central component of long-term sustainability goals.
As new construction represents a small fraction of the building
stock of a given city in any year, city energy policies are increasingly
focused on ways to improve the efficiency of existing buildings3.
Informational energy regulations, which are premised on the idea
that an absence of data and transparency causes suboptimal invest-
ment in energy efficiency4, have become popular policy instruments
to encourage market-based interventions for energy use reductions.
More than 20 cities in the United States, including Austin, Chicago
and San Francisco, have adopted energy informational policies in
recent years, and the pace of adoption continues to increase5.
NYC has been a leader in this regard, and has implemented two
informational regulations in its efforts to reduce building energy
use and cut GHG emissions. The first, set forth in Local Law 84 of
2009 (LL84), requires property owners of large buildings to release
annual energy consumption data used to benchmark building
energy performance. The second, known as Local Law 87 (LL87),
introduced a mandatory energy audit requirement for buildings
larger than 50,000 ft2. Each covered property must conduct an audit,
also referred to as an Energy Efficiency Report, once every 10 years
and report its findings, which include detailed energy end-use infor-
mation and recommended energy conservation measures (ECMs).
Roughly 10% of regulated buildings have been required to conduct
an audit each year since 2013, and annual deadlines are randomly
assigned based on the last digit of the property’s Borough-Block-Lot
(BBL) tax parcel identifier. LL87 also requires owners to implement
certain retrocommissioning measures to ‘tune-up’ existing systems
at the time of audit, such as to ensure that light fixtures are clean and
water pumps are operating as designed6.
Early studies indicate that energy disclosure is correlated with
meaningful reductions in building energy use7–9, although recent
evidence suggests that there are differential impacts of the policy10.
Yet few, if any, studies have examined the effect of a mandatory
audit policy on energy use in commercial and multifamily resi-
dential buildings. This is an important omission: if information
gained from the required energy audits leads property owners to
invest in energy efficiency improvements that they would not other-
wise implement, policymakers may be justified in expanding audit
requirements to a broader range of properties and other cities may
consider these requirements in their carbon-reduction-policy tool-
kit. A positive effect of mandatory audits on energy efficiency may
also provide support for requiring more rigorous audits, including
the consideration of ‘deep energy retrofits’, which will be necessary
to achieve citywide 80%, and greater, carbon emissions reduction
targets. However, if mandatory audits—which can be quite costly
and time-consuming for property owners, especially of smaller
buildings11—do not meaningfully influence behaviour or invest-
ment decisions, policymakers may do well to simply adopt an
energy performance standard and allow the market to determine
the optimal way to meet the requirement. Notably, in April 2019
NYC passed a law that caps the amount of fossil-fuel based energy
that large buildings can consume without paying a fine12. This law,
which effectively established performance standards for the build-
ings it covers, may obviate the need for additional mandates.
This study seeks to inform urban energy policy decisions by
comparing energy use in properties that have performed a manda-
tory energy audit with those that have not. Specifically, we anal-
ysed energy benchmarking data collected by NYC annually under
LL84 from 2011 to 2016 to investigate whether properties that
conducted an audit exhibited greater average reductions in energy
use than similar, non-audited properties. We collected detailed
audit report data from mandatory LL87 audits conducted in 2013
and 2014 through a randomly assigned allocation process, which
resulted in an integrated sample of 3,981 buildings. The analy-
sis examines two primary building types (office and multifamily
The impact of mandatory energy audits on
building energy use
Constantine E. Kontokosta   1 ✉, Danielle Spiegel-Feld2 and Sokratis Papadopoulos3
Cities are increasingly adopting energy policies that reduce information asymmetries and knowledge gaps through data trans-
parency, including energy disclosure and mandatory audit requirements for existing buildings. Although such audits impose
non-trivial costs on building owners, their energy use impacts have not been empirically evaluated. Here we examine the effect
of a large-scale mandatory audit policy—New York City’s Local Law 87—on building energy use, using detailed audit and energy
data between 2011 and 2016 for approximately 4,000 buildings. This specific policy context, in which the compliance year is
randomly assigned, provides a unique opportunity to explore the audit effect without the self-selection bias found in studies
of voluntary audit policies. We find energy use reductions of approximately –2.5% for multifamily residential buildings and
–4.9% for office buildings. The results suggest that mandatory audits, by themselves, create an insufficient incentive to invest
in energy efficiency at the scale needed to meet citywide carbon-reduction goals.
Nature Energy | VOL 5 | April 2020 | 309–316 | www.nature.com/natureenergy
309

Articles
NATure EnergY
housing), while it controls for multiple time-invariant and time-
varying attributes to evaluate whether energy audits have dif-
ferential impacts across market segments. We also attempt to
contextualize the audit effect by disaggregating the potential
impacts of retrocommissioning activities from those attributable
to ECMs. Our results indicate postaudit energy use reductions of
2.5% for multifamily and 4.9% for office buildings compared to
non-audited properties. The magnitude of savings is found to be
consistent with what could be achieved through low-cost and no-
cost energy efficiency improvements.
Energy retrofits and the energy efficiency gap
Numerous studies have demonstrated that energy efficiency retro-
fits can substantially reduce building energy use13,14. In fact, building
science research indicates that ‘deep’ retrofits can reduce building
energy use by 50% or more15,16. Given that commercial and resi-
dential buildings are responsible for approximately 40% of the total
energy consumption in the United States17, to implement even a
fraction of these measures would make a material contribution to
governments’ GHG reduction goals. As a case in point, one study
reports that, by 2050, more than 11% of US energy use could be
saved through building retrofits alone18.
Many of the available energy efficiency retrofit measures, whether
technological or behavioural, are shown to have relatively short
payback periods, after which they are expected to confer net cost
savings19. For larger commercial or multifamily buildings, several
common ECMs, such as upgrading building controls and replacing
lighting, are estimated to have payback periods of five years or less
and, in many cases, less than one year20,21. Despite the apparent eco-
nomic case for retrofits, adoption has been slow22,23. This disconnect
between the availability of cost-effective energy efficient technolo-
gies and their diffusion and use in the marketplace is often referred
to as the ‘energy efficiency gap’4.
Scholars have long theorized that the energy efficiency gap is
due, at least in part, to information deficits11,24–26. Three distinct
types of deficits are believed to be at play. First, building own-
ers may lack information about the energy performance of their
buildings, which includes information about strategies to reduce
consumption, and thus may not be knowledgeable about energy
efficiency investment opportunities. Second, even where building
owners are so informed, they may withhold relevant information
from their tenants. Landlords may have an incentive to withhold
such information where tenants pay their own utility bills because,
in such instances, landlords will bear the cost of energy efficiency
upgrades, whereas their tenants receive the benefits through
lower utility payments27. This misalignment of incentives between
landlords and tenants is typically referred to as the ‘split incen-
tive’ problem25. Finally, the absence of energy use information in
the marketplace creates barriers to accounting for building energy
efficiency in investment or tenant and homebuyer locational deci-
sions. Labelling and rating systems, such as the US Environmental
Protection Agency’s ENERGY STAR programme and the US Green
Building Council’s Leadership in Energy and Environmental Design
rating system, are designed to overcome this knowledge gap, similar
to nutrition labelling or vehicle fuel efficiency ratings28,29. However,
these programmes, and others, are constrained by the self-selection
of participants and non-trivial flaws in data quality and the statisti-
cal methods used to calculate scores30,31.
The literature contains substantial evidence to support the exis-
tence of persistent information deficits32,33. Even more concern-
ing is how information asymmetries might impact lower-income
households, for whom a lack of knowledge about efficiency oppor-
tunities or mismatched utility subsidies and payment programmes
may hinder measures for energy cost reduction34–36. Recent stud-
ies of energy efficiency in subsidized housing found these proper-
ties to be far less efficient than similar market-rate housing, which
suggests that significant opportunities exist to improve the quality
of the low-income housing stock37,38.
Effects of energy disclosure and audit policies
Policies that require energy audits, such as LL87, and the disclosure
of energy performance information, like LL84, are designed to help
overcome information gaps39. Yet the two types of policies employ
different tactics to do so. In particular, although audit requirements
aim to improve building owners’ awareness of cost-effective retrofit
opportunities, disclosure policies seek to reduce information asym-
metries between building owners and prospective investors or renters
so that this information can factor into decision-making processes40.
A growing body of evidence suggests that disclosure policies can
generate efficiency improvements. For instance, a recent study by
Meng et al. revealed that LL84 produced a 6% reduction in energy
use intensity (EUI) in the first three years after the policy took effect,
and a 14% reduction over the first four years8. Papadopoulos et al.
also found energy use reductions over time in the buildings covered
by LL84, but identified distinct clusters of buildings that actually
increased their consumption during the same time period10. A num-
ber of other studies also demonstrate that energy information dis-
closure impacts real-estate prices, which is assumed to reflect rental
premiums, higher occupancy rates or lower operating expenses in
more efficient buildings41,42.
In contrast, relatively few scholars have examined the impact of
energy audits on energy use, and most of those who have done so
focused on voluntary audits in the industrial sector, rather than on
mandatory audits of buildings43,44. With respect to the building sec-
tor, a study of municipalities in northern Italy found that municipal
authorities were significantly more likely to make energy efficiency
improvements to public buildings after having conducted an energy
audit45. Conversely, Murphy46 found that voluntary energy audits in
private homes in the Netherlands had little, if any, impact.
The paucity of scholarship that evaluates the impact of manda-
tory building energy audits is surprising given the increasing preva-
lence of such policies across the United States. As of 2017, at least
eight American jurisdictions require building owners to conduct
energy audits in some form40. Some of these cities have released
compliance data and early results from their audit programmes, but
this information has not been formally studied. The lack of analysis
is striking given the costs associated with energy audits. In NYC, for
example, the audits required by LL87 have been estimated to cost
approximately $0.15 ft–2 (ref. 11). Applying this cost estimate to our
dataset of properties that were audited in 2013 or 2014 indicates
that an average size multifamily building (126,368 ft2) and office
building (412,430 ft2) would pay $17,691 and $57,740, respectively,
for their audits. Although these are relatively small figures when
compared to the value or net operating income of buildings of this
size, the time, cost and regulatory oversight create the possibility of
undue burdens for some building owners.
The question, then, is whether mandatory energy audits lead
to energy use reductions over time. If so, does the magnitude of
the effect justify the cost and time associated with conducting and
reporting the results of the audit? We use a two-way mixed design
analysis of variance (ANOVA) and Bayesian regression to evalu-
ate the impact of building audits on energy consumption pre- and
postaudit compared with a control group not subject to the audit
requirement. The analysis is based on energy use and audit compli-
ance data for 3,981 large multifamily and office buildings in NYC
from 2011 to 2016.
Evidence of postaudit energy use reductions
From Fig. 1, we observe a clear decrease in EUI for audited (treat-
ment) multifamily buildings in the post-audit period, when com-
pared to non-audited buildings. For office buildings, there is a
reduction in EUI for audited properties between the pre-audit and
Nature Energy | VOL 5 | April 2020 | 309–316 | www.nature.com/natureenergy
310

Articles
NATure EnergY
intervention periods, after which EUI values between the control
and treatment groups converge, with audited properties exhibit-
ing greater uncertainty. Given higher initial average EUI, office
buildings experience larger absolute decreases, on average, in
energy use over time when compared to multifamily residential
buildings. The ANOVA results, as shown in Table 1, demonstrate
statistically significant coefficients for the interaction term (time
period and intervention) for both office and multifamily buildings
at the 95 and 90% confidence levels, respectively. Although these
results suggest that audits do have an impact on energy use over
time, they represent a relatively coarse quantification of the audit
policy’s impact because they do not account for other factors that
could influence building energy use in the postaudit period. The
Bayesian regression results discussed below account for both the
effect of energy audits and the dynamic control variables that affect
building energy use over time.
In Table 2, we show that Bayesian regression model coefficient
means and the 95% highest posterior density (HPD) intervals for
NYC’s office and multifamily housing building stock. We note
that in both cases, when controlling for factors that might affect
changes in energy consumption over time, the average value of the
coefficient related to audits (that is, audited property) is negative.
Specifically, audited office buildings tend to reduce their EUI by
4.86% compared to non-audited properties, and multifamily prop-
erties are found to have a 2.47% reduction. From the intercept of the
two models, however, we notice diverging results. Although office
buildings, on average, reduced their energy consumption, multi-
family properties, on average, increased their energy consumption
during the study period. Therefore, in the multifamily housing case,
audits result in a smaller increase in energy use than would other-
wise be expected.
Focusing on the coefficient of interest for the effect of energy
audits on change in energy consumption over time, we show the
coefficient posterior distributions for multifamily and office build-
ings, respectively, in Figs. 2 and 3. The three Markov Chains con-
verged well and are stationary, which means that there are no
drifts and discrepancies in the mean and standard deviation of
the distributions. To answer the question of ‘how certain are we
that audits have a negative impact on energy consumption over
time?’, we calculate the density of the posterior distribution that
Table 1 | Mixed design ANOVA results—statistical significance
Multifamily housing
Office
Treatment (P value)
0.352
0.749
Time (P value)
0.706
0
Interaction (treatment × time)
(P value)
0.085
0.047
86
a
b
115
110
105
100
95
90
85
85
Energy use intensity (kBtu ft–2)
Energy use intensity (kBtu ft–2)
84
83
82
Control
Audit
Control
Audit
Pre
Intervention
Time period
Post
Pre
Intervention
Time period
Post
Fig. 1 | EUI distribution pre-, during and postaudit period. a,b, Multifamily housing (a) and office buildings (b). The error bars correspond to the 95%
confidence intervals. Although there is an observable difference in control and audited office buildings during the pre-audit period, the overlapping
confidence intervals make this difference not statistically significant. The t-test results in Table 3 further validate the above argument. Btu, British
thermal units.
Table 2 | Bayesian regression results demonstrate the effect of
audits on energy use
Multifamily housing
Office
Mean
coefficient
95% HPD
interval
Mean
coefficient
95%
HPD
interval
Intercept
4.27
(2.32, 6.22)
–7.81
(–19.46,
4.99)
Audited property
–2.47
(–4.45,
–0.44)
–4.86
(–10.49,
0.83)
Number of floors
–0.01
(–0.12, 0.10)
0.01
(–0.28,
0.26)
Building age
–0.02
(–0.05, 0.01)
0.07
(0, 0.15)
Gross floor area
0
(0, 0)
0
(0, 0)
Electricity as
primary fuel
source
8.28
(5.83, 10.84)
–3.58
(–9.26,
1.86)
Mean pre-audit
EUI
–0.36
(–0.39,
–0.33)
–0.14
(–0.19,
–0.09)
Property market
value
0
(–0.01, 0.01)
–0.01
(–0.04,
0.02)
Worker density
difference
–
–
0.04
(–0.10,
0.17)
Computer density
difference
–
–
0.04
(–0.09,
0.18)
Operating hours
difference
–
–
–0.04
(–0.14,
0.06)
Nature Energy | VOL 5 | April 2020 | 309–316 | www.nature.com/natureenergy
311

Articles
NATure EnergY
falls below the threshold of zero, which for multifamily housing is
P(βaudited < 0) = 0.990 and for office is P(βaudited < 0) = 0.958, where β is
the coefficient for the effect of energy audits on energy use.
In an attempt to associate the observed savings with potential
retrofit actions, we estimate from the audit report data the average
expected EUI improvement possible through recommended low-
cost ECMs (those with payback periods of less than two years) and
retrocommissioning. The expected savings from low-cost ECMs are
found to be 4.56% for multifamily and 1.87% for office buildings,
with retrocommissioning activities associated with approximately
2% savings in both building types. Based on the magnitude of the
audit-impact coefficients, these figures suggest office buildings
exhibit, on average, energy savings that are consistent with those
expected from recommended low-cost measures. For residen-
tial buildings, however, as the audit coefficient is lower than that
expected from low-cost ECM adoption, the impact of the manda-
tory audit is negligible in relation to identified savings opportunities.
Finally, we link the savings associated with energy audits to a
financial consideration that is often overlooked in the retrofit
decision: the cost burden of the audit itself. According to the US
Department of Energy, the cost of a building energy audit ranges
between $0.12 and 0.50 ft–2 (ref. 47), whereas NYC market-specific
estimates set the cost at $0.15 ft–2 (refs 11,48). Given the average
energy savings attributed to audits from the Bayesian model dis-
cussed above, combined with building fuel mix and energy cost esti-
mates from the US Environmental Protection Agency49, we find that
the average annual energy cost savings due to auditing for the NYC
building stock are $0.121 ft–2 for office and $0.038 ft–2 for residential
buildings. Therefore, especially for residential properties, the rela-
tively high payback period of the energy audit (four years or more,
on average) is an important consideration in the cost-benefit analy-
sis of mandatory audit policies.
Discussion and policy implications
Our analysis finds that buildings that conduct a mandatory audit
reduce their energy use over time more than non-audited buildings.
However, it is important to note that the magnitude of the audit
effect is consistent with, or less than, the expected savings from low-
cost ECMs and retrocommissioning. This effect is particularly lim-
ited for multifamily buildings, which exhibited percentage decreases
in EUI postaudit of approximately half those of office buildings. We
consider two explanations for the relatively modest effect of manda-
tory audits.
First, it is possible that LL87 audits are, in fact, influencing prop-
erty owners’ plans for energy efficiency improvements, but capital
cycles for investment in the relevant measures are too long to be
captured by the temporal period of our analysis. Stated otherwise, it
may be that after reviewing a required LL87 audit, a property owner
decides to eventually invest in some relatively costly recommended
improvement, such as new heating, ventilation and air conditioning
equipment, but plans to do so (or did do so) after 2016, which is the
last year for which we have EUI data in this analysis.
Notably, however, we did not find a greater difference between
energy consumption in audited and non-audited properties when
we included only those buildings that conducted audits in 2013,
rather than the original 2013 and 2014 combined treatment group.
This finding suggests that the investment cycle hypothesis may not
be a significant factor, as these buildings had an additional year to
implement audit recommendations. Of course, capital cycles may
still be too long for certain capital-intensive ECMs to be captured
during this time period. However, many ECMs with non-trivial
energy savings have relatively low first costs and/or have short pay-
back periods.
The second possible explanation is that energy audits do not
motivate property owners to invest in ECMs that they would not
otherwise pursue. There are at least three reasons why this might
be the case.
Poor audit quality. The most straightforward explanation as to why
LL87 audits may not be encouraging meaningful energy savings is
that the audit process and reports are not of sufficient quality to
reduce the uncertainty that buildings owners have regarding the cost
and energy savings of particular ECMs50. A case study of 30 commer-
cial and residential audits conducted in buildings across the United
States revealed widespread shortcomings, which included missed
ECMs and overestimated savings51. It is possible that the audits being
produced in compliance with LL87 are similarly lacking.
Insufficient economic incentives for investment. Assuming man-
datory audits are effective in identifying substantial cost-effective
ECMs, other capital improvements, such as renovating a lobby or
adding new amenities, may generate larger risk-adjusted returns on
investment. This may be driven, in part, by the perceived uncer-
tainty around energy retrofit investments, and thus inflate the
required rate of return (or hurdle rate) for energy investments when
compared with more traditional building improvements. Building
owners, then, may lack sufficient economic incentive, in the absence
of strong energy pricing signals, public incentives or regulatory
mandates, to implement energy improvements over other capital
improvements33.
It is important for local policymakers to understand whether this
is, indeed, a barrier to the implementation of recommended ECMs.
If so, cities will need to strengthen the incentives for efficiency
improvements before audit mandates can have a significant impact.
For instance, to require property owners to assess more ambitious
ECMs in their audits, as has been proposed (for example, NYC’s
One City Built to Last sustainability plan52), will not lead to actual
energy savings if property owners do not have an incentive, and the
access to capital, to invest in energy efficiency in the first place.
Motivation of property owners. A final reason as to why LL87
audits may not generate substantial energy savings relates to the
mandatory nature of the programme. Clearly, not all property own-
ers are equally interested in energy efficiency and those who are
Table 3 | Mean values and t-test results
Multifamily housing
Office
Variable (mean value)
Control
Treatment
Matched
Control
Treatment
Matched
Building age
65
66
66
75
77
77
Number of floors
11
11
11
21
24
22
Gross floor area (ft2)
133,801
146,207
132,196
410,318
470,741
371,380
Electricity as primary fuel source
0.11
0.1
0.1
0.72
0.8
0.8
Property market value ($ ft–2)
64
60.6
61.5
139.7a
165.4
144.4
Mean pre-audit EUI (kBtu ft–2)
84.7
84.5
84.1
96
101.9
90.3
aDifference significant at 95% confidence level (P value<0.05)
Nature Energy | VOL 5 | April 2020 | 309–316 | www.nature.com/natureenergy
312

Articles
NATure EnergY
most motivated will probably begin exploring energy saving oppor-
tunities voluntarily, without waiting for the obligation to conduct
an audit26,53. It can be argued that much of the reduction in energy
use over time observed in cities with energy disclosure laws can be
attributed to this self-selection by owners more sensitive to envi-
ronmental concerns or, conversely, the public perception of a lack
of interest in environmental responsibility. If this assumption holds,
then the information produced by mandatory audits should be most
useful for those owners who are not otherwise motivated to reduce
their energy consumption. Requiring property owners to devote
time and money to conduct an audit may provide little value in
shifting behaviour. Indeed, it may further discourage energy invest-
ments out of contempt for the mandatory nature of the process and
create an incentive to simply do the minimum reporting necessary
to comply with the law. These attitudinal factors could potentially
explain the discrepancy between the findings in our study, which
investigates the impact of mandatory energy audits on energy con-
sumption, and previous studies that have examined responses to
voluntary, subsidized audit programmes and found a robust adop-
tion of the recommended measures43,54.
Independent of the exact cause, if mandatory audits do not
encourage investment in ECMs proportionate to the time and cost
of conducting the audit, policymakers should consider whether a
mandatory policy is the most efficient strategy to encourage energy
efficiency in buildings. Although mandatory audits do produce
substantial amounts of data about building systems and energy
reduction potentials, which can have a significant value for policy-
making, it may be possible to collect these data through less costly
means, and complete coverage of a city’s building stock may not
be necessary to make reasonable inferences about where savings
opportunities exist. For example, a recent study demonstrates how
machine learning can be used to predict ECM recommendations
and cost-savings opportunities with a high degree of certainty based
on simple surveys of building systems55. Such ‘automated’ audits
could significantly reduce the time and cost associated with build-
ing assessments.
Conclusions and future work
Climate action is becoming an increasingly pressing priority for city
leaders. Given the substantial contribution building energy use has
on GHG emissions in urban areas, cities are focusing their efforts on
increasing the efficiency of the building sector through a range of
data-driven policy and regulatory tools. Although energy reporting
is a relatively low-cost mandate, required audits impose non-trivial
costs in terms of capital and time for building owners. The question
we explore here is whether this requirement actually advances the
goal of reducing energy consumption in buildings.
We find that, for the time period studied between 2011 and
2016, mandatory energy audits had a modest negative impact on
energy consumption in office and residential buildings in NYC,
0.40
a
b
1
0
–1
–2
–3
–4
–5
–6
–7
0
250
500
750
1,000 1,250
Sample
1,500 1,750 2,000
MC 1
MC 2
MC 3
MC 1
MC 2
MC 3
0.35
0.30
0.25
Density
0.20
0.15
0.10
0.05
0.00
–8
–6
–4
Audit coefficient
Audit coefficient
–2
0
2
Fig. 2 | Audit coefficient distribution (multifamily housing). a,b, Multifamily housing audit coefficient distribution (a) and sampled values (b) for the
three Markov Chains (MC) used to train the regression model.
0.14
0.12
0.10
0.08
Density
0.06
0.04
0.02
0.00
–15
–10
–5
Audit coefficient
0
5
0
–5
–10
–15
5
0
250
500
750
1,000
Sample
1,250
1,500
1,750
2,000
MC 1
MC 2
MC 3
MC 1
MC 2
MC 3
Audit coefficient
a
b
Fig. 3 | Audit coefficient distribution (office). a,b, Office audit coefficient distribution (a) and sampled values (b) for the three Markov Chains used to
train the regression model.
Nature Energy | VOL 5 | April 2020 | 309–316 | www.nature.com/natureenergy
313

Articles
NATure EnergY
at a magnitude consistent with the savings potential of low-cost
ECMs and retrocommissioning activities. The result reinforces the
hypothesis that audits, by themselves, provide only limited incen-
tive to invest in energy efficiency upgrades. Ultimately, building
owners remain constrained by factors that audit information alone
may not overcome, such as limited access to capital, uncertainty in
savings projections, opportunity costs and weak pricing signals in
energy markets.
To begin to address these economic and behavioural barriers,
cities need to develop a comprehensive strategy to support energy
efficiency in the building sector that starts with a foundation of data
transparency and rigorous analytics. Energy disclosure mandates
are an important first step: once the data are available, buildings can
be evaluated on their energy performance and compared with their
peers, which creates a ‘grading’ scheme that can help to shift indi-
vidual and collective decision-making56. Following this, cities can
consider performance targets and provide financial and regulatory
incentives to motivate building owners to improve their energy effi-
ciency and also ensure that regulations are in place to require poorly
performing buildings to improve when owners do not respond to
incentives. Audit requirements, then, could be used to target deep
retrofits, focusing on ECM opportunities that could achieve 30%
or greater savings, and automated or virtual audits could replace
the existing need for traditional audit mandates. Similarly, because
audit policies produce significant data on building systems and
operating characteristics—information that is useful for a range of
city agencies, but often difficult to collect–mandatory requirements
could be replaced by incentives for voluntarily reporting audit data.
Our analysis here is an initial attempt to empirically investigate
the impact of a citywide mandatory energy audit law. As we con-
tinue our research, future work will extend this analysis as addi-
tional years of data become available. A detailed survey of building
owners and managers would help to understand the motivations
for and barriers to implementing energy efficiency upgrades and
provide the needed context to develop a comprehensive city energy
strategy. A systematic evaluation of audit quality should also be con-
ducted to ensure the accuracy of audit reports, which would help
to provide owners with greater certainty on the validity of the pro-
posed recommendations. Finally, machine learning and artificial
intelligence applications should be explored more fully to under-
stand if data-driven modelling can enhance, complement or even
replace more manual processes for collecting relevant building data
and developing ECM recommendations. Together, these steps could
be used to develop evidenced-based and data-driven policies for
city climate action.
Methods
Data description. Our energy performance data (total energy use and EUI) and
audit data were collected by the NYC Department of Finance and the Department
of Buildings pursuant to LL84 and LL87. Data were provided by the NYC Mayor’s
Office of Sustainability subsequent to a data sharing request. The LL84 dataset
includes all covered buildings, which are defined as buildings with greater than
50,000 ft2 of gross floor area, that submitted energy use data in calendar years 2011
through 2016. EUI is defined as the total annual energy consumption divided by
the gross floor area of the building, and we utilized weather-normalized site EUI
to capture the direct consumption reported through utility bills adjusted for the
total number of heating degree days and cooling degree days in a given year57. In
addition to the total energy use and EUI values, the dataset contains building-
specific features, which include physical (age, gross floor area and so on) and
operational (occupancy density, weekly operating hours, conditioned spaces and so
on) characteristics.
We performed preprocessing of the LL84 dataset prior to our analysis. First, as
EUI data are self-reported, we identified and removed misreported and erroneous
entries. Specifically, we applied a logarithmic transformation to the EUI data and
filtered outlier values that fell outside the threshold of two standard deviations
from the mean58. Second, using the unique Borough Block Lot property identifier,
we merged energy and building attribute data with tax lot and zoning information
provided by the NYC’s Primary Land Use Tax Lot Output database to identify
additional building characteristics, such as assessed value. Finally, we integrated
the merged dataset with information from individual audit reports submitted to
the Department of Buildings as per LL87 requirements to identify properties that
conducted an energy audit in calendar years 2013 or 2014, and to analyse building-
specific ECM recommendations and savings potentials.
After our data processing steps, we analysed whether the audited properties
in our sample demonstrated larger percentage reductions in site EUI between the
pre- and postaudit period than those of similar buildings that did not perform an
audit during the study period. We defined the EUI percentage change for each
building as the difference between the mean EUI during the two years prior to
the audit (2011 and 2012) and the two years after the audit (2015 and 2016). We
used the two-year average to account for anomalous variations in building energy
consumption that could occur in a given year (for example, Hurricane Sandy had
a non-trivial impact on energy use in buildings in the impacted areas in 2012). We
focused on multifamily and office buildings, as the two types account for more
than 90% of the total LL84 covered properties by quantity and aggregate energy
consumption. The merged dataset contains 3,981 properties, which include 3,563
multifamily buildings and 418 office buildings.
Methodology. To assess the impact of energy audits on building EUI over time,
we split the data into three time periods: pre-audit (average EUI in years 2011 and
2012), intervention (average EUI in years 2013 and 2014) and postaudit (average
EUI in years 2015 and 2016). As a first step to test the significance of the audit
effect, we used a two-way mixed design ANOVA in which the dependent variable
is EUI, the within-group variable is time (with three levels as mentioned above)
and the between-group variable is the intervention (with two levels, which indicate
audited and non-audited buildings).
One significant limitation of this approach is that it does not account for
additional variables, besides time and intervention, that might be associated with
changes in EUI, such as changes in occupancy characteristics.
Bayesian regression. To more comprehensively examine the impact of the audit
policy, we used a Bayesian regression model to quantify the impact of energy
audits on energy reduction over time while controlling for additional time-varying
(dynamic) covariates that can influence changes in energy consumption. Bayesian
statistics and probabilistic programming allow us not only to assess the effect
of audits on energy use, but also to quantify the uncertainty of the estimated
parameters59,60. Bayesian regression is a useful alternative to frequentist methods
for policy analysis, as it provides a more intuitive, probabilistic output rather than
relying on fixed P-value thresholds. We formulate our problem as follows:
y NðβTX; σ2
yÞ
ð1Þ
The output y, which is the percentage difference in EUI between the pre- and
postaudit period, is generated from a normal distribution (N) with a mean equal
to the transpose (T) of the coefficients’ (β) matrix multiplied by the independent
variables’ (X) matrix and variance σ2
y
I
.
The frequentist approach to this problem estimates the model parameters using
the maximum likelihood estimation method as static values: ^β ¼ ðXTXÞ
1XTy
I
. This constitutes the fundamental difference between Bayesian and frequentist
statistics; Bayesian makes a statement of probability about a parameter value
given a fixed credible region ( Pðθj credibleregion Þ
I
), whereas frequentist makes
a statement of probability about the confidence interval given a fixed parameter
value ( Pð confidenceinterval jθÞ
I
)61,62. Therefore, in the context of our policy
evaluation research problem, Bayesian statistics allow us to ask and answer
the question ‘how certain are we that the effect of energy audits on energy
consumption over time is negative?’.
The independent variables we use in our model cover a range of building
characteristics, changes in occupancy and the audit treatment effect. We control
for non-varying characteristics, which include building age, size and EUI levels
prior to the audit, as well as time-varying occupancy factors, which include worker
density and operating hours, that might cause changes in the EUI over time. Note
that occupancy variables are not available for multifamily buildings, and hence
they are omitted from the residential model.
We used the No-U-Turn Sampler developed by Hoffman and Gelman63, an
efficient Markov Chain Monte Carlo algorithm, to draw 2,000 posterior samples
for the office and multifamily housing stock models. Monte Carlo is a general
approach of drawing random samples, whereas Markov Chain revolves around
the concept that the next sample to be drawn is independent of the past, based
solely on the present sample (the Markov process). We repeated the process for
three chains to assess the robustness of our estimated model parameters. Another
advantage of Bayesian regression is the ability to include prior knowledge regarding
the model parameters’ distribution. In this work, as NYC is among the first
large cities to enact a mandatory energy audit law, we do not provide any prior
information in the algorithm (an uninformative prior).
To measure the uncertainty in the model estimates, we calculated the HPD
intervals, which is the range of values that cover the maximum distribution
density under a predefined probability. Let f(x) be the density function of
the random variable X; then the 95% highest density region is the subset
Rðf 95Þ ¼
x : f ðxÞ≥f 95

I
, where P X 2 Rðf 95Þ

≥0:95
I
 (ref. 64).
Nature Energy | VOL 5 | April 2020 | 309–316 | www.nature.com/natureenergy
314

Articles
NATure EnergY
Prior to fitting the Bayesian model, we examined the treatment and control
groups to verify that the inherent variability between audited and non-audited
building characteristics is random and not attributable to some unobserved
selection bias. As the year a building must comply with LL87 and submit its audit
data is determined based on the last digit of its tax lot number, the selection and
reporting process is essentially random. However, the concern is that the treatment
(audited) group is not a randomized sample of the population and therefore
fundamentally different than the control (non-audited) group. To account for
this, Table 3 shows the mean values of the model covariates that do not change
over time for audited (treatment), non-audited (control) and a set of matched
non-audited properties using propensity scores with 1:1 nearest-neighbour
matching65,66. We ran t-tests to examine the difference in means between the
control and matched sample with that of the treatment group. With the exception
of the assessed value per square foot for office buildings, we found no statistically
significant differences in the treatment and control groups, either with or without
matching. Therefore, we are confident in the random assignment of buildings to
the treatment and control samples.
Data availability
All data, except LL87 data, are available through the NYC Open Data Portal at
https://opendata.cityofnewyork.us/. LL87 data are available upon reasonable
request, and with permission, from the NYC Mayor’s Office of Sustainability. The
data that support the plots within this Article and other findings of this study can
be obtained from the NYC Mayor’s Office of Sustainability or the authors, upon
permission from the NYC Mayor’s Office of Sustainability who own the data.
Code availability
Any applicable code relevant to the findings is available from the authors upon
reasonable request.
Received: 27 August 2019; Accepted: 26 February 2020;
Published online: 30 March 2020
References
	1.	 Glaeser, E. L. & Kahn, M. E. The greenness of cities: carbon dioxide
emissions and urban development. J. Urban Econ. 67, 404–418 (2010).
	2.	 Inventory of New York City Greenhouse Gas Emissions (City of New York,
2017); https://www.dec.ny.gov/docs/administration_pdf/nycghg.pdf
	3.	 Burr, A., Keicher, C. & Lawrence, J. Commercial Building Energy Rating and
Disclosure Policies (Institute for Market Transformation, 2013).
	4.	 Jaffe, A. B. & Stavins, R. N. The energy-efficiency gap What does it mean?
Energy Policy 22, 804–810 (1994).
	5.	 Map: U.S. City, County, and State Policies for Existing Buildings: Benchmarking,
Transparency and Beyond (Institute for Market Transformation, 2017);
https://www.imt.org/resources/map-u-s-building-benchmarking-policies
	6.	 LL87: energy audits & retro-commissioning NYC Mayor’s Office of
Sustainability http://www.nyc.gov/html/gbee/html/plan/ll87.shtml (2020).
	7.	 New York City’s Energy and Water Use 2014 and 2015 Report (City of New
York, 2017); http://www.nyc.gov/html/gbee/downloads/pdf/UGC-
Benchmarking-Report-101617-FINAL.pdf
	8.	 Meng, T., Hsu, D. & Han, A. Estimating energy savings from benchmarking
policies in New York City. Energy 133, 415–423 (2017).
	9.	 Palmer, K. & Walls, M. Using information to close the energy efficiency gap:
a review of benchmarking and disclosure ordinances. Energy Effic. 10,
673–691 (2017).
	10.	Papadopoulos, S., Bonczak, B. & Kontokosta, C. E. Pattern recognition in
building energy performance over time using energy benchmarking data.
Appl. Energy 221, 576–586 (2018).
	11.	Hsu, D. How much information disclosure of building energy performance is
necessary? Energy Policy 64, 263–272 (2014).
	12.	Spiegel-Feld, D. Local Law 97: Emissions trading for buildings? NYU Law
Review Online 94, 148–168 (2019).
	13.	Chidiac, S., Catania, E., Morofsky, E. & Foo, S. Effectiveness of single and
multiple energy retrofit measures on the energy consumption of office
buildings. Energy 36, 5037–5052 (2011).
	14.	Ardente, F., Beccali, M., Cellura, M. & Mistretta, M. Energy and
environmental benefits in public buildings as a result of retrofit actions.
Renew. Sust. Energy Rev. 15, 460–470 (2011).
	15.	Hestnes, A. G. & Kofoed, N. U. Effective retrofitting scenarios for energy
efficiency and comfort: results of the design and evaluation activities within
the office project. Build. Environ. 37, 569–574 (2002).
	16.	Ürge-Vorsatz, D., Harvey, L. D. D., Mirasgedis, S. & Levine, M. D. Mitigating
CO2 emissions from energy use in the world’s buildings. Build. Res. Inf. 35,
379–398 (2007).
	17.	Use of Energy in the United States Explained (U.S. Energy Information
Administration, 2017); https://www.eia.gov/energyexplained/index.
cfm?page=us_energy_use
	18.	Nadel, S. Pathway to Cutting Energy Use and Carbon Emissions in Half
(American Council for an Energy-Efficient Economy, 2016).
	19.	Bradshaw, J. L., Bou-Zeid, E. & Harris, R. H. Greenhouse gas mitigation
benefits and cost-effectiveness of weatherization treatments for low-income,
American, urban housing stocks. Energy Buildings 128, 911–920 (2016).
	20.	Fulton, M. et al. United States Building Energy Efficiency Retrofits: Market
Sizing and Financing Models (Deutsche Bank Climate Change Advisors, 2012).
	21.	Chidiac, S., Catania, E., Morofsky, E. & Foo, S. A screening methodology for
implementing cost effective energy retrofit measures in Canadian office
buildings. Energy Build. 43, 614–620 (2011).
	22.	Im, J., Seo, Y., Cetin, K. S. & Singh, J. Energy efficiency in US residential
rental housing: adoption rates and impact on rent. Appl. Energy 205,
1021–1033 (2017).
	23.	Ryghaug, M. & Sørensen, K. H. How energy efficiency fails in the building
industry. Energy Policy 37, 984–991 (2009).
	24.	Hirst, E. & Brown, M. Closing the efficiency gap: barriers to the efficient use
of energy. Resour Conserv. Recycl. 3, 267–281 (1990).
	25.	Gillingham, K. and Palmer, K. Bridging the Energy Efficiency Gap Discussion
paper 13–02 (Resources for the Future, 2013).
	26.	Kontokosta, C. E. Energy disclosure, market behavior, and the building data
ecosystem. Ann. NY Acad. Sci. 1295, 34–43 (2013).
	27.	Hope, A. J. & Booth, A. Attitudes and behaviours of private sector landlords
towards the energy efficiency of tenanted homes. Energy Policy 75, 369–378
(2014).
	28.	Elbel, B., Kersh, R., Brescoll, V. L. & Dixon, L. B. Calorie labeling and food
choices: a first look at the effects on low-income people in New York City.
Health Affairs 28, w1110–w1121 (2009).
	29.	Davis, L. W. & Metcalf, G. E. Does better information lead to better choices?
Evidence from energy-efficiency labels. J. Assoc. Environ. Resour. Econ. 3,
589–625 (2016).
	30.	Newsham, G. R., Mancini, S. & Birt, B. J. Do LEED-certified buildings save
energy? Yes, but. Energy Build. 41, 897–905 (2009).
	31.	Kontokosta, C. E. A market-specific methodology for a commercial building
energy performance index. J. Real Estate Finance Econ. 51, 288–316 (2015).
	32.	Niemeyer, S. Consumer voices: adoption of residential energy-efficient
practices. Int. J. Consum. Stud. 34, 140–145 (2010).
	33.	Kontokosta, C. E. Modeling the energy retrofit decision in commercial office
buildings. Energy Build. 131, 1–20 (2016).
	34.	Laskari, M., Karatasou, S. & Santamouris, M. The design of an energy and
water advice programme for low-income households. Energy Build. 110,
426–434 (2016).
	35.	Kontokosta, C. E., Reina, V. J. & Bonczak, B. Energy cost burdens for
low-income and minority households: evidence from energy benchmarking
and audit data in five US cities. J. Am. Plann. Assoc. 86, 89–105 (2020).
	36.	Santamouris, M. et al. On the relation between the energy and social
characteristics of the residential sector. Energy Build. 39, 893–905 (2007).
	37.	Bird, S. & Hernandez, S. Policy options for the split incentive: increasing
energy efficiency for low-income renters. Energy Policy 48, 506–514 (2012).
	38.	Reina, V. J. & Kontokosta, C. E. Low hanging fruit? Regulations and energy
efficiency in subsidized multifamily housing. Energy Policy 106, 505–513
(2017).
	39.	Ramos, A., Gago, A., Labandeira, X. & Linares, P. The role of information for
energy efficiency in the residential sector. Energy Econ. 52, S17–S29 (2015).
	40.	Mims, N. et al. Evaluation of US Building Energy Benchmarking and
Transparency Programs: Attributes, Impacts, and Best Practices (Lawrence
Berkeley National Laboratory, 2017).
	41.	Jensen, O. M., Hansen, A. R. & Kragh, J. Market response to the public
display of energy performance rating at property sales. Energy Policy 93,
229–235 (2016).
	42.	Fuerst, F. & McAllister, P. Eco-labeling in commercial office markets: do
LEED and energy star offices obtain multiple premiums? Ecol. Econ. 70,
1220–1230 (2011).
	43.	Anderson, S. T. & Newell, R. G. Information programs for technology
adoption: the case of energy-efficiency audits. Resour. Energy Econ. 26, 27–50
(2004).
	44.	Fleiter, T., Gruber, E., Eichhammer, W. & Worrell, E. The German energy
audit program for firms—a cost-effective way to improve energy efficiency?
Energy Effic. 5, 447–469 (2012).
	45.	Annunziata, E., Rizzi, F. & Frey, M. Enhancing energy efficiency in public
buildings: the role of local energy audit programmes. Energy Policy 69,
364–373 (2014).
	46.	Murphy, L. The influence of energy audits on the energy efficiency
investments of private owner-occupied households in the Netherlands. Energy
Policy 65, 398–407 (2014).
	47.	Baechler, M.C. A Guide to Energy Audits PNNL-20956 (Pacific Northwest
National Laboratory, 2011).
	48.	Varone, S. & Varsalona, P. Complying with Local Law 87/09 (Rand
Engineering and Architecture, 2018); https://randpc.com/ask/mep/
local-law-8709
Nature Energy | VOL 5 | April 2020 | 309–316 | www.nature.com/natureenergy
315

Articles
NATure EnergY
	49.	Estimating Fuel Mix and Energy Cost (Environmental Protection Agency,
2018); https://portfoliomanager.energystar.gov/pdf/reference/FuelMixand
Cost.pdf
	50.	Krutwig, M. C. & Tantău, A. Obligatory versus voluntary energy audits: are
there differences in quality?. Proc. Int. Conf. Bus. Excellence 12, 522–532 (2018).
	51.	Shapiro, I. 10 common problems in energy audits. ASHRAE J. 53, 26–31 (2011).
	52.	One City Built to Last (City of New York, 2014); https://www1.nyc.gov/assets/
builttolast/downloads/OneCity.pdf
	53.	Nair, G., Gustavsson, L. & Mahapatra, K. Factors influencing energy efficiency
investments in existing Swedish residential buildings. Energy Policy 38,
2956–2963 (2010).
	54.	Harris, J., Anderson, J. & Shafron, W. Investment in energy efficiency: a
survey of Australian firms. Energy Policy 28, 867–876 (2000).
	55.	Marasco, D. E. & Kontokosta, C. E. Applications of machine learning
methods to identifying and predicting building retrofit opportunities. Energy
Build. 128, 431–441 (2016).
	56.	Papadopoulos, S. & Kontokosta, C. E. Grading buildings on energy
performance using city benchmarking data. Appl, Energy 233, 244–253 (2019).
	57.	Energy Star Portfolio Manager: Climate and Weather Technical reference (US
Environmental Protection Agency, 2019).
	58.	Kontokosta, C. E. & Tull, C. A data-driven predictive model of city-scale
energy use in buildings. Appl. Energy 197, 303–317 (2017).
	59.	Efron, B. Bayesians frequentists, and scientists. J. Am. Stat. Assoc. 100,
1–5 (2005).
	60.	Clark, J. S. Why environmental scientists are becoming Bayesians. Ecol. Lett.
8, 2–14 (2005).
	61.	Murphy, K. P. Machine Learning: a Probabilistic Perspective (MIT press, 2012).
	62.	Gelman, A. et al. Bayesian Data Analysis (Chapman and Hall/CRC, 2013).
	63.	Hoffman, M. D. & Gelman, A. The No-U-Turn Sampler: adaptively
setting path lengths in Hamiltonian Monte Carlo. J. Mach. Learn. Res. 15,
1593–1623 (2014).
	64.	Hyndman, R. J. Computing and graphing highest density regions. Am. Stat.
50, 120–126 (1996).
	65.	Dehejia, R. H. & Wahba, S. Propensity score-matching methods for
nonexperimental causal studies. Rev. Econ. Stat. 84, 151–161 (2002).
	66.	Rosenbaum, P. R. & Rubin, D. B. The central role of the propensity score in
observational studies for causal effects. Biometrika 70, 41–55 (1983).
Acknowledgements
We thank the NYC Mayor’s Office of Sustainability for sharing the LL84 benchmarking
and LL87 energy audit data used in this study. We also thank participants at the ‘Next
Generation Building Efficiency Policies’ workshop hosted by the NYU Law School and
NYU Marron Institute for their feedback on preliminary versions of this work, and
K. Hoffman for useful discussions on methodology. The research was supported by
National Science Foundation Grant no. 1653773 and by the Sloan Foundation. All errors
remain our own.
Author contributions
All the authors contributed equally to all aspects of the work presented in this article.
Competing interests
The authors declare no competing interests.
Additional information
Correspondence and requests for materials should be addressed to C.E.K.
Reprints and permissions information is available at www.nature.com/reprints.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in
published maps and institutional affiliations.
© The Author(s), under exclusive licence to Springer Nature Limited 2020
Nature Energy | VOL 5 | April 2020 | 309–316 | www.nature.com/natureenergy
316

    Policy Implications:

Answer:
• Cities should consider a comprehensive strategy to support energy efficiency in the building sector that starts with a foundation of data transparency and evidence-based decision-making.
• Energy disclosure mandates are an important first step: once data are available, buildings can be evaluated on their energy performance and compared to their peers.
• Audit requirements should be used to target ‘deep’ retrofits, while automated or virtual audits could replace the existing need for traditional audit mandates.
• Since audit policies generate substantial data on building systems and operating characteristics, mandatory audit requirements could be replaced by incentives for voluntarily reporting of audit data.