GBA Logo horizontal Facebook LinkedIn Email Pinterest Twitter Instagram YouTube Icon Navigation Search Icon Main Search Icon Video Play Icon Plus Icon Minus Icon Picture icon Hamburger Icon Close Icon Sorted
Musings of an Energy Nerd

GBA Prime Sneak Peek: Is Weatherization Cost-Effective?

A recent study claims that weatherization measures cost more than they’re worth — but the study’s analysis is flawed

The most recent International Energy Conservation Code increases requirements for attic insulation in some climate zones. One GBA reader isn't so sure the change will make a big difference in how comfortable a house might be. Photo courtesy Office of Energy Efficiency and Renewable Energy.

GBA Prime subscribers have access to many articles that aren’t accessible to non-subscribers, including Martin Holladay’s weekly blog series, “Musings of an Energy Nerd.” To whet the appetite of non-subscribers, we offer a “GBA Prime Sneak Peek.” This GBA Prime blog was originally published on July 3, 2015.

A recent paper on the cost-effectiveness of weatherization work has received much more attention in the popular press than have similar studies in the past. The researchers concluded that weatherization measures performed at five nonprofit community action agencies in Michigan weren’t cost-effective. Newspaper headline writers have had a field day, trumpeting generalizations that aren’t supported by the limited data collected by researchers. (A typical headline: “Study: Home efficiency upgrades fall short, don’t pay.”)

Energy experts familiar with weatherization programs have suggested that the headline writers should have dug a little deeper into this story than they apparently did, but these measured voices have gained less attention than the “Weatherization Doesn’t Pay” headlines.

The controversial paper, “Do Energy Efficiency Investments Deliver?,” was written by three economists: one from the University of Chicago, Michael Greenstone, and two from the University of California at Berkeley, Meredith Fowlie and Catherine Wolfram. (Some of the paper’s findings were summarized by Scott Gibson in a recent GBA news story, “Blows Against Two Carbon Reduction Strategies.”)

This decades-old program helps low-income Americans

The federal low-income weatherization program — officially known as the Weatherization Assistance Program, or WAP — is now 39 years old. Sponsored by the U.S. Department of Energy, the program pays for weatherization workers to perform air sealing work, to add insulation, to seal leaky duct seams, and to replace inefficient appliances in the homes of low-income Americans. The services are provided at no charge to recipients.

The weatherization program is administered by state employees; in most states, however, the actual weatherization work is performed by employees of local nonprofit groups known as community action agencies.

One of the guidelines promoted by WAP is that weatherization measures should be selected based on cost-effectiveness. Over the past decades, many researchers have looked at whether the program is successful at achieving this cost-effectiveness goal.

For example, in a 2010 paper, “Cost-Effectiveness of Weatherization in Low-Income Urban Housing Stock,” author Jonathan L. Bradshaw reported, “Most weatherization treatments examined are profitable. Although cost-effectiveness varies significantly between cities and treatment scenarios, almost all treatments in the cities examined were NPV-positive [net-present-value-positive] over either a 7 or 15 year period. These results indicate that we would expect the present value of energy savings earned through weatherization to exceed weatherization installation costs for programmable thermostats, attic insulation, and air sealing.”

Summing up several decades of program evaluations, Fred Sissine’s 2012 paper, “DOE Weatherization Program: A Review of Funding, Performance, and Cost-Effectiveness Studies,” reported, “Over the program’s history, DOE’s Oak Ridge National Laboratory (ORNL) and the Office of Management and Budget have used process and impact evaluation research methods to assess WAP operations and estimate cost-effectiveness. Virtually all the studies conducted through 2005 showed that the program was moderately cost effective.”

Does weatherization pass the cost-benefit test?

The three economists who authored “Do Energy Efficiency Investments Deliver?” came to a different conclusion from previous researchers. In their abstract, the authors wrote, “Conventional wisdom suggests that energy efficiency (EE) policies are beneficial because they induce investments that pay for themselves and lead to emissions reductions. … The findings suggest that the upfront investment costs are about twice the actual energy savings.”

In the section of the paper headed “Conclusion,” the authors wrote, “From a policy perspective, WAP does not appear to pass a conventional cost-benefit test, although its full set of goals may not be reflected in such tests.”

Basic facts about the study are hidden

The paper by Fowlie, Greenstone, and Wolfram is poorly written and opaque. It neglects to report information of obvious interest to those concerned with weatherization issues: for example, how many weatherized houses were included in the study, the average pre-weatherization utility bills for the studied houses, and the average post-weatherization bills of weatherized houses.

I contacted one of the researchers, Meredith Fowlie, to ask how many weatherized houses were studied. “If you want to know the number of households in our experimental sample that appear in all data sources needed for the estimation of causal impacts, the total number is 616,” she answered. “The 616 number conditions on a perfect match across all of our data sources. A small share of households in our initial experimental sample could not be perfectly matched with our billing data and thus were dropped from the sample.”

Although she also wrote, “You can find these numbers in Table 3 of the Appendix,” the numbers in the table don’t match her tally. The table lists a total of 436 “experimental encouraged” households, 1,473 “other weatherized” households, and 180 “experimental control” households.

Although the number of studied weatherized houses is not clearly reported in the researcher’s paper, I have no reason to doubt Fowlie’s statement that the researchers evaluated the cost-effectiveness of weatherization measures performed at 616 houses. Because it took me so much digging to determine how many houses were studied, it’s hard to fault the journalists who latched on to the prominently featured statement at the beginning of the researchers’ report: namely, the claim that the paper “reports on the results of an experimental evaluation of the nation’s largest residential EE [energy efficiency] program conducted on a sample of more than 30,000 households.”

To repeat: the number of weatherized houses that were evaluated by the researchers was 616, not 30,000.

Fuzzy numbers

Only by careful digging does one discover, in a confusingly labeled table, the type of weatherization measures that were implemented at the studied houses. The information can be found in the Appendix, Table 1. It appears that 85% of the houses received new attic insulation; 76% received a measure called “infiltration reduction” (more accurately called “air sealing”); 44% received new wall insulation; and 34% received new furnaces.

The authors wrote, “Recipient households in our study received approximately $5,150 worth of home improvements on average, at zero out-of-pocket costs. The most common measures included furnace replacement, attic and wall insulation, and infiltration reduction.” (Note that the authors didn’t list these measures in order of frequency.)

They also wrote, “We estimate that the WAP energy efficiency investments reduce monthly energy consumption by 10-20% on average.” It’s unclear why, if the researchers are reporting an “average,” they were unable to pin down energy savings with any better precision than the wide range cited here.

The authors continued, “Although this surely provides a substantial assist to participating low-income households in the form of reduced energy bills, the upfront investment costs are about twice the realized energy savings. Further, the model-projected savings are roughly 2.5 times the actual savings.”

Needless to say, most journalists reported that the cost of the weatherization measures was greater than the anticipated energy savings.

Almost all weatherization researchers report average energy savings in weatherized houses. The method of reporting chosen by the three economists, however, is surprisingly vague. “Our estimates imply energy savings of 17.2 MMBtu [of natural gas] per year,” the authors wrote. (That estimate of per-family annual savings would be worth about $155 in Michigan.)

It’s unclear why the authors report that their “estimates imply” this level of savings; why not just tell us the actual weather-normalized savings?

Putting the data into perspective

The three researchers made broad generalizations about residential energy efficiency improvements based on limited data from one state. It is worth noting:

  • The researchers only looked at the work of 18% of the low-income weatherization agencies working in Michigan. The researchers didn’t evaluate weatherization programs in any other state.
  • The researchers only analyzed heating fuel savings in homes that were heated with natural gas, excluding homes heated by propane, fuel oil, and electricity. Of course, weatherization work is much more likely to be cost-effective in homes heated by propane, fuel oil, or electricity, because these fuels are generally more expensive than natural gas.
  • While natural gas is currently inexpensive, it may not be inexpensive forever. Rising energy prices would improve the cost-effectiveness of the weatherization work that was studied.
  • The researchers only looked at WAP. They did not evaluate any energy-efficiency work performed at the homes of middle-class or upper-class Americans. Because low-income families often live in substandard housing, weatherization work in low-income homes is often less cost-effective than similar work in homes owned by middle-class Americans. Moreover, WAP has to serve every low-income family that qualifies for the program, even when the family’s home may not be an ideal candidate for cost-effective weatherization measures — a fact that tends to lower cost-effectiveness numbers.
  • The researchers’ conclusions show that they didn’t appreciate the fact that low-income weatherization programs often perform work for reasons other than energy savings; and that while the non-energy measures reduce the cost-effectiveness of the work, the measures provide important benefits to the families who are served.
  • The researchers sought and obtained funding that they used to encourage low-income families to enroll in the weatherization program. This “encouragement design” may have lowered the savings compared to regular WAP work. Low-income households apply for WAP voluntarily, presumably because they think they can benefit — for example, because their homes are drafty and poorly insulated. By pushing the program on people and increasing application rates by six times, the researchers’ marketing efforts may have resulted in a group of participants who were less likely to benefit from the program than the normal flow of applications. The “encouragement design” changed the treatment group to include more households that didn’t feel as though they needed the program. Moreover, the “encouragement group” was found to have smaller homes and more elderly occupants than other weatherized homes and apparently had much lower gas use (about 25% lower than the average home in Michigan).
  • As Martin Kushler, a senior fellow at ACEEE, has pointed out, “This study looked at the WAP during the ‘stimulus package’ years. … WAP was extremely stressed at the time, with tremendous pressure to push money out to the field. Job creation was at least as big a goal as energy savings, and they were functioning with a lot of new and inexperienced employees in order to handle the huge increase in funding and the deadlines to get it spent. … The stimulus package demands on the system drove up the average cost per home in the program, in an effort to ‘get the money spent.’ This would naturally tend to diminish the apparent cost-effectiveness in terms of energy savings per dollar spent.”
  • The researchers apparently made their cost-effectiveness calculations without performing any weather normalization. Weather normalization is the adjustment of fuel cost numbers to account for differences between colder-than-average winters, average winters, and warmer-than-average winters; without weather normalization, it’s impossible to compare one winter’s fuel bill with the next winter’s fuel bill. Instead of weather normalization, the researchers included month-specific effects which indirectly account for weather through how the control group’s usage shifted month by month. But these month-specific effects doesn’t actually capture the weather that each customer experienced and do not adjust the usage for a typical weather year.
  • When trumpeting their findings in a press release, at least one of the three authors didn’t just report the data; he used the Michigan data to make sweeping generalizations about residential energy efficiency improvements. “Energy efficiency investments hold great potential as a means to fight climate change,” Michael Greenstone declared. “However, we found that, at least in the case of residential energy efficiency investments, the projected savings overestimate the reality on the ground.”

One of Greenstone’s co-authors appears to disagree with Greenstone’s generalizations. Asked about the study, Meredith Fowlie was quoted in the Washington Post as saying, “This is one study in one state looking at one subpopulation and one type of measure. I would not feel comfortable generalizing from our study in Michigan.”

Furnace replacements aren’t cost-effective

After the paper by Fowlie, Greenstone, and Wolfram was published, I called up a prominent weatherization expert known for his data-analysis skills and extensive experience evaluating energy-efficiency programs. Because of his current employment status, he asked that I withhold his name — a request to which I reluctantly acceded. I’ll have to call him “John Doe.”

“These researchers looked at the weatherization work of just five agencies in Michigan,” John Doe began. (There are a total of 28 community action agencies that provide weatherization services in Michigan.) “WAP is a national program in 50 states,” Doe continued. “Different agencies have different program designs and different levels of training. There are variations in energy savings between agencies that can vary from 2 or 3 to 1, and there are variations in energy savings between states that can vary from 2 or 3 to 1. So this is not a representative sample. You can’t draw any conclusions about the WAP program based on this sample.”

John Doe noted that replacing an old furnace with a new furnace usually has a long payback period. When weatherization jobs include furnace replacement — as opposed to just air sealing and insulation improvements — the furnace replacement jobs tend to drag down the cost-effectiveness numbers. Of the weatherization jobs studied by these researchers, 34% included furnace replacement.

“In some cases, weatherization agencies estimate the efficiency of the existing furnace at a really low number, low enough that the program rules will permit the furnace to be replaced, because they know that the furnace is on its last legs and isn’t particularly safe,” Doe told me. “It’s a stopgap approach, but these decisions are made for the best of reasons. Some of these programs have strict criteria for when certain measures can be performed, so they end up doing furnace replacements which are really health and safety measures, even though they are categorized as an energy-efficiency measure. These programs want to fix up a house as much as they can as well as reduce energy bills. We know that furnace replacement is almost never cost-effective, especially now that natural gas is cheap, but that doesn’t mean that you shouldn’t ever replace a furnace. … Remember, these agencies are doing an amazing job, juggling multiple funding streams, trying to help poor people have better energy-efficient homes.”

Doe continued, “How do you account for the costs and benefits? Maybe we shouldn’t be replacing these furnaces. Maybe $2,000 of the $5,000 that was spent on a weatherization job was invested in things that we never really thought would pay for themselves. But a cost-effectiveness analysis doesn’t account for the value of the improved quality of life and improved safety for the people who live there. There are a wide array of benefits when we weatherize people’s homes. When there is a leaky roof, maybe they patched the roof and fixed the attic. The work sometimes includes asbestos abatement or repairs for faulty wiring. It’s not fair to say this work isn’t cost-effective.”

Thinking like economists, not energy experts

The fact that authors of the paper are all economists affected their method of analysis.

“The authors did a terrible job of analyzing the data, so we don’t really know what the savings are,” Doe told me. “They didn’t use the standard analytical methods used in the energy-savings field. They ignored the fact that this is physics, not economics. They actually don’t model energy use, but instead model the logarithm of monthly energy use — which makes an additive process (heating use adds onto baseload use in the winter — it doesn’t multiply it by some percent) into something that makes no sense and can lead to unpredictable answers.”

An expensive way to reduce carbon emissions

The three economists — Fowlie, Greenstone, and Wolfram — noted that spending on WAP is not the best way to reduce carbon emissions. That’s true, but WAP was not established to lower carbon emissions. The program was established to help low-income families who are struggling with high energy bills.

One of the papers I cited at the beginning of this article — “Cost-Effectiveness of Weatherization in Low-Income Urban Housing Stock” by Jonathan L. Bradshaw — addresses the question of carbon emissions. “Weatherization strategies aimed at energy savings, carbon savings, and cost-effectiveness may not lead to the same conclusion,” Bradshaw wrote. “Because average energy consumption, carbon intensity of energy consumed, and energy prices all vary geographically and largely independently, energy savings, carbon savings, and cost-effectiveness are not necessarily aligned. Weatherization strategies that seek to minimize residential energy use may not be the same strategies that seek to minimize residential carbon emissions. Additionally, there are different ways to consider cost-effectiveness, including net present value or by abatement cost for energy or carbon. In evaluating existing and designing new weatherization programs, it will be important for policy-makers to recognize these differences and decide the priorities of weatherization programs.”

Taking a broader view

Is the low-income weatherization program cost-effective? It’s hard to say. The weatherization work performed by some community action agencies is almost certainly cost-effective, while the work of other agencies may not be. These differences depend on climate, fuel costs, housing stock types, worker skills, and a variety of assumptions, including assumptions about future energy cost increases.

“The WAP program is running out of high users,” John Doe told me. “Average natural gas usage has dropped. A few decades ago, the average house used 1300 therms of gas a year; now it is 900 therms. There used to be a lot of houses with inefficient refrigerators and naturally aspirated furnaces. The programs are running out of that type of house. Over time, the worst of the housing stock gets knocked down, or it has been insulated by some program or another. The housing stock is getting more efficient. Furnace efficiency standards, water heater efficiency standards, improved insulation — it has all added up to significant reductions in energy use.”

While it may be harder in 2015 to show that the work performed by WAP is cost-effective than it was in 1985 or 1995, that doesn’t mean that the money invested in the program is wasted. After a house is weatherized, the house becomes more comfortable to live in, and the low-income family living in the house sees lower fuel bills. All kinds of social benefits flow from this type of work.

“These programs are typically judged by including the associated ‘non-energy benefits’ in addition to the direct energy savings,” wrote Martin Kushler. “These multiple benefits include things like the effects on comfort, health, safety (e.g., WAP typically installs smoke detectors, CO detectors, fixes wiring problems, fixes gas leaks, etc.), increased value of the improved housing stock, reduced utility-bill payment arrearages and non-payment collection costs (which saves money for all ratepayers), improved ability to remain in the dwelling and not have to be relocated, etc. Studies exist in the industry that quantify these types of variables, and when taken in aggregate, the non-energy benefits’ value can nearly equal, or even exceed, the direct energy savings value… The study [by Fowlie, Greenstone, and Wolfram] … simply ignores those other multiple benefits, and does not quantify them in the analysis. This methodological flaw pre-ordains the conclusion.”

Setting up a straw man

Kushler worries that this paper might affect funding for programs aimed to help low-income families.

Kushler wrote, “The authors conducted a study of one particular low-income program (the federal Weatherization Assistance Program, or WAP), as implemented in portions of one state (Michigan), and somehow ended up with the sweeping headline ‘Study Finds Costs of Residential Energy Efficiency Investments are Double the Benefits.’ Some of the popular press is already picking up on this theme, and the concern is that a misunderstanding (or misuse) of this study will lead to low-income families having less access to important programs that drive down their utility bills. Or worse yet, as a broad-brush attack on all types of energy efficiency programs.”

He continued, “In short, this study cherry-picked the worst possible program for comparing total costs to just direct energy savings, then set up a straw man to knock down, then tried to suggest, from an extremely limited sample of one program type, that all ‘residential energy efficiency investments’ are suspect.”

Energy savings don’t amount to much when energy is cheap

In my opinion, WAP is an excellent program. I’m proud that some of my federal tax dollars are used to perform energy-efficiency improvements in the homes of low-income families. (Unfortunately, I can’t really say the same thing about some of the other ways that the federal government spends our tax money — but that’s another story.)

According to Merrian Borgeson, a senior scientist for energy for the Natural Resources Defense Council, the study by Fowlie, Greenstone, and Wolfram has not been peer-reviewed. The three researchers might have been able to repair some of the paper’s flaws if it had undergone the usual peer-review process. Moreover, Greenstone’s attempt to generalize from the study’s limited data is indefensible.

That said, the researchers make some valid points — some of which are frequently mentioned on GBA. Here are two:

  • When weatherization workers enter data into existing energy modeling programs, the software usually overpredicts future energy savings, for two reasons: the inputs are often wrong, and the software programs have flaws.
  • Energy retrofit work on existing houses is not the fastest or most cost-effective way to address our looming climate change crisis.

Weatherizing old houses isn’t easy work. The most cost-effective examples of this work occur when all of the stars are aligned: when the house being worked on is poorly insulated and leaky; when there is good access to the basement and attic; when fuel costs are high; when the work is not impeded by hazards like asbestos, faulty electrical wiring, or a wet basement; when the crew is well trained and well motivated; and when the implemented measures are limited to measures with a proven track-record for cost-effectiveness.

When it comes to weatherization work, things have gotten a little tougher recently. The average house is using less energy than it used to, so there are fewer opportunities for dramatic savings. At the same time, energy — especially natural gas — is cheap. Meanwhile, labor isn’t getting any cheaper. And many older homes still have the same grab-bag of problems that have always plagued weatherization workers: water entry into basements, small access hatches, asbestos duct insulation, lead paint, knob-and-tube wiring, and cramped attics filled with long-forgotten possessions.

Fourteen years ago, the Environmental Protection Agency launched Home Performance with Energy Star, a program that hoped to lure contractors into a lucrative new field: providing energy retrofits for middle-class and upper-class Americans. As Nate Adams (among many others) discovered, it’s hard to make money doing this sort of work. (Nate Adams wrote a series of blogs on this topic for GBA; the first blog in his series was titled How an Efficiency Program Killed My Business.) Customers for this type of work are hard to find, so client-acquisition costs are high. Homeowners require lots of hand-holding. And most old houses have a lot of problems that need to be fixed before energy retrofits can start.

While these facts about weatherization work need to be acknowledged, it’s important not to throw out the baby with the bathwater. In many areas of the country, there are lots of opportunities to perform cost-effective work: for example, by replacing incandescent bulbs with CFLs or LEDs, or by swapping old single-speed swimming pool pumps for new variable-speed pumps. It still pays to seal up the large holes in our nation’s attics, and the attic insulation in many homes is still appallingly thin. We need to continue to work at addressing these problems, while simultaneously investing in a variety of other measures designed to reduce our nation’s carbon emissions.


GBA Prime subscribers can read a great many posted comments at the page where this article was originally published: Is Weatherization Cost-Effective? If you are a GBA Prime subscriber, that page remains the best place to post comments. Non-subscribers are invited to post comments below.


  1. AlanB4 | | #1

    Reality denial, starting with
    Reality denial, starting with a conclusion and working backwards, propaganda and massaging data to achieve the desired outcome and poor data acquisition and analysis are common tactics, and extremely effective ones.

  2. user-1107451 | | #2

    My own experience, and are you telling me the media manipulates
    I can say from my own experience, I never used the weatherization program just hints and advice from GBA
    I air sealed my attic and added 6" of blown in cellulose (done by myself) to an attic with R38 batts in it.
    The 2750sq ft house was built in 2007 and is in Zone 5 NY
    The summer I saw a 50% reduction in my central a/c bill from 550Kwh to 275Kwh, and in NY by the time you add "service" charges it works out to be 25c per KWH, so the cellulose and spray foam I expect to see it this pay for itself in just under 4 years.
    Then again the media manipulation of a story of badly collected statistical data is by no means unique.
    write a flawed survey of course you will get bad data, did they ask if the recipients then used their a/c heating etc any differently than they did before? Did they add into it the cost of energy? For example Con Ed (or Conned as I like to call them) asked in NY for a 15% increase in cost of electricity, they got 6% due this month, were those figures added into the savings this program was established for?
    No they are the media facts dont come into it

  3. LucyF | | #3

    I did the same thing - air sealed and insulated attic
    I did the same thing only I've gone from a poorly air sealed R-5 attic to fairly well air sealed R-20 attic and my energy use is half. I'm in a more temperate zone that you are - zone 3 Upstate SC.

    And the economic studies don't mention comfort. My house is SO much nicer to live in now.

    I have some structural things to do in the attic (like reinforce the flimsy site-built trusses and make stronger connections to the framing of my house), then I'm going to blow in as much cellulose as I can.

  4. JonathanBeers | | #4

    Authors response to criticisms
    Good to see some dialog about the study. They posted a response to criticisms at:

  5. Expert Member
    Dana Dorsett | | #5

    The problem is in the overpromising, not the underdelivering...
    The NEAP audit tool used by the programs they studied regularly under-estimates the efficiency of existing heating equipment, and overestimates heat losses from infiltration (as do most industry-standard heat load tools used in the HVAC trades). In the original paper they estimate that the error in the NEAP tool to be about 25%, since it was predicting heating energy use that was 25% higher than actually measured in those homes.

    So, a 39% savings from 125% of reality is still a savings of about 24% below the true baseline use, but changes the net-present-value of more expensive upgrades such as furnace swaps.

    This is a long known problem with the accuracy of energy audit tools. So while the energy programs may still be viable, it's worth better due-diligence on the baseline when going for more than the dead-obvious and inexpensive low-hanging fruit such as air sealing an insulation, and working harder at determining the true impact of window swaps and furnace upgrades.

    It's not that efficiency investments or efficiency programs aren't cost effective, it's that not ALL efficiency are cost effective, especially if you assume record low natural gas prices are going to be here forever.

    In the defensive article they state:

    "In calculating program benefits, we used real 2013 residential energy prices for electricity and natural gas in Michigan and assumed that these figures would increase at the rate of inflation over the lifetime of the investments. While some have criticized this as too conservative, it is standard to use current energy prices as a predictor of future energy prices."

    That's complete BS when looking at the dynamics of natural gas price volatility in recent decades. Without a world oil price sufficient to keep the fracking craze going, the natural gas glut that has recent prices running near all time record lows is guaranteed to dry up, since dry-gas fracked wells are (on average) nowhere near profitable at current wholesale prices. To ignore that is complete cherry-picking. Not even those drinking the frack-water really believe natural gas will stay this cheap for decades.

    About 1/4 of all natural gas going into the US gas grid is from fracked oil wells. Now that oil prices have fallen, even most fracked wells with substantial liquids have negative (or very thin) return, which has cut the drilling rates considerably. The average fracked well is 95% depleted in only about three years, so unless the oil prices rise to where its financially rational for the drilling rates to resume, we have at best three more years of the super-cheap natural gas the US has enjoyed for the past 5-odd years. If sanctions are lifted and Iranian oil starts flowing onto the market it'll be "game over" for most of the shale fracking operations for at least a decade, since it's a lot cheaper to expand oil production in Iran than it is to expand it the Marcellus or Eagle Ford shales, etc.

    The inflation adjusted residential retail price of natural gas in 2006, before the fracking boom was fully 60% higher than it is right now ( ). Without a high oil price to drive high drilling rates in the shale fields we could easily be at $1.60/therm (in 2015 dollars) or higher natural gas in under a decade, which changes the NPV of a condensing gas furnace considerably. If they are going to intentionally put blinders on how these markets work and not present realistic high & low price scenarios, focusing only on the low-price present, they're not really doing a very good job. There is hedge value in efficiency investments that does not show up in their analysis.

    Predictions of near-future electricity prices being steady is more reasonable, but in fact electricity prices are likely to fall as more renewables go onto the grid, since it cuts into expensive peak-power, and has a levelized cost lower than new fossil fired generation. (UBS, Barclays, CitiGroup et al are predicting deflationary pressures on electricity pricing in the US to begin in earnest in a decade or so.) But to declare a 25 year lifecyle investment such as a gas furnace as not cost effective based on zero energy price inflation is a dubious assertion at best.

    So, if they have a point (and I think they do), it's that the NEAT tool is too crude, inadequate for making the investment decisions, and that something better needs to be used. It's neither rocket science nor time consuming to compare NEAT predictions of the present energy use with the ACTUAL present energy use, which seems like an obvious first hint to when more analysis is necessary to predict which upgrades pay off, and which might not.

  6. krom | | #6

    Don't forget the thumb on the scale..
    Dana, not only is the tool flawed, but the guys using it make money by selling goods and services. If they can skew the data a little, then they can make a sale. The homeowner won't complain because they are getting a considerable discount, or free services.

  7. GBA Editor
    Martin Holladay | | #7

    Response to Dana Dorsett
    As I'm sure you know, the problem with software that overpromises savings -- a problem that GBA has reported on for years (see, for example, Energy Modeling Isn’t Very Accurate) -- is independent of the challenges facing researchers into weatherization cost-effectiveness.

    For decades, researchers into weatherization cost-effectiveness have compared (a) the average weather-normalized pre-retrofit energy use data for a group of houses with (b) the average weather-normalized post-retrofit energy use data for the same group of houses. This isn't that hard to do. Once these data are gathered, there is no need to pay any attention to modeling software, unless you are a researcher focusing on whether the predictions of energy modeling software are accurate. (Hint: They aren't.)

  8. dankolbert | | #8

    Thanks, Jonathan
    Fascinating comments section in the link to the authors' response.

  9. Expert Member
    Dana Dorsett | | #9

    I understand that it's been covered here (response to Martin #7)
    A criticism leveled by the economists was that the savings were only 10-20%, which they in fact normalized against heating degree-days, and analyzed fairly competently to rule out rebound effect. They were in fact correct that A: The savings were only 10-20%, and B. That there was effectively zero increased energy use due to rebound effects.

    So the problem boiled down to the unrealistic promises made based the flawed tool set, resulting in poorer-than-anticipated financial returns. The way they spun it wasn't particularly useful or enlightening but they weren't just making stuff up. Yes, the actual savings were less than half what was promised, but the actual baseline was ~20% lower than the audit tool.

    The fact that the savings were only 25-50% of what was advertized isn't a problem with the program or investing in energy efficiency in general, except for the fact that the operators of the program continued to use the tool without correcting it to match the actual baseline, or figuring out where the tool was in error. So to headline it as "Efficiency investments are not cost effective", is a political spin. A more nuanced & correct interpretation would be "Poor but popular energy auditing tool leads to less-economic expenditure of public funds."

    Where the program is at fault is in over-reliance on a flawed tool for decision making. It's particularly egregious when sanity checking of predicted vs. actual current energy use shows large discrepancies, which should have prompted them to figure out where the errors are. That is a problem with the way the efficiency program is conducted, and it does indeed lead to poorer financial performance.

    But I have to say, if I hired a group of economists to figure out the financial return on efficiency investments, if they didn't show me high, low and middle scenarios for the return at different possible energy price inflation levels I wouldn't be paying them in full. Their assertion that it's standard practice to assume flat real energy prices for up to 20 years is completely out to lunch. I've never seen an energy price/cost projection from competent that didn't show a range of scenarios on time scales that long, except for worst-case scenarios projections. Some of the electricity price inflation numbers I[ve seen bandied about in rooftop PV proposals are downright laughably high worse than any possible worst-case scenarios. (Yeah, right, electric rates are going to be running 2-3% ahead of general inflation for the next 25-30 years, despite the fact that they have gone DOWN in real terms over the past 25-30 years.)

    Sticking with a flat energy price number, a number that happens to be at a 25 year low with a price-volatile commodity like natural gas is a fatal flaw in their analysis. If they don't actually get that, they should go do something else with their careers where they might do something useful. But I think they DO get it, and are playing a different game by spinning the numbers that way.

Log in or create an account to post a comment.



Recent Questions and Replies

  • |
  • |
  • |
  • |