Ronald Reagan once joked that the game Trivial Pursuit had a special economists' edition: It came with 100 questions and 3,000 answers. Economists are notorious for being unable to agree on anything. So it's striking that on the finances of the Olympics, they almost all agree.
"Investing in the Olympics is not worth the investment," says Andy Zimbalist of Smith College.
"You build all these facilities that are perfect for the Olympics, that are not really as desirable once the circus leaves town," says Allen Sanderson of the University of Chicago.
"You end up with a very indebted city or host nation long after the confetti is cleaned up; someone has to pay the bills for it," says Bob von Rekowsky of Fidelity Investments.
This analysis runs counter to the assurances of host cities, which invariably promise new investment, tourism, and development. Governments insist that the games will pay financial dividends.
The Government Claims Olympic Benefits
London, like other Olympic hosts, says its investment paid off. The British government went further than most to back up its claims, citing a massive five-year, 1,000-page study conducted by a respected team of consultants.
The report concluded that by last summer, Britain had already earned at least $1 billion more than the $15 billion it spent on the 2012 Summer Games. Forecasts for the future went into the tens of billions of dollars.
Business Secretary Vince Cable was all over the news when the report came out, trumpeting the payback on investment.
"Certainly a very considerable net benefit," he told one BBC interviewer.
In every TV and radio appearance, Cable emphasized that this was an independent report.
"It's been independently audited," he told one interviewer. "An anonymized survey, methods approved by the national audit office," he told another.
The media picked up the same language. Most British newspapers described the report as an "independent analysis." Some added, "Commissioned by the government."
A Government-Funded Report
That commission was a $2 million contract. The check went to a firm called Grant Thornton, where a manager named Rob Turner oversaw the project.
"Government did not say you have to reach a certain figure at any point, and they wouldn't have," Turner insisted when I met him at Grant Thornton's headquarters in Central London.
Nor, he said, did anyone in the government suggest that the report should broadly reach a positive conclusion rather than a negative one.
"The modeling was done on the basis of trying to understand the economic impacts," he said. "If it had been a negative figure, then that would have been equally in the report."
That "modeling" is key. In a study like this, economists create an exclusive model, like an original equation or recipe. The model shapes the conclusions.
The authors might have avoided potential criticism that they skewed the model in the government's favor if they'd published this study in a peer-reviewed academic journal. But Turner says everyone wanted this document to be usable, not just for academics.
"It was about trying to make it accessible," he says, "Which is why the thousand pages, which were read by some, were distilled into a 30-page document, which will be read by a large number of people."
The government office in charge of this report would not say whether there was any implicit or explicit expectation that the analysis reach a positive conclusion.
A spokesman at the Department for Culture, Media, and Sport provided a written statement, saying the analysis was "supported by robust and comprehensive evidence."
Turner and his colleagues did bring in peer advisers outsiders who were invited to consult on the study.
"Invariably we took those views on board," Turner says.
When asked whether the study came with the endorsement of the outside peer reviewers, Turner replied, "Very much so. The peer reviewers felt that it was a robust and comprehensive study."
A Contrary Voice
That does not match the description of peer reviewer Stefan Szymanski, who told NPR, "I thought this was tantamount to a whitewash."
Szymanski, who is a professor of sports management and economics at the University of Michigan, says, "The report provided a very bullish view, refused to comment on any of the negatives, or even to really qualify any of the results, and I was very unhappy about this."
In an email to Turner before the report was published, Szymanski wrote, "I'm very uncomfortable about the triumphalist tone of the report, which does not reflect what the data is saying." He added, "It should be stated clearly that the opinions expressed in the report are yours and not mine."
Szymanski says the authors sent him the final draft with one week to make comments. Crucially, they wouldn't let him preview the model the formula that produced the numbers in the report. Szymanski says that's "extremely unusual."
Even so, the authors did make some changes based on his feedback.
"For example," says Szymanski, "The initial report claimed that the number of tourists arriving in Britain had increased during the period of the games, when the official government numbers showed that tourist numbers during June-July-August had actually fallen."
There were three other peer reviewers. One did not assess the final report. Another looked at the redevelopment of East London, but not the financial investment. The third peer reviewer is not an expert in economic analysis.
So Szymanski was the only person with expertise in statistical number crunching who looked at this study before it was published. And he describes it as a "political document" that ignores massive amounts of economic research showing Olympics are almost always unprofitable.
"Any serious published academic work is almost entirely skeptical about the potential of these events to generate economic benefits. And there has been a lot of research done by now," he says.
Is It Too Soon To Judge?
Some of that work has been done by a London economist named Max Nathan, at the National Institute for Economic and Social Research.
"There's a natural temptation to add up all the positive numbers you can find and discount or ignore all the negative numbers," says Nathan.
He says countries are never satisfied with throwing a great party. Every Olympic host promises an economic benefit, too.
"And I think if you look at official evaluations of the Games, not just in London, but in many other host cities over the years, you see that tendency," he says.
In Nathan's view, it's just too soon to know whether the London Games were worth the investment. Economists can say there was a small increase in job creation; home prices went up; development of East London happened faster than it would have without the Olympics.
Is all of that worth the $15 billion price tag? Nathan says, "It will be years before we can see that."