The American Coalition for Clean Coal Electricity is running an advertisement at the Washington Post and The Hill websites which makes the following claim: 72% of opinion leaders support coal electricity. The ACCCE touts this claim repeatedly at their various websites, but there is so little information available about the study that produced this claim that it’s literally impossible to verify. However, given the number of inconsistencies in what little information is available, we can make an educated guess as to the accuracy of the 72% claim.

If you click on the “America’s Power” advertisement (screen shots shown at right), you’re taken to this page, where the ACCCE claims “it’s easy to see why 72 percent of American opinion leaders support the use of coal.” On this page, however, there are four links on the page that all go to the same press release that describes the ACCCE study that produced this 72% number.

The Election Day press release makes a number of claims about an earlier survey that are inconsistent based exclusively on the information presented in the press release. The claims all rest on the study’s definition, provided at the bottom of the press release, of “opinion leaders,” aka “opinion elites.” They are defined as follows:

The poll interviewed 600 opinion elites nationwide. Elites are defined as adults with $80,000 or more in household income and a four-year college degree or more and a professional or managerial job title or a business owner and a high degree of involvement in politics and policy matters.

According to the Census Bureau economic data, there were 116.8 million households in the U.S. in 2007 (the latest year for which there’s complete data), of which only 34.1 million meet the income requirement described for an “opinion elite” above. That’s just 29% of all households.

According to the Census Bureau educational attainment data, there are approximately 60.5 million people in the U.S. with four-year college degrees more more. That’s just under 28% of the estimated population of the U.S. in 2007 that was at least 18 years old.

According to the Bureau of Labor Statistics, if we take all occupations in management, legal, computers and mathematics, architecture and engineering, healthcare providers and technicians, and business and finance as individuals who might qualify as “opinion elites” by profession, that’s a total of approximately 26.1 million people out of a total of approximately 253 million people employed in 2007. That’s only about 10% of the workforce.

Finally, we’ll have to make an educated guess about how someone was defined as having “a high degree of involvement in politics and policy matters.” It’s probably fair to say that the nation’s 65,000 legislators and 15,000 lobbyists in 2007 qualified, but they are certainly not alone (and they’re already counted in the workforce numbers just described). If we assume that there are about 20x more people in the U.S. who meet the survey’s political participation threshold than there is legislators and registered lobbyists, then that’s approximately 1.6 million people who meet this criteria as well. That’s only about 0.4% of the population.

As you can see, there’s a significant problem with this definition – it applies to such a small number of individuals in the U.S. that it’s difficult to generalize the results. At best there’s between 1 and 2 million people who meet all the requirements. In reality, however, the number of “opinion elites” will be much smaller given the number of people who meet most but not all of the criteria of the definition. Unfortunately, without more information, we can’t know how much smaller, except that there are at least 600 “opinion elites” in the U.S. – that’s how many were surveyed, after all.

So much for the available survey methodology notes – on to the survey’s results.

The first claim based on the definition of “opinion elite” is that the survey “found that 72 percent of opinion leaders nationwide support the use of coal for generating electricity.” Right away this claim raises an important question: what is the margin of error in this number? The press release doesn’t say.

The second claim is that the 72% represented “a significant increase over the past year and the highest level of support since the group began polling almost 10 years ago.” How is this possible when the ACCCE didn’t exist 10 years ago – it was founded in 2008. Is the press release talking about the polling organizations, AmericanPublic.us and RT Strategies, or one of the ACCCE’s precursor organizations?

The third claim is that “the poll shows that Americans are very optimistic about the future for coal. When asked the question ‘do you believe coal is a fuel for America’s future?’ — 69% of Americans agreed (compared to only 26% who disagreed).” This claim is an outright lie. According to the survey’s own methodology, they polled only 600 people who qualified as “opinion elites.” It is not possible to generalize the results of a survey from a tiny selected minority out to the entire population at large. It would be like polling just my immediate family about our car buying habits and then trying to apply that to my entire community.

The last two claims related only to “those surveyed,” and you can always draw conclusions about the survey’s respondents from their answers. But without a statistical method to correlate your data with the wider population, you cannot draw any conclusions for the wider population. And there’s no suggestion in these last two claims that there is any correlation to all American or even just to all “opinion elites.”

In summary, the survey as its described in the ACCCE press release suffers from a number of critical problems: an amazingly tight definition of the survey’s population, no defined margins of error for any of the questions individually or for the survey in its entirety, an unexplained discrepancy regarding how many years the poll has been taken, and an outright lie. I requested a copy of the survey’s methodology last week and did not receive any response.

So what can we surmise about the survey and the related web advertisement from these survey claims? Since we don’t have the actual survey as a reference, we have to make educated guesses from what information we do have.

First, we can probably say that the participants of this survey were either a) selected by the polling organizations or b) self-selected their own participation in the survey. We can make this educated guess because of the lack of any margins of error and because of the phrase “those surveyed” for the last two claims. Selected polls are only valid for the individuals selected and so statistics cannot be used to extrapolate the poll data to the rest of the target population.

Second, we can probably say that this press release is an attempt to manipulate people, and for more reasons than “that’s what press releases exist for.” There’s one blatant lie, possibly a second (the “10 years” claim), and a huge number of unanswered questions. In addition, the page that the advertisement links to has four different links on it – that all go to the exact same survey press release. This suggests that the ACCCE online communications expert wanted to make that first page look like there was more research supporting for the ACCCE advertisement than there really was.

And third, given the first two educated guesses, it’s almost certainly accurate to say that the ACCCE advertisement is unsupported bunk.