If you have a pulse, you’ve probably been asked to answer questions for a poll or six this season. Most polls I’ve been asked to participate in have been political polls, but with oil prices high, there have been poll commissioned by the GOP, the Democratic Party, and various third parties, each of which is hoping that the polls will support their particular political position.

Unfortunately, one of the polls I came across recently is one that I’d really, really like to believe in. But the poll results themselves are all but meaningless. Specifically, a Wilderness Society poll produced by Belden Russonello & Stewart, purports to show that the American public isn’t buying into the GOP lies that drilling on the outer continental shelf, in the Arctic National Wildlife Refuge, and mining for oil shale in 2018 will lower prices today. Unfortunately, the language of the poll questions is sufficiently biased that the poll is nearly useless for anything but manipulating the media into spouting bogus statistics in an attempt to counter other bogus statistics.

Let’s look at the questions in detail.

Question #1 says the following:

The U.S. has certain public lands and offshore areas that are protected from oil and gas drilling. Should Congress open the protected areas to oil and gas drilling or should Congress continue to protect these areas? Do you feel that way strongly or somewhat?

There are a few problems with this question. First, there’s a “don’t know” answer, but there isn’t a “don’t care” response. The question forces people who may honestly not care about either option to choose to open areas to drilling or to continue to protect them. Second, the language itself is biased by the use of the words “protect” and “protected”. This language defines a frame – protection – instead of permitting the polled person to define their own frame. It’s just as biased as a question framed from the opposing viewpoint, such as the one I constructed below:

The U.S. has certain public lands and offshore areas where oil and gas drilling isn’t allowed. Should Congress open those areas to gas and oil drilling or should Congress continue to block development of those areas?

If the Wilderness Society had wanted an actual, unbiased response to this question, they could have had the question reworded by their consulting firm to not use biased language at all. Third, this is really two questions merged into one – a person could easily be in agreement with protecting offshore areas and open to drilling in protected lands, yet the answerer is required to choose one answer for both areas. And fourth, words like “somewhat” and “strongly” are intended to gage a person’s strength of feeling on an issue, with “somewhat agree” being between “agree” and “don’t care”. Similarly, “strongly agree” measures a level of agreement that is greater than “agree”. So, where are the “agree” or “disagree” options to answer this question? The options are:

• Strongly open to drilling
• Somewhat open to drilling
• Somewhat continue to protect
• Strongly continue to protect
• don’t know/refused to answer

So much for an accurate spectrum of possible answers.

A better question would be the following:

State whether you agree, somewhat agree, don’t care, somewhat disagree, or disagree with the following statements:

1. Congress should open up presently closed public lands to oil and gas drilling.
2. Congress should open up presently closed offshore areas to oil and gas drilling.

Question #2 says the following:

Looking to the future, which one of the following do you think should be a more important priority for government: Investing in new energy technology including renewable fuels and more efficient automobiles; or expanding exploration and drilling for more oil?

As with question #1, there’s not a “both” or a “neither” option for this question. And while it’s technically not an either/or question (it’s asking about what’s more important, not whether one is important and the other unimportant), a “both are equally important” option would still be a good idea. Without it, it sets the two positions up as opposing each other, when in fact they don’t have to be.

Question #3 says the following:

Do you think that allowing oil companies to drill in public lands and offshore areas that are currently off limits to drilling will result in lower gas prices for American consumers or not?

This question would be reasonably clean – if it wasn’t after two skewed questions and didn’t seem like it was trying to paint oil companies as land grabbers. Perhaps a better yes/no question might be the following:

The U.S. Energy Information Administration says that expanding oil development into lands and offshore areas will not increase oil supplies until 2018 at the earliest. Do you think that opening those areas today to development will lead to lower gasoline prices tomorrow?

Questions 4 through 7 are all agree/disagree/don’t know questions:

Q4. Since we have already opened up most of our public lands to oil drilling and gas prices have not gone down, there is no reason to believe that opening more land to drilling will lower gas prices.

Q5. The U.S. has less than 3 percent of the world’s oil reserves, therefore, opening protected public lands to oil drilling will not produce enough oil to affect the price of gas.

Q6. The small percentage of public lands still protected from oil drilling should remain off limits because they are valuable natural resources that cannot be replaced.

Q7. The President’s proposal to open up public lands to oil and gas drilling is more likely to enrich oil companies than to lower gas prices for American consumers.

These questions aren’t even trying to gauge public opinion and have no business being in a poll that supposedly does.

Question $4 is true, but so oversimplified that it barely resembles reality. Let’s create a hypothetical – the U.S. has 50% of it’s public land open to oil and gas drilling and the price is at $50 per barrel. Then the U.S. opens an additional 45% of its public lands to oil and gas drilling, but those lands have no oil or gas under them – that last 5% that’s still protected does. Naturally opening up those 45% of lands didn’t do anything for oil prices, because there wasn’t any oil there. The situation in the U.S. doesn’t resemble this hypothetical in the least, but the hypothetical makes my point as to why this question is meaningless.

Question #5 is true, but also oversimplified. Included in the GOP’s “public lands” is oil shale in the Green River Formation of Utah, Colorado, and Wyoming, and the shale is estimated to hold WAY more oil than Saudi Arabia ever did. Shale oil isn’t included in “oil reserves”. Environmentalists and residents of those states (such as myself) usually oppose shale development because a) mining and refining shall requires 3 barrels of water for every barrel of oil extracted, b) the oil requires a great deal of energy to refine since it’s in the form of kerogen, not Arabian light sweet crude, c) in-situ extraction of kerogen from the ground requires huge power plants to heat up the rocks to 700 degrees Fahrenheit to melt the kerogen out and we’re worried that the process will poison ground water and wells and create huge plumes of air pollution from the likely coal-burning power plants.

Question #6 is more or less fine because it’s so simple. It tests for classical environmentalist views on things, and it’s hardly a crime to hold such views. The survey is so bad that the results are effectively meaningless, but whatever.

And Question #7 is very nearly pure propaganda. A yes/no question about enriching oil companies instead of lowering gas prices? Please, if you’re going to try and frame a statement for your press release, make it a little less blatant, or better yet, don’t include it in the bogus survey at all.

Speaking of the press release, let’s look a little closer at how The Wilderness Society (TWS) is using this lovely little poll. TWS talks about how the public doesn’t want the GOP to drill in ANWR, mine oil shale, or open the outer continental shelf (OCS) to drilling, yet nowhere in the 7 poll questions are any of those three mentioned directly. In other words, they’re twisting the results of the poll already, and we’re not even out of the first paragraph.

TWS also says “the public overwhelmingly believes (76% to 19%) that policymakers should focus on investing in new energy technologies including renewable fuels and more efficient vehicles rather than expanding exploration and drilling for more oil.” This is a semi-accurate representation of the question and results, although this comes from Question #2, which lacks enough answer options to be a good question. Question #2 also asks which option is more important, while TWS phrases it as an either/or result, tweaking the question’s results. What I find most interesting about this press release is that its author choose to focus on the results to Question #2 twice, both in the first paragraph and again in the last, instead of detailing the answers to the other questions.

Most people lack the time or energy to verify that they’re not being manipulated by unscrupulous organizations intending to make political hay with a poll, and this goes for journalists as well as readers. And too many “experts” on polling are the ones creating the bogus polls in order to further their own particular ideology or to pay the bills. And yet polling is used all the time, precisely because it can be so easily manipulated by people with axes to grind. In this case, TWS used a poll to make a political point and to provide environmentally-friendly and anti-carbon energy/pro-renewable energy politicians something to point to as they resist the GOP drilling machine. Unfortunately, the TWS poll is nothing more than a political tool, and a blunt one at that. A good quality poll would have impaled the GOP upon a punji stick coated in their own lies – this poll skewered its creators, The Wilderness Society.

Other posts on the subject of oil prices you might be interested in at S&R:

• GOP to Dems – solve energy crisis by next week
• July 23 Carboholic
• July 16 Carboholic
• Republicans and the oil weapon – what Obama should do
• July 9 Carboholic
• Gingrich’s “Energy Independence Day” makes false promises

Thanks to JSO and Denny for help teasing out some of the biases in the questions above