Let’s start with a basic observation: Over the past few years, investors have decided that commodities are a valid asset class, just like stocks and bonds.

In a traditional portfolio an investor might decide to allocate some percentage of his or her assets into stocks, some into bonds of various types, and to keep some cash. For more and more investors, however, commodities are becoming part of that allocation.

As a result, financial institutions have created a number of products designed to track the performance of specific commodities or of popular commodity indexes such as the Goldman Sachs Commodity Index (GSCI).

Because crude is the most heavily weighted commodity in the majority of commodity indexes, the argument goes that the steady flood of cash into commodities indexes is behind the big surge in prices of crude oil since 2004-05.

Those who make the simple argument that speculators are to blame for higher oil prices will tell you that, despite bearish oil fundamentals right now, money blindly invested in commodity indexes is causing oil prices to rally far above where they otherwise would be.

By extension, regulations limiting position sizes for futures speculators would result in a permanent lower price for crude.

One of the more eloquent and well-researched proponents of this speculation theory is Michael Masters, who testified before the Senate Committee on Homeland Security and Government Affairs in May 2008. The following chart is taken from Masters’ testimony.

Source: Michael W. Masters. Chart from: US Senate. Hearing of Committee on Homeland Security and Government Affairs. 05/20/2008.

Masters used data from the Commodity Futures Trading Commission’s (CFTC) weekly Commitment of Traders Report to construct this chart. The bars at the bottom show the cash being invested into these passive commodity index funds; Masters even breaks down these billions in investment funds by the particular commodity index tracked.

At first glance, this chart looks compelling because the explosion in speculation appears to occur after 2003, just as oil prices began to explode to the upside.

Undoubtedly, speculators can influence crude oil prices in the short term. But the simple argument just doesn’t make economic sense. Consider the following quote from Senator Bernie Sanders (D-VT) on Tuesday at a CFTC hearing to consider tighter regulations of futures markets:

[T]he bottom line is that we have got to make sure that Americans are no longer ripped off at the gas pump by the same Wall Street gamblers responsible for the worst economic crisis since the Great Depression.

Mr. Chairman, we all learned in our economics 101 text-book that when supply is low and demand is high, prices are supposed to go up; and when supply is high and demand is low prices are supposed to go down. That is a concept that most Americans can understand.

Source: Sen. Sanders (VT). Quote from: Commodity Futures Trading Commission. Hearing on Energy Position Limits and Hedge Exemptions. 07/28/09.

It’s ironic that Sen. Sanders discusses Economics 101 as basic economics explains why the speculation argument is rubbish and why oil prices rose from 2004 through mid-2008.

Those who argue speculators are to blame are essentially saying that there are two markets for oil, a physical market composed of producers and consumers of oil and a paper market made up of speculators on New York Mercantile Exchange (NYMEX) or other futures exchanges. These pundits and politicians conclude that activity in the paper oil market is pushing up prices paid in the physical market.

But there aren’t two markets for crude oil. The paper and physical markets are intimately related. For Example, producers use the futures markets to hedge their exposure to oil prices; right now, crude oil futures for September 2010 are trading at USD76 per barrel, so a company could sell these futures right now to lock in USD76 oil a year from now.

Consumers have to buy gasoline to put in their cars, trucks and SUVs. Because the retail price of gasoline is related to the price of oil and gasoline futures traded on NYMEX, physical oil consumers’ behavior is also influenced by NYMEX oil prices.

In fact, this appears to be the kernel of Sander’s argument that the American consumer is being “ripped off” because futures market speculators are pushing oil prices higher.

Logically, if no one thought futures market speculators were pushing prices above the level at which the physical market would clear, no one would bother holding a hearing about the practice.

For illustrative purposes, let’s say that oil prices would be at USD50 without the influence of speculators, but futures market buying activity pushed crude oil to USD100.

Economics is essentially the study of how scarce goods (like oil) are allocated by society. Prices are nothing more than a signal for people who consume a good and for those who produce that good.

“Price” is the signal that allows the free market to allocate scare goods such that the quantity of goods demanded equals the supply of that good available.

At an artificially high price of USD100 a barrel, producers would want to supply more oil to the market. Rising prices would act as a signal for producers to spend more money on crude oil exploration so that they could bring more oil to market.

It would also act as an incentive for producers to tap more expensive reserves such as Canada’s oil sands or deepwater Brazil–at higher prices these resources become economic.

But when it’s at USD100, consumers want to use less oil. Consumers might decide sell their SUVs and buy a Prius or simply drive less. The quantity of crude demanded at USD100 is less than it would be at USD50.

In other words, rising prices also signal a need to reduce demand and bring prices back in line with supply.

If oil prices were artificially inflated by speculators, basic economics suggests we’d see some or all of the following three major effects:  rising supplies as producers produce more oil to take advantage of higher prices; falling demand as consumers try to use less; and a build in crude oil inventories as the artificial price signal leads to too much supply and too little demand to clear the physical market.

Let’s apply the theory to recent facts. Those who blame speculators for the rise in oil prices in recent years suggest speculation became a problem in 2004 and reached a fever pitch in mid-2008 when oil prices neared USD150 a barrel.

If these observers were correct, during this period the world should have experienced a massive jump in oil inventories, increased oil output from producers, and relatively sluggish demand for oil. These factors would cause oil inventories to build globally.

Here’s a chart of US oil inventories from the end of 2003 through the middle of 2008.

Source: International Energy Agency, Bloomberg

To create this chart I added US stocks of oil, gasoline and distillate (heating oil). As you can see, inventories bounced up and down considerably, but there’s no obvious trend toward inventory buildup from the end of 2003 through the middle of 2008. Data from the International Energy Agency (IEA) show a similar picture for the developed world as a whole.

Obviously oil producers can’t respond immediately to rising prices because it takes time to mobilize rigs and identify new projects. But if the run-up in commodity prices from 2003 to 2008 were truly artificial, we should have seen oil inventories build at some point.

This brings us to supply. The chart below shows the number of active natural gas drilling rigs outside North America.

Source: Bloomberg

As you can see, global producers responded to rising oil prices by drilling for more oil. In fact, oil and gas producers both within and outside OPEC poured record sums into new oil and gas development projects over this time period.

Unfortunately, that investment wasn’t enough. Non-OPEC oil production increased from 44.85 million barrels a day in 2003 to 45.12 million in 2008, with all of that increase coming from the former Soviet Union countries. That’s an increase of less than 300,000 barrels a day in a global oil market of 84.5 million barrels a day, not quite literally a drop in the bucket but close.

Of course, with non-OPEC production growth sluggish, OPEC did fill the gap, increasing its production by more than 4.5 million barrels a day between 2003 and 2008. But OPEC’s production increases came directly out of spare capacity, oil production capacity OPEC can bring on-line quickly and sustain. More broadly, spare capacity is the world’s cushion against sudden demand or supply shocks.

According to the IEA OPEC’s effective spare capacity stood at no more than 1.7 million barrels a day in July 2008, down from 2003-04 levels and well off the 3 million to 4 million barrels a day common in the 1990s.

As for demand, check out the chart below.

Source: BP Statistical Review of World Energy

This chart shows the year-over-year increase in global oil consumption from 1974 through 2007. As you can see, global oil demand growth on a percentage basis hasn’t been at all sluggish in recent years. In fact, you have to go back to the ’70s to find a similarly significant period of strong growth in demand.

Another way to look at this is that in the ’70s global oil demand growth averaged 1.58 million barrels a day followed by 500,000 in the ’80s and 950,000 in the ’90s.

From 2000 through 2007 global oil demand grew at 1.25 million barrels a day, the fastest pace since the ’70s.

More broadly, the price signal did appear to work in the developed world. Countries in the Organization of Economic Cooperation and Development (OECD), including the US, saw consumption decline between 2003 and 2008. The wild car was, of course, the developing world, where consumption soared.

As incomes rise in the developing world, consumers want to buy cars and, therefore, use more oil. Ironically, poorer developing countries are less sensitive to price than the developed world. The power of the wealth effect to influence consumer behavior overcame higher gasoline prices.

Consumer and producer behavior from 2004 through mid-2008 is not consistent with artificially high oil prices. The speculation argument has little basis in reality.

The real cause of rising prices is unusually strong demand growth coupled with sluggish supply response despite record spending.

As Sen. Sanders noted at yesterday’s CFTC hearing, prices rising in the face of high demand and low supply is simply Economics 101.

Speaking Engagements

Eight score and one year ago, with the onset of the California Gold Rush, San Francisco earned a reputation as a prospector’s town. It’s time again to seek paths to prosperity–and to enjoy one of the most beautiful natural settings in the US, if not the world.

Venture west for the San Francisco Money Show Aug. 21-23, 2009, at The San Francisco Marriott and discover how Elliott Gue, Roger Conrad and GS Early can help you profit in these adventurous times.

Elliott will detail the new direction for Personal Finance and provide his forecast on energy markets for 2009. Roger will discuss utilities, Canadian income and royalty trusts as well as his new service focused on exploiting the greatest spending boom in history, New World 3.0. GS, a constant at PF for two decades, will be there to speak on emerging tech, nanotech and defense tech.

Click here or call 800-970-4355 and refer to priority code 014315 to register as a guest of The Energy Letter.