A remarkable thing happened in Silicon Valley during the past decade. Venture capitalists and entrepreneurs set their sights on clean energy as the Next Big Thing. They audaciously hoped to reinvent energy by harnessing the incredible innovation that had transformed information technology and biotechnology.

Some of the best venture capitalists in the business, including my friends Bill Joy and Vinod Khosla, detached from their computing roots and focused on energy startups. The result was a staggering surge of capital into clean-energy technologies. Worldwide, from 2006 to 2010, about $535 billion in venture capital, private equity and initial public offerings as well as mergers and acquisitions flowed into 4,236 clean-tech businesses, according to a recent analysis by GlobalData.

Venture-capital investing is inherently high-risk, so it shouldn’t surprise or bother anyone that many of these startups failed ― some rather spectacularly. Solyndra LLC, the solar-cell company, for example, went bankrupt even after receiving a $535 million in loan guarantees from the U.S. Energy Department. But similar failures happened during the dot-com bubble. Remember pets.com and its infamous sock-puppet TV ads?

What is worrying is that almost a decade of energy investing hasn’t produced any home runs ― no green-energy equivalents of eBay, Amazon, Google or Facebook. The modest, incremental advances we have seen don’t perceptibly move the needle on the energy problem.

That’s not to say there aren’t good companies that swear they are just about to produce their miracle; in fact, my own company has spawned a startup ― called TerraPower ― that has developed a pretty amazing set of advanced technologies for nuclear energy. Let’s hope a few of us turn out to be right.

In the meantime, however, a real revolution has happened in traditional energy ― one that poses a serious challenge to companies and investors betting on alternative energy. This breakthrough is arguably one of the greatest advances in energy production since the 1960s. And it came not from a Silicon Valley company, or from MIT or Stanford, but from the son of a Greek goatherd who immigrated to the U.S.

George Mitchell was born in Galveston, Texas, went to Texas A&M University and, in 1946, founded an oil-drilling and real-estate business. The company did well, and in the 1980s, Mitchell decided to take on a major technical challenge: He would try to coax gas out of a portion of the Barnett shale, which lies deep under Fort Worth and 15 counties in north-central Texas.

People told Mitchell he was wasting his money; you can’t squeeze blood from a stone, and you can’t squeeze oil or gas out of shale, which is essentially fossilized mud. Huge amounts of natural gas have formed in layers of shale, but it’s trapped within the rock and doesn’t flow toward a borehole.

The same is true of vast gas deposits that are stuck in coal beds too deep to mine, and gas that saturates spongelike sandstones and other semiporous rocks. Pulling out this “tight gas,” as drillers call it, is like trying to suck a thick milkshake through a thin cocktail straw or to breathe through a pillow.

But Mitchell was stubborn. He and his roughnecks doggedly tinkered with a variety of long-known techniques that had never been used in combination. One of these was horizontal drilling, which originated in the 19th century, was adapted for oil production by the Soviets in the 1930s and was perfected by oil drillers in the 1980s.

A second idea was to inject fluid into the rock to fracture it into lots of pieces, thus allowing the gas and oil inside to flow more easily. In 1865, Colonel Edward Roberts, a Civil War veteran, demonstrated (and patented) the use of explosive nitroglycerin for this purpose ― which worked amazingly well, but was quite dangerous. By the 1940s, engineers had developed a gentler approach that uses high-pressure water and chemicals rather than explosions to break up the rock. It became a standard practice for some oil, gas and water wells.

A third technique that Mitchell tried was adding sand to the water to help prop open the cracks that formed in the rock. Together these approaches, collectively called hydraulic fracturing, or “fracking,” allowed drillers to inexpensively recover gas from the tight Barnett shale. Mitchell earned nothing for developing the technology, but his company went on to make a lot of money on gas leases.

A great many others in the energy industry have done the same, as the arrival of fracking unlocked enormous deposits of shale gas, tight gas and coal-bed methane across the U.S. and in other countries as well. Mitchell’s miracle has more than doubled the known reserves of natural gas.

The new resources are so vast that they would last for a century at current rates of gas consumption. And this cheap form of energy isn’t under the control of a foreign dictator, stuck in the Arctic or submerged miles below the sea ― it lies in the farmlands of New York, Pennsylvania and Texas.

The location turns out to be something of a mixed blessing; yes, these places are secure and politically stable, but they are also home to not-in-my-backyard activists who claim fracking threatens to pollute groundwater, in some cases to the point where flames will spew from people’s shower heads.

Industry experts says that is unlikely because the gas-containing rock typically lies 2 miles or more underground, in strata that are geologically isolated from groundwater. However, any form of natural-gas production can produce some environmental issues because it must be piped to the surface, and gas sometimes leaks into groundwater through breaks in the pipe, or through abandoned wells.

Another problem is that the water pumped underground for fracking gets contaminated in the process, and much of that waste comes back up and must be stored. As for the rare flaming faucet, it’s hard to tell whether fracking is to blame, because the same regions where this technique is most profitable tend to have shallower, natural deposits of gas that can contaminate groundwater without any help from industry.

So far, scientific evidence has not clearly linked the fracking procedure itself to groundwater contamination. But if there is anything we learned from the BP oil spill in the Gulf of Mexico last year, it is that any technology that produces energy in large quantities poses some environmental risk. So as a society, we face an interesting question: Would we rather depend for our energy on distant suppliers in the Mideast and elsewhere? Or is it better to produce the energy ourselves and accept the risk of creating some messes in our backyards?

Before answering that, it’s important that we understand a few other important caveats to Mitchell’s gas miracle: First, natural gas is not as useful as oil. Although some cars and buses run on natural gas, their range is limited because, even in its compressed form, natural gas doesn’t have the energy density of gasoline or diesel fuel. Also, you can’t use natural gas very efficiently for boats or airplanes. So until researchers come up with a cheaper gas-to-liquid conversion process, we will still need oil for some things.

A more serious problem is that both collecting and burning natural gas produce carbon dioxide, the main culprit in global warming. Gas producers never miss a chance to point out that burning gas emits less CO2 than burning coal does. But detailed research (including some of my own) shows that, in practice, switching from coal-fired electricity plants to gas-fired ones would have almost no effect on global warming. In fact, some scientists who have tried to do a full accounting of the emissions due to gas produced by fracking have concluded that it actually ends up putting more CO2 into the atmosphere than coal does.

What’s more, cheap gas just reinforces our use of carbon-based energy. New technologies are always risky and ― as George Mitchell found ― they almost never work perfectly from the start. So investors need an incentive to take that risk. Getting an alternative-energy technology off the ground is much easier if the price of conventional energy is high, either because it is taxed or because the fuel is genuinely scarce.

A carbon tax could provide this price incentive, but with the world teetering on the edge of financial ruin, the political appetite for new taxes has evaporated.

Not so long ago, many people believed that the cost of oil and gas would rise indefinitely, thus supporting the market for alternatives. Mitchell’s miracle has changed that calculus, much to the chagrin of the Silicon Valley venture capitalists who caught the green-energy bug. 

By Nathan Myhrvold

Nathan Myhrvold, the former chief strategist and chief technology officer at Microsoft Corp. and the founder and chief executive officer of Intellectual Ventures, is a Bloomberg View columnist. The opinions expressed are his own. ― Ed.

(Bloomberg)