From the beginning — that is, since the advent of the net neutrality concept early last decade — I argued that a strict neutrality regime would have outlawed, among other important technologies, CDNs, which prioritized traffic and made (make!) the Web video revolution possible.

So I took particular notice of this new interview (sub. required) with Akamai CEO Paul Sagan in the February 2011 issue of MIT’s Technology Review:

TR: You’re making copies of videos and other Web content and distributing them from strategic points, on the fly.

Paul Sagan: Or routes that are picked on the fly, to route around problematic conditions in real time. You could use Boston [as an analogy]. How do you want to cross the Charles to, say, go to Fenway from Cambridge? There are a lot of bridges you can take. The Internet protocol, though, would probably always tell you to take the Mass. Ave. bridge, or the BU Bridge, which is under construction right now and is the wrong answer. But it would just keep trying. The Internet can’t ever figure that out — it doesn’t. And we do.

There it is. Akamai and other content delivery networks (CDNs), including Google, which has built its own CDN-like network, “route around” “the Internet,” which “can’t ever figure . . . out” the fastest path needed for robust packet delivery. And they do so for a price. In other words: paid priority. Content companies, edge innovators, basement bloggers, and poor non-profits who don’t pay don’t get the advantages of CDN fast lanes.

So important are CDNs in today’s Internet architecture that the FCC felt the need to explicitly exempt them. So much for a “neutral” policy.

In footnote 235 and elsewhere, the FCC seems to think CDNs are all about geographic advantages (to overcome speed-of-light delay) and server-consolidation (yielding infrastructure efficiencies to content and app providers). True enough, but the FCC ignores what Sagan believes is his key service — real-time prioritization.

The net neutrality Order allows CDN prioritization but says paid priority by broadband service providers on the customer link “would raise significant cause for concern,” and “as a general matter, it is unlikely that pay for priority would satisfy the ‘no unreasonable discrimination’ standard.”

Numerous legal infirmities are likely to kill the entire Order when reviewed by the courts. But the substantive technological problems and inconsistencies embodied in the paid priority section need to be examined. Parsing these items should help us all understand how this very complex thing we call the Internet works . . . perhaps chasten the FCC when it attempts to enforce its new rule . . . and hopefully inform a better policy should this Order be vacated.

The FCC fails to explain persuasively why CDN prioritization is much different from last-mile broadband prioritization. Even if a meaningful distinction were demonstrated, we’d still be left with a probable ban on last-mile prioritization, which is a problem because last-mile prioritization will likely prove essential for a variety of real-time communication services.

The FCC’s case really rests on Section II.B, which is all about the supposed motives of broadband service providers. Apparently, the FCC thinks CDNs should be exempt because they have no malign intent whereas broadband service providers are awash in “incentives to limit Internet openness.”

The FCC says broadband providers may have incentives to

(1) disadvantage some edge providers by blocking or controlling the transmission to end-users;

(2) charge edge providers for access or priority to end-users; and

(3) “degrade or decline to increase the quality of the service they provide to non-prioritized traffic.”

The “no blocking” principle that all parties agreed to way back in 2005 would seem to take care of ominous incentive (1). And a much simpler reliance on existing consumer and/or antitrust law could easily defang ominous incentives (2) and (3).

But these specifics fail to address the larger point: Is it even true that most incentives steer broadband service providers toward a less open Internet? Did the FCC even consider incentives that point in the opposite direction — toward a more open Internet? I can’t find any evidence they did.

The FCC apparently cannot see that the vast bounty of content and apps on the Web created an entirely new market for telcos and cablecos. Namely, broadband Internet access. (Of course, in the typical two-sided coin of positive-sum innovation, it was broadband that created a new market for content and apps.)

As I wrote in The Wall Street Journal way back in 2006:

Blocking and degrading Internet access would quite simply be business suicide for incumbent service providers. Compared to cable’s other content operations like basic and premium TV channels, its broadband cable modem services are more than 50 times as profitable per unit of bandwidth consumed. This means that with just a tiny sliver of the usable bandwidth in its pipes, cable’s Internet services supply about 20% of the revenue and the majority of their net income. Does anyone really think the bandwidth providers are going to kill their golden goose?

Yes, it’s true that popular broadband Internet services also cannibalize some existing products. Long distance voice went away, and Web video is now beginning to compete with cable TV. But the telcos and cablecos know they will never be the key creators of content and apps — not compared to the rest of the world. They get the wonderful Web for free. Compared to hefty sums they must pay for TV content (e.g., ESPN) or movies, broadband access is a simple business. Why decimate the value of one of their three basic products (the others being TV and mobile) by closing off big chunks of the Web to their customers? It would be dumb. It won’t happen. It hasn’t happened.

So the FCC twists itself in pretzels trying to use a high-minded “net neutrality” or “open Internet” policy to do what it really wants — the much more vulgar task of regulating the telcos and cablecos — while exempting (for now) technologies and services that it knows are absolutely essential to a well-functioning Internet but are not at all “neutral” or “open” according to the FCC’s own criteria.

I’m glad CDNs are not covered by the rules, but in its feeble attempt to explain the broadband-CDN distinction, the FCC overlooked the fact that the no-priority rules could limit or block innovation in key real-time communications services.

CDNs are good for speeding static content and even putting some broadcast content (like live sports events) onto fast-lanes to consumers. (Akamai’s record traffic day to that point was its Web broadcast of the Masters golf tournament last spring.) But when it comes to decentralized, real-time, interactive, unpredictable, transactional content — like high-resolution video conferencing or online gaming — CDNs won’t do. We will need some form of in-stream prioritization.

But this is a matter of engineering and economics. There are several ways to achieve quality-of-experience for multimedia applications. We can create virtual (logical) channels using packet priority technologies. We can dedicate frequency (analog) channels using cable-TV like banding or, in the optical realm, wavelength division multiplexing (WDM) within a wire. Or we can deploy more wires (or wireless spectrum). There are technical and financial tradeoffs between using computer power (switching) and communications power (bandwidth) to achieve these ends. They depend on the state of the existing infrastructure, the cost-performance ratios of the technologies and resources (which change over time), and the strategic architecture and business model of the network. In the end we will either prioritize digitally on the last-mile link or prioritize incoming/outgoing traffic onto/from a frequency channel just outside the FCC “no priority” zone. But that will then raise the question of whether providing such channels is itself paid priority. Do we begin to see why these are not questions that can be answered by crude politics?

By exempting CDNs, the FCC acknowledges their herculean task in delivering multimedia to the masses. Akamai cogently described the state of play when unveiling its new HD video service:

Why is it so difficult to deliver an optimal end-user experience? Simply put, it’s challenging to reliably deliver high-throughput data streams across the Web – especially to large audiences.

The fundamental challenges of online video delivery lie within the Internet itself. Given that the Internet is made up of over 13,000 competing networks, it works surprisingly well. But its many bottlenecks and capacity limitations are unpredictable and difficult to manage – lying outside the control of any single entity or group. While these problems affect the delivery of all types of Internet content, they are particularly challenging for video, which requires the transfer of large data volumes at very high rates.

But if delivering static and broadcast content is a challenge, doubly so for the coming wave of real-time interactive cinema-quality video. It’s a challenge CDNs cannot conquer and will require new technologies, architectures, and business models — many of them requiring some form of broadband prioritization — to master.

It would be a shame if the FCC’s new Order interrupted this essentially technological and economic story because of a dubious analysis that narrowly focuses on the supposed political motivations of broadband service providers and completely ignores powerful incentives pushing all parties toward Internet openness.

What the per capita metric really measures is household size. And because the U.S. has more people in each household than many other nations, we appear worse in those rankings. But as the Phoenix Center has noted, if each OECD nation reached 100% broadband nirvana — i.e., every household in every nation connected — the U.S. would actually fall from 15th to 20th. Residential connections per capita is thus not a very illuminating measure.

But look at the actual Internet traffic generated and consumed in the U.S.

The U.S. far outpaces every other region of the world. In the second chart, you can see that in fact only one nation, South Korea, generates significantly more Internet traffic per user than the U.S. This is no surprise. South Korea was the first nation to widely deploy fiber-to-the-x and was also the first to deploy 3G mobile, leading to not only robust infrastructure but also a vibrant Internet culture. The U.S. dwarfs most others.

If the U.S. was so far behind in broadband, we could not generate around twice as much network traffic per user compared to nations we are told far exceed our broadband capacity and connectivity. The U.S. has far to go in a never-ending buildout of its communications infrastructure. But we invest more than other nations, we’ve got better broadband infrastructure overall, and we use broadband more — and more effectively (see the Connectivity Scorecard and The Economist’s Digital Economy rankings) — than almost any other nation.

The conventional wisdom on this one is just plain wrong.

Hooray. We live in a nation of laws and elected leaders, not a nation of unelected leaders making up rules for the rest of us as they go along, whether in response to besieging lobbyists or the latest bandwagon circling the block hauled by Washington’s permanent “public interest” community.

This was the reassuring message yesterday from the D.C. Circuit Court of Appeals aimed at the Federal Communications Commission. Bottom line: The FCC can abandon its ideological pursuit of the “net neutrality” bogeyman, and get on with making the world safe for the iPad.

The court ruled in considerable detail that there’s no statutory basis for the FCC’s ambition to annex the Internet, which has grown and thrived under nobody’s control.

. . .

So rather than focusing on new excuses to mess with network providers, the FCC should tackle two duties unambiguously before it: Figure out how to liberate the nation’s wireless spectrum (over which it has clear statutory authority) to flow to more market-oriented uses, whether broadband or broadcast, while also making sure taxpayers get adequately paid as the current system of licensed TV and radio spectrum inevitably evolves into something else.

Second: Under its media ownership hat, admit that such regulation, which inhibits the merger of TV stations with each other and with newspapers, is disastrously hindering our nation’s news-reporting resources and brands from reshaping themselves to meet the opportunities and challenges of the digital age. (Willy nilly, this would also help solve the spectrum problem as broadcasters voluntarily redeployed theirs to more profitable uses.)

International comparisons in such a fast-moving arena as tech and communications are tough. I don’t pretend it is easy to boil down a hugely complex topic to one right answer, but I did have some critical things to say about a major recent report that got way too many things wrong. A new article by that report’s author singled out France as especially more advanced than the U.S. To cut through all the clutter of conflicting data and competing interpretations on broadband deployment, access, adoption, prices, and speeds, however, maybe a simple chart will help.

Here we compare network usage. Not advertised speeds, which are suspect. Not prices which can be distorted by the use of purchasing power parity (PPP). Not “penetration,” which is largely a function of income, urbanization, and geography. No, just simply, how much data traffic do various regions create and consume.

If U.S. networks were so backward — too sparse, too slow, too expensive — would Americans be generating 65% more network traffic per capita than their Western European counterparts?

As Washington and the states pile up mountainous liabilities — $3 trillion for unfunded state pensions, $10 trillion in new federal deficits through 2019, and $38 trillion (or is it $50 trillion?) in unfunded Medicare promises — the U.S. needs once again to call on its chief strategic asset: radical innovation.

One laboratory of growth will continue to be the Internet. The U.S. began the 2000’s with fewer than five million residential broadband lines and zero mobile broadband. We begin the new decade with 71 million residential lines and 300 million portable and mobile broadband devices. In all, consumer bandwidth grew almost 15,000%.

Even a thriving Internet, however, cannot escape Washington’s eager eye. As the Federal Communications Commission contemplates new “network neutrality” regulation and even a return to “Title II” telephone regulation, we have to wonder where growth will come from in the 2010’s . . . .

Later this week the FCC will accept the first round of comments in its “Open Internet” rule making, commonly known as Net Neutrality. Never mind that the Internet is already open and it was never strictly neutral. Openness and neutrality are two appealing buzzwords that serve as the basis for potentially far reaching new regulation of our most dynamic economic and cultural sector — the Internet.

I’ll comment on Net Neutrality from several angles over the coming days. But a terrific essay by Berkeley’s Jaron Lanier impelled me to begin by summarizing some of the big meta-arguments that have been swirling the last few years and which now broadly define the opposing sides in the Net Neutrality debate. After surveying these broad categories, I’ll get into the weeds on technology, business, and policy.

The thrust behind Net Neutrality is a view that the Internet should conform to a narrow set of technology and business “ideals” — “open,” “neutral,” “non-discriminatory.” Wonderful words. Often virtuous. But these aren’t the only traits important to economic and cultural systems. In fact, Net Neutrality sets up a false dichotomy — a manufactured war — between open and closed, collaborative versus commercial, free versus paid, content versus conduit. I’ve made a long list of the supposed opposing forces. Net Neutrality favors only one side of the table below. It seeks to cement in place one model of business and technology. It is intensely focused on the left-hand column and is either oblivious or hostile to the right-hand column. It thinks the right-hand items are either bad (prices) or assumes they appear magically (bandwidth).

We skeptics of Net Neutrality, on the other hand, do not favor one side or the other. We understand that there are virtues all around. Here’s how I put it on my blog last autumn:

Suggesting we can enjoy Google’s software innovations without the network innovations of AT&T, Verizon, and hundreds of service providers and technology suppliers is like saying that once Microsoft came along we no longer needed Intel.

No, Microsoft and Intel built upon each other in a virtuous interplay. Intel’s microprocessor and memory inventions set the stage for software innovation. Bill Gates exploited Intel’s newly abundant transistors by creating radically new software that empowered average businesspeople and consumers to engage with computers. The vast new PC market, in turn, dramatically expanded Intel’s markets and volumes and thus allowed it to invest in new designs and multi-billion dollar chip factories across the globe, driving Moore’s law and with it the digital revolution in all its manifestations.

Software and hardware. Bits and bandwidth. Content and conduit. These things are complementary. And yes, like yin and yang, often in tension and flux, but ultimately interdependent.

Likewise, we need the ability to charge for products and set prices so that capital can be rationally allocated and the hundreds of billions of dollars in network investment can occur. It is thus these hard prices that yield so many of the “free” consumer surplus advantages we all enjoy on the Web. No company or industry can capture all the value of the Web. Most of it comes to us as consumers. But companies and content creators need at least the ability to pursue business models that capture some portion of this value so they can not only survive but continually reinvest in the future. With a market moving so fast, with so many network and content models so uncertain during this epochal shift in media and communications, these content and conduit companies must be allowed to define their own products and set their own prices. We need to know what works, and what doesn’t.

When the “network layers” regulatory model, as it was then known, was first proposed back in 2003-04, my colleague George Gilder and I prepared testimony for the U.S. Senate. Although the layers model was little more than an academic notion, we thought then this would become the next big battle in Internet policy. We were right. Even though the “layers” proposal was (and is!) an ill-defined concept, the model we used to analyze what Net Neutrality would mean for networks and Web business models still applies. As we wrote in April of 2004:

Layering proponents . . . make a fundamental error. They ignore ever changing trade-offs between integration and modularization that are among the most profound and strategic decisions any company in any industry makes. They disavow Harvard Business professor Clayton Christensen’s theorems that dictate when modularization, or “layering,” is advisable, and when integration is far more likely to yield success. For example, the separation of content and conduit—the notion that bandwidth providers should focus on delivering robust, high-speed connections while allowing hundreds of millions of professionals and amateurs to supply the content—is often a sound strategy. We have supported it from the beginning. But leading edge undershoot products (ones that are not yet good enough for the demands of the marketplace) like video-conferencing often require integration.

Over time, the digital and photonic technologies at the heart of the Internet lead to massive integration — of transistors, features, applications, even wavelengths of light onto fiber optic strands. This integration of computing and communications power flings creative power to the edges of the network. It shifts bottlenecks. Crystalline silicon and flawless fiber form the low-entropy substrate that carry the world’s high-entropy messages — news, opinions, new products, new services. But these feats are not automatic. They cannot be legislated or mandated. And just as innovation in the core of the network unleashes innovation at the edges, so too more content and creativity at the edge create the need for ever more capacity and capability in the core. The bottlenecks shift again. More data centers, better optical transmission and switching, new content delivery optimization, the move from cell towers to femtocell wireless architectures. There is no final state of equilibrium where one side can assume that the other is a stagnant utility, at least not in the foreseeable future.

I’ll be back with more analysis of the Net Neutrality debate, but for now I’ll let Jaron Lanier (whose book You Are Not a Gadget was published today) sum up the argument:

Here’s one problem with digital collectivism: We shouldn’t want the whole world to take on the quality of having been designed by a committee. When you have everyone collaborate on everything, you generate a dull, average outcome in all things. You don’t get innovation.

If you want to foster creativity and excellence, you have to introduce some boundaries. Teams need some privacy from one another to develop unique approaches to any kind of competition. Scientists need some time in private before publication to get their results in order. Making everything open all the time creates what I call a global mush.

There’s a dominant dogma in the online culture of the moment that collectives make the best stuff, but it hasn’t proven to be true. The most sophisticated, influential and lucrative examples of computer code—like the page-rank algorithms in the top search engines or Adobe’s Flash—always turn out to be the results of proprietary development. Indeed, the adored iPhone came out of what many regard as the most closed, tyrannically managed software-development shop on Earth.

Actually, Silicon Valley is remarkably good at not making collectivization mistakes when our own fortunes are at stake. If you suggested that, say, Google, Apple and Microsoft should be merged so that all their engineers would be aggregated into a giant wiki-like project—well you’d be laughed out of Silicon Valley so fast you wouldn’t have time to tweet about it. Same would happen if you suggested to one of the big venture-capital firms that all the start-ups they are funding should be merged into a single collective operation.

But this is exactly the kind of mistake that’s happening with some of the most influential projects in our culture, and ultimately in our economy.

Professors at a leading research unit put suspect data into a bad model, fail to include crucial variables, and even manufacture the most central variable to deliver the hoped-for outcome.

Climate-gate? No, call it Berkman’s broadband bungle.

In October, Harvard’s Berkman Center for the Internet and Society delivered a report, commissioned by the Federal Communications Commission, comparing international broadband markets and policies. The report was to be a central component of the Administration’s new national broadband Internet policy, arriving in February 2010.

Just one problem. Actually many problems. The report botched its chief statistical model in half a dozen ways. It used loads of questionable data. It didn’t account for the unique market structure of U.S. broadband. It reversed the arrow of time in its country case studies. It ignored the high-profile history of open access regulation in the U.S. It didn’t conduct the literature review the FCC asked for. It excommunicated Switzerland . . . .

See my critique of this big report on international broadband at RealClearMarkets.

As the Journal wrote in its own editorial on Saturday:

Our own view is that the rules requiring traditional phone companies to connect these calls should be scrapped for everyone rather than extended to Google. In today’s telecom marketplace, where the overwhelming majority of phone customers have multiple carriers to choose from, these regulations are obsolete. But Google has set itself up for this political blowback.

Last week FCC Chairman Julius Genachowski proposed new rules for regulating Internet operators and gave assurances that “this is not about government regulation of the Internet.” But this dispute highlights the regulatory creep that net neutrality mandates make inevitable. Content providers like Google want to dabble in the phone business, while the phone companies want to sell services and applications.

The coming convergence will make it increasingly difficult to distinguish among providers of broadband pipes, network services and applications. Once net neutrality is unleashed, it’s hard to see how anything connected with the Internet will be safe from regulation.

Several years ago, all sides agreed to broad principles that prohibit blocking Web sites or applications. But I have argued that more detailed and formal regulations governing such a dynamic arena of technology and changing business models would stifle innovation.

Broadband to the home, office, and to a growing array of diverse mobile devices has been a rare bright spot in this dismal economy. Since net neutrality regulation was first proposed in early 2004, consumer bandwidth per capita in the U.S. grew to 3 megabits per second from just 262 kilobits per second, and monthly U.S. Internet traffic increased to two billion gigabytes from 170 million gigabytes — both 10-fold leaps. New wired and wireless innovations and services are booming.

All without net neutrality regulation.

The proposed FCC regulations could go well beyond the existing (and uncontroversial) non-blocking principles. A new “Fifth Principle,” if codified, could prohibit “discrimination” not just among applications and services but even at the level of data packets traversing the Net. But traffic management of packets is used across the Web to ensure robust service and security.

As network traffic, content, and outlets proliferate and diversify, Washington wants to apply rigid, top-down rules. But the network requirements of email and high-definition video are very different. Real time video conferencing requires more network rigor than stored content like YouTube videos. Wireless traffic patterns are more unpredictable than residential networks because cellphone users are, well, mobile. And the next generation of video cloud computing — what I call the exacloud — will impose the most severe constraints yet on network capacity and packet delay.

Or if you think entertainment unimportant, consider the implications for cybersecurity. The very network technologies that ensure a rich video experience are used to kill dangerous “botnets” and combat cybercrime.

And what about low-income consumers? If network service providers can’t partner with content companies, offer value-added services, or charge high-end users more money for consuming more bandwidth, low-end consumers will be forced to pay higher prices. Net neutrality would thus frustrate the Administration’s goal of 100% broadband.

Health care, energy, jobs, debt, and economic growth are rightly earning most of the policy attention these days. But regulation of the Net would undermine the key global platform that underlay better performance on each of these crucial economic matters. Washington may be bailing out every industry that doesn’t work, but that’s no reason to add new constraints to one that manifestly does.

— Bret Swanson

Send the bits with lasers and chips
See the bytes with LED lights

Wireless, optical, bandwidth boom
A flood of info, a global zoom

Now comes Lessig
Now comes Wu
To tell us what we cannot do

The Net, they say,
Is under attack
Stop!
Before we can’t turn back

They know best
These coder kings
So they prohibit a billion things

What is on their list of don’ts?
Most everything we need the most

To make the Web work
We parse and label
We tag the bits to keep the Net stable

The cloud is not magic
It’s routers and switches
It takes a machine to move exadigits

Now Lessig tells us to route is illegal
To manage Net traffic, Wu’s ultimate evil

The FCC in Oh Eight bought into the theory
And Comcast, the Chairman determined to bury

Then on Nine Twenty One
Genachowski declareth
Thou shalt not give primacy
Networks, you must shareth

The Net’s great virtue was decentralization
But Julius from on high decreed concentration
of power and decisions over all peers
Substitute bureaucrats for Net engineers

You may not
Shall not
Cannot cap
It’s discrimination!
Don’t you know that?

You may not give
the bits a weighting
No matter importance, voice or data

You may not
Shall not
Cannot charge
Even though
the bytes are large

In what other sector
is pricing illegal?
All for nothing as business model
is laughably feeble

Silicon, wires, software glue
It takes tens of billions to make bits move

You cannot charge
You cannot price
You cannot cap
It won’t suffice

No, no, they say, to bit analysis
Though it may lead to Net paralysis

We will not tolerate Q. O. S.
Not Weighted Round Robin
Nor flow-based service

Not Q-in-Q
Not PBT
Nor SLAs
at Layer Three

Storewidth and caching to speed the bits through
Akamai, Limelight cut latency to you
Though the result is customer speed
Don’t you get it? You may not give priority

Content, conduit, Googleplex
This neutrality thing may lead to bit wrecks

We thought all was well
3G, 4G, then WiMax!
But the once free Net
suffered bureaucratic attacks

When zetta visual bits
and mobile nets collide
Will the Chairman answer his phone?
. . . Or run and hide?

Out: speed, gadgets, quantum innovation
In: lobbyists, lawyers, mass litigation

Come to Washington
Ask us permission
Don’t they know it’s abundance they’re missin’?

But abundance is over
Scarcity is cool
We thought the Web was different
Now who’s the fool?

Moore’s law transcends
the cramped, dire fears
of misnamed Free Press
They’re no high tech seers

Mr. Chairman, please
might you cut us a break?
Should D.C. micromanage
the guts of cyberspace?

If you ban the codes that make the Net work
DiffServ, IPsec, and Dual Leaky Bucket
Net investors may revolt
and with fury say ————!

Who knows what the Web will bring
Which business models or networks will reign
With Net traffic growing at 60 percent
Shouldn’t techies experiment?

With fiber and cable
to bandwidth enable
Googleplex servers
a giant content stable

We do useful things like
BRED and Draft Martini
But once Neutrality is loosed,
might we never bottle this genie?

What about MPLS
and Inter Frame Gap?
Utopians now rule
Get used to that fact

Traffic, apps, digital value exploding
So tell me, FCC
What the heck is the problem?

The Net keeps on giving
Like water to wine
No artificial limits
Abundance sublime

But D.C. takes this great bounty for granted
Reap others’ rewards, assume innovation planted

Tera, peta, exa, zetta
So long free Net, see ya later

Miracle though the Net may be
The global telecosm is not free

— Bret Swanson