BGI, based in China, is the world’s largest genomics research institute, with 167 DNA sequencers producing the equivalent of 2,000 human genomes a day.

BGI churns out so much data that it often cannot transmit its results to clients or collaborators over the Internet or other communications lines because that would take weeks. Instead, it sends computer disks containing the data, via FedEx.

“It sounds like an analog solution in a digital age,” conceded Sifei He, the head of cloud computing for BGI, formerly known as the Beijing Genomics Institute. But for now, he said, there is no better way.

The field of genomics is caught in a data deluge. DNA sequencing is becoming faster and cheaper at a pace far outstripping Moore’s law, which describes the rate at which computing gets faster and cheaper.

The result is that the ability to determine DNA sequences is starting to outrun the ability of researchers to store, transmit and especially to analyze the data.

We’ve been talking about the oncoming rush of biomedical data for a while. A human genome consists of some 2.9 billion base pairs, easily stored in around 725 megabytes with standard compression techniques. Two thousand genomes a day, times 725 MB, equals 1,450,000 MB, or 1.45 terabytes. That’s a lot of data for one entity to transmit in a day’s time. Some researchers believe a genome can be losslessly compressed to approximately 4 megabytes. In compressed form, 2,000 genomes would total around 8,000 MB, or just 8 gigabytes. Easily doable for a major institution.

Interested to know more.

We’re nearing 300 million mobile subscribers in the U.S., and Strategy Analytics estimates by 2014 we’ll add an additional 60 million connected devices like tablets, kiosks, remote sensors, medical monitors, and cars. All this means more connectivity, more of the time, for more people. Mobile data traffic on AT&T’s network rocketed 8,000% in the last four years. Remember that just a decade ago there was essentially no wireless data traffic. It was all voice traffic. A few rudimentary text applications existed, but not much more. By year-end 2010, AT&T was carrying around 12 petabytes per month of mobile traffic alone. The company expects another 8 to 10-fold rise over the next five years, when its mobile traffic could reach 150 petabytes per month. (We projected this type of growth in a series of reports and articles over the last decade.)

The two companies’ networks and businesses are so complementary that AT&T thinks it can achieve $40 billion in cost savings. That’s more than the $39-billion deal price. Those huge efficiencies should help keep prices low in a market that already boasts the lowest prices in the world (just $0.04 per voice minute versus, say, $0.16 in Europe).

But those who focus only on the price of existing products (like voice minutes) and traditional metrics of “competition,” like how many national service providers there are, will miss the boat. Pushing voice prices down marginally from already low levels is not the paramount objective. Building fourth generation mobile multimedia networks is. Some wonder whether “consolidation of power could eventually lead to higher prices than consumers would otherwise see.” But “otherwise” assumes a future that isn’t going to happen. T-Mobile doesn’t have the spectrum or financial wherewithal to deploy a full 4G network. So the 4G networks of AT&T, Verizon, and Sprint (in addition to Clearwire and LightSquared) would have been competing against the 3G network of T-Mobile. A 3G network can’t compete on price with a 4G network because it can’t offer the same product. In many markets, inferior products can act as partial substitutes for more costly superior products. But in the digital world, next gen products are so much better and cheaper than the previous versions that older products quickly get left behind. Could T-Mobile have milked its 3G network serving mostly voice customers at bargain basement prices? Perhaps. But we already have a number of low-cost, bare-bones mobile voice providers.

The usual worries from the usual suspects in these merger battles go like this: First, assume a perfect market where all products are commodities, capacity is unlimited yet technology doesn’t change, and competitors are many. Then assume a drastic reduction in the number of competitors with no prospect of new market entrants. Then warn that prices could spike. It’s a story that may resemble some world, but not the one in which we live.

The merger’s boost to cell-site density is hugely important and should not be overlooked. Yes, we will simultaneously be deploying lots of new Wi-Fi nodes and femtocells (little mobile nodes in offices and homes), which help achieve greater coverage and capacity, but we still need more macrocells. AT&T’s acquisition will boost its total number of cell sites by 30%. In major markets like New York, San Francisco, and Chicago, the number of AT&T cell sites will grow by 25%-45%. In many areas, total capacity should double.

It’s not easy to build cell sites. You’ve got to find good locations, get local government approvals, acquire (or lease) the sites, plan the network, build the tower and network base station, connect it to your long-haul network with fiber-optic lines, and of course pay for it. In the last 20 years, the number of U.S. cell sites has grown from 5,000 to more than 250,000, but we still don’t have nearly enough. CEO Randall Stephenson says the T-Mobile purchase will achieve almost immediately a network expansion that would have taken five years through AT&T’s existing organic growth plan. Because of the nature of mobile traffic — i.e., it’s mobile and bandwidth is shared — the combination of the two networks should yield a more-than-linear increase in quality improvements. The increased cell-site density will give traffic planners much more flexibility to deliver high-capacity services than if the two companies operated separately.

The U.S. today has the most competitive mobile market in the world (second, perhaps, only to tiny Hong Kong). Yes, it’s true, even after the merger, the U.S. will still have a more “competitive” market than most. But “competition” is often not the most — or even a very — important metric in these fast moving markets. In periods of undershoot, where a technology is not good enough to meet demand on quantity or quality, you often need integration to optimize the interfaces and the overall experience, a la the hand-in-glove paring of the iPhone’s hardware, software, and network. Streaming a video to a tiny piece of plastic in your pocket moving at 60 miles per hour — with thousands of other devices competing for the same bandwidth — is not a commodity service. It’s very difficult. It requires millions of things across the network to go just right. These services often take heroic efforts and huge sums of capital just to make the systems work at all.

Over time technologies overshoot, markets modularize, and small price differences matter more. Products that seem inferior but which are “good enough” then begin to disrupt state-of-the art offerings. This was what happened to the voice minute market over the last 20 years. Voice-over-IP, which initially was just “good enough,” made voice into a commodity. Competition played a big part, though Moore’s law was the chief driver of falling prices. Now that voice is close to free (though still not good enough on many mobile links) and data is king, we see the need for more integration to meet the new challenges of the multimedia exaflood. It’s a never ending, dynamic cycle. (For much more on this view of technology markets, see Harvard Business School’s Clayton Christensen).

The merger will have its critics, but it seriously accelerates the coming of fourth generation mobile networks and the spread of broadband across America.

— Bret Swanson

Mobile, perhaps not coincidentally, has also been historically a quite lightly regulated industry. But emerging is a sort of slow boil of small but many rules, or proposed rules, that could threaten the sector’s success. I’m thinking of the “bill shock” proceeding, in which the FCC is looking at billing practices and various “remedies.” And the failure to settle the D block public safety spectrum issue in a timely manner. And now we have a group of  rural mobile providers who want the FCC to set prices in the data roaming market.

You remember that “roaming” is when service provider A pays provider B for access to B’s network so that A’s customers can get service when they are outside A’s service area, or where it has capacity constraints, or for redundancy. These roaming agreements are numerous and have always been privately negotiated. The system works fine.

But now a group of provider A’s, who may not want to build large amounts of new network capacity to meet rising demand for mobile data, like video, Facebook, Twitter, and app downloads, etc., want the FCC to mandate access to B’s networks at regulated prices. And in this case, the B’s have spent many tens of billions of dollars in spectrum and network equipment to provide fast data services, though even these investments can barely keep up with blazing demand.

The FCC has never regulated mobile phone rates, let alone data rates, let alone data roaming rates. And of course mobile voice and data rates have been dropping like rocks. These few rural providers are asking the FCC to step in where it hasn’t before. They are asking the FCC to impose old-time common carrier regulation in a modern competitive market – one in which the FCC has no authority to impose common carrier rules and prices.

In the chart above, we see U.S. info-tech investment in 2010 approached $500 billion. Communications equipment and structures (like cell phone towers) surpassed $105 billion. The fourth generation of mobile networks is just in its infancy. We will need to invest many tens of billions of dollars each year for the foreseeable future both to drive and accommodate Internet innovation, which spreads productivity enhancements and wealth across every sector in the economy.

It is perhaps not surprising that a small number of service providers who don’t invest as much in high-capacity networks might wish to gain artificially cheap access to the networks of the companies who invest tens of billions of dollars per year in their mobile networks alone. Who doesn’t like lower input prices? Who doesn’t like his competitors to do the heavy lifting and surf in his wake? But the also not surprising result of such a policy could be to reduce the amount that everyone invests in new networks. And this is simply an outcome the technology industry, and the entire country, cannot afford. The FCC itself has said that “broadband is the great infrastructure challenge of the early 21st century.”

Economist Michael Mandel has offered a useful analogy:

new regulations [are] like  tossing small pebbles into a stream. Each pebble by itself would have very little effect on the flow of the stream. But throw in enough small pebbles and you can make a very effective dam.

Why does this happen? The answer is that each pebble by itself is harmless. But each pebble, by diverting the water into an ever-smaller area,  creates a ‘negative externality’ that creates more turbulence and slows the water flow.

Similarly, apparently harmless regulations can create negative externalities that add up over time, by forcing companies to spending  time and energy meeting the new requirements. That reduces business flexibility and hurts innovation and growth.

It may be true that none of the proposed new rules for wireless could alone bring down the sector. But keep piling them up, and you can dangerously slow an important economic juggernaut. Price controls for data roaming are a terrible idea.

The very success of smartphones, tablets, and all the new mobile form-factors fuels frustration. They are never fast enough. We always want more capacity, less latency, fewer dropped calls, and ubiquitous access. In a real sense, these are good problems to have. They reflect a fast-growing sector delivering huge value to consumers and businesses. Rapid growth, however, necessarily strains various nodes in the infrastructure. At some point, a lack of resources could stunt this upward spiral. And one of the most crucial resources is wireless spectrum.

There is broad support for opening vast swaths of underutilized airwaves — 300 megahertz (MHz) by 2015 and 500 MHz overall — but we first must dispose of one spectrum scuffle known as the “D block.” Several years ago in a previous spectrum auction, the FCC offered up 10 MHz for commercial use — with the proviso that the owner would have to share the spectrum with public safety users (police, fire, emergency) nationwide. This “D block” sat next to an additional 10 MHz known as Public Safety Broadband (PSB), which was granted outright to the public safety community. But the D block auction failed. Potential bidders could not reconcile the technical and business complexities of this “encumbered” spectrum. The FCC received just one D block bid for just $472 million, far below the FCC’s minimum acceptable bid of $1.3 billion. So today, three years after the failed auction and almost a decade after 9/11, we still have not resolved the public safety spectrum question.

Washington is considering two principal options: (1) redo the previous D block auction with the private-public sharing scheme; or (2) grant the D block to public safety, creating a nationwide contiguous 20 MHz public safety band.

The shared-spectrum scheme was perhaps worth a try the first time ’round. Although many doubted it would succeed, the concept was interesting, and it offered at least theoretical efficiencies — commercial users get most of the bandwidth during public tranquility, first responders get priority in emergency. Plus, it’d be nice to pick up a few billion dollars for the U.S. Treasury. Better than allocating it all (for free) to public safety where much of the bandwidth may lay fallow much of the time, right? Wrong. A failed auction yields no dollars and no usable spectrum.

It turned out the technical complexities of building for two very different purposes — high efficiency for commercial versus highly robust and redundant for public safety — and the likely conflicts of authority and responsibility (inherent in any entity with no true “owner”) posed too great a risk for anyone to bid a serious amount. Many of the technical and ownership risks still apply today.

Why not just award the D block to public safety and get on with the plan to allocate spectrum in amounts many times larger than the D block? Supporters of a re-auction say the government might leave money on the table and deny this particular band to commercial users. True, as far as it goes. Which isn’t very far.

A new paper by the Phoenix Center attempts to analyze this tricky subject from an economic perspective. Phoenix examines the costs and benefits of each option and concludes it is mostly timing of the allocation and the location of the spectrum in the frequency band that count.

allocating the contiguous D Block to public safety only postpones the allocation of an additional 10 MHz for commercial purposes (which the “new” block comes from the 500 MHz of spectrum promised by the FCC and the Obama Administration).  In the second option, the D Block is auctioned for commercial purposes now, precluding its assignment for public safety purposes.  In this case, the incremental benefits and costs from commercial use accrue now, but the benefits and costs of public safety’s use are postponed.  Framed in this way, the relevant issue is not whether the 10 MHz is used for public safety or used for commercial use, but rather when and which 10 MHz is put to use in both, and how the size and timing of benefits compare between these two alternatives.

Although the FCC says a new D block auction would be unencumbered, specific rules have not yet been proposed, and it’s likely some technical and even “open access” rules could accompany the block, thus reducing its market value. Further, in the new wireless broadband networks using LTE technology, contiguous 10 MHz bands are highly desirable. The physical separation of the D block with other commercial bands would mean a likely auction value of between $2-6 billion less than a contiguous 10 MHz block elsewhere. Phoenix estimates the cost of delaying a commercial auction of the D block is around $600 million annually, essentially the consumer surplus of having more spectrum available, which translates into lower prices for mobile services.

On the other hand, a public safety D block would be contiguous with the existing PSB block. This is important because it would substantially reduce the cost of building a public safety network. (It’s easier technically to build a system that covers one contiguous frequency band rather than two bands that are “far apart.” Phoenix estimates network equipment savings for contiguous bands of $4 billion.)

The cost of delaying the public safety network buildout is awfully hard to estimate. Can we get by on what we have now? What if a disaster hits? What is the value of having a real broadband network for combating everyday crime, for the next Katrina? Who knows, but it is likely to be a significantly large number. Leaving this probably significant number aside, however, it’s still easy to see from Phoenix’s estimates that $4 billion > $600 million. That is, once the costs and benefits are added up, the $4 billion in extra capital to build a noncontiguous public safety network is higher than the $0.6 billion in value generated by the commercial network over a five-year window, given all the challenges of this particular block. Any marginal public safety value would just make the gap wider and argue further for a grant to public safety.

If the U.S. weren’t about to open up hundreds of megahertz of much needed new spectrum, the argument may be different. Perhaps then we would have to force commercial and public safety to share a cramped apartment. But because we are opening up lots of new spectrum; because we do have this rare opportunity to give public safety a contiguous band, which should last basically forever; because it should preclude public safety from seeking impositions on commercial spectrum in the future; and because it could pave the way politically for the next huge spectrum expansion, it looks like the smart thing to do.

The fact is, with hundreds of megahertz of new spectrum being unleashed to the commercial world, an unencumbered grant of 10 MHz to the public safety world is probably worth it. Let’s get on with the show.

(Thanks to @AdamThierer for flagging the WashPost article.)

• Mobile data traffic was even higher in 2010 than Cisco had projected in last year’s report. Actual growth was 159% (2.6x) versus projected growth of 149% (2.5x).
• By 2015, we should see one mobile device per capita . . . worldwide. That means around 7.1 billion mobile devices compared to 7.2 billion people.
• Mobile tablets (e.g., iPads) are likely to generate as much data traffic in 2015 as all mobile devices worldwide did in 2010.
• Mobile traffic should grow at an annual compound rate of 92% through 2015. That would mean 26-fold growth between 2010 and 2015.

Here’s a new exaflood metric for you — tweets per second.

From the Twitter blog:

Folks were tweeting 5,000 times a day in 2007. By 2008, that number was 300,000, and by 2009 it had grown to 2.5 million per day. Tweets grew 1,400% last year to 35 million per day. Today, we are seeing 50 million tweets per day—that’s an average of 600 tweets per second. (Yes, we have TPS reports.)

“Striving to Map the Shape-Shifting Net” – by John Markoff – The New York Times – March 2, 2010

“Data, data, everywhere” – The Economist – Special Report on Managing Information – February 25, 2010

“Managing the Exaflood” – American Association for the Advancement of Science – February 19, 2010

“Professors Find Ways to Keep Heads Above ‘Exaflood’ of Data” – Wired Campus – The Chronicle of Higher Education – February 24, 2010