Hi Lonnie, I posted on this topic earlier today on the Cook Report on Internet (http://www.cookreport.com) online symposium, and I thought it was suitable for sending your way, as well, since you’ve so eloquently set the stage. I enjoyed reading your account of how you ‘survived’ the effects of quakes, btw, causing me to reflect on my own dependence on connectivity, which, I fear, may also be reaching a level of clinical proportions 😉 I wonder if you would provide a citation for the photo at the top of your blog. Which outage does it depict, do you know?

My earlier post, slightly edited for venue, follows:
—

“When we go dead, the world goes dead.”

Those are the words of David Lassner, President and Chairman of the Board of Governors for the Pacific Telecommunications Council, highlighting the recent and still ongoing debacle that resulted from last week’s massive cable failures caused by a series of quakes off Taiwan.

During the past week I’ve commented on Verizon’s entry into a newly announced transpacific venture with roughly a half-dozen Asian partners, and the fact that the cable to be installed has been designed as a single, linear link between North America and China. What stands out as an obvious-, some might even suggest objectionable-, trait here is the fact that the cable’s design does not take into account any means of backing itself up in the event of an interruption. Verizon does state in its press release, however, that it would enhance the survivability of services already riding over other cables, as well as its own traffic, through mutual assistance measures with other operators whose cables enter and leave the region. Hm.

A new cable of the size described in the Verizon release cannot be expected to support all of its own working services in a failover condition, even if it fills only half of its potential capacity with live traffic, by folding onto other cables possessing far less capacity. Arguably, another link placed by the new venture could be added some years out for robustness in order to remedy this apparent imbalance by the time the first one fills to any appreciable level. But, who is to say what the traffic growth on the first link will be over the next two or three years, prior to the second one going in? The capacity of the first cable could easily undergo exhaust without much prior notice, and from the looks of traffic growth resulting from Outsourced Business Processing and other offshoring operations, not to mention the region’s internal growth, it probably will. The Verizon release from the WSJ:
—
“Big Phone Firms Delve Undersea For Asian Growth
By Li Yuan – December 18, 2006; Page A1
http://tinyurl.com/y7trem

Verizon Communications, Starhub, Vodafone Group, Chunghwa Telecom, Johnson Controls.” Some of the world’s biggest telecom companies are racing to tap China and other rapidly growing Asian markets by building faster pipelines for the surging volumes of Internet and phone traffic produced by multinational corporations and the region’s consumers. Verizon Communications Inc. signed an agreement today with five major Asian telecom carriers to build the first high-speed trans-Pacific undersea cable system directly linking the U.S. and China. According to the company, the planned $500 million project will offer an alternative to the single low-capacity cable that now provides the only direct link between mainland China and the U.S. Currently most …
–snip

After questioning the wisdom of this single-threaded strategy publicly on another forum, I came to learn that, despite the havoc and grief caused by the latest earthquake off Taiwan causing multiple cable failures and placing most of Asia, at a minimum, into a state of stupor that it is still trying to extricate itself from today (a/o 01-01-07), the continent’s major service providers have nevertheless rejected the notion of bolstering the region with additional links for link fail-overs and disaster recovery purposes. Their justification for this is based on cost-benefit. They say that major outages of the type encountered last week do not occur all that frequently, further stating that any benefits that might be derived from having the additional capacity in place could not justify the additional costs of provisioning it.

[See: “Asian Telecoms Companies Unwilling to Invest More in Backup Lines to Cope with Disasters”
By Peter Enav | AP | Dec. 29, 2006 http://tinyurl.com/umf4t ]

I see a need to point out the shortsightedness, if not some of the errors that are inherent in this argument, on several grounds. First, the stakes that are riding over the region’s submarine cables are rising every day. Straight line projections alone would suggest that, what might appear feasible today will be largely inconsequential one or two years from now, which is the time that implementing a new link would take if planning commenced immediately, which, I feel assured, it will not.

Secondly, with the amount of internal, organic growth taking place within many parts of Asia, as opposed to growth that might be attributed to supplementary links for recovery purposes, even if one counted only the new cables that are planned at this time and ignoring additional links for reliability’s sake, those additional cables through organic expansion will only add to the probability of a greater number of future failures. This heightened probability could actually take on a form geometric progression if submarine cable builders continue to follow the same submarine corridors, landing stations and in-country, overland routes as those in existence today, which at some point becomes unavoidable based purely on spatial constraints when a certain threshold of cables being placed onto the same, limited footprint is reached.

Third, and perhaps most interesting of all for a number of geo-political and ethical reasons is this: In a global community where communications traffic on any international cable can impact just about anyone without regard to where the communicating parties are situated on the planet, it is illogical, if not a form of chutzpah, to suggest that the level of reliability and survivability of any segment on that network could justifiably be decided by a single carrier or group of regional carriers. In other words, as one enjoys the benefits of community, they are then bound at the same time by being obligated to act responsibly.

Some may argue that the Asian carriers are indirectly making a good point, arguing that restoration could be effected through the existing mesh of facilities, as is the case with Internet traffic using IP in the main. Well, the first and most obvious response to this latter position would be, Why didn’t it work last week? Or even today, when much of the continent is still struggling with limited backup accommodations via European optical routes strung through the Middle East (SEA-ME-WE) and via Satellite paths that take two space hops in order to reach the United States?

But more to the point, the international traffic that comprises most of today’s enterprise and public services are not based on the same form of fungible IP fabric associated with the larger Internet, which otherwise could handle IP routed traffic over the larger public mesh, including International backbones already in place. I’m not suggesting that the bulk of those services couldn’t be made to operate in such a model with adequate planning and accommodations made in advance. But I am saying that those accommodations haven’t been made, and I doubt they will.

Consider, at the present time if you set aside the public Internet’s international backbone routes, the majority of remaining services are not “IP-routable” today in the same sense that public Internet traffic would be. Rather, they consist of the global switched telephone network (GSTN) in conjunction with C7 (SS7), private lines, IRUs, VPNs, extranets and community-of-interest networks, cellular roaming services and myriad other managed services (both private and public) that have not been programmed to follow the rules of Internet Protocol on the public ‘Net.

By virtue of its sheer girth – i.e., “sixty times the capacity of the existing cable” to China – the new transpacific cable would break the calculus for backing up the traffic that it signs on, because it is not self-healing in itself, and the cable that it will have to depend on for backup is only one-sixtieth of its size in the working condition. In the absence of a self-healing mechanism – which could be either ring-based or A/B switched, or even a mesh-architecture with sufficient capacity for failover relationships programmed in – performing “gang” switching of all affected services at the time of disruption — without any of the foregoing, each of the individual service types mentioned above would require individual planning and discreet handling during outages, and such could entail an almost infinite number of failure permutations. This would hardly be considered feasible for contingency planning and recovery purposes.

In short, the bulk of international traffic does not at the present time lend itself to being routed over “public” IP facilities, i.e., what is generally referred to as “the public Internet,” hence demands special planning at the discreet service level for contingency situations until either one of the following two occurs: either the universe of traffic is quickly converted to a form of traffic that is conducive to routing over the Internet (or a similar form of arrangement by proxy), or facilities are designed to function in a self-healing manner without effecting external route changes.

The article that prompted the above outpouring follows.

Enjoy, and have a healthy and prosperous 2007, everyone!

Frank A. Coluccio
DTI Consulting Inc.
New York City
—-

Earthquake Exposes Worldwide Telecommunications Vulnerabilities; Disaster Planning Gurus Speak at PTC’07
Pacific Telecommunications Council 2007

HONOLULU–Wednesday’s earthquake and aftershocks jolted Asia for hours but the telecommunication disruptions may reverberate for weeks. When transpacific fiber optic cables were damaged, internet and phone call volume plummeted by half and web browsing slowed to a crawl. This, in the world’s fastest-growing communications marketplace.

The disaster highlights the vulnerability of international telecommunications in a global economy that has grown dependent on real-time communications. It also raises the stakes for $500 million in planned investments in new transpacific undersea cables.

“Natural disasters can expose weaknesses in global communications,” said Ken Zita, who served as a telecommunications advisor to the U.S. Government following the Asian tsunami. Zita, conference chairman of PTC’07: Beyond Telecom, will host telecommunications executives from over 60 countries at PTC’07 next month where emergency communications and disaster management will be highlighted. “Despite the latest network management technologies, traffic concentration remain susceptible to strong natural hazards,” he said noting session on rapid deployment of communications tools in times of disasters and the establishment of emergency tactical plans for critical communications facilities.

According to David Lassner, President and Chairman of the Board of Governors for PTC and CIO at the University of Hawaii, underscores the importance of emergency planning. “Global telecommunications cannot be underestimate. Everything from billions of dollars in international trade to personal communication with family is silently carried by our industry,” he said. “When we go dead, the world goes dead.”

PTC’07 will be held in Honolulu January 14 – 17. Click http://www.ptc07.org for online registration. Leading sponsors include Stealth Communications, Telus, Tyco, Verizon, VSNL International, Arbinet, Asia Netcom, AT&T, CPCNet Hong Kong Ltd, France Telecom Group, GCI, Intelsat, LoralSkynet, Macquarie Telecom, Paul, Hastings, Janofsky & Walker LLP, Qwest Communications, Korea Telecom (KT), PacketFront, REDCOM Laboratories, Redline Communications, Boeing Satellite Systems International, China Telecom USA, CITIC 1616, Level 3 Communications Inc., Pacific Crossing Limited (PCL), and SubOptic.

Founded in 1980, Pacific Telecommunications Council (PTC) is an international, non-profit, non-government membership organization based in Honolulu, Hawaii, USA. Its mission is to promote the development, understanding, and beneficial use of telecommunications and information technology throughout the Pacific region.

——