I'm glad the company I work for is not the only one to have outages, er, "unscheduled improvements" to its network.
But 12 hours is a long time to have an outage affecting _everybody_.
Wed Apr 18, 5:02 PM
By David Friend
TORONTO (CP) - BlackBerry addicts quickly slipped back into the habit Wednesday after technical problems shut down the portable e-mail device's key capabilities for more than 12 hours, though questions lingered about the cause and whether it could happen again.
Research In Motion (TSX:RIM), developer of the BlackBerry, provided few answers and little comfort to users in the wake of an unknown glitch that resulted in a massive system failure, affecting Internet connections and e-mail for millions of users.
Reports of trouble with BlackBerry e-mail began to surface as early as 8 p.m. ET Tuesday. Phone service through the handhelds was unaffected.
It took the company more than half a day to issue a statement and even then RIM said only that the "root cause is currently under review."
That left technical analysts to speculate that what likely started as an isolated problem later careened out of control, possibly worsened by RIM's atypical server setup.
RIM runs its North American servers through an operations centre at its Waterloo headquarters and has done so since the company's infancy, according to Carmi Levy, senior research analyst at InfoTech Research Group.
"The way we see it, is that it's one large monolithic facility but within that facility there are backups and failovers and redundancies," he said.
"What essentially happened is there was a failure in that facility that cascaded out of control. It resulted in the operation centre's inability to keep up with e-mail transmissions."
A backup system was likely also in place to prevent such minor outages, said Jesse Hirsh, a technology industry watcher for Openflows Networks. Either the backup system didn't kick in or failed along with the first system, he said.
"This suggests something went horribly wrong," he said.
Hirsch pointed to RIM's decision to keep all of its servers in house as part of the problem.
"This is definitely a proprietary approach that they've taken which is not the industry norm," he said.
Other major companies like IBM and Google use the Internet instead of their own private servers, which lessens the effect from any technical crashes, Hirsch said.
"The Internet has to go down to affect them in such a way. Because the Internet is distributed so largely it just doesn't happen anymore," he said.
RIM representatives didn't return calls to either confirm or deny exactly what happened.
The company's lack of complete disclosure sparked fury on Internet message boards as some BlackBerry users questioned why the company hadn't issued a statement earlier.
"For a company that revolves around communication technology... they sure are doing a crap job in communicating directly with their clients in a situation like this," wrote one forum user.
The urgency of a BlackBerry outage, and the affect it could have on RIM's future growth, became glaringly apparent.
Often humorously referred to as the"CrackBerry" by ardent users, the BlackBerry is known as the device of choice in the business community, where they are used to discuss deals, schedule plans and monitor after-hours trading.
"When a lot of people bought BlackBerrys they no longer took their laptops when they travelled," said Nick Bontis, a marketing professor at the DeGroote School of Business at McMaster University.
In Ottawa, some Canadian politicians expressed their dismay with the down time, while others said it proved they were addicted.
Liberal MP Garth Turner, known for his prolific Internet blogging, said politicians were scrambling to find other ways to communicate when the system went down.
"We all lost our data when we were in the House of Commons last night. The sound of BlackBerrys being thrown against the desk was deafening for a while," he said.
This isn't the first time that RIM has been hit by a major server issue.
In June 2005, an outage of its e-mail services appeared to only affect devices on GSM cellular networks, the wireless standard dominant in Europe and other regions. That outage primarily impacted big U.S. carriers Cingular and T-Mobile, while others only saw a slowdown in message traffic.
Some industry watchers suggest that if server failures continue to plague the company, RIM could lose current users, and potential future customers, to competitors like Motorola's Smartphone and the upcoming Apple IPhone.
"The response will depend on how concretely and proactively RIM responds to this outage," said Levy. "If they do nothing and these outages continue to occur, it will ultimately damage their ability to grow their subscriber base because of reduced confidence in the platform."
On Wednesday, investor confidence seemed unshaken. RIM shares closed the day up $2.80 to $151.62, after dropping as much as $3.36 earlier in the day on the Toronto Stock Exchange.
As of 4 p.m. ET, The firm's customer support centre, at 1-877-255-2377, carried a brief recorded message reporting that the company is "closely monitoring systems to maintain normal service levels."
RIM has said that about eight million people use the BlackBerry worldwide. About one million U.S. federal government and emergency workers are believed to use the devices.