North American Network Operators Group

Date Prev | Date Next | Date Index | Thread Index | Author Index | Historical

History: lengthy outages

  • From: Sean Donelan
  • Date: Thu Jan 25 02:52:07 2001

On Wed, 24 January 2001, [email protected] wrote:
> Somebody bitched a router config, and it took 22.5 hours to figure it out?
> That's the sort of goof you might expect from a mom and pop ISP with a
> hundred customers and virtually no IP clue.  I'll be shocked if multiple
> people (at multiple levels) aren't fired over this.  Screwing up happens.
> Taking this long to figure out what you (or even for others to figure out
> what someone else) screwed up is just absolutely unbelievable.

That's a bit unfair.

There have been a number of lengthy outages.

AS7007 router configuration problem: April 25 1997 lasted 2 hours
AOL (ANS router configuration problem): Aug 7 1996 lasted 19 hours
ATT frame-relay switch errors: April 13, 14 1998 lasted 26 hours
BBN standard power failure: October 11, 1996 lasted about 12 hours(off and on)
NETCOM router configuration error: June 20 1996 lasted 13 hours
Sprint database problems: September 3 1996 lasted 5 hours
NSI root server corruption (operational error): July 16 1997 lasted 4 hours
PacBell configuration problems: January 30-31 1997 lasted 48 hours
UUNET frame-relay problems: July 1 1997 lasted over 24 hours
UUNET cisco/bay router problems: November 7 1997 lasted 5 hours
Worldcom frame-relay switch errors: August 1999 lasted 9-10 days

If I skipped your favorite provider, have no fear.  More than likely
they've also had (or will have) a lengthy outage at one time or another.
One interesting thing I found was providers seem to have substantial
outages shortly after announcing "100% Uptime."

But somehow the net continues to stumble forward.