Ooops! Forgot to Back Up Horror Stories

Subject: Ooops! Forgot to Back Up Horror Stories
From: Kelley <kwalker2 -at- gte -dot- net>
To: "TECHWR-L" <techwr-l -at- lists -dot- raycomm -dot- com>
Date: Wed, 26 Dec 2001 12:09:54 -0500


We're doing an in-house newsletter about all the little ways the firm can lose data through mistakes and "cascading failures". For the feature on remembering to back up your documents as you work, we're hoping to collect some "Ooops, I forgot to backup horror stories!!". I've appended a sample of what I wrote for last year's issue to give you an idea of the range of stories we included.

I'm not writing the newsletter this year; I've tasked it out. It's her first time writing content like this--she's a online help techwriter I hired because she needed a job and we need a writer! So, to ease her transition from one type of writing to another, I thought I would do the work of collecting stories. I'm (and most of the folks I would ask) are fresh out of new ideas/stories, so I figured that TECHWRirlers might have some memorable stories. I want to ask for "humorous" stories, but I was scolded by a sysadmin friend who told me that backup failure stories are NEVER funny. :)

Anyway, if you would be so kind, I'd love to hear your stories offlist I presume, since it's not exactly "on topic". (Although, I'm sure some of us could angle it into an "on topic" discussion... :p ) Identifying details--names and orgs--will be anonymized.



Sample Stories:

Backup is so crucial and so often overlooked that Geek Culture has a "law" known as Huber's law: The chances of losing your data are directly proportional to the length of time since your last backup.

Once upon a time, there was a company called Kaypro computers. They ran their entire business on 10MB hard disk computers, which, despite management's errant beliefs, were terribly prone to failure. And so, the day before an IRS filing was due, the key accounting computer's hard disk crashed. It cost the company almost $400,000 in fines and penalties because they had no backups and had to recreate the records from hard copy files. At least they had hard copy, but they could have bought a lot of backup devices for $400K.

One of XXX's consultants does a lot of writing, and occasionally writes in bed late at night. He laid the computer on the floor next to the bed one night and his cat spilled a water glass onto the computer. The computer was fried and the hard disk was trashed, too. He did have a backup made a week earlier, but all of the articles he had written had to be rewritten. Several days of work gone for triple stupidity: Computer on floor, water near the computer, and no back up.

When a fire swept through Oakland and Berkeley Hills in Northern California in 1991, Maxine Hong-Kingston lost her home as well as a half-finished manuscript for her book, which is now being published as Fifth Book of Peace. Asked about the experience she said, "It's horrifying how long it takes." It was a struggle to try to write about the loss to the fires. In her subsequent re-writing of the 1800 page book, her thoughts continually turned to water because it seemed the only thing that might have saved her work, her home, her neighborhood, and the 25 people who died in the fires.

Another XXX consultant recalls why she backs up five different ways. A colleague, a Ph.D. candidate at Cornell University, had finished her masterpiece: a 500 page tome that her five member committee had ostensibly read in various stages throughout the five year long project. The week prior to her dissertation defense, the apartment building she lived in burned to the ground. She stood outside, dressed in a nightgown, holding her cat, screaming about her dissertation. Five firefighters had to hold her back. She had kept all three copies of her work in her home. Although her committee had read her work, none of them had kept the e-mailed attachments she'd sent them. As far as they were concerned, it was her job to have foreseen the possibility of such a mishap. She had to reproduce all her work. They refused to confer the Ph.D. without a hard copy of it.

A proud papa, a systems administrator, decided to take his 2-year-old son to visit his workplace. It was the Thanksgiving holiday, most people were heading toward the company festivities-everyone except our proud papa and forty employees busily wrapping up projects before heading to the party themselves. "Being the brilliant 2-year- old he was at the time," according to his father, the young geek reached out as he toddled past the Novell server and pressed the shiny button on the Compaq fileserver. The result: a crashed network, lost files since the backup tape was from the night before, an hour of work to get all the services back on-line, and 40 irate employees who lost all the work they'd been doing.

The Home Shopping Network is amazing. Located in St. Petersburg, FL, they are subject to more lightening strikes than anywhere on earth. So they built in backup and redundancy everywhere. They have a bank of batteries that fill an entire building for the regular blackouts. They have two mainframe computers for sales; the second is there just in case the first dies. The TV studio you see is only one of two. There is an exact duplicate of it that can be turned on in minutes in case the first one dies. They spend a lot of money on backup, but have you ever seen them off the air?

Back Biting Y2k+1 Bug: The chain of convenience stores, 7-Eleven, felt the sting the first day of the New Year, 01/01/01. A glitch that read the date as January 01, 1901 halted all computer mediated credit-card sales at 5,200 stories, despite having spent nearly $9 million on a Y2K upgrade. Computer glitch halts credit card sales at stores across the U.S. It took nearly two days to get all the systems back on-line because the credit-card sales system does more than simply record sales information. It is also integrated with a tracking system that examines weather forecasts, inventory, traffic conditions, holidays and other data in order to keep an adequate supply of goods on hand for changing weather patterns, road conditions, holiday and seasonal needs.

A few days after the New Year, the owner of a PDA discovered that he'd been bitten by the Y2K+1 bug. The electronic organizer displayed an error message the first time he turned it on after the New Year. He lost four years worth of notes and phone numbers. He could have avoided the problem with a backup kit, which costs the same as the PDA. Still, it was well worth the money since the costs of restoring four years worth of data certainly exceeded the cost of the equipment needed to backup in the first place. What would happen to you if you lost your PDA? How much information do you store there? Is any of it backed up? Backing up is not so hard to do!

Perhaps the experiences of two systems administrators, neophytes back in the early days of campus computing, are illustrative. Joe was hired as a systems administrator for an engineering department. He was given a choice. His boss said, "We have a budget; I can give you a higher salary and we can buy some new disk drives. Or, we can buy a tape drive and provide backup service for the graduate students. What's your preference?" Many years later, Joe says sardonically, "I can't tell you how fun it was to get mail asking for help restoring dissertations." Since then Joe has never chosen disk drives over backup upgrades.

Paul, on the other hand, feels pretty strongly about the topic, insisting that one simply cannot describe an "I failed to plan for the worst" story as anything short of a nightmare. His worst experience came one day when he unmirrored a drive in the morning with the intent of replacing it later in the day. In the meantime, however, a second drive crashed. Half a day's work was lost for everyone.

Jack Carrol, a network security pro, characterized the most common backup disasters as "cascading failures": a well-intentioned but poorly-planned effort to fix one problem results in the problem getting worse, possibly drastically worse. She insists that its not that the systems administrator or average user made a stupid mistake per se. Rather, it's just that the situation is generally high-pressure and the decision-making is muddled as a result. They panic, in other words. The best thing to do in that situation is try to count to ten and take several-many-deep breaths. If possible, walk away from the situation and try to clear your head. Get help from another person or a third party. Even if they can't help, sometimes sharing the burden with a trusted colleague can help. Then go back to the task and try to look at the situation with fresh eyes and a clear head.

Copyright Interpact, Inc. 2001

Be a published author! iUniverse gives you: a high-quality paperback, a
custom cover design, and distribution to 25,000 retailers. And it's
affordable. Join our almost 10,000 published authors today.

Your monthly sponsorship message here reaches more than
5000 technical writers, providing 2,500,000+ monthly impressions.
Contact Eric (ejray -at- raycomm -dot- com) for details and availability.

You are currently subscribed to techwr-l as: archive -at- raycomm -dot- com
To unsubscribe send a blank email to leave-techwr-l-obscured -at- lists -dot- raycomm -dot- com
Send administrative questions to ejray -at- raycomm -dot- com -dot- Visit for more resources and info.

Using References in your work: From: Sue
Re: Using References in your work - comments on Web site: From: Daniel Kamman, Information Consultant

Previous by Author: Word Master documents
Next by Author: better safe than sorry.
Previous by Thread: Re: Using References in your work - comments on Web site
Next by Thread: RE: Using References in your work

What this post helpful? Share it with friends and colleagues:

Sponsored Ads