Thursday, October 15, 2009

I have been reading articles about the immense data outage that's taken place for users of T-Mobile's Sidekick device. Having worked in IT for over 16 years, I still can't wrap my mind around the idea that NO usable backups existed from which the corrupted file could be restored.

When I worked for the bank, we had backups of our backups. Literally. There were two parallel files containing the data, always updated simultaneously, and both files had backups both onsite and offsite. We had things set up so that, IF the main data file got corrupted, the only record that would have to be recreated would be the ONE record that was in the process of being written to the file at the moment the crash occurred. If the system didn't fail at a time when a record was being written to the file, we wouldn't lose anything at all, because everything else was backed up in multiple places.

Now, granted, I'm talking about a BANK and there are about a bazillion Federal regulations governing everything banks do, including maintaining data integrity. So that instills a particular "Safety First. And Second, Third, Fourth, and Fifth" mindset in the people in the bank's IT department, as well it should.

But IMO, it shouldn't take the looming threat of Federal Regulators Getting In Your Face for a company to take the stance that preserving data integrity is more important than life itself. That is an attitude that SHOULD be a given.

I am just flabbergasted that any company anywhere could suffer a significant, possibly permanent data outage. Whatever money they may have saved by cutting corners, and downsizing away the people who knew the system best (which happened as part of a recent MSFT takeover of Danger) is now going to be spent paying damages on class action suits. So not only did they not save money by restructuring, they lost a lot of consumer trust into the bargain. To me, this looks like a worst-case-scenario for what can happen when a corporation thinks solely with its wallet.

This morning, I saw an article proclaiming that they think they can restore "most" of customers' data. Color me unimpressed until I see a percentage to quantify this nebulous "most". After all, 50.0001% would qualify as "most", as would 99.9999%.

There's one part of me, as a person who was downsized out of a longtime IT position, that wants to say, "NYA NYA NYA, corporate bigwigs, that's what happens when you cut your staff to a skeleton crew." But I can't, because the people this REALLY hurts are the consumers whose data was lost. The freaking bigwigs who gutted the department, even if their heads roll as a result of this, will get their golden parachute severance package and land with some other big corporation. The consumers are the ones who'll be up a creek if that data can't be restored.

Argh. I probably shouldn't read the news before my morning coffee. As you can see, it makes me think too much.

No comments: