This project involves statistical analysis of some massive government-collected data files -- so big that kicking unneeded fields out of the working database is a priority, reports a pilot fish in the know.
"Hotshot database analyst had the boss call a meeting so everyone could go through the file record documents, item by item," says fish. "Do we need this field? Do we need this one? Basically, any field no one spoke up for would get omitted from the big database load.
"The chief statistician on the project objected: 'Most of those fields we're eliminating are only one or two characters. If we don't know what they're used for, wouldn't it be safer to keep them?'
"His suggestion was, of course, brusquely overruled, and the useless little fields were left out.
"But these government data files were rather old -- they dated back to when storage was at a premium and many strange things were done to conserve space.
"Upon considerable further review, it turned out that the little one-character fields are switches. If this variable is a 1, the next field is, let's say, annual household income. If it's a 3, the next field is total Social Security benefits received. And so on.
"So omitting the little 'useless' fields made analysis impossible until they did a complete reload.
"But neither the boss nor the database analyst ever acknowledged that the chief statistician had been right."
Big stories, little stories, Sharky wants 'em all. So send me your true tales of IT life at sharky@computerworld.com. You'll snag a snazzy Shark shirt every time I use one. Add your comments below, and read some great old tales in the Sharkives.
Get your daily dose of out-takes from the IT Theater of the Absurd delivered directly to your Inbox. Subscribe now to the Daily Shark Newsletter.