Via the Washington Post comes an interesting article on data mining and the airline industry. Apparently, airplanes are not crashing enough for the airlines to be able to determine the sources and causes of accidents. That is, there is not enough variability in the outcomes (the last crash was August 2006) to do forensic analysis.
Instead, airlines are turning to ‘precursor’ anlayses, data mining a whole slew of events that have not led to accidents: unstabilized approaches, pitch rates at takeoff, pilot scheduling. The article suggests but does not detail the sheer number of variables and flights being analyzed, saying that Southwest has ‘mined data on more than 1 million flights’, but not really talking about what that means.
Organizationally, this is fascinating because so much more often we see organizations respond to events rather than trying to predict them. Or rather, as the vice chairperson of the NTSB put it, mining for precursors is like “‘reading tea leaves’ because it can require imagination to tie together incidents that don’t seem hazardous at first blush.” Arguably, it’s the imagination part that is so tricky in seeing what to make of precursors to mistakes and accidents. Even if you find them, often precursors only matter when they happen in conjunction (ie. in systems that are tightly coupled). So you can actually imagine a series of events that still would not result in a crash unless those events were temporally and organizationally tied together.
I would say that this is what we’re seeing now in the finance world, but it’s not. It’s worth another post, but there we’re seeing deliberate profit-seeking and many (though not nearly all or homogeneously) firms knowing that things could blow-up but not really caring.
h/t: Paul Kedrosky