The hole story: What you don't see will kill you

Parsing through Howard Wainer's wonderful Graphic Discovery: A Trout in the Milk and Other Visual Adventures, I came across my new favorite looking vs. seeing anecdote.

During WWII, Hungarian-born mathematician Abraham Wald undertook a study with the British Air Ministry to use statistical analysis to help protect bombers flying over enemy territory. The data to be crunched included the number and location of bullet holes on returning aircraft, and the goal was to use this information to determine where to best add armor to the plane's structure.

A nifty little chart was created to show where the maximum number of bullet holes were located on returning aircraft. This chart showed the greatest damage not on the main wing and tail spars, engines, and core fuselage areas, but rather on the aircraft extremities. Based on this, the Air Ministry suggested adding armor to those extremities. Wald suggested they were dead wrong.

Wald said more armor should go on the places that had the least holes. Huh? What was he thinking?

Wald was keeping the Air Ministry from falling into the "survivorship bias": they were forgetting the their data did not include the planes that had been lost. If the returning planes had no holes in their wing spars and engines, the better assumption to make is that even a few holes in those places were deadly: no damage was recorded in those areas because those planes were the ones that had crashed. Wald recommended more armor in those data-free areas.

The lesson: the data that isn't there may tell as important a story as the data that is.

PS. Amazon has no more copies of Wald's original study. Does anybody have any idea where to find:
A reprint of "A method of estimating plane vulnerability based on damage of survivors" by Abraham Wald (CRC) (CRC)
by Abraham Wald
Operations Evaluation Group, Center for Naval Analyses (January 1, 1980)
ASIN: B0006Y8P9I

data visualization, Abraham Wald , Howard Wainer

Comments

[sildenafil]

very interesting how these monsters and know the physical machines and logic with which are huge and heavy machinery after the heavens am extremely attention, congratulations on the excellent blog topic.