In ww2, the royal airforce was tasked with adding armour to spitfires as too many were getting shot down. The commander decided to take account of each plane that returned from flight and record where it had been shot. They decided to look at the data and add armour to the areas that had the most shots recorded, as these were the areas that were shot the most.
The reality is that the planes that returned with many bullet holes had returned because those areas were acceptable to get shot in…. the plane could survive being shot in those areas.
The armour should have instead gone to the locations without any bullet holes. As each plane that was shot in these areas likely didn’t make it back to the base.
Some data that we are presented with is only a subset of the total data that we should be looking at. But through some self-filtering process, we can not retrieve. We are only able to look at data points that survive the process and not the ones that fail
examples:
- start-ups that are successful are easier to find than ones that fail
- actors that are successful are easier to find than ones that fail
- musicians that are successful are easier to find than ones that fail
- footballers that are successful are easier to find than ones that fail