It turns out that there are a number of indices that measure forecast accuracy. Here are two examples.
The Citigroup Economic Surprise Index measures how much estimates for a large number of economic indicators differ from reported values. Unfortunately there is no clear description of how the index is constructed, but positive index values mean reality was better than expected and negative ones mean it was worse.
|Economic Surprise Index|
It is clear that economist’s forecasts are a lot like broken clocks. They are correct only when it is unavoidable, twice a day for the clock, and whenever economists transition between being too optimistic and too pessimistic for the forecasts.
There are two things to be learned form this index: 1. Don’t take economic predictions seriously. The only thing certain about them is that they will be wrong. (The median absolute index value is 29.4.) 2. Economists are typically too pessimistic. The median index value is 8.4, indicating that the economy generally did better than predicted.
|Percentage of Companies Beating Estimates|
Another big part of the forecasting industry is predicting company earnings. Let’s see how well analysts do.
This index tracks the percentage of companies inthe S&P 500 that reported earnings above the Bloomberg average analyst estimate.
If analyst made accurate predictions, this index should hover right around 50% most of the time. Clearly that’s not the case. The index was below 50% only once in the last fourteen years. In every other quarter analysts underestimated company earnings. The estimates were too low for as much as 80% of the companies in the S&P 500 coming out of the recent recession, a time when good estimates would have been especially useful.
This index confirms our findings above: 1. Don’t take economic forecasts seriously. They are almost always wrong. 2. Analysts tend to be too pessimistic.
Now go back to watching your favorite pundits on TV. Just remember it’s TV. It’s not real.