only about 14% (111 months out of 792) of the time since 1953. We’re using a lot of data to forecast a rare event. There are bunch of solutions to this problem. We’ll keep it simple for now.1
Lowering the threshold is the simple solution. Here we say if the model’s probability of recession is above the historical frequency (14%), then we say that the economy is in recession.
Let’s look at some metrics:
- Accuracy: 49%
- Specificity: 48%
- False positive rate: 52%
Lowering the threshold produces better results than the previous attempt, but it’s still not much better than a coin flip. If we lower the threshold below 14% then we’re more likely to predict recessions than actually occur. The specificity goes up, but accuracy goes down.
An easy way to see this trade-off is run the range of thresholds through the prediction algorithm and graph the results. As the threshold increases toward the same frequency of recessions, specificity increases, but accuracy decreases.
We hope to experiment with more advanced solutions in other posts.↩