Models behaving badly

Two popular GDP models did not do well this quarter

Models are tools that help people analyze things better. At least, that is ideally how it should be. It’s no secret that models sometimes don’t work right and can actually be counterproductive. Unfortunately, they are often least accurate in the times they are most needed.

Take the 2020Q1 release of GDP, which came in at -4.8%. Now look at what GDPNow and the FRB Nowcast predicted:

As you can see, both models severely underestimated the impact coronavirus would have on GDP in the first quarter. That’s because these models take data in, run an algorithm, and produce output. Most of the data going in did not account for the unprecedented pandemic.

However, these GDP prediction models aren’t useless just because they missed pretty badly one quarter. They still provide a consistent methodology for turning raw data into an estimate. They still provide a benchmark. But, no model is perfect and an extreme outlier event (in this case) caused them to underperform.

It’s a great example of why automation can’t always be a 100% replacement for people - you need an experienced second check that can assess for reasonableness. People still provide value. Tools definitely benefit when used by experienced practitioners.