Why does the forecast on my weather app always seem to be wrong?

I've been asked this, or a variation of it, many times so I figured I would unravel the mystery behind the forecast data on most weather-related mobile apps/websites (from here on I'll just refer to them as apps).

Two categories of forecasts

Forecasts that you see in weather apps basically fall into one of two categories: those that are manually produced (typically by a meteorologist who works in the area the forecast is for) and those that are automated.  Most national apps, such as those produced by huge news and weather companies like Yahoo! and The Weather Channel among others, fall into the latter category.  In fact, there are now also companies that mass-produce "local" weather apps that are nothing more than a template with automated forecasts.

A range of blended forecasts

There are also a few "in-between-ers" that have local forecasts that are slightly-tailored versions of an automated forecast.  In the forecasting world, we have terms for these 'tweeners depending on how much tailoring is done.  A "human over the loop" forecast is one in which a meteorologist (or many times just a data quality checker) monitors the forecasts produced but only edits them if they are grossly in error, usually due to a computer error of some sort, not because the automated forecast is wrong. A "human in the loop" forecast is one in which the meteorologist (typically) monitors the automated forecast but modifies or edits it based on meteorological reasoning. In other words, in these 'tweener forecasts, automation produces the forecasts, but the output is quality checked for obvious computer errors or meteorological reasoning.  We consider "human in the loop" forecasts of generally acceptable quality, while "human over the loop" and strictly automated forecasts to be of unacceptable quality.

Automation and forecast models

In all but those manually generated, automation plays a significant role in the mass production of local forecasts.  The automated forecasts are produced by weather computer models - often the same models that meteorologists use to prepare their forecasts. So why is that bad?  It's not always. But the forecasts typically get worse the farther into the future you're looking.  This is mainly because there are so many variables that can change even a little bit that could set off a completely different set of outcomes.  (Think of the pebble that is dropped in a tranquil lake in which the ripples expand out from the center, or the butterfly effect in which a tiny wind current is disturbed, triggering a series of events downstream).

First a short background: there are MANY computer models available that could be used to produce a forecast. The ones that go farthest out (past 3-5 days) are generally global models (they produce data for the whole world) and are run 2-4 times a day and typically produce data at 3-6 hours intervals. In the short-term (0-3 days), there are many more options, they  can be run anywhere from hourly to every 6 hours, and many times produce forecast data at 1-3 hour intervals.  When a meteorologist talks about a "run" of a particular model, he/she is referring to the time that the model initialized, which for most models, except the high-resolution short-term models, is 00Z, 06Z 12Z, or 18Z (or 6am, noon, 6pm, or midnight CST).  So for each computer model of this type there are 4 "forecast runs" produced each day and each can vary (sometimes widely) from the previous run - even for the same model!

Model output example

Let's take the example of Halloween night (6 1/2 days out from this writing).  A fairly significant storm system is expected to be in the area mid-week, so it's a good example (and pertinent since many people's plans will depend on the forecast).  The model output below was produced by two different computer models (the U.S.-based GFS and U.K.-based European model) and three different runs (2 from the European and 1 from the GFS).  They all show precipitation accumulation over the preceding 6 hours leading up to the valid times shown.  Commentary on each is below the graphic.  (All graphics courtesy of WeatherBell - click any image to enlarge it.)

The European (Euro) model is only run twice a day so we start with yesterday morning's run.  It shows rain, not heavy but an organized area, moving through Memphis Halloween afternoon/evening.  Enough to mess up some costumes!  If your weather app is based on this model, it tells you your trick-or-treaters will get wet.

The next run of the Euro, from last night, shows the weather system a little wetter, but also slower.  The chances of getting wet in Memphis Halloween night are pretty good (remember this precip accumulation ends at 7pm, so we would presume - correctly - that the heavier rain would be directly over Memphis after 7pm). Your Euro-based weather app though might indicate only a "chance of rain" Wednesday, because most of it falls after 7pm that night. 

Shifting to last night's overnight run of the GFS model (which is produced 4 times a day), the above graphic is the precip expected Thursday afternoon and early evening.  Obviously it shows the weather system well east of Memphis.  A dry evening for trick-or-treating!  (We could also assume it would be cooler than the Euro model shows since the front pushing the precip has moved through versus the Euro which would have the cold front still to our west.)  Your weather app says rain chances are zero!

Finally, we look at the same GFS run as above, but the scenario on Wednesday evening (24 hours earlier and the night the Tigers take on Cincinnati on national TV at the Liberty Bowl). Compare that image to the Euro images above. It looks like almost the same solution, only 24 hours earlier!  The two models are handling the same system in a similar manner, only the Euro is about a day later than the GFS!

[Caveat: European data is rarely used for weather apps, but the point is the same.  Different models can and often do have vastly different results.]

Crap apps -- garbage in, garbage out

Automated forecast apps are captive to one particular forecast model and will change every time the model runs and produces new output (up to 4 times a day).  If the computer model has a good handle on the forecast, they can be very accurate.  If the model is wrong ("garbage in"), the output is "garbage out."  The farther into the future you look, the more likely you are to see wildly-varying forecast and forecasts that are just plain wrong.  These automated forecasts come from what a noted Birmingham broadcast meteorologist calls "crap apps" and I completely agree.  No skill, no human intervention, just a pile of manure!

Automated forecasts can do a decent job, especially in the first day or two of a forecast, but also rarely provide any level of detail.  "40% chance of t'storms" with a cloud/lightning icon doesn't tell you whether your morning soccer game, or afternoon outdoor chores, or evening outing will get wet!

Take the example below for one.  If you have both apps, which do you use?

So is my weekend a washout or sunny?

In the same category, you get stuff like this (even "local" sources can sometimes be misleading):

Which is it - sunny or mostly cloudy?  And what is a 17% chance of rain? 

How Cirrus-produced forecasts differ

The most accurate forecasts tend to be those that are locally-produced and manually-generated by a degreed meteorologist that knows the area he/she is forecasting for like the back of their hand.  It is also preferable that he/she has forecasted for the region (as well as verified forecasts against actual conditions) for a number of years.

The forecast you will find in mobile apps produced by Cirrus Weather Solutions (MemphisWeather.net and StormWatch+) are of two varieties, and neither are automated.  First, MemphisWeather.net's "MWN Forecast" meets the above criteria - hand-written by a degreed meteorologist (ME) that has been in the same area for a number of years (27) and verified years of forecasts against actual conditions.  The forecasts that appear on StormWatch+ are all produced by National Weather Service meteorologists in the local area that the forecasts are generated for.  No haphazard computer models that change run to run or are way off base.

The benefit of a forecast generated by a qualified meteorologist is that that person can look at ALL of the model data (not just one as in an automated crap app), use their training and skills, and produce a forecast that is a blend of data that is more realistic than any one model.  When searching for just the right weather app from a forecast standpoint, know the source, and more importantly TRUST the source.  If it costs a small amount to get the opinion of a trusted source over a free, mass-produced automated forecast, you're probably much better off in the long run!

Do you have experience with "crap apps" or forecasts from sources that seem to always be changing or wrong? We'd love your comments.

--Erik Proseus, Meteorologist, Cirrus Weather Solutions, LLC

----
Follow MWN on Facebook, Twitter, and Google+
Visit MemphisWeather.net on the web or m.memphisweather.net on your mobile phone.
Download our iPhone or Android apps, featuring StormWatch+ severe weather alerts!