Relative model performance in predicting thunderstorms

[poseidon]

Just wondered if anyone had come across or had any knowledge on relative performance of the various models (e.g. UKMO, GFS, WRF-ARW, WRF-NMM) in predicting potential thunderstorm spots [or conditions] across Europe (or more specifically the UK)? It seems to be sparse as far as I can see.

 

I wouldn't have thought it would be that hard to keep a statistical track of successful predictions for each model?

 

Are some models better at certain scenarios than others, if so what?

Edited May 26, 2014 by poseidon

[knocker]

Do you not require a time frame for the predictions?

[poseidon]

Or several time frames (all presumably fairly short).

[Vorticity0123]

Just wondered if anyone had come across or had any knowledge on relative performance of the various models (e.g. UKMO, GFS, WRF-ARW, WRF-NMM) in predicting potential thunderstorm spots [or conditions] across Europe (or more specifically the UK)? It seems to be sparse as far as I can see.

 

I wouldn't have thought it would be that hard to keep a statistical track of successful predictions for each model?

 

Are some models better at certain scenarios than others, if so what?

 

I don't have any data available about convective performance, so sorry for that.

 

Regarding the last question, it is possible to say something about the performance without actually having any statistics.

As a general rule, it can be assumed that models with a high level of detail (like the WRF) are better in predicting convective activity than models with a more coarse grid (like the GFS). This has to do with the fact that convective acitivity is a small-scale process.

 

Perhaps this helps a little?

Edited May 28, 2014 by Vorticity0123

[poseidon]

Vorticity0123, on 28 May 2014 - 14:03, said:Vorticity0123, on 28 May 2014 - 14:03, said:

I don't have any data available about convective performance, so sorry for that.

 

Regarding the last question, it is possible to say something about the performance without actually having any statistics.

As a general rule, it can be assumed that models with a high level of detail (like the WRF) are better in predicting convective activity than models with a more coarse grid (like the GFS). This has to do with the fact that convective acitivity is a small-scale process.

 

Perhaps this helps a little?

 

Hi Vorticity, thanks for replying to my extremely wide-ranging question. Yes, you should be able to assume that I guess, although I've seen a number of posts saying things like "I don't like WRF" without giving reasons. I just find it odd that all these different models can be used to forecast CAPE, rainfall, etc, etc and we don't seem to know the relative reliability [or do we??]. For instance, is WRF-ARW better than WRF-NMM, and under what circumstances or visa-versa? Is Euro4 better than WRF at forecasting rainfall, or not? Has the usage of these WRF models been evaluated for European or UK topographies & climatic conditions?

 

Given that WRF should be better than GFS at predicting convective activity why would you bother to look at GFS CAPE values [for instance], instead of just going to WRF off the bat?

Edited May 28, 2014 by poseidon

[Vorticity0123]

Hi Vorticity, thanks for replying to my extremely wide-ranging question. Yes, you should be able to assume that I guess, although I've seen a number of posts saying things like "I don't like WRF" without giving reasons. I just find it odd that all these different models can be used to forecast CAPE, rainfall, etc, etc and we don't seem to know the relative reliability [or do we??]. For instance, is WRF-ARW better than WRF-NMM, and under what circumstances or visa-versa? Is Euro4 better than WRF at forecasting rainfall, or not? Has the usage of these WRF models been evaluated for European or UK topographies & climatic conditions?

 

The liking or disliking of a model probably has nothing to do with the model itself; it might just be based on personal experience. 

Regarding the comparison of individual model performances (fine gridded), I don't know if there is any data available. A meteorologist may be able to give a more thorough view of inter-model comparison. 

 

For mid-range models, comparison data is available, see the graph below:

 

Anomaly correlation (a measure of skill) for a variety of mid-range models regarding height anomalies. 

 

In the link below, the mid-range models shown above may be correlated with other variables as well. 

 

http://www.emc.ncep.noaa.gov/gmb/STATS_vsdb/

 

 

Given that WRF should be better than GFS at predicting convective activity why would you bother to look at GFS CAPE values [for instance], instead of just going to WRF off the bat?

 

An explanation for this question might be that the WRF only goes out to 3 days, while the GFS gives results up to 16 days. Therefore, if one would be willing to see whether there is any storm potential in, say, 5 days, it would only be possible to look at the GFS output. In short, it could have to do with the various timescales of the different models.

 

Sources:

http://www.emc.ncep.noaa.gov/gmb/STATS_vsdb/

http://www.meteociel.fr/modeles/wrfnmm.php

http://www.wetterzentrale.de/topkarten/fsfaxsem.html

Edited May 28, 2014 by Vorticity0123

[poseidon]

The liking or disliking of a model probably has nothing to do with the model itself; it might just be based on personal experience. 

 

Quite possible.

 

Thanks for the anomaly info, looks interesting - I shall take a look at it.

 

 

An explanation for this question might be that the WRF only goes out to 3 days, while the GFS gives results up to 16 days. Therefore, if one would be willing to see whether there is any storm potential in, say, 5 days, it would only be possible to look at the GFS output. In short, it could have to do with the various timescales of the different models.

Yes, that did cross my mind, but then a number of sites offer European/UK WRF data out to 5 days anyway. I can see the value of using GFS to look at the longer time-frame synoptic outlook but if a higher resolution mesoscale model should be more accurate for short-term convective forecasting then using GFS for shorter timescales in convective forecasting is sub-optimal.