[b]Introduction[/b]

Hi all, since 2011 I've done a yearly weather model experiment to find the best performing one.

The previous winners were,

2011 ECM
2012 UKMO
2013 GEM

This time the experiment took part during December 2014. We will be testing the nine weather models listed below,

European Centre for Medium Range Weather Forecasts = ECM
Japan Meteorological Agency = JMA
Met Office = UKMO
Canadian Weather Service = GEM
Global Forecast System = GFS
Global Forecast System Parallel = GFS(P)
Navy Global Environmental Model = NAVGEM
Global weather forecast model = GME
China Meteorological Administration = CMA

So how does it work? To make it fair all of the models are tested by using their 12z runs from the same day. All they have to do is forecast from 24 to 144 hours for a random location I've chosen the more accurate they are the more points are awarded and bonus points are awarded for how far they forecast out. For example the points awarded for getting something correct at 144 hours is much higher than at 24 hours.

[b]24 Hours[/b]

As you would expect its pretty close between them with the ECM and JMA doing the best here.

[attachment=239658:24hours.png]

[b]48 Hours[/b]

The ECM and both the new and old GFS models all performed well here with the UKMO and NAVGEM not too far behind them.

[attachment=239659:48hours.png]

This chart below shows the total scores so far and who has performed the best upto the current time scale. The ECM currently leads with both the GFS models behind.

[attachment=239660:Upto48.png]

[b]72 Hours[/b]

The JMA performed the best here while the ECM and UKMO are not too far behind.

[attachment=239661:72hours.png]

The latest total scores show the ECM still leading with the JMA and UKMO just behind. The GFS models are still close and last years winner the GEM seems to be performing below expectations.

[attachment=239662:Upto72.png]

[b]96 Hours[/b]

Surprisingly its the old GFS model that tops the scores here and is comfortably ahead of second place. The CMA done a good job as well beating the UKMO and last years winner.

[attachment=239666:96hours.png]

The latest total scores show the ECM still leading but the old GFS model has caught up with it and is only a few points behind.

[attachment=239667:Upto96.png]

[b]120 Hours[/b]

The new GFS model done a great job here and performed much better than the older version. The ECM and NAVGEM also performed well at this time range.

[attachment=239672:120hours.png]

The top scores up to this point show the ECM still leading with the JMA and the new GFS model behind it. Last years winner GEM is going to have to perform greatly at 144 hours if it wants the top spot again.

[attachment=239673:Upto120.png]

[b]144 Hours[/b]

Once again the new GFS model performs much better here than not only the older version of the GFS model but than the rest as well it's so far ahead in points here. The CMA also done a good job while the ECM performed slightly poorer than expected.

[attachment=239675:144hours.png]

The final results are shown below,

[attachment=239676:FinalResults.png]

So it's the brand new GFS model that's the winner and it managed to beat the ECM at the last part of the experiment however the ECM still done a great job and was for most of the experiment the best performing model. The old GFS finished in third nearly 100 point behind the new version so this may be evidence than the new GFS model is indeed better. Next the JMA and UKMO finish still near the top and are close in points. NAVGEM finishes in the middle spot while last years winner GEM done terrible this time. CMA and GME finish in the final two spots at the bottom.