The criteria of the scientific nature of some theory is ability of researches to approve or disapprove on practice the consequences that can be deduced from the proposed theory. (so called verification principle suggested by K. Popper). When verification principle is applied to economic forecasting it can be rephrase as following. “The ultimate test of the validity of economic propositions is their ability to forecast” (Klein L.R. (eds.) (2009). The Making Of National Economic Forecasts. Edward Elgar Publishing Inc.)
Nevertheless, accuracy of economic forecasting can not be applied as a test of validity/quality to all economic forecasts/models. There are two types of forecasts: forecasts that are made to predict the future; and scenario forecasts that are developed to answer the question – What if ? In the second case the quality of economic forecasts is defined by the coverage of various theoretically realized variants of scenario conditions and economic policy, and the adequacy of the description of the mechanisms of interaction of scenario parameters, options for economic policy and the internal logic of the development of the forecasting object.
We assume that 1 year ahead economic forecasts from international agencies are produced to predict future of separate economies since these forecasts have only one variant and they are revised 2 or more times a year in order to be closer to actual data by taking into account all recent economic information. That is why the comparison of forecast accuracy in this case is acceptable.
The main directions of the analysis of errors in economic forecasts are: a comparison of the accuracy of forecasts of different sources; comparison of forecast accuracy of individual sources with consensus forecast (Novotný, Raková 2010); assessment of the presence of systematic overstatement or underestimation of forecasts (Cabanillas, Terzi, 2012); consideration of various indicators of estimation of prediction errors; Comparison of the accuracy of forecasts for individual countries and at certain time periods (OECD, 2014).
The publications of five international organizations were selected as a statistical basis for assessing the accuracy of economic forecasts:
– World Economic Outlook from International Monetary Fund
– Global Economic Prospects from The World Bank
– Global Economic Outlook from United Nations LINK Project
– Economic Outlook from Organisation for Economic Co-operation and Development
– European Economic Forecast from European Commission
The choice of these sources is also due to the availability of projections for a long retrospective period and broad country coverage. Past forecasts of these organizations have been publicly available since 1999 and for some organizations from an earlier period. The World Bank, the IMF and the United Nations publish forecasts for more than one hundred countries in the world. OECD and the European Commission published forecasts for more than 40 countries.
Date of Publication
All sources release their forecasts at least two times in a year. From each source we take only one publication, which was released closer to the beginning of the forthcoming year. For example, when we calculate one year ahead forecast error for 2010, we compare forecast from the following publications:
– IMF World Economic Outlook – October 2009
– WB Global Economic Prospects – January 2010
– UN LINK Global Economic Outlook – October 2009
– OECD Economic Outlook – December 2009
– EC European Economic Forecast – November 2009
As a basis for comparing projections and actual values, we took the actual data from the IMF World Economic Outlook databases, which were published at the end of the year following the year of comparison.
This way of choosing the basis for comparison is determined by the desire to reduce the impact of revisions of actual data related to changes in methodology, refinements and other factors.
The list of projected indicators from the five organizations listed above crosses entirely only in terms of GDP growth in real terms.
As the forecasts of economic growth rates are published by all five international organizations, and since this indicator is the most important among other economic indicators, the analysis of forecast errors will mainly focus on forecasts of economic growth rates. Also we estimated forecast accuracy for consumer price index and unemployment rate.
In total, the database contains more than 12,000 forecasts of international agencies for 1997-2016 period.
At the first step for each source, indicator, country and year we calculate absolute error (AE) of forecast.
AE = | Actual value for year t – predicted value for year t | / Actual value for year t
At the second step for each source, indicator, country and available time period we calculate mean absolute error (MAE) of forecast.
MAE= ∑AE/n , where n is a number of years, for which absolute error was calculated