ATTACHMENT D: MACROECONOMIC FORECASTING PERFORMANCE
This attachment extends the analysis in Section 3 of the performance of Treasury’s macroeconomic forecasts over the past two decades.
Choice of ABS data benchmark: first published or most recent outcomes
A summary of Treasury’s forecasting performance for forecasts of year‑average nominal GDP growth against the first‑published outcomes and estimates of the most recent, outcomes (the June quarter 2012 National Accounts) are shown in Tables D.1 and D.2 respectively.
Table D.1: Performance of nominal GDP growth forecasts against first published outcomes

Table D.2: Performance of nominal GDP growth forecasts against most recent estimated outcomes

The mean error in the forecasts depends importantly on whether the forecasts are compared with the first‑published outcomes or with the most recent estimated outcomes. On average over the full sample, Treasury’s nominal GDP growth forecasts are broadly in line with the first‑published outcomes, but around ¼ of a percentage point lower than the most recent estimated outcomes. The differences arise because the ABS has revised up estimated year‑average nominal GDP growth outcomes over time, largely due to upward revisions to growth in real GDP, rather than the GDP deflator.
All subsequent tables and charts compare the economic forecasts with the most recent estimated outcomes. This is for two reasons. Firstly, the most recent estimated outcomes presumably represent the ABS’s current best estimates of the true outcomes. And, secondly, Treasury’s revenue mapping models use the most recent estimates of the nominal economy in order to forecast tax revenue. 
Terms of trade
Treasury’s forecasting performance for growth in the terms of trade against the most recent estimated outcomes is shown in Table D.3 and Figure D.1.
The large size of the absolute errors in Treasury’s forecasts of the terms of trade is a reflection of the difficulty in forecasting commodity prices. The forecasts have tended to underestimate growth in the terms of trade since the early years of the 2000s, and particularly during the first mining boom. The absolute error has been largest over the period post the GFC, reflecting heightened volatility in global prices of coal and iron ore.
Table D.3: Performance of terms of trade growth forecasts

Figure D.1: Evolution of terms of trade growth forecasts

Figure D.1 suggests that the direction of change in the terms of trade is usually forecast correctly, although the size of the change has been typically underestimated. 
Correlation of the forecast errors
As nominal GDP is the product of real GDP and the GDP deflator, a strong positive correlation between nominal GDP and these two components should be observed.
Table D.4 shows correlation coefficients between Budget forecast errors for nominal GDP growth and a selection of its main components over the period 1996‑97 to 2011‑12. For each variable, the forecast error is the difference between the Budget forecast and its estimated outcome from the June quarter 2012 National Accounts release.
Table D.4: Correlation coefficients between Budget forecast errors, 1996‑97 to 2011‑12

Consistent with expectations, the correlations found in Table D.4 indicate nominal GDP growth forecast errors are strongly correlated with price growth forecast errors (growth in the GDP deflator and the terms of trade) and, to a lesser extent, real GDP growth forecast errors. Figure D.2 summarises the forecast errors for nominal and real GDP and the terms of trade.
Figure D.2: Summary of Treasury Forecast Errors

Expenditure components of real GDP
Treasury adopts a sectoral team based approach to forecasting, with aggregate GDP forecasts built from components of the expenditure measure of GDP. The relative volatility of the various expenditure components makes some expenditure components easier to forecast than others. For example.
Figure D.3 illustrates the volatility of annual dwelling and business investment growth compared with the relative stability of household consumption growth.
Figure D.3: Growth in Household Consumption, Dwelling and business Investment 

Table D.5 below shows the mean error and mean absolute percentage error (MAPE), standard deviation, and share of the economy (relative importance) of the components of GDP(E) over the period 1998‑99 to 2011‑12. As expected, forecasts for the most volatile expenditure series tend to have the largest forecast error. The results also indicate that Treasury has been too optimistic forecasting exports growth and too pessimistic forecasting business investment growth and correspondingly imports growth.
Table D.5: Performance of GDP expenditure component forecasts (1998‑99 to 2011‑12, all forecast rounds)

Income components of GDP
To provide macroeconomic series for forecasting tax revenue, it is necessary to split nominal GDP into its various income components, in particular, compensation of employees, profits earned by companies, and profits earned by unincorporated businesses.
The largest income component is compensation of employees, which measures wages and other payments made to employees. Table D.6 and Figure D.4 show Treasury’s forecasting performance for this series. The forecasts show the same pattern of overestimation common to the forecasts of nominal GDP growth in the early years of the forecasts. Forecast accuracy improved considerably after 1994‑95, before deteriorating again during Mining Boom Mark I and then improving again over the post‑GFC period.
Table D.6: Performance of forecasts of growth in compensation of employees against most recent estimated outcomes

Figure D.4: Evolution of compensation of employee’s growth forecasts

Errors in nominal GDP growth forecasts that are not reflected in forecast errors of growth in compensation of employees manifest themselves in forecast errors of growth in the other components of income, in particular gross operating surplus and gross mixed income. Growth in compensation of employees is relatively more stable than growth in either gross operating surplus or gross mixed income.
Table D.7 and Figure D.5 shows the forecasting performance for corporate gross operating surplus (GOS), which is the sum of GOS for non‑financial private companies, financial private companies and public trading enterprises.
Table D.7: Performance of forecasts for growth in corporate GOS against most recent estimated outcomes

Figure D.5: Evolution of corporate GOS growth forecasts

Note: The definition for private financial corporation gross operating surplus was changed from the 1998‑99 national accounts onwards, creating a break in series that forecasts cannot be compared across.
Over the past 22 years, Treasury has underestimated corporate gross operating profits on average by around ¼ of a percentage point when compared with recent published outcomes, with a mean absolute percentage error of around 3 percentage points. The size of the underestimation has increased since the start of Mining Boom Mark I. Over the post‑GFC period, the negative bias has fallen slightly; however, there has been a continued increase in the mean absolute percentage error, which largely reflects the very large forecast error in 2009‑10.
Serial correlation of aggregate forecasts
Revisions to Treasury’s forecasts do not display serial correlation. As Table D.8 shows, the average serial correlations of revisions to the forecast aggregates are close to zero across the full sample period. This indicates that the revisions to Treasury’s forecasts are unpredictable. In other words, revisions to Treasury forecasts between rounds occur as a result of (completely) incorporating new information. 
Table D.8: Serial correlation of forecast revisions (1990‑91 to 2011‑12)

Identifying directional changes in forecasts
Table D.9 summarises Treasury’s performance in correctly identifying accelerating (decelerating) year‑on‑year growth for the key economic aggregates. Acceleration (or deceleration) in growth is defined as a year‑on‑year change in growth in excess of ½ of a percentage point.
The results in Table D.9 indicate that Treasury is generally successful in identifying the direction of changes in the key aggregate forecasts. With the exception of the GDP deflator (where forecasts have been less accurate), changes in growth have been correctly predicted around 60 to 75 per cent of the time over the past 20 years.
Table D.9: Correctly predict acceleration (1992‑93 to 2011‑12)