ATTACHMENT E: REVENUE FORECASTING PERFORMANCE
E.1: Revenue Forecasting Performance
Introduction
This attachment extends the analysis in Section 3 of the performance of Treasury’s forecasts of tax revenue. It first looks briefly over the entire revenue forecasting history since Federation (110 years) for general trends in forecasting performance. It then takes a more detailed look at the revenue forecasting performance over the past two decades, consistent with the time period chosen for the analysis of the economic forecasting performance.
Long run analysis (1901‑02 to 2011‑12)
Figure E.1 below shows the forecast error (f* — f) for each Budget forecast of total revenue since 1901‑02 (the bars), as well as the 15‑year rolling mean absolute percentage error (MAPE, the light blue line) and the 15‑year rolling mean error (the red line). 
Figure E.1: Budget Forecast Errors For Growth In Total Revenue

As can be seen from Figure E.1, the forecasts have become more accurate (lower MAPE) and less biased (smaller mean error) over the second half of the past century. The rolling average MAPE has fallen from around 7 percentage points to around 2 percentage points, indicating a substantial improvement in accuracy. The MAPE has risen again to around 3 percentage points in recent years, with the 2008‑09 Budget forecast for 2008‑09 being particularly inaccurate, overstating actual revenue by around 7 percentage points, the largest overstatement since 1930‑31. This forecast was made before the GFC, which had a very significant negative impact on tax revenue in 2008‑09. The issues surrounding revenue forecasting during the GFC are discussed in more detail in Section 4 of this report. 
In terms of bias, the forecasts were on average nearly 5 percentage points below the actual outcomes up until the start of the 1960’s, indicating a conservative tendency to significantly underestimate government revenue. Since the 1960’s, the forecasts have shown only a slight conservative bias, underestimating revenue by ½ of a percentage point on average.
Recent Revenue Forecasting Performance
A summary of Treasury’s performance over the past two decades for forecasts of tax revenue against Final Budget Outcomes are shown in Tables E.1 and E.2. A comparison of Treasury forecasts with those of Access Economics, as well as those of official agencies overseas, is presented below.
Table E.1: Performance of forecasts of growth in tax revenue against Final Budget Outcomes (percentage points)
(a) Budget forecast for the financial year which starts in July (two months later). In 1990‑91 to 1993‑94 and 1996‑97 the Budget was published in August and so it is the Budget forecast for the financial year which had started one month earlier.
(b) MYEFO forecast for the financial year which started in July (four months earlier). Available from 1996‑97.
(c) Budget forecast for the financial year which started the previous July (ten months earlier). In 1990‑91 to 1993‑94 and 1996‑97 the Budget was published in August and so it is the Final Budget Outcome.
As would be expected, the accuracy of the forecasts generally improves as the forecast horizon shortens. The MAPE for Budget forecasts of the coming financial year (made two months prior to the start of the relevant financial year) is 2.7 percentage points. In contrast, the MAPE for Budget forecasts of the current financial year (made around ten months after the start of the relevant financial year) is 0.6 of a percentage point. This is due to the fact that more accurate information on revenue and economic activity becomes available as the forecast horizon shortens. 
Over the full sample, Treasury’s forecasts of tax revenue growth have only a minor negative bias, being on average only 0.3 of a percentage point below the outcomes. This figure has not been found to be statistically different from zero. However, within the sample there are two time periods during which tax revenue was generally overestimated — the early 1990’s recession, and the recent period since the GFC — and one phase during which tax revenue was generally underestimated — the period of continuous economic expansion from 1994‑95 to 2007‑08. 
Figure E.2 below shows the evolution of Treasury’s forecasts for taxation revenue growth over the past 22 years, and the Final Budget Outcomes. 
Figure E.2: Evolution of Taxation Revenue Forecasts

Moving from annual growth rates to levels of annual revenue in dollars (Table E.2), Treasury’s taxation revenue forecasts are, on average, $1.1 billion below Final Budget Outcomes (or 0.4 per cent of total 2011‑12 tax revenue) while the Budget forecasts are, on average, $0.4 billion below outcomes (or 0.1 per cent of total 2011‑12 tax revenue). 
Table E.2: Performance of forecasts of tax revenue levels against Final Budget Outcomes (normalised to 2011‑12 values, $billion*)
*Normalised using nominal GDP growth, to calculate the level of tax revenue error if the nominal economy had been at its 2011‑12 size for the whole period.
(a) Includes the Budget forecasts, MYEFO forecasts, and Budget within year forecasts.
(b) Budget forecast for the financial year which starts in July (two months later). In 1990‑91 to 1993‑94 and 1996‑97 the Budget was published in August and so it is the Budget forecast for the financial year which had started one month earlier.
(c) MYEFO forecast for the financial year which started in July (four months earlier). Available from 1996‑97.
(d) Budget forecast for the financial year which started the previous July (ten months earlier). In 1990‑91 to 1993‑94 and 1996‑97 the Budget was published in August and so it is the Final Budget Outcome.
Figure E.3 shows the Budget error in billions of dollars for each year, and the mean error in each of the four sub periods. 
Figure E.3: Errors in Budget Forecasts of the Level of Tax Revenue 
(normalised to 2011‑12 values, $billion*)

* Normalised using nominal GDP growth, to calculate the level of tax revenue error if the nominal economy had been at its 2011‑12 size for the whole period.
E.2: Comparisons with other Forecasters
Access Economics
Comparisons of Treasury total tax revenue forecasts with those of Access Economics from their Budget Monitor publication are presented in Table E.3 below.
Table E.3: Performance of Treasury and Access forecasts of revenue growth against Final Budget Outcomes (percentage points)
Note: Access forecasts are on an accrual (not cash) basis from 1999‑00, and are compared with Final Budget Outcomes on an accrual basis. Access forecasts are generally taken from the May Budget Monitor (for Budget) and the November Budget Monitor (for MYEFO), which are usually released around a week in advance of the Budget and MYEFO publications.
(a) Includes the Budget forecasts, MYEFO forecasts, and Budget within year forecasts.
(b) Budget forecast for the financial year which starts in July (two months later). In 1990‑91 to 1993‑94 and 1996‑97 the Budget was published in August and so it is the Budget forecast for the financial year which had started one month earlier.
(c) MYEFO forecast for the financial year which started in July (four months earlier). Available from 1996‑97.
(d) Budget forecast for the financial year which started the previous July (ten months earlier). In 1990‑91 to 1993‑94 and 1996‑97 the Budget was published in August and so it is the Final Budget Outcome (this applies to Treasury and Access).
Treasury’s Budget forecasts of tax revenue have been comparable to those of Access Economics over the past 22 years. The differences in forecasting accuracy between Treasury and Access are small and were found not to be statistically significant at the 10 per cent level. In this regard, it is worth noting that Access Economics publish their forecasts around a week in advance of the Budget, and have access only to the tax policy information that is made publicly available at this time. An adjustment is made to Access’ forecasts to allow for Budget costings of policy measures in an attempt to remove the information advantage that Treasury forecast would otherwise have over Access in relation to new policy measures announced in the Budget. Figure E.4, below, shows the Budget forecast errors of the two organisations for each of the past 22 years.
Figure E.4: Treasury and Access Economics Errors for Budget Forecasts of the Growth of Tax Revenue

Note: Access forecasts are on an accrual (not cash) basis from 1999‑00. They are generally taken from the May Budget Monitor (for Budget) and the November Budget Monitor (for MYEFO), which are usually released around a week in advance of these publications. 
Official Agencies Overseas
This section compares the performance of Treasury total tax revenue forecasts with the forecasts prepared by official agencies overseas — HM Treasury (United Kingdom), Department of Finance Canada, New Zealand Treasury, the Office of Management and Budget (United States). These countries have broadly similar tax systems to Australia. 
Figure E.5 below shows the outcomes for tax revenue growth in each country over the past decade. Tax revenue growth in Australia has been quite similar to the other countries over this period, with the exception of the United States, where growth has been more variable. All countries experienced falling tax revenue during the GFC due to the adverse economic impacts on production, consumption, profits and employment. 
Figure E.5: Annual Tax Revenue Growth by Country

Different institutional environments in which the revenue forecasts are prepared may have implications for forecast accuracy. For example, the United States may be at a particular disadvantage as their Budget is released relatively far in advance of the start of the fiscal year (around eight to 10 months prior), and the legislature has the power to alter any of the tax policies set out in the Budget. In contrast, in the other countries we examine, Budgets are released relatively close to the start of the fiscal year (two months or less prior), and the legislature has limited power to adjust tax policies, unless they wish to reject the entire Budget. Our results for the United States attempt to adjust for any forecast disadvantage due to the impact of post‑Budget changes to tax policies.
Figure E.6 shows the forecast errors (f* — f) for growth in tax revenue across the five agencies, over the past decade. The red diamonds are the errors observed for 2008‑09 or equivalent financial year for each country — these errors highlight the universal difficulties experienced in forecasting at this time. The revenue forecasting performance of the Australian Treasury is comparable to that of official agencies in the United Kingdom, Canada and New Zealand. The United States forecast performance is much worse, with errors almost three times as great.
Figure E.6: International tax revenue forecast errors (2001‑02 to 2010‑11)

(a) Data for Canada excludes 2002‑03, as no Budget is available for this year.
(b) Total revenue forecasts, including tax and other miscellaneous revenue. Adjusted for post‑Budget changes to policies.
Table E.4 below shows summary statistics for forecast performance for each country over the past decade. The United Kingdom forecasts have been the most accurate, with a MAPE of around 2 per cent, with Australia, Canada and New Zealand displaying similar levels of forecast accuracy with MAPEs of between 2½ to 3½ per cent. The United States forecasts are far more inaccurate, with a MAPE of around 9 per cent. The differences in forecasting accuracy between Australia and official agencies overseas were found not to be statistically significant at the 10 per cent level, except for the United States.
In terms of bias, the Canadian and New Zealand forecasts show some negative bias, with an average error of around ‑1¼ to ‑1½ per cent, whereas the Australian forecasts have displayed a smaller negative bias of around ‑¾ per cent over this period. The United Kingdom forecasts have a slight positive bias of around ½ of a per cent, and the United States forecasts have a substantial positive bias of around 4½ per cent.
Table E.4: International performance of tax revenue forecasts against outcomes (2001‑02 to 2010‑11)
(a) Data for Canada excludes 2002‑03, as no Budget is available for this year.
(b) Forecasts for total on‑Budget revenue, including tax and other miscellaneous revenue. Adjusted for post‑Budget changes to policies.
Trend estimate forecast comparison
This section compares the performance of Treasury’s Budget tax revenue forecasts with those from a simple trend approach. All data have been adjusted to remove tax policy changes, creating underlying tax revenue series and forecasts. These adjustments attempt to remove the policy advantage that Treasury’s forecasts would otherwise have over the trend forecasts (for example, the Treasury forecasts would factor in the introduction of the GST in 2000‑01, whereas the trend forecast would not).
For the purposes of this analysis, ‘trend’ is defined as the average annual growth rate over the previous X years for which an outcome is available, consistent with the approach taken in the economic section of this report. For example, the 3 year trend estimate in 2011‑12 is the average of tax revenue growth in 2007‑08, 2008‑09 and 2009‑10. The trend results are compared with Treasury’s Budget forecasts.
Figure E.7 below shows the outcome for underlying tax revenue growth compared with the Budget forecasts and the 10‑year trend. The Budget forecasts are clearly much better at capturing the cyclical influences on tax revenue growth than using a trend approach. In particular, they capture the two major downturns in tax revenue in 1991‑92 and 2009‑10 and the subsequent recoveries.
Figure E.7: Budget and Trend Estimates Versus Actual Tax Revenue Growth 
(growth rates, underlying)

Table E.5 shows the performance of Budget forecasts against trend estimates of varying lengths.
Table E.5: Performance of Budget forecasts of growth in tax revenue against trend estimates (underlying)

Over the full sample, Treasury’s forecasts of tax revenue growth are more accurate than the trend estimates. Treasury forecasts have a lower MAPE for the period as a whole, and for three of the four sub‑periods. The one and three year trend estimates are more accurate over the ‘Mining Boom Mark I’ period from 2003‑04 to 2007‑08, a period over which tax revenue was consistently underestimated by Treasury.
The trend estimates do particularly badly in terms of accuracy over the two recession and recovery periods, from 1990‑91 to 1993‑94 and from 2008‑09 to 2011‑12. Treasury’s forecasts predict these downturns and the bounce back in tax revenue during the recovery phase, whereas the backward‑looking trend estimates do not. 
E.3: Relationship between Revenue Errors and Nominal Economy Errors
This section examines the correlations between the forecast errors for tax revenue and the nominal economy (or economic base), over the past decade. 
There is not an exact relationship between the overall growth of the nominal economy and growth in total tax revenue. The historical relationship between economic and tax growth over the past decade is shown in Figure E.8 below. The correlation coefficient between these two series is 0.55. One reason why the series will not track exactly is that revenue collections will be impacted by tax policy changes — for example, a reduction in a tax rate or a broadening of a tax base. The red line below shows underlying tax revenue growth (adjusting for tax policy changes), and it does map slightly better with economic growth than the grey line (which is not adjusted for policy changes), with a correlation coefficient of 0.73. However, there are still obviously other sources of differences between economic and tax growth, including timing differences, which are discussed further below.
Figure E.8: Nominal GDP growth versus tax revenue growth (annual average)

In terms of a particular head of revenue (such as company tax), there are several reasons why growth in the head of revenue may not track closely with growth in the corresponding economic base. These include:
•	Policy decisions which lead to variations in the growth of the head of revenue. 
•	Timing differences between economic activity and the receipt of the associated tax revenue.
•	Differences in scope between the tax base for the head of revenue, and the associated economic base. For example, fringe benefits tax is levied on the value of fringe benefits provided to employees, but the closest economic base is a wages measure (see Table C.3 in Attachment C for a description of the main head of revenue mapping models).
•	Miscellaneous factors such as changes in compliance activities of the ATO.
Table E.6 below shows the major heads of revenue and the primary associated economic base. It also notes how much of the total tax base each head of revenue comprises, and the correlation between the head of revenue and its associated economic base.
All of the remaining analysis in this Attachment excludes capital gains tax (CGT), whereas CGT is included in the rest of the report. The economic bases do not include capital gains. CGT is forecast separately using a stock model which incorporates assumptions regarding the timing of gain realisation and loss utilisation.
Table E.6: Major heads of revenue
Note: All numbers exclude capital gains tax.
(a) In 2011‑12.
(b) Correlation between growth rates in revenue and associated economic base, between 2002‑03 and 2011‑12. GST calculations start from 2003‑04, as 2002‑03 was a transitional year when revenue accelerated due to increased compliance and education.
(c) Income tax withholding has been adjusted for tax policy measures in calculating the correlation coefficient.
As can be seen from Table E.6, the individual economic bases do not map perfectly with the major heads of revenue. In particular, the economic bases used as an input in forecasting company tax and other individuals tax do not have a good relationship with these revenue heads, with correlation coefficients of 0.26 and 0.05. This is one source of difficulty in translating the economic bases into tax revenue forecasts. There are also several other potential sources of revenue forecast error, such as:
•	forecast errors in the economic base;
•	errors in tax policy costings;
•	errors in tax revenue timing estimates; and
•	miscellaneous factors such as post‑Budget policy decisions and court decisions relating to tax law interpretation.
The contribution of an individual head of revenue to the overall tax revenue forecast error will depend upon its share of the tax base (its relative importance), and the error in the forecasts for that head of revenue. Figure E.9 shows the contributions of the major heads of revenue to the total Budget tax revenue forecast error, over the past decade. 
Figure E.9: Contribution to total tax revenue error by head of revenue (Budget forecasts)

The main contributors to total tax revenue error over the past decade have been company tax, income tax withholding and GST. The company tax contribution reflects larger percentage forecast errors, while the income tax withholding contribution reflects smaller percentage forecast errors which are amplified due to its relative importance (since it comprises nearly 50 per cent of the tax base). Section 4 of the Report discusses in greater detail some of the main forecast errors by head of revenue since the start of ‘Mining Boom Mark I’ (2003‑04).
Table E.7 below looks at the relationships between the major pairs of tax head of revenues and associated economic bases. It shows the mean error and MAPE for each of these pairs — in general, the error on the head of revenue should be at least as high as the economic base error, since this error is just one of many potential sources of revenue error (other sources of error have been briefly outlined above — tax policy costing error, timing error etc). The standard deviation of each series is also shown, with a higher number indicating a more volatile series which is more difficult to forecast.
The correlation between the historical growth for each pair is also shown (which was also reported in Table E.6), as is the correlation between the Budget forecast errors. The forecast errors should generally be well correlated, except where there are significant differences in timing or scope between the head of revenue and the economic base which need to be adjusted for in the head of revenue model (for example, in the case of company tax and other individuals tax). Appendix A of Section 4 of the Report shows the significant adjustments that take place to the Corporate GOS economic base in order to generate the company tax forecasts.
Table E.7: Relationship between errors in the economic base and taxation revenue (Budget forecasts)

Note: All numbers exclude capital gains tax.
(a) Standard deviation of series growth rates, from 2002‑03 to 2011‑12.
(b) Income tax withholding has been adjusted for tax policy measures in calculating the correlation coefficient.
(c) GST calculations start a year later, in 2003‑04, as 2002‑03 was a transitional year when GST revenue accelerated due to increased compliance and education.
In the case of the largest head of revenue, income tax withholding, the majority of the forecast error over the past decade has been driven by error in the associated economic base (compensation of employees). The correlation coefficient between the two error series is 0.85, indicating that there are not significant timing or scope differences between income tax withholding and compensation of employees. In addition, the two series have very similar mean errors and MAPEs over this period, indicating that there is not much additional revenue forecasting error beyond the economic base error.
Figure E.10 below shows the forecast errors between income tax withholding and the associated economic base. Most of the errors fall in the top right quadrant (the errors on both forecasts are positive) or the bottom left quadrant (the errors on both forecasts are negative).
Figure E.10: Budget forecast errors on compensation of employees and income tax withholding growth

In contrast, the error in the second largest head of revenue, company tax, has only been weakly correlated with the error in its corresponding economic base, gross operating surplus (correlation coefficient of 0.18). 
Figure E.11 shows the forecast errors between company tax and gross operating surplus. Several of the recent errors on company tax have been positive (in the top left and right quadrants), with the mean error on company tax revenue forecasts being 2.3 percentage points over the past ten years. This overestimation of company tax revenue has been driven by factors including longer than usual lags in the recovery of company tax payments following the GFC, and an increasing share of the economy being accounted for by the mining sector, which currently has a relatively low corporate tax‑to‑GOS ratio. 
The relationship between company tax and gross operating surplus is complex. Although the correlation coefficient between the errors is low, this is in a large part due to two factors. Firstly, the outlier 2009‑10 result, which is difficult to explain, has a significant effect. Without this data point, the correlation coefficient is 0.65. Secondly, the lag between the economic activity (as measured by GOS) and company tax caused by the company tax payments system, means that the relationship should not be expected to be exact. The timing model for company tax attempts to account for this lag, and is described in more detail in Section 4 of this Report. 
Figure E.11: Budget forecast errors on gross operating surplus and company tax

The third largest head of revenue, GST, has a reasonably good historical relationship with its main economic base (consumption subject to GST), indicating that the series have similar scope. The forecast errors are also reasonably well correlated (correlation coefficient of 0.80), as shown in Figure E.12, with a notable exception in 2011‑12 when there was a large forecast error on GST collections and almost zero error on the economic base. This was due to the growth outcomes for these series diverging significantly, with GST collections falling by 0.5 per cent while the economic base grew by 5.4 per cent. This issue is discussed further in Section 4 of this Report. 
Figure E.12: Budget forecast errors on consumption subject to GST and GST

The final and smallest individual head of revenue which we examine, other individuals taxes, comprises a variety of items including taxes on interest, dividends and small business income. Partly because of the diversity of this head of revenue, it is difficult to find a corresponding economic base. The main economic base used, gross mixed income, does not map particularly well with the head of revenue in history (correlation coefficient of 0.05). Other smaller economic bases are also utilised in the other individual’s taxes model, including other business income and property income economic bases such as interest and rent. Figure E.13 shows the forecast errors are also not well correlated with the economic base (correlation coefficient of 0.25).
Figure E.13: Budget forecast errors on gross mixed income and other individuals tax

Overall, the total tax revenue errors are reasonably well correlated with the error on the forecast for the nominal economy (correlation coefficient of 0.62), as shown in Figure E.14 below.
Figure E.14: Budget forecast errors on nominal GDP (non‑farm) and total tax revenue