A Quick Question
Posted by Jacques Chester on Monday, June 16, 2008
For those of you who’ve moved in public policy circles such as the RBA, Treasury departments and the like: how are the much referred-to policy costings carried out?
This entry was posted on Monday, June 16th, 2008 at 6:02 PM and filed under Blegs, Economics and public policy.
Follow comments here with the RSS 2.0 feed.
Apologies. Comments and trackbacks are both currently closed.
Well, I have personally witnessed a State Treasurer doing the final costings on a half billon dollar policy by adding up the various subtotals quite literally on the back of an envelope.
It was an A4 size envelope though and he did find a $200,000 or so discrepancy. Because of that 0.4% or whatever error, he demanded everything be rechecked from scratch which turned up some much more serious errors in some of the subtotals.
Bit like that anecdote in Stephen Fry’s “The Hippopotamus” where a Hungarian sugar beet entrepreneur, reading through a dense technical report on a proposed sugar beet processing plant, scribbles “but will the machinery take the load?” at random in the margins, forcing the engineers to check and recheck everything and so discover other issues that needed more work.
So while I may not know much about the minutiae of policy costings, I can certainly recommend ways of getting those that do to get it right.
Posted on 16-Jun-08 at 8:17 pm | PermalinkIs that really a quick question?
Sounds more like a trick question to me…
Posted on 16-Jun-08 at 8:54 pm | PermalinkThey have ‘models’ of stuff. So they have big spreadsheets with all they hope they need to know. How many tax payers on this income, how many on that. Then if their numbers are good enough, they can say what will happen to revenue if we cut this tax rate by that amount etc. But . . . they generally are in a poor position to make judgements about the behavioural effects of policy changes, and so costings rarely contain them. And they get asked a lot of questions they don’t know and that their models are not good enough to give very good answers on.
So they do what we all do in such situations – that is, the best they can.
Posted on 17-Jun-08 at 12:05 am | Permalinkdepends on the policy to be costed.
some are very simple. eg a policy to give a grant of $25m costs $25m. costing done
still pretty simple – a program of grants of $25m p.a. in real terms. need to determine the correct index to apply (and that is almost never CPI; could be non-farm GDP, or a combined index of wages and growth, or some other) and cost that. But nevertheless, just a quick mathematical exercise. Likewise a proposal to make an acquisition of property or equipment – not just the actual purchase price but all the associated costs of contracting and administration have to be added in.
slightly more complex – costing a proposed building or road. need to find out how much similar roads or buildings have cost, what special factors apply to this one, what the movements in costs in the industry have been, when the project is anticipated to start and finish.
more complex – regulatory proposals. costs, benefits are harder to estimate and there is a detailed process for doing this in most governemnts, sometimes with rules of thumb for calculating some of these
much more complex – any changes to the tax or social welfare systems. Requires detailed spreadsheets and models that also will try to the extent possible to take account of behavioural changes in response to the policy (Nick, when I was involved my experience was that people doing costing are mostly aware of the impact of behaviour: if they can make a reasonable judgement on them, the costings will try to include them, but will omit them if there’s no basis on which to make assumptions. A good costing will however document this). These models also need to take account of the fact that often new taxes or payments substitute for others. People spend years developing the models, and still it is virtually impossible to get them right.
virtually impossible – innovative policy that takes government in directions it has not gone before: without a base of past spending or revenue patterns on which to base the policy, very hard indeed to cost. this is why often it is best to pilot test such policies in some way.
Posted on 17-Jun-08 at 8:06 am | PermalinkThe limitations to costing quality are time (you often have to churn them out incredibly quickly), data availability and quality, and your negotiating skills with the gatekeepers at the Dept of Finance who in effect have veto powers over your costing. Often Finance adds value by making you be rigorous, but occasionally they prefer to push a politically convenient (for them) line and hence make you less rigorous.
As an old public service tax and welfare coster, I use the following scientific methodology:
- get good quality data, build a good model on it (yep, developing the model can take years) and use that model as much as possible.
- if the right data and/or model is unavailable, get what data you can and build a quick and simple model on it.
- if even mediocre hard data is unavailable, make educated guesses as to the parameters and do back of the envelope calculations.
- in the rare case where there’s not even enough to make educated guesses, then make them up. Because no-one will ever accept an answer of “we don’t know”.
The other interesting thing with costings is that, as Stephen implied, the bigger the proposed policy change the more uncertain the costing. This is simply because more and more parameters – especiaslly behavioural parameters – matter as the size of the change increases. The truth is that even with good data and good models things like a major restructuring of the entire welfare system are virtually uncostable. Which is one good reason why radical policy change is rare.
On behavioural parameters, it’s true that most costings try and avoid them but this is a reflection of their uncertainty rather than a lack of awareness that they matter. And the more uncertain the parameters are, the more likely there will be pressure from some pollie or bureaucrat who wants to kill or promote a particular policy.
In general I trust public service costings more than outside costings, mainly because of the extra specialised resources available to the public service and the much superior access they have to the relevant data. And most bureaucrats see it as an important aspect of their professionalism to make the costings honest, which is more han can be said for costings emanating from lobby groups.
Posted on 17-Jun-08 at 9:10 am | PermalinkSo am I to understand that it’s all Excel spreadsheets and a bit of guesswork?
And do these costings include confidence levels?
Posted on 17-Jun-08 at 11:52 am | PermalinkI used to work for a South Australian oil and gas producer, and you’d be surprised (or perhaps not) at how much business-critical information was sitting in a bunch of spreadsheets on various peoples’ (not backed up) hard drives. I’m sure that that particular company is not unique (particularly as the place I now work has a number of managers who do much the same thing).
So Jacques, I reckon it’s probably spreadsheets and making shit up all the way down …
Posted on 17-Jun-08 at 12:31 pm | PermalinkDepends a bit on what is being costed and at what stage.
There are forward works programs that may be costed on the basis of the costs of similar programs running now in the same jurisdiction or interstate. If these programs are similar enough, they can be scaled up on some unit basis.
For actual approval purposes they are normally be costed on the usual basis of numbers of people required and the resources. However, it depends on what is being asked and how soon the estimates are required. Once that is compromised, then anything can happen…including the back of the envelope. Those costings are normally done in the same manner as costings everywhere else, except that in Govt, tax considerations normally do not count, nor do revenue in a strict sense, since most revenue goes into consolidated revenue rather than the agency coffers.
It is my opinion though that the number of cost overruns the public sees is symptomatic of some systematic error in costing frankly. There was a Harvard Business Review article on this a couple of years ago (also went into time overruns as well) which opined that people wanted to give their higher ups the best news early on, so that estimate and time variable were always trimmed to the most optimistic. Unfortunately I dont have the ref with me in Prague
It is quite common for Treasury to tell Depts how much they can have, the politicians to tell Depts what they must do, so official estimates are then cut and sliced to fit (or worse expanded to fit). Which of course leads in some cases to the cynical attitude of “They pretend to pay for it, and we’ll pretend to do it.” Or, we’ll do it, and then when the cost comes in over estimate, we shrug our shoulders and tell them, thats what it cost. There really is an issue of tension between what is required to be done and what it costs. It is a brave Dept head who goes to a Minister and tells them that what they want cannot be done for the price.
Posted on 21-Jun-08 at 7:16 pm | Permalink