Accurate estimation is impossible for complex technical projects, but keeping to agreed budgets is still possible using feedback and change.
Version: initial draft: 28 December 2006, Latest =29 12 06
Accurate estimation of complex systems and software projects is seemingly impossible to guarantee. There are many unavoidable reasons for this. Even when estimation seems to work, this might just be a case of stopping the effort when the budget runs out, but as a result - delivering systems of unacceptably low quality.
The main idea of this paper is that there is a constructive alternative to bad estimation processes.
You use ‘process control’ (do a little, learn quickly, change quickly) to quickly and continuously tune anything and everything about your project, so that prioritized budgets (like time to market, money, human resources) can be met.
You consciously sacrifice less-holy things for your more-holy things
We are better off stipulating reasonable resource constraints (deadlines and cost budgets) and then learning to live within them. This is acceptable as long as we get our highest priorities satisfied when resources run out. The rest is by definition, marginal.
Why we cannot estimate project costs accurately:
We do not define requirements well enough to estimate their costs
We do not collect or have access to experience data that would allow us to estimate costs, even if we did have clear requirements
Even if we did have historic data for the costs of meeting clear requirements, it would probably not help much because our new project would be different in some way.
We do not really need time and cost estimates:
They are an old custom, intended to prevent overruns, and to give management some feeling that the job will get done in time at a reasonable cost. But they do not in fact prevent overruns or assure value.
What management needs is delivery of some critical system requirements in a predictable time frame
And they need to be sure that the project will be profitable, and not embarrass them with unexpected losses.
This paper describes an alternative way to achieve these management needs.
Principles of Resource Control
The Risk Principles
1. DRIVERS: If you have not specified all critical performance and quality levels numerically – you cannot estimate project resources.
2. EXPERIENCE: If you do not have historic data about the resources needed for your technical solutions - you cannot estimate project resources.
3. ARCHITECTURE: If you do your project solutions all at once without learning their costs and interactions – you cannot expect to be able to estimate the results.
4. STAFF: If a complex and large professional project staff is an unknown set of people, or changes in mid project – you cannot expect to estimate the costs.
5. SENSITIVITY: If even the slightest change is made after an ‘accurate’ estimation, to any requirements, designs or constraints – then the estimate might need to be changed radically. And – you probably will not have the information necessary to do it, nor the insight that you need to do it.
The Control Principles
6. LEARN SMALL: Do projects in small increments of delivering requirements – so you can measure results and costs against estimates.
7. LEARN ROOT: If incremental costs for a given requirement deviate negatively from estimates – analyze the root cause, and change anything about next increments that you believe might get you back on track.
8. PRIORITIZE CRITICAL: You will have to prioritize your most critical requirements and constraints: there is no guarantee you can achieve them all. Deliver high-value for resources-used first.
9. RISK FAST: You should probably implement the highest-risk ideas early, so you have plenty of resource to deal with their uncertainties.
10. APPLY NOW: Learn early, learn often, learn well, and apply the learning to your current project.
Some Constructive suggestions to people who want estimates.
- Stipulate one or more useful deadlines from you point of view.
- Be specific about what has to be done by each deadline.
- Ask if these deadlines seem reasonable for the tasks prioritized.
- necessary make adjustments.
Stipulate the value to you of reaching each major (top 10) requirement. Make an outsourcing contract and pay part of that value only when that requirement is successfully delivered.
Do business with suppliers who consistently deliver value for money. Don’t waste your money on suppliers who make excuses. ‘No cure, no pay’ is one way to motivate suppliers to give you value for money – otherwise their motivation is to keep billing you forever.
 Peter J. Morris, The management of projects, Telford, London, ca 1995
 Gilb, Tom, Principles of Software Engineering management (chapter on Estimation), Addison-Wesley, 1988. In Print, 20th Printing 2006
 Gilb, Tom. Competitive Engineering: sub title, 2005, Elseveir
 Boehm, Barry. CoComo and C II
 Larman and Basili: History of IID methods, IEEE Computer June 2003
 Mills, Harlan IBM System Journal NO. 4 1980 (give website)
“In mathematics, approximation or estimation typically means finding upper or lower bounds of a quantity that cannot readily be computed precisely. While initial results may be unusably uncertain, recursive input from output, can purify results to be approximately accurate, certain, complete and noise-free.”
Personal website: http://www.gm.fh-koeln.de/~bundschu
“Estimation of IT-Projects”
Metrics News, Vol. 4, Nr. 2, December 1999, pp. 29 - 37
A Holistic Dynamic Classification Framework for Software Estimation
FESMA Conference, Heidelberg, May 2001
FUNCTION POINT APPROXIMATION WITH THE FIVE LOGICAL COMPONENTS
FESMA Conference in Madrid, Oct. 18 – 20, 2000
copy to Magne Jørgensen, who will do research in the area
2008 Incose submission website, and blog and newsletter