Inaccurate cost estimating has dogged transportation projects for years. B. Flyvbjerg’s noted study on the results of transportation project estimating found that for the past 70 years, the cost of transportation projects has been consistently underestimated in many parts of the world, including the U.S.
J.J. Reilly documented in 2001 that a significant number of complex infrastructure projects substantially exceeded their budgets. He presented his findings at the March 2001 American Underground Construction Conference. Fryvbjerg published his results in “Underestimating Costs in Public Works, Error or Lie?” in the summer 2002 American Planning Association Journal. Flyvbjerg reported on the cost performance of 258 international transportation infrastructure projects, with the following results. Cost comparisons are from the published cost at decision to proceed—normally related to the Final Environmental Impact Statement or Record of Decision in the U.S.—to the final delivered cost.
- Construction costs are underestimated in almost nine out of 10 transportation infrastructure projects;
- Actual costs are on average 28% higher than estimated costs;
- Road project costs averaged 20% higher than estimated;
- Tunnel and bridge costs averaged 34% higher than estimated; and
- Rail project costs averaged 45% higher than estimated.
Even though there may be good reasons for the increase in cost relative to that estimated, the public perception is that estimates of project costs and schedules are unreliable and not to be trusted. The public wants to know how much a project costs, how long it will take and why. The public also wants to know why estimates change over time. Across the country, the difficulty in answering the public’s questions is a consequence of many factors, including poor cost estimating practices, poor project management and poor communication within the design and construction community and with the various public bodies.
The Washington State Department of Transportation (WSDOT), like many agencies grappling with large, complex projects, began a critical self-examination after one particularly sensitive project, the S.R. 167 Freeway Extension, experienced significant increases in cost estimates during the early stages of project development. Politicians asked Secretary Doug MacDonald to explain. In answer, WSDOT has developed a new methodology to better estimate the cost of large projects.
Uncertain future
Those close to the transportation industry may recognize the following scenario: During planning, early in project definition, an estimate of a project cost and schedule is made known. Early figures based on a rudimentary scope of work with few project details are meant to be a rough approximation of the project. But somehow this early estimate can become a public “pronouncement” of a project’s cost and schedule. As the project develops and moves through scoping and early design phases, much more knowledge about the project becomes available. With the rising knowledge about a project’s scope comes an understanding that contending with some elements of the project will require significant additional resources. Such elements could be related to scope, environmental mitigation and permitting, rising cost of right-of-way as corridors develop in advance of the project, utilities, seismic and other considerations.
Why were early scoping and planning estimates deficient? Traditional estimating practices tend to produce “the number” for a project. But the single number masks the critical uncertainty inherent in a particular project. It implies a sense of precision beyond what can be achieved during planning, scoping or early design phases.
One answer WSDOT found to these fundamental questions is the realization that an estimate is more accurately expressed not as a single number but as a range. To put this into practice, WSDOT developed the Cost Estimate Validation Process (CEVP) for projects over $100 million, and subsequently the less intense Cost Risk Assessment (CRA) workshops for projects valued between $25 million and $100 million.
Estimating contingencies
To determine an accurate estimate range for both cost and schedule, risk must be measured. Formerly, WSDOT measured risk based on the estimator’s experience and best judgment, without explicitly identifying the project’s uncertainties and risks. That has changed. Estimates now comprise two components: the base cost component and the risk (or uncertainty) component. Base cost is defined as the most probable cost of the planned project that can be expected if no significant problems occur. The base cost does not include contingency. Once the base cost is established, a list of uncertainties is created of both opportunities and risks, called a “risk register.” The risk assessment replaces general and vaguely defined contingency with explicitly defined risk events and with the probability of occurrence and the consequences of each potential risk event. Scope control is necessary but is not addressed in this short article.
Once a project manager has determined that a CEVP or CRA workshop is required for the project, the process goes as follows: At the prep session the project team and cost risk team review the project for the upcoming workshop, determine participants and draft a high-level project schedule that takes the project to completion. This is translated into a basic flowchart of the significant project activities.
At the workshop, which typically runs from two to five days depending on the complexity and size of the project, the cost estimate is reviewed and validated, and a base cost for the project is determined. Potential risk events are identified and, through elicitation of the project team and subject-matter experts, the probabilities and consequences of risk events are quantified. After the workshop, the flowchart activities, the base cost and risk events (as determined in the workshop) are then entered into a Monte-Carlo simulation model. The model produces an estimated range, with probabilities, for project cost and schedule; results are conveyed in terms of confidence levels for costs and time to complete the project. The simulation also ranks the risk events in order of significance so project managers can focus on the top-ranked risks for risk-mitigation planning to gain the best cost benefit from the risk management actions. The workshop results are compiled into a report, which includes a spreadsheet that can be used for risk-management planning and a one-page summary of the modeling results with the most essential information and project data.
For example, a road interchange project in Kitsap County, Wash., went through a CRA workshop with results summarized as follows:
- Project estimate prior to workshop: $16 million;
- CRA workshop estimate range: $20.2 million to $25.3 million; and
- Total project amount at award: $22.2 million.
The actual awarded amount fell within the estimates of the workshop. Additionally, the workshop provided the project team with a risk register of events with estimated probabilities and consequences. Using this tool for pre-emptive management, the project team could address ways to manage the risks through avoidance, mitigation or transference. Risks that were accepted were known and monitored. This identification allows the project team to be anticipatory in responding to risk.
Dodging costs
Two of the risks identified in the workshop for the interchange project in Kitsap County are described as follows. One risk involved uncertainty in construction of the bridge widening due to aesthetic and technical concerns surrounding the need to widen a bridge and provide a retaining wall. Because of the detailed discussion that ensued in the workshop, one of the subject-matter experts suggested a retaining wall that could also serve as the bridge support or abutment. The design team ultimately adopted this suggestion and developed a design for the combined retaining wall and bridge abutment, thereby avoiding an estimated $260,000 in costs.
The second risk involved uncertainty in storm-water collection and treatment. As a result of identifying this risk and estimating its effect and probability, the design team responded by avoiding a combined-use detention pond and instead designed a project-specific pond, thereby avoiding an estimated $220,000 in costs. Experience indicates that costs of changes during construction are managed to around 6-7% of bid price.
The benefits of risk-based estimating utilizing the CEVP workshops are manifold and include:
- Increased and improved communication within the project team, in and among cross-functional groups, stakeholders, management, the legislature and the public at large;
- Known risks are identified and quantified; new risks are revealed;
- The risks are quantified in terms of probability and effect;
- The risk-based modeling analysis prioritizes the risks and provides guidance for project managers needing to know where to focus their risk-mitigation resources; and
- Provides the information necessary for developing effective risk-management plans.
As with any new process, lessons are learned through experience and practice. Observations are offered here for those considering implementation of CEVP or similar risk-based estimating processes.
Risk-based estimating is only possible if the owner truly wants to know the realistic range of probable costs and is prepared to communicate these contingencies to the public and decision makers.
High-profile projects benefit from the open and robust review of estimated cost and schedule.
Contingency estimating also can increase public confidence at a time when chronic underestimation of project costs on large projects has led to a significant erosion of public confidence in infrastructure agencies, in Washington state and elsewhere in the U.S. Fortunately, WSDOT has been able to show that it is addressing this problem, leading to a recent public vote in 2005 that approved the use of increased gas taxes for critical transportation projects.
The added openness and truthfulness about what we know and do not know with regard to project estimates and schedule is a healthy thing. The CEVP/CRA process has led to increased accountability with regard to public declarations of cost estimates and better management of resources.
