As director of snow and ice operations for the Massachusetts Department of Transportation, Paul Brown often recommended investments and process improvements to upper management.
“Lots of times, they’d come back with, ‘What’s the cost-benefit?’” Brown said. “There was nothing out there to help practitioners demonstrate the return on their investments to upper managers.”
To begin addressing that need, Brown served as project champion for a research project by the Clear Roads national winter-maintenance research program (www.clearroads.org). The project developed a Cost-Benefit Analysis Toolkit for 11 winter-maintenance practices, equipment options and operations strategies in 2010. That toolkit proved valuable to practitioners—so much so that Clear Roads commissioned a second phase of the project. The updated toolkit, which added 10 more winter-maintenance tools for analysis and made several technical refinements, was recently completed.
“There was a recognition by winter-maintenance professionals that there’s a need to implement the latest tools to provide a good level of service,” said David Veneziano, research scientist at Montana State University’s Western Transportation Institute (WTI) and part of the research team for this project. “But with budgets being what they are, they needed to justify expenditures, and a cost-benefit analysis is the way to do that.”
With the web-based Cost-Benefit Toolkit, users select the technology they want to run an analysis for and then input a series of data specific to their agency’s situation. The toolkit then calculates benefits and costs for the agency, motorists and society as a whole, as well as an overall benefit-cost ratio. That ratio provides a clear indicator of return on investment: If it is more than 1.0, the investment will have greater benefits than costs, and the bigger the ratio, the greater the difference.
Given the intense budget pressures that winter-maintenance programs are facing, the toolkit can be a big asset in determining whether a given investment is worthwhile—and in demonstrating value to bottom-line-conscious decision makers.
“It can have major impacts on an agency’s priorities and programs,” Brown said.
“Winter-investment choices were traditionally based on anecdotal information,” said Xianming Shi, senior research scientist at WTI and the toolkit project’s principal investigator. “The toolkit provides a framework for evaluating options in a more scientific manner.”
Without the toolkit, a cost-benefit analysis is possible, but “not everybody has the time to sit down and do one,” Veneziano said. Practitioners would need to identify the costs and benefits related to the equipment or practice they are evaluating, work out how to calculate those costs and benefits, determine what data they need to make those calculations, gather that data in one place and make the calculations. The toolkit simplifies these steps.
For each of the 21 items included in the toolkit, the research team scoured published literature to identify the costs and benefits connected to that item, as well as methodologies for quantifying those costs and benefits. They then developed the toolkit around those findings. Users input data as directed on a series of web-based forms, and the toolkit makes the necessary calculations.
Running an analysis is a fairly quick process. Wyoming DOT Maintenance Staff Engineer Cliff Spoonemore reported that it took about 15 minutes to fill in the web forms for an analysis for tow plows. The data required is extensive, however. For example, the tow plow module is a five-page form that requires annual operating costs per plow, the number and cost of injury and property damage crashes per storm season, plow purchase and modification costs, annual maintenance costs and quite a bit more. Compiling that data for Wyoming’s analysis took about a week, although most of that time was spent gathering existing data from sources around the agency rather than conducting original research.
That level of detail is needed for a thorough analysis, though. The quality of any analysis is dependent on the accuracy and precision of the information it is based on, and winter maintenance is complex enough that many factors need to be taken into account.
“You can’t just guess at values,” Spoonemore said. “Without real numbers, you can’t be sure if the output is legitimate.”
The toolkit doesn’t leave users on their own to guess at how to obtain data.
“You can click the Help button to figure out how to derive everything you need,” said Annette Dunn, winter-operations administrator for the Iowa DOT. With a new user-management system introduced in Phase II of the toolkit (see sidebar for details), “if you don’t have an answer, the system doesn’t lock up,” she said. “You can come back and enter it later.”
Brown noted that in most cases, an agency will be collecting the information that the toolkit requires in some form, so running an analysis requires bringing those numbers into one place rather than gathering them from scratch.
“It forces an agency to become responsible and really look at its operations,” Brown said.
With that said, the toolkit can be used for preliminary evaluations as well. For many pieces of data, the toolkit offers suggested values based on published literature, values provided by agencies or equipment manufacturers, or FHWA figures. These suggested values offer a good starting point when a practitioner does not yet have actual figures to feed into the system.
They do not, however, provide an end point.
“Costs and benefits often vary with local needs and conditions,” Shi noted.
Reasonable estimates can differ from actual values enough to significantly affect the benefit-cost ratio.
In Iowa, Dunn’s toolkit analysis initially estimated that installing GPS technology on plows would yield a 6.4 benefit-cost ratio. That preliminary analysis actually underestimated the ratio, because Dunn had overestimated costs. Rerunning the analysis with reduced system-communication costs and better supplier information based on quotes from actual bids yielded a ratio closer to 10.
Armed with that information, Dunn was able to demonstrate the value of the GPS installation to both the state’s information technology governance board and to her management. In 2011, IDOT’s director fast-tracked implementation.
“Based on our projections, we knew that we could save several million dollars, so the director said to implement it immediately,” Dunn said. “That’s significant in this tight economy.”
Importantly, the projected benefits have materialized. Material savings and overall benefits derived from giving supervisors the ability to monitor trucks and application rates from their desks and make changes as conditions warrant are in line with what the toolkit projected.
“We couldn’t have done this in any other way,” Dunn said, noting that there is no other tool to automate the cost-benefit process and it would not have been practical to dedicate the efforts of a team to conduct the analysis.
Wyoming’s efforts are still in the early stages. The state is currently in the budgeting process, and Spoonemore hopes that the 5.1 benefit-cost ratio for two tow plows will lead to approval of the investment. The executive staff and chief engineer have seen the analysis and were impressed by it.
“They thought it was good, solid information,” he said. “Most of the numbers came from Wyoming DOT’s accounting and record systems or from actual vendor costs. Our values are not based on theories.”
In Massachusetts, Brown has used the toolkit for several analyses.
“Our state is privatized, so we’re forced to look at how much we can do ourselves,” he said. “Tow plows can eliminate a contractor or let one employee do more.”
MassDOT used the toolkit to evaluate the return on investment for tow-plow purchases, and also began prewetting chemicals based on a 3-to-1 benefit-to-cost ratio.
States with large fleets are naturally especially interested in the cost-benefit ratios of their investments, as even a small improvement percentage-wise can translate into a large number of dollars. But Veneziano noted that smaller agencies also can use the toolkit to evaluate and justify their winter-maintenance expenditures.
“This toolkit can help any practitioner in the winter-maintenance field make their lives a little bit easier,” he said.
The toolkit was designed to be relevant for nonexperts as well.
“It’s intuitive enough that even a nonpractitioner can plug the numbers in and get a flavor of how a given tool can save money,” Brown said.
One for all
Since the updated toolkit was just released, there are no specific plans yet for the next edition. But there are several winter-maintenance tools that Clear Roads members have expressed interest in adding to the toolkit, such as fixed automated spray technology, agricultural-based chemicals vs. nonagricultural-based chemicals, and living and temporary snow fences.
“The goal is to continue developing the toolkit until we have a cost-benefit analysis for all of the tools in snow management,” Brown said.
The toolkit also has the potential to help fill current gaps in winter-maintenance information. One of the difficulties in analyzing the costs and benefits of winter-maintenance tools is the fact that many of these costs and benefits are not directly financial in nature. As a result, their values have to be derived by estimates of the price of time, safety, environmental impact or other factors.
Many of these tools have been thoroughly studied, so there are established methods for calculating costs and benefits. But for relatively new tools that do not yet have an extensive body of research, estimating the impact can be difficult. Toolkit users may help solve that problem: As agencies run cost-benefit analyses, they input data based on their actual experiences, and it may be possible to extract and utilize that information.
“These will be the real numbers that we need to improve understanding,” Shi said. WM