Still waiting

The Federal Motor Carrier Safety Administration's (FMCSA) Motor Carrier Safety Status Measurement System better known as SafeStat has found its way into the news again. Unfortunately, it appears that some of the most recent developments may have an impact on when SafeStat's numerical results will again be displayed publicly. Last month, I reported on the Government Accountability Office analysis of

Jim York

Aug. 1, 2007

3 min read

The Federal Motor Carrier Safety Administration's (FMCSA) Motor Carrier Safety Status Measurement System — better known as SafeStat — has found its way into the news again. Unfortunately, it appears that some of the most recent developments may have an impact on when SafeStat's numerical results will again be displayed publicly.

Last month, I reported on the Government Accountability Office analysis of SafeStat's modeling techniques for identifying high-risk motor carriers. That analysis found that SafeStat's effectiveness in targeting carriers with high crash rates could be increased about 9 basis points by converting the system's calculation method from an “expert judgment” approach to a statistical regression model.

In July, Calvin Scobel III, the Dept. of Transportation's Inspector General (DOTIG), reported to the House Transportation and Infrastructure Committee on the results of an audit that examined improvements in the quality of SafeStat's underlying crash data.

The audit was conducted at the behest of House of Representatives member Thomas Petri (R-WI), who asked the DOTIG staff to judge whether the crash data quality had improved sufficiently to merit the restoration of publicly displayed SafeStat results.

The quality of crash data is a long-standing problem that potentially hinders SafeStat's ability to accurately identify carriers that pose high crash risks. Primary issues include the following:

Timeliness. The amount of time required for states to report the data to FMCSA's central crash database.
Completeness. The percentage of all fatal and non-fatal crash data that is actually reported.
Accuracy. The percentage of crashes correctly matched to the appropriate motor carrier

Both the DOTIG and GAO reports acknowledge that FMCSA and the states have made great progress in the timeliness of crash reporting. The GAO report, for example, noted that the percentage of crashes reported within 90 days has jumped from 32% in FY 2000 to 89% in FY 2006.

With respect to the completeness of crash data, fatal and non-fatal crashes must be evaluated separately. FMCSA's most recent analysis indicates that nearly all of the fatal crashes in 2005 were reported to the database. However, they represent just 4% of all crashes. Non-fatal crashes, on-the-other-hand, represent 96% of the total crash universe. FMCSA now acknowledges that reporting of this particular subset of data is still incomplete.

In fact, the DOTIG analysis cited a recent University of Michigan Transportation Research Institute's (UMTRI) Center for National Truck and Bus Statistics report which revealed that 15 states are reporting only 64% of their total non-fatal crashes.

This is, indeed, troubling. Consequently, DOTIG has recommended that FMCSA withhold its plans to restore the public display of SafeStat motor carrier safety rankings until such time as the “completeness gap” in non-fatal crash data is rectified.

While UMTRI is currently conducting state-by-state evaluations of non-fatal crash completeness, their analysis will not be completed until FY2008. Until such time, the agencies do not know what they do not know.

This is unfortunate, since stakeholders (e.g., motor carriers, shippers, traveling public and insurers) deserve a benchmarking system, which compares the relative safety performance among motor carriers.

Jim York is the ass't. vice president of technical services for Zurich Services Corp. Risk Engineering in Schaumburg, IL.