Data quality is the key issue addressed within the CSA program, according to the NAS study. (Photo: Sean Kilcarr/Fleet Owner)

NAS report highlights flaws in SMS data quality

June 28, 2017
Findings don’t surprise trucking industry groups, which long contested the “inconsistencies” within the FMCSA’s now seven-year old CSA program.

While the revamped safety measurement system (SMS) deployed by the Federal Motor Carrier Safety Administration (FMCSA) within its Compliance, Safety, Accountability or “CSA” program is “conceptually sound,” a long-awaited report from the National Academies of Sciences, Engineering, and Medicine (NAS) concludes that several features need improvement.

The report concluded that, over the next two years, FMCSA should develop a more “statistically principled approach” based on an item response theory (IRT) model to identify motor carriers that “need interventions” from a safety performance perspective.

The NAS also identified a number of “data quality issues” within the information gathered by the FMCSA to make safety fitness determinations.

Thus it recommended that FMCSA should continue to collaborate with states and other agencies to improve the collection of data on vehicle miles traveled (VMT) and on crashes; data which is often missing and of unsatisfactory quality. 

“Including vehicle miles traveled data by state and month will enable SMS to account for varied environments where carriers travel – for example, in icy winter weather in the North,” NAS said in its report. “In addition, there is information available in police narratives not represented in the data used that could be helpful in understanding the contributing factors in a crash.”

David Heller, vice president of government affairs for the Truckload Carriers Association (TCA), stressed to Fleet Owner that the “data inconsistencies” within the CSA program is what has long concerned the industry, not the CSA program itself.

“TCA supports, in theory the notion of CSA. However, [this] recent study … reaffirms what our industry has been saying all along; that there are inconsistencies with the SMS,” he noted. “Safety minded, forward thinking carriers should be able to stand behind their safety accomplishments and being judged on those achievements, in an equitable manner. [That] should be the basis of what CSA is all about.”

Chris Spear, president and CEO of the American Trucking Associations (ATA), echoed that same sentiment.

“This report has confirmed much of what we have said about the program for some time,” he said in a statement. “The program, while a valuable enforcement tool, has significant shortcomings that must be addressed and we look forward to working with FMCSA to strengthen the program.”

Specifically, ATA noted the NAS study validated the trucking industry’s concerns about the inclusion of certain types of violations in the CSA system, that geographic enforcement disparities can have a significant impact on carriers’ scores and that the collection and use of clean inspections is critical to the accuracy of the program.

“We strongly agree with the NAS conclusion that CSA should be based less on the ‘subject-matter expertise’ of enforcement and more on the empirically validated data,” added Sean Garney, ATA’s director of safety policy. “We also see great potential in the recommendation that FMCSA overhaul the current CSA methodology in favor of a new, more adaptive, data-centric model with the potential to address serious flaws in the system.”

He noted that, to maximize CSA’s potential, there is an “urgent need” to address issues regarding data sufficiency and accuracy – specifically when looking at crash and carrier exposure data.

“Until more study – and correction – of these issues is complete, ATA strongly believes FMCSA should continue to keep CSA scores out of the public domain,” Garney pointed out.

To a degree, the NAS report backs up ATA’s position on the public sharing of such safety data.

“The committee that conducted the study and wrote the [NAS] report was unable to recommend whether SMS percentile ranks should be made public because it would require a formal evaluation to understand the consequences of public knowledge of the information,” the group said in a statement.

NAS also highlighted several other items as part of its report on SMS data protocols:

  • About 100,000 fatality- or injury-causing crashes involving large trucks and buses occur in the U.S. each year.
  • Approximately 3.5 million CMV roadside inspections are conducted yearly by specially trained inspectors on approximately 900 potential violations of safety regulations.
  • Those violations fall into six categories: unsafe driving, hours of service compliance, vehicle maintenance, controlled substances/alcohol use, hazardous materials compliance, and driver fitness. 
  • In addition to that roadside inspection data, NAS urged FMCSA to research ways of collecting data on carrier characteristics – including driver turnover rates, type of cargo, and method and level of compensation. 
  • For example, compensation levels are relevant because it is known that drivers who are better-compensated, and those not compensated based on miles traveled, have fewer crashes. 

Sponsored Recommendations

Way Beyond Weight: 5 Ways Truck Weights Affect Fleet Operations

Truck weights affect everything from highway safety and operational costs to back-office efficiency. Here’s how.

Heavy-Duty Maintenance Checklist

A maintenance checklist can help ensure you hit everything necessary during an inspection. Check out our free downloadable checklist to help streamline your repairs.

Five Ways a Little Data Can Save Your Company Millions

While most trucking and logistics companies rely on cellular to keep their work fleet connected, satellite has the ability to connect anywhere and through small data transmission...

Fleet Case Study: 15% YOY Growth for ITDS

Learn how this small trucking company scaled significantly and maintained outstanding customer service without adding additional people. Sylectus TMS can automate operations and...

Voice your opinion!

To join the conversation, and become an exclusive member of FleetOwner, create an account today!