Data quality is the key issue addressed within the CSA program according to the NAS study Photo Sean KilcarrFleet Owner

NAS report highlights flaws in SMS data quality

June 28, 2017
Findings don’t surprise trucking industry groups, which long contested the “inconsistencies” within the FMCSA’s now seven-year old CSA program.

While the revamped safety measurement system (SMS) deployed by the Federal Motor Carrier Safety Administration (FMCSA) within its Compliance, Safety, Accountability or “CSA” program is “conceptually sound,” a long-awaited report from the National Academies of Sciences, Engineering, and Medicine (NAS) concludes that several features need improvement.

The report concluded that, over the next two years, FMCSA should develop a more “statistically principled approach” based on an item response theory (IRT) model to identify motor carriers that “need interventions” from a safety performance perspective.

The NAS also identified a number of “data quality issues” within the information gathered by the FMCSA to make safety fitness determinations.

Thus it recommended that FMCSA should continue to collaborate with states and other agencies to improve the collection of data on vehicle miles traveled (VMT) and on crashes; data which is often missing and of unsatisfactory quality. 

“Including vehicle miles traveled data by state and month will enable SMS to account for varied environments where carriers travel – for example, in icy winter weather in the North,” NAS said in its report. “In addition, there is information available in police narratives not represented in the data used that could be helpful in understanding the contributing factors in a crash.”

David Heller, vice president of government affairs for the Truckload Carriers Association (TCA), stressed to Fleet Owner that the “data inconsistencies” within the CSA program is what has long concerned the industry, not the CSA program itself.

“TCA supports, in theory the notion of CSA. However, [this] recent study … reaffirms what our industry has been saying all along; that there are inconsistencies with the SMS,” he noted. “Safety minded, forward thinking carriers should be able to stand behind their safety accomplishments and being judged on those achievements, in an equitable manner. [That] should be the basis of what CSA is all about.”

Chris Spear, president and CEO of the American Trucking Associations (ATA), echoed that same sentiment.

“This report has confirmed much of what we have said about the program for some time,” he said in a statement. “The program, while a valuable enforcement tool, has significant shortcomings that must be addressed and we look forward to working with FMCSA to strengthen the program.”

Specifically, ATA noted the NAS study validated the trucking industry’s concerns about the inclusion of certain types of violations in the CSA system, that geographic enforcement disparities can have a significant impact on carriers’ scores and that the collection and use of clean inspections is critical to the accuracy of the program.

“We strongly agree with the NAS conclusion that CSA should be based less on the ‘subject-matter expertise’ of enforcement and more on the empirically validated data,” added Sean Garney, ATA’s director of safety policy. “We also see great potential in the recommendation that FMCSA overhaul the current CSA methodology in favor of a new, more adaptive, data-centric model with the potential to address serious flaws in the system.”

He noted that, to maximize CSA’s potential, there is an “urgent need” to address issues regarding data sufficiency and accuracy – specifically when looking at crash and carrier exposure data.

“Until more study – and correction – of these issues is complete, ATA strongly believes FMCSA should continue to keep CSA scores out of the public domain,” Garney pointed out.

To a degree, the NAS report backs up ATA’s position on the public sharing of such safety data.

“The committee that conducted the study and wrote the [NAS] report was unable to recommend whether SMS percentile ranks should be made public because it would require a formal evaluation to understand the consequences of public knowledge of the information,” the group said in a statement.

NAS also highlighted several other items as part of its report on SMS data protocols:

  • About 100,000 fatality- or injury-causing crashes involving large trucks and buses occur in the U.S. each year.
  • Approximately 3.5 million CMV roadside inspections are conducted yearly by specially trained inspectors on approximately 900 potential violations of safety regulations.
  • Those violations fall into six categories: unsafe driving, hours of service compliance, vehicle maintenance, controlled substances/alcohol use, hazardous materials compliance, and driver fitness. 
  • In addition to that roadside inspection data, NAS urged FMCSA to research ways of collecting data on carrier characteristics – including driver turnover rates, type of cargo, and method and level of compensation. 
  • For example, compensation levels are relevant because it is known that drivers who are better-compensated, and those not compensated based on miles traveled, have fewer crashes. 
About the Author

Fleet Owner Staff

Our Editorial Team

Kevin Jones, Editorial Director, Commercial Vehicle Group

Cristina Commendatore, Executive Editor

Scott Achelpohl, Managing Editor 

Josh Fisher, Senior Editor

Catharine Conway, Digital Editor

Eric Van Egeren, Art Director

Voice your opinion!

To join the conversation, and become an exclusive member of FleetOwner, create an account today!

Sponsored Recommendations

Proactive Fuel Risk Management Guide

Download this informative guide to explore innovative techniques to prevent fuel fraud and misuse before it happens. Understand how to save 11% or more in fuel-related costs while...

Going Mobile: Guide To Starting A Heavy-Duty Repair Shop

Discover if starting a heavy-duty mobile repair business is right for you. Learn the ins and outs of licensing, building, and marketing your mobile repair shop.

Increase your fleet’s fuel economy with the right lubricants

See how Mobil Delvac™ oils boosted GP Transco's fleet.

Career Minded

The Peterbilt Technician Institute teaches you the skills needed for a lucrative and fulfilling career, transforming students into certified diesel technicians. Maximize your ...