Study: SMS BASIC Scores not valid crash-frequency predictors

June 27, 2012

The SMS BASIC scores within the Federal Motor Carrier Safety Administration’s (FMCSA) Compliance Safety Accountability (CSA) rules are not valid predictors of crash frequency, according to a new study.  “Regression analysis shows that SMS percentile scores account for less than one percent of the variation in crash frequency for each of these BASICs,” stated the Alliance for Safe, Efficient, and Competitive Truck Transportation(ASECTT), which commissioned the study.

ASECTT stated that “an examination of the study demonstrates that FMCSA’s data cannot be used to predict the crash performance of individual carriers, even though the FMCSA claims SMS scores are correlated to the average crash frequency of hundreds of carriers at each percentile integral.

“Consumers of freight transportation do not select ‘average’ carriers,” the group added, “they select individual carriers and the Agency study offers no proof that SMS methodology is a predictor of individual carrier safety performance at any percentile level.”

The study was conducted by Inam Iyoob, PhD, director of Engineering for Transplace, whom ASECTT pointed out holds a doctorate in engineering and is regarded as a mathematical expert.

“I can’t see any useful purpose in averaging the crash data of hundreds of carriers in each of 100 different percentiles and then calculating a regression of the average values,” said Iyoob. “The purpose of regression analysis is to explain variation. Averaging hundreds of carriers at each percentile eliminates most of the variation in the data.

“It is not statistically accurate to say the SMS methodology and BASIC percentile scores are an accurate predictor of carrier safety predicated upon the crash data the Agency uses to justify its conclusions,” he added.”

According to Iyoob, while it is logical to assume that unsafe driving and driver fatigue do impact crashes, “the way the SMS BASICs Unsafe Driving and Fatigued Driving are captured, calculated and interpreted by FMCSA does not show any correlation to crashes. Hence usage of SMS data for carrier selection will unduly favor some and penalize others, and thus should be avoided.”

Iyoob stated that in advising shippers and brokers to use SMS methodology, FMCSA concluded: “Internal, external, and independent (University of Michigan’sTransportation Research Institute) evaluations have all shown that, of the six BASICs based on regulatory compliance (the Crash Indicator BASIC is based on actual crashes), the Unsafe Driving BASIC and the Fatigued Driving (HOS) BASIC have the strongest relationships to future crash risk.”

But he pointed out that a separate study, conducted by Wells Fargo, of the 200 largest carriers, “for which there is actually sufficient data,” found no perceptible correlation between safety and SMS percentiles in Unsafe Driving or in Fatigued Driving, “the two BASICs the Agency proclaims as most definitive.”

Iyoob reported that the Wells Fargo Study concluded that: “Quite simply, we found very little relationship (i.e., not statistically significant) between Unsafe Driver or Fatigued Driver scores and actual Accidents per Power Unit.”

Voice your opinion!

To join the conversation, and become an exclusive member of FleetOwner, create an account today!

Sponsored Recommendations

The Road Ahead: 2025 Trucking and Fleet Insights

Discover how fleet operators are impacted by challenges like driver onboarding delays and complex compliance, and the critical need for technology to boost efficiency and cut ...

Driving Growth: How to Manage More Freight

Ready to grow your trucking business? Whether you have 25 or 200 trucks, this guide offers practical tips and success stories to help you expand with confidence. Discover how ...

How to Maximize Fleet Management with Vehicle Bypass

Join us on February 18th to learn how truck weigh station bypass systems boost fleet performance and driver satisfaction.

Optimizing your fleet safety program using AI

Learn how AI supports fleet safety programs with tools for compliance monitoring, driver coaching and incident analysis to reduce risks and improve efficiency.