The SMS BASIC scores within the Federal Motor Carrier Safety Administration’s (FMCSA) Compliance Safety Accountability (CSA) rules are not valid predictors of crash frequency, according to a new study. “Regression analysis shows that SMS percentile scores account for less than one percent of the variation in crash frequency for each of these BASICs,” stated the Alliance for Safe, Efficient, and Competitive Truck Transportation(ASECTT), which commissioned the study.
ASECTT stated that “an examination of the study demonstrates that FMCSA’s data cannot be used to predict the crash performance of individual carriers, even though the FMCSA claims SMS scores are correlated to the average crash frequency of hundreds of carriers at each percentile integral.
“Consumers of freight transportation do not select ‘average’ carriers,” the group added, “they select individual carriers and the Agency study offers no proof that SMS methodology is a predictor of individual carrier safety performance at any percentile level.”
The study was conducted by Inam Iyoob, PhD, director of Engineering for Transplace, whom ASECTT pointed out holds a doctorate in engineering and is regarded as a mathematical expert.
“I can’t see any useful purpose in averaging the crash data of hundreds of carriers in each of 100 different percentiles and then calculating a regression of the average values,” said Iyoob. “The purpose of regression analysis is to explain variation. Averaging hundreds of carriers at each percentile eliminates most of the variation in the data.
“It is not statistically accurate to say the SMS methodology and BASIC percentile scores are an accurate predictor of carrier safety predicated upon the crash data the Agency uses to justify its conclusions,” he added.”
According to Iyoob, while it is logical to assume that unsafe driving and driver fatigue do impact crashes, “the way the SMS BASICs Unsafe Driving and Fatigued Driving are captured, calculated and interpreted by FMCSA does not show any correlation to crashes. Hence usage of SMS data for carrier selection will unduly favor some and penalize others, and thus should be avoided.”
Iyoob stated that in advising shippers and brokers to use SMS methodology, FMCSA concluded: “Internal, external, and independent (University of Michigan’sTransportation Research Institute) evaluations have all shown that, of the six BASICs based on regulatory compliance (the Crash Indicator BASIC is based on actual crashes), the Unsafe Driving BASIC and the Fatigued Driving (HOS) BASIC have the strongest relationships to future crash risk.”
But he pointed out that a separate study, conducted by Wells Fargo, of the 200 largest carriers, “for which there is actually sufficient data,” found no perceptible correlation between safety and SMS percentiles in Unsafe Driving or in Fatigued Driving, “the two BASICs the Agency proclaims as most definitive.”
Iyoob reported that the Wells Fargo Study concluded that: “Quite simply, we found very little relationship (i.e., not statistically significant) between Unsafe Driver or Fatigued Driver scores and actual Accidents per Power Unit.”
