Between 2012 and 2017, fatalities involving large trucks and buses grew by 10.2 percent. While it isn’t fair to lay blame at the feet of CSA, clearly the program hasn’t had the intended effect of improving crash statistics. If you count the pilot testing phase, Compliance Safety Accountability is more than 10 years old, and it has been a full eight years since CSA (then called Comprehensive Safety Analysis 2010) replaced its predecessor, SafeStat.
By now, the wrinkles should have been ironed out of the Federal Motor Carrier Safety Administration’s overhaul of commercial vehicle safety compliance. After a number of revisions and changes to its methodology, CSA remains a flawed system plagued by enforcement disparities, lack of inspection data on certain carrier groups and crash correlation discrepancies.
Earlier this year, FMCSA withdrew 2015-proposed CSA revisions, clearly signaling that another major overhaul of safety compliance is in the process as the agency continues its quest for a fair carrier safety scoring system and accurate crash risk methodology.
Last month, CSA officially went back to the drawing board when the U.S. Department of Transportation’s Office of Inspector General began an internal audit of FMCSA’s corrective action plan based off findings of a congressionally mandated study by the National Academies of Science in 2017. According to an OIG memo to FMCSA, the audit will “assess the extent to which FMCSA’s corrective action plan addresses the NAS recommendations and relevant OIG and GAO recommendations and identify challenges FMCSA may face when implementing the corrective action plan.”
Based largely on recommendations from NAS, FMCSA is forging ahead to create a new CSA Fast Act Score Model that repurposes the Item Response Theory methodology – currently used to evaluate programs and standardize testing in the healthcare industry – and applies it to carrier safety performance. FMCSA says the IRT model “can evaluate motor carriers with varying safety data and characteristics by using violations to assess the safety performance of each carrier.”
Furthermore, the agency says the IRT model can determine the “prevalence of each violation and the violation’s effectiveness in distinguishing between carriers with and without a strong safety culture.”
FMCSA also plans to use the IRT model to improve data collection and evaluate the potential of adding new data sets, including “carrier characteristics” data such as driver turnover rates, compensation levels and types of cargo hauled.
SambaSafety, owner of Vigillo, has been working behind the scenes to develop a new CSA scorecard, based on the IRT methodology for the CSA Fast Act Score Model, to give its customers a sneak peek at how they might fare in the new system.
“Our analysis shows that the IRT model is an effective means of identifying a poor safety culture at motor carriers and will represent a significant improvement in the overall effectiveness of the CSA program,” said Steve Bryan, executive vice president and general manager for SambaSafety Transportation. “However, IRT… almost completely changes the building blocks of CSA. There are no longer any violation weights, CSA points, BASIC measures or safety event groups. The new, single CSA BASIC Score is a very different way of representing the safety culture of a motor carrier.”
Carriers and industry stakeholders can take hope in SambaSafety’s early findings, but the years-long process of overhauling CSA is just getting started. Let’s hope this time around, FMCSA is able to make a change that answers more questions than it raises.
