When I first started covering the trucking industry in early 2010, the U.S. DOT’s Federal Motor Carrier Safety Administration was working toward the roll out an all-new system intended to rank carriers on their safety performance and, ultimately, predict their crash risk and intervene if necessary.
The Comprehensive Safety Analysis 2010 (which in 2011 became the Compliance, Safety, Accountability program) would replace the SafeStat system with what FMCSA billed as a more comprehensive and more safety-focused scoring program. The scores would be public, meant to rope in the private sector to avoid carriers with poor scores and add accountability and transparency to the system.
Reliant on big sets of data and new algorithm-based software, CSA would be the great precog of the industry – predicting the future and heading it off.
However, the CSA program was plagued with issues that made it, at best, questionable in FMCSA’s use as a carrier rating and intervention system. At worst, CSA’s Safety Measurement System was an inaccurate and unfair tool for judging carriers’ safety and crash risk — and was used by shippers and brokers in carrier hiring decisions and by insurers to determine premiums. These concerns led Congress to pull CSA scores from public view and require FMCSA to address the program’s deficiencies before scores could be made public again.
Nearly a decade into the CSA experiment, there’s still more questions than answers about its future and whether FMCSA can pull off a system meant to accurately score a wildly diverse industry with hundreds of thousands of companies operating across 50 states. That’s despite FMCSA having taken CSA back to the drawing board and proposing a new path forward for the program last year.
Fresh concerns were highlighted last week in a report from the U.S. DOT’s Office of Inspector General (OIG), who pointed to hard truths about the agency’s plan to overhaul the CSA system in an attempt to make it more accurate and more fair. The agency’s plan, announced last July, was based on recommendations from the National Academies of Science (NAS), whose researchers and engineers drafted a blueprint they said would be ideal in addressing CSA’s deficiencies and its mission.
Easier said than done, said DOT’s OIG. Their conclusions, in short: Great plan; now, how are you going to pull it off?
In its response to those concerns, included in the OIG report, the agency mostly shrugged, acknowledging the major hurdles it faces in implementing the plan and mostly punting in addressing OIG’s concerns.
In a letter to the OIG, FMCSA Administrator Ray Martinez said the agency “has no immediate plans” to find new sources of data to feed CSA. He also said the agency won’t decide for up to another year whether it can pursue a revised scoring system, called item response theory, proposed by the National Academies last year and included in FMCSA’s corrective action plan.
Insufficient data was one of the key elements identified by both CSA detractors and the NAS as limiting CSA’s accuracy. If the agency doesn’t plan to find new data sources, as suggested by NAS, it seemingly seriously jeopardizes efforts to repair CSA and redeploy the system.
To its credit, the agency did point to recent efforts to try to adhere to NAS’s recommendations and its own corrective action plan. FMCSA began small-scale testing the item response theory scoring model last year and completed that testing just last week. It’s also held public meetings over the past year to seek input from fleets, drivers and other stakeholders about how it should proceed in its reforms.
But those efforts seem meager compared to the task at hand of earnestly repairing a broken system. If the OIG report and FMCSA’s responses are any indication, it may be years before CSA is fixed and redeployed — if FMCSA can even get there.
Background on CSA’s troubles
The agency didn’t take up a CSA overhaul under its own volition. It was required to by Congress, who, after less than a half-decade after CSA was deployed, intervened and forced FMCSA to pull carriers’ scores from public view. It also required the agency to work with NAS to develop a plan to fix the program, should FMCSA wish to make its scores public again.
As noted above, since its deployment in 2011, carriers and trade groups pointed to major concerns around accuracy and fairness within CSA’s Safety Measurement System and its BASICs.
Particularly for small carriers, and especially for the huge swath of carriers with fewer than six trucks, CSA has been a stab in the dark. Such carriers have trouble registering enough inspection data to reach the threshold to be ranked in the system. And if they do, they had so little inspection data still that the math was tilted against them.
And that’s not to mention inconsistencies in geographical enforcement, rightful concerns over lack of crash fault and accountability, the peer-to-peer ranking system or the underlying methodology used to tabulate the rankings.
