Garbage in, garbage out

Avery Vise is editorial director of Commercial Carrier Journal. E-mail [email protected].

Imagine a computer program that could analyze every cost and revenue item and recommend changes that would dramatically improve your operating ratio. Suppose that when you tried to implement this plan, it hardly changed your financial position at all. You probably would assume that the system is junk. But it could be that the data was bad.

That could be the situation with SafeStat – the Federal Motor Carrier Safety Administration’s system for assessing the safety risk of motor carriers. FMCSA uses SafeStat to target carriers for compliance reviews. That alone makes accuracy critical. But SafeStat is no longer just an enforcement tool.

Through the magic of the Internet, anyone – shippers, insurance companies, competitors, trial attorneys and so on – can read lots of lurid things about your company at the A&I website, which hosts detailed SafeStat data.

What can someone learn about your company at A&I Online? The website provides, for example, the dates, locations, accident report numbers, vehicle identification number, whether there were fatalities and other data – but not driver names – on every accident recorded during that 30-month period. It also includes driver and vehicle out-of-service violations and moving violations and the reason for them. And anyone can see your SafeStat score – the number that determines your likelihood for an audit.

Is it appropriate for this much information to be available so freely? It probably doesn’t matter because once a federal agency makes such information available, it isn’t likely to reverse the decision. But as a matter of fairness, carriers should have the opportunity to correct inaccurate or misleading data. The American Trucking Associations and others have advocated such a process as part of a government-wide effort to ensure data quality.

It’s hard to assess the degree to which SafeStat suffers from poor data quality. Last month, Rep. Thomas Petri (R-Wis.), who chairs the House subcommittee that oversees motor carriers, asked the Department of Transportation’s inspector general to investigate issues surrounding data quality and the validity and objectivity of the SafeStat methodology.

One flaw in SafeStat is obvious. The system reflects all accidents that are recordable as defined by the federal safety regulations. It doesn’t matter whether your truck ran a red light and slammed into a car or a speeding car rear-ended your truck while it was stopped at a red light.

FMCSA, on the other hand, does consider preventability when it conducts compliance reviews or reviews safety ratings on appeal. In fact, accident preventability is the most common reason FMCSA modifies a carrier’s safety rating following a compliance review.

According to a CCJ analysis of 111 decisions on petitions for safety rating review since early 2001, accident preventability was an issue in 97 cases. Of those, FMCSA found that in 62 cases at least one of the accidents challenged was not preventable. In 46 of those 62 cases, FMCSA upgraded the carrier’s safety rating, usually from conditional to satisfactory.

None of this matters in SafeStat, however. In fact, even when FMCSA declares an accident not preventable, it removes that accident only for the purposes of the safety rating. The accident remains in SafeStat.

ATA has asked FMCSA to make preventability the standard in SafeStat as well as compliance reviews. Also, ATA wants FMCSA to establish a process that allows carriers to routinely challenge crashes that it believes are preventable.

ATA’s request is more than reasonable, and making the data more precise upfront would benefit carriers and FMCSA. Plus, misleading SafeStat information damages carriers’ reputations unjustly. If FMCSA wants to air carriers’ dirty laundry, it should at least give them a fair chance to wash it first.