Data quality and disparity were the chief topics of conversation at a public listening session held Aug. 29 by the Federal Motor Carrier Safety Administration as part of its efforts into overhauling its Compliance, Safety, Accountability program, which Congress directed the agency to do in late 2015.
The agency is leaning on a report issued in June 2017 by the National Academies of Science, which urged the agency to build a new statistical model to underly CSA’s Safety Measurement System, one based on so-called “item response theory,” to better prioritize at-risk carriers for investigation or other intervention.
David Yessen of FMCSA’s compliance office outlined steps the agency has taken recently, including building small-scale versions of a new statistical model to test and considering new ways to utilize the data it has — its MCMIS database of inspections and violations, crash data, etc. — as well as new data inputs.
As it identifies a potential new source of data to use over the coming year as it builds the system, the agency will be dropping the data into the IRT model to see if it has the appropriate effects. Yessen likened the model to “a funnel, and you can keep dropping information into that funnel, and we’re looking for opportunities to do that.”
The agency was accepting feedback on what new data sets it could use in the CSA system, including metrics on driver pay methods and carriers’ turnover rates. However, representatives from fleets and the American Trucking Associations cautioned the agency against using such data in determining carriers’ CSA scores.
“If a carrier has adopted a proactive culture of safety, which I’ve been told that’s what they want us to do, if I have methods in place to identify bad eggs … and I get them out of my fleet before they have an accident, I still have to replace the driver. That’s a kind of positive turnover,” said Irwin Shires of Panther Expedited. Shires is also a member of the Motor Carriers for Regulatory Reform group. The agency shouldn’t automatically assume a relationship between driver turnover and carrier safety, he said. “There are a number of carriers out there who churn through drivers,” he said, but the agency should “take into account the exceptions to the rule before you implement them into an IRT model.”
Though there were some commendations for the agency taking into account method of compensation and turnover as potential data points for safety assessment, getting carriers to disclose such information to the system would require rulemaking for the FMCSA to actually require that disclosure, Director of Enforcement Joe DeLorenzo acknowledged.
Dan Horvath, safety policy VP for the American Trucking Associations, provided a counterpoint when he noted that, in his association’s view, “from a compensation perspective, we don’t think it’s reliable to judge a carrier on that.”
Former ATA rep Dave Osiecki, now with the Scopelitus firm, underscored the variation that is fairly common today from a per-mile, hourly and/or percentage or per-trip basis all within the same company, to say nothing of the reality of an independent owner-operator’s varying contracts with brokers, shippers and other freight parties.In addition, on turnover rates and other carrier-specific data, Horvath said, “there’s definite hesitation from carriers to provide those perspectives.”
FMCSA’s Barbara Baker, also of the compliance office, acknowledged as much: “We get into a catch 22,” she said. “If we have an IRT model based on new sources liked driver turnover, age,” and etc., “that ends up being company confidential and/or personal information” that needs to be protected from dissemination to the public. The raw information in the MCMIS database and its use by public and private research and other entities was another topic of discussion that received less attention at this morning’s session.
