Earlier this year, news surfaced that employees at two of the world’s largest companies were listening to audio recordings of private citizens with Amazon Alexa and Google Home devices.
The fact that any company would have employees review data, audio and video captured by Internet-of-Things (IoT) devices is not surprising. Human review is an integral part of the process for software to “learn” automation.
In this instance, Amazon and Google employees were transcribing words and phrases that software on the IoT devices did not automatically recognize. Some employees were caught sharing funny recordings with co-workers and admitted to overhearing some troubling content, however.
Aside from these controversies, many software developers regularly use human review for machine learning, a form of artificial intelligence (AI), as a feedback and validation loop for the “machines” to grow smarter at a rapid pace.
In the transportation industry, advancements in autonomous vehicles, driver safety systems, digital freight matching and other applications are all powered by machine learning technology. Perhaps more importantly, transportation companies are using the technology to better manage risk by changing future outcomes rather than reacting to present events.
Managing more risk
As the amount of data continues to expand, fleet owners and executives are concerned about having more data than they can properly manage. This concern emanates from the real dangers of having information about drivers and other areas of risk but failing to act.
During an accident investigation, a plaintiff attorney might discover instances that prove management knew about a problem but did nothing to resolve it. For example, data that shows a driver had an elevated risk of an accident might be ignored.
Another concern is that technology may have unintended consequences. Putting new devices in a vehicle to warn drivers about unsafe following distances, lane departures and objects in their blind spots, for example, could erode their trust in technology or cause distractions.
Driver safety systems are designed to be accurate most of the time, but “when a system gives false positives, those who are trying to be affected by it lose confidence, which means the system has zero value,” says Steve Mitgang, chief executive of SmartDrive Systems, which offers a video-based driver safety and telematics platform.
Human review and machine learning are making it possible for driver safety systems to continuously improve the accuracy of risk detection and alert drivers and fleet managers at the right moment.
“There is so much in the real world that the machine doesn’t know yet,” Mitgang says. “You need human intervention until you can do it correctly.”
Stand-alone driver safety products that do not have the benefit of human review may lack context for understanding risk, he says. A device that detects drowsy driving, for example, needs to alert drivers at the right moment to trigger a change in the risky behavior. The driver has to respect the accuracy of the alert and the intervention has to happen far enough in advance to change the outcome, he explains.
“You don’t want the driver to ignore it,” he says.
SmartDrive recently added a driver feedback display option for its in-vehicle SR4 safety platform that can provide drivers with various audible text-to-speech alerting and visual icons. The alerting can come as a message from a fleet manager based on review of driver behaviors, such as speeding, and can also be used for instant alerts and visual cues for a variety of behaviors that its machine vision camera and sensors detect, he says.
Instant risk detection
Some camera-based driver safety systems utilize a video review service. The companies have behavior analysts that review video footage captured by their in-vehicle IoT devices to note risk factors that the software did not recognize, such as driver distractions and weather.
Increasingly, technology companies are bypassing this video review service in an effort to lower costs for their customers and provide more immediate information to drivers and fleet managers.
A driver safety platform from Nauto, for example, uses in-vehicle artificial intelligence and multi-sensor fusion to perform real-time risk assessment, the company says.
The Nauto platform continuously analyzes sensor data from driver behaviors, exterior hazards, and vehicle movement using an in-vehicle device. The device runs its proprietary risk detection algorithms to identify distracted driving, drowsiness, tailgating, hard acceleration, braking and cornering.
Nauto also detects fleet policy violations for cell phone and cigarette use. The on-device algorithms are augmented by additional cloud algorithms for further analysis and refinement. The company says it continuously improves its algorithms with real-world data, machine learning techniques and human review.
Rather than offer a review service, Verizon Connect uses real-time feedback from its customers to power the machine learning for its Integrated Video system. The system is an option for fleets using its Reveal management platform.
“We really want the inputs from our customer’s eyes,” says Kevin Aries, head of global product success for Verizon Connect. When reviewing event videos, its customers rate the accuracy of the AI-generated classification of harsh driving events by using a scale of 1-5. Verizon Connect uses this feedback to continuously improve the accuracy of its algorithms in the IoT device and its cloud-based software to detect crash risk.
Driving safety improvements
With the latest machine learning technology, especially in the area of driver safety, motor carriers and logistics providers are taking a more prescriptive and measured approach to solving problems and focusing human resources on tasks that will create better outcomes.
Such has been the experience of National Grid, a provider of natural gas and electricity in the northeast region of the U.S., since it first deployed a video-based driver safety and telematics system from Lytx in 2010.
National Grid deployed the Lytx DriveCam program across its gas operations fleets, made up of almost 4,000 vehicles, to reduce unsafe driving incidents and provide drivers with greater awareness of their own risky driving behaviors.
From 2010 to 2018, its utility fleets saw a 94% reduction in collisions, an 84% reduction in avoidable near-collisions, an 83% reduction in severity of events, and an 84% reduction in frequency of events.
“Over the past decade, the Lytx Driver Safety Program has helped us make safe driving a top priority by coaching and reducing unsafe driving habits,” says Samantha Murray, vehicle safety manager at National Grid. “Additionally, and just as important, we are shifting toward a culture where we emphasize the safe driving habits from our good drivers through positive recognition programs.”
During the last year, the company has accelerated its risk reduction. From April-June 2018 to April-June 2019, it saw unsafe following distance behaviors decrease by 51%, collisions by 24%, and handheld cell phone use decrease by 13%, among other results.
This year, National Grid began using Lytx’s open application programming interface to create its own internal vehicle safety performance dashboard that combines data from the Lytx Driver Safety Program and Fleet Tracking Service with its own internal and third-party data.
With this central data warehouse and user interface, National Grid users can access real-time and historical fleet data tailored to what matters most to the team.
“The new dashboard reconfigures the information we’re getting from Lytx to highlight the improvements of our driving behaviors even further, while continuing to note areas for growth,” said Tyler Virkler, National Grid program manager, who spearheaded the design of the data project.
One of the benefits of technology powered by machine learning is that the accuracy of driver safety and risk management systems are improving continuously, and motor carriers are able to take actions on areas that might otherwise go unnoticed.