One of the most iconic scenes in Stanley Kubrick’s movie 2001: A Space Odyssey, is where David Bowman is “negotiating” with the HAL 9000 computer to open the door of the spaceship. Hal says, “I’m sorry, Dave. I’m afraid I can’t do that.” Anyone who has argued with voice-activated “help” probably can relate to Dave.
Automation is inevitable. As the Borg Collective say in Star Trek: The Next Generation, “resistance is futile.” We seem to have a collective need to replace human processes with automated ones, sometimes without any logic.
I have wondered for years why my washing machine has to have an internet connection. Like I’m going to suddenly realize that the wash needs to be done while I’m away from home. The manufacturer does not explain how my dirty laundry is supposed to automagically get into the washer, or how the detergent will automatically load itself into the machine, or how the clean wet clothes get into the dryer on their own. But rest assured, all that missing functionality will be addressed by over-the-air software updates at the next opportunity.
One inventive company has solved one of my problems by introducing a washer that also dries clothes. Still not sure how the detergent gets in or the lint filter gets cleaned from the application on my phone, but I’m sure they are working on that.
In the late 1990s I sat through more than my fair share of marketing presentations from would-be-dot-com companies offering the next best thing since sliced bread in automated software solutions. The running joke was to count how many times the word “absolutely” would be uttered in answering any “can it do this or that” question. Of course, it couldn’t. I knew it and they knew, but you can’t sell something without promising the world, right? Early symptoms of the future “fake it till you make it” paradigm.
I am amazed that automotive product recalls seem to now be shadowed almost instantly with marketing press releases about how the problem will be solved by over-the-air software updates. This seems to be irrespective of whether the problems originate in hardware or software, or even before they understand the root causes of the problems.
I keep thinking of Dave and HAL, out there on the edge of space, negotiating, while ground controllers with a 43-minute signal lag time to Earth are irrelevantly trying to work the problem.
One of the greatest challenges facing vehicle operators is understanding what the software engineers actually built into the software. I would bet there are even software engineers wondering this as well, since software is now getting into millions of lines of code, some of it actually created by AI systems, and few, if anyone, actually knows how it all works as a system.
The reason you get nearly continuous software updates on your phone, computer and countless devices is that companies can no longer test 100% of the real-world permutations of situations, so they put it in your hands to do their live-testing. This is the software equivalent to a phrase often stated in films: “shoot them all and let God sort them out”
When you were born, did your birth certificate instruct you on becoming a software tester for life?
Research on more than 80 years of aircraft automation has shown that regardless of well-trained pilots are, they can often get “surprised” by not knowing how the plane will react in a given situation. Sometimes the details of the software are not even discussed with the pilots. Other times, the software proves inadequate to the unexpected combination of failures. Turns out that software engineers are not omniscient. Go figure.
The scariest aspect of automation is a seemingly inescapable human faith that somehow a computer and its software are above reproach compared to a person. The world seems blissfully ignorant that humans designed and developed the systems that created the hardware and software. It’s like dollar bills should have the statement “In computers we trust” printed on them.
I marvel at the belief that software updates can fix all ills. I marvel at the faith that an automated vehicle will always do right. I marvel that all accidents now, and in the future, will be solely the responsibility of careless human drivers, because, clearly, computers can’t make mistakes. The operative word there is Marvel, the publisher of graphic novels, or what ignorant older people foolishly call comic books.
Drivers will need to know what software engineers built into controlling the vehicles. They will need to know what the vehicle can — and more importantly, cannot – do in every situation. As with aircraft automation and pilots, the vast void between training and the unexpected real-world will require on-the-spot thinking. Filling that capability void with intuition and experience when the software and hardware prove incapable of handling the situation.
I have said that automated vehicles will reduce the accident rate, but those accidents that do happen are likely to be catastrophic.
I expect that 90% or better of airline flights are easily handled by 80 years of automation maturation. Maybe even 99%.
I still want a skilled and capable pilot on-board when stuff happens and I’m on that one flight in 100 that has a series of “off-nominal” events.
The push to automate trucking has some great potential, but software engineers are people and they work for companies that need to make profit. Tradeoffs and unknowns are real. Software is never, as Mary Poppins said, “practically perfect in every way.” Dave and HAL are a prescient lesson for the automated truck developers and their marketing teams to park their hubris. Humans, however flawed, are needed to deal with the unexpected. If not in your truck, then in the car next to it.
