🔗 AV and Predictability
text Ido PerlmuterFirst published: .
As I was writing my latest crybaby rant about why Autonomous Vehicles Aren't Autonomous, I once again made a reference to "badly written JavaScript APIs." This got me thinking, because it's kinda become a go-to sentence for me. I don't really know if any of the current companies working on autonomous vehicles are actually writing JavaScript code (it seems some of them do at least for their infotainment systems UIs), but it's very prevalent in many of the fancy new progressive technologies, such as crypto-currencies and other blockchain applications, where code for "smart contracts" tends to be so badly written that people regularly lose millions of dollars (or fake money, by this point I still don't know but am too afraid to ask).
Now, JavaScript isn't necessarily bad as a programming language, and perhaps it's not JavaScript's fault that people are writing bad code with it. But I do believe it is completely unfitting for the domain. So I started thinking: if I were to write software for an autonomous vehicle, which language would I use?
The thing about driving, is that it's all about predictability. The entire point of all traffic laws is to make traffic predictable. Humans are difficult to predict, expect for certain aspects in big enough boundaries. So the laws are there to eliminate unpredictability as much as possible.
Humans may not be easily predictable, but they are flexible and adaptable, which is very important when driving, because at the end of the day, traffic can be quite unpredictable at small ranges. Even flexibility, though, needs to be predictable: given a certain unexpected situation, we should be flexible enough to react, but our reaction needs to be predictable. Take all those idiots that like to brake check other drivers: they introduce an unexpected issue to the other driver's drive, and predict that the driver will slam on the brakes and not ram into them (and I do admire those that do anyway).
The thing about unpredictability, is that it cascades, and cascading failures are often catastrophic failures. I mentioned in my earlier rant that AVs depend on many factors, such as local hardware and software, remote servers, and GPS satellites. Let's look at how unpredictability cascades: an AV's computer needs to make a certain request to a remote server in order to function. So the vehicle's program, written in a certain programming language, makes a call to the operating system (written in a certain programming language) using its API, which in turns makes a call to the network adapter using its API, which accepts the request and sends it to the remote server, where it's accepted by its network adapter, parsed by its operating system, handled by its computer program, which makes a request to a database program, that makes a request to the OS to access the filesystem using its API, parses the result, and prepares a response that will go through the entire way back to the vehicle's computer program. And this is an oversimplification, mind you. When the server's filesystem reacts to the database in an unpredictable way, it is sometimes the case that the database program will also react in an unpredictable way, which in turn causes the server's computer program to react in an unpredictable way, and so on it cascades until your car's computer program loses its shit and kills you.
So our main goal when choosing a programming language for AV-development should be to eliminate (or avoid introducing) as many levels of unpredictability as possible. So right out of the bat, dynamic languages like JavaScript and Python are out. Both have standards/specifications, but this is not enough, and any written code, by nature, will be unpredictable. Try to find out what a "file object" is in Python, I dare you. Neither the language reference nor the documentation can really say.
Most statically/strongly-typed languages are out too, probably. Go, for example, is a case study of cascading failures. It happily lets you ignore errors, a practice which always results in cascading failures that are extremely difficult to diagnose, and neither the standard library documentation nor the top open-source Go projects teach you how to properly handle errors, which remain one of the language's most divisive aspect. By the way, not even the documentation of Go's standard library can say what you should do if a database cursor fails to close. C? I can practically feel myself crashing already. Furthermore, almost all of the commonly used programming languages have "side-effects". Running a function with the same exact input over and over again will not always produce the same output, because there's often a source of unpredictability, which can be as basic as the current date.
Of course, you can say that writing safe and predictable software in these languages is possible, even if hard. And you'd be right. You can adopt very strict development methodologies and processes; you can hire better programmers; you can utilize static code analysis to catch the most egregious kinds of mistakes; you can do fuzzy testing; and more. We all know, however, that this is not what's happening at all those tech companies these days, at least not well, and developers are an unpredictable bunch.
Should we turn then to one of the purely functional programming languages, such as Haskell? Will an unconventional solution reduce unpredictability or increase it?
Unpredictability (and complexity) is the main reason why I distrust autonomous vehicle technology, and anything that utilizes Artificial Intelligence and Machine Learning. The latter two definitely have their uses cases, but I fear they are being shoved into places where they shouldn't be. I will probably never trust a car whose behavior depends on an AI system, because such a system—in my eyes—is entirely unpredictable. I have no idea what it's going to do. This is why I don't buy Tesla's spiel about Autopilot and FSD decreasing the driver's cognitive load. On the contrary, I think they increase it, because the driver—who serves as a fail-safe for the AI-based system—needs to be able to predict incorrect AI behavior, or otherwise react and correct it virtually immediately. That's just not possible.
There are other factors to take into account for our choices, of course. They all eventually somehow relate to safety. Predictability is required because it increases safety. Good performance is required because it allows the vehicle to react quickly to unexpected situations, thus increasing safety.
So I don't really know which programming language to use, but I do know that I would make reduction of sources of unpredictability my top priority, or one of them. Not only to make my autonomous vehicle safer and more predictable, but to also make it easier to diagnose when a failure does occur. Are you really able to tell with 100% accuracy what the cause of a failure was in a system whose based on machine learning and AI?
Maybe I should just read NASA's Software Design Principles, if there's anyone who knows unpredictability, it's them.