Driverless cars vs Elon Musk: Is it best to trust your eyes?

Driverless cars are the future, but how they will work is a problem that Elon Musk is finding harder to solve than he thought.

"When radar and vision disagree, which one do you believe? Vision has much more precision, so better to double down on vision than do sensor fusion" Elon Musk, April 2021.

As the years and months pass by, it's fair to ask whether driverless cars are getting closer to reality. It seems almost like a paradox: the closer we get to having fully autonomous cars, the longer it takes and the further away we seem to get. It's because the last bit is hard; really hard. 

Scientists actually call this phenomenon a "hard" problem. 

Some problems just require a solution. Is there water on Mars? Send a probe there. That's difficult, but it's far from impossible. That's not a hard problem. What is the source of consciousness? That really is a hard problem. Scientists have been looking for consciousness for decades and they've never found it. For example, where's the control centre of your brain? There doesn't seem to be one. There's no single area of the brain that issues commands to the rest of the brain, starting and stopping processes and synchronising them to avoid chaos. This matters because you'd probably expect consciousness to be associated with something central to the brain's functioning. But it isn't. You can see why it's a hard problem.

The problems of designing a driverless car

What makes the problems around driverless cars different from designing, let's say, a conventional racing car, is that traditional engineering - even modern, advanced engineering - is deterministic. Give it the same set of inputs and you will get the same set of outputs for any given circumstances. It goes without saying that when you design racing cars, you're not expecting them to drive by themselves. 

There are dozens of companies trying to design practical self-driving cars. They all have their own approach. It is perhaps unfair to focus on Tesla, but the company of which Elon Musk is the CEO is possibly taking the most advanced, and the most - ahem - adventurous  approach. I have to say that it's the one that I find most intriguing. 

Not only does Tesla have a big head start with fully electric vehicles, it has a big advantage in its ability to collect data from drivers. Essentially Tesla's task is to "solve reality" with its technology (that's Musk's own description of the task). This is a hard problem. It needs the car to "understand" the world. 

Other approaches use LIDAR, understandably so. LIDAR builds a 3D model of a car's surroundings in detail and in real time. Musk's argument against this in justifying his all-visual approach is both that humans can drive perfectly well and that it is actually harder to do the computing necessary to integrate LIDAR and camera inputs than it is to rely on cameras entirely. 

Driverless cars and machine learning

This might sound either lazy or naive, but Musk's reasoning is sound. Tesla has an incredible resource that it uses to improve its cars all the time: existing drivers. Tesla frequently has its own AI "driver" running in the background. Human drivers are unaware of it and it's perfectly safe, because it's not connected to the control systems of the car. But it does have access to all the real-time status information. Essentially, it knows how the car is being operated by the human driver. 

The car's automatic driving software "shadows" the real driver and makes a note every time it comes to a different decision. These hidden disagreements are frequently OK and might just be caused by differences in driving technique. But where there are significant divergences, the data is sent back to Tesla so that the circumstances can be analysed. Maybe it was a road sign obscured by the branch of a tree or by bad weather. Perhaps the autonomous software underestimated the stopping distance of the car because of wet roads. It could be a million different things and for every one of those error reports, it needs to "learn" the answer, which is indeed what it is likely to do. 

This autonomous "shadowing" is also what will allow Tesla to say confidently that their cars are safer driving autonomously than driving with a human in charge. It will be a matter of record. And don't forget that the driverless car doesn't have to be perfect - just "better". 

But there will still be loose ends - significant ones. One of the hardest aspects of using a ride-sharing service like Uber is meeting your driver. With no precise way to specify where you want to meet you have to walk up and down a crowded street to see where your ride is. It will be the same for driverless cars at the end of their journeys. Have you ever tried getting a taxi to stop exactly where you want it to in London or New York?

There are all kinds of "edge" cases. There are a myriad ambiguities. These will be solved but it's hard to say whether that's going to be this year or this decade. My feeling is that driverless cars will pass the threshold of acceptability for general use sooner rather than later, but that might just be unreasonable optimism. 

Meanwhile, Tesla's research and development in taking life-critical data from multiple cameras and feeding this converged picture into a gigantic AI system is going to be transformative for everyone.  Once again, video is the technology that is defining the future.