Tesla: a hacker accesses the “augmented vision” of the Autopilot and reveals how the software works

Tesla has an option called Autopilot, which allows the car to drive automatically under user supervision. A hacker, Green, managed to access the program’s “augmented view” in order to see how it works. We thus understand what the car “sees” in this mode.

You're here
Credit: Green

Tesla’s Autopilot mode allows semi-autonomous driving, the car being able to steer automatically on the roads. A feat made possible not only by the many sensors fitted to the vehicle, but also by the on-board software. A hacker was able to access “augmented vision” of this software, in order to see what the Tesla “sees” in this mode.

We thus understand how the Tesla analyzes its environment. We can see the lines on the ground highlighted, but also squares which represent other vehicles on the road as well as blue dots which indicate dangers, such as road edges, signs or other obstacles. Other data is analyzed in real time, such as road markings or traffic lights. We also notice the sensors used during maneuvers thanks to a diagram on the side.

A mode not available to the general public

This mode is not available to the general public, but hacker Green managed to access it anyway. He also wonders why Tesla is blocking him. We can imagine that the manufacturer seeks to keep the mystery and also not to distract the driver, who must always remain vigilant in Autopilot mode.

Read also – Model S and Model X: Tesla increases their price by € 5,000 without any explanation

However, nothing says that it will be accessible one day. Tesla indeed often updates its vehicles, offering new features to owners. Recently, the Sentinel mode, which is used to monitor the car with the sensors, has been improved, allowing the user to see his vehicle remotely on his smartphone.

This augmented mode is in any case very interesting to watch, almost hypnotic, and is revealed through several visuals and videos through hacker Green’s Twitter thread.