A new generation of sensors, increased computing power and artificial intelligence are making it possible to digitally map the topography of the entire world and paving the way for a self-driving future.
- Marc Deckert
- Xurbia_Xendless Ltd., still from the video “House of Cards”(2008), directed by James Frost
- renishaw/Dynascan S250
Knowing your way around a place is extremely useful, not only when you’re in urgent need of a shot of espresso or have to find a tailor to alter your trousers in an unfamiliar town, or when you’re in search of that perfect secluded bay. It’s even more important when you’re at the wheel, as it’s much easier to follow a route if you know it like the back of your hand. You’re aware that a hidden bend is soon coming up, you know which hills the lorries slow down to a crawl on, or where an emergency exit is suddenly going to appear. And you might well decide to drive past that emergency exit because you remember that the espresso at the next exit has a better crema.
Without personal experience of a route, even the best drivers will be more stressed by the time they get to their destination. Contextual awareness of the environment and local knowledge add to the driver’s direct perception, which makes them feel better – and safer.
Billions of pixels
“Vision, although our most dominant sense, has its limits,” says Sanjay Sood. “That cannot be the case with self-driving vehicles, however, which need to see through buildings, around corners and 20 miles in advance to manoeuvre safely.” Sood is head of Highly Automated Driving at HERE, a company specialising in digital maps that are far more precise than those currently used for navigation. He raises a point that is often underestimated in the debate about “autonomous” or, to be more accurate, highly automated driving. Usually the focus is on the intelligence behind the next generation of vehicles and the dramatic advances being made in sensor technology. But what applies to good drivers also holds true for the vehicles themselves. “To fully realise the vision of autonomous driving, vehicles will need to understand the road environment beyond the range of their on-board sensors,” declares Sood.
The latest HD maps contain a wealth of information – not just on roads and routes. Their billions of pixels contain, as it were, the entirety of the surrounding environment, from trees by the roadside to details that are accurate to a quarter of an inch, such as exact lane boundaries or even the height of the kerb. All of this data is captured and displayed in three-dimensional images. The raw material for the map is not provided by a camera but by Lidar (Light Detection and Ranging) technology. A highly sensitive, vehicle-mounted laser scanner shoots high-frequency pulses of laser light and then uses sensors to detect reflections from any objects. By measuring the time it takes between sending out a pulse and the light bouncing back, the sensor can gauge the distance to each individual point. Viewed in the raw, the pixelated landscapes look more like a futuristic computer game.
NASA once used Lidar to map the exact topography of the moon, and archaeologists are now embracing the technology to identify and build up an exact picture of excavation sites. British rock band Radiohead have demonstrated that Lidar can be used to create art, too. Their Grammy-nominated music video for “House of Cards” was created entirely using data captured by the technology.
However, the most important advantage the HD maps have for cars and their drivers is a significant increase in safety levels. Every piece of information is present in back-up form, as sensors cannot always provide a complete picture of the vehicle’s surroundings. There will always be times when the lane markings are difficult to make out, a road sign is covered or has even been knocked over. This is where the map comes into play. It also creates redundancy so that self-driving vehicles no longer have to “trust to luck”. They know what lies ahead of them at all times and can check the real-time information against the data provided by the map.
“We see the map as a sort of additional sensor,” explains Klaus Büttner, BMW Group’s Vice President of Autonomous Driving Projects. The map has the advantage of delivering important information about road conditions beyond the range of the on-board sensors, which is key to anticipatory driving. At BMW, Büttner’s focus is on making vehicles intelligent enough to react appropriately to any conceivable traffic situation. That is far removed from conventional coding. It is, in effect, a sort of training, and indeed the experts at BMW talk of the algorithms being “trained” in the process.
“There is a new generation of sensors, and computing power is constantly increasing. But the real breakthrough has been in the algorithms we use to develop driving strategies and recognise the driving environment. These algorithms have made remarkable progress, and it’s all driven by artificial intelligence.”
Klaus Büttner, Vice President of Autonomous Driving Projects at BMW
Even today, assistance systems in production models such as the current BMW 5 Series can regulate the speed of the vehicle depending on traffic, make sure that the vehicle stays in its lane, and assist in manoeuvres such as lane changes. However, drivers have to keep their hands on the steering wheel and remain alert and ready at all times to take back control of the vehicle. But the roadmap for the next steps forward has already been laid out. In 2021, BMW will offer a package for highly automated driving on motorways. This will mean that level three of the current classification has been reached, and drivers will only have to remain ready to intervene if a problem is reported by the vehicle itself.
Only a few years ago, experts were warning people not to have unrealistic expectations. The fact that highly automated driving is now so close to becoming a reality can be attributed to recent technological advances, according to Büttner. “There is a new generation of sensors, and computing power is constantly increasing. But the real breakthrough has been in the algorithms we use to develop driving strategies and recognise the driving environment. These algorithms have made remarkable progress, and it’s all driven by artificial intelligence.”
Speaking to the project manager for Autonomous Driving at BMW, one soon realises how multifaceted his job is. He and his team are working on taking not only all the solutions that are already available but also some that are still at the R&D stage and integrating them into a safe, series-ready car. But in addition to the work on actual products, there is another fascinating side to the project that sounds like the stuff of science fiction. At the Munich Research and Innovation Centre, Büttner uses computers that are so powerful that it is possible to create various layers of “deep neural networks”. The plan is for the software to be able to understand all the details a human driver processes intuitively. “We’re working with reinforcement learning,” explains Büttner. “In other words, we run through as many traffic scenarios as possible with the computer along with an assessment of those situations. Gradually, the computer develops its own understanding of which driving strategies are optimal. It starts to think abstractly.”
Intelligent maps in the new BMW 5 Series
Purely in terms of perception, the vehicle of the future will have several senses that complement one another. Cameras will recognise the driving environment as well as road signs and traffic lights; radar will measure distances to other vehicles and objects, while a series of laser scanners, miniature versions of the large Lidar devices currently mounted on the data-capture vehicles, will deliver real-time 3D images of the surrounding area. Highly precise maps are another part of the package, says Büttner. In 2015, BMW, Audi and Daimler jointly acquired HD cartography experts HERE. Their highly precise maps are considered key to the success of autonomous driving. However, the HD material that has been produced so far is only the first step.
“We’re thinking in two phases,” explains Dietmar Rabel, Director of Product Management Autonomous Driving at HERE. “Right now, we’re sending out our proprietary data-capture vehicles to create images of the roads that are as precise as possible.” HD maps are already used by many vehicles on the road today. “Take trucks, for example. For them, it’s really useful to know about inclines in the road to help optimise gear changes. That can save enormous amounts of diesel fuel and it also prolongs the life cycle of the transmission.” The maps are currently also being used in the BMW 5 Series, for instance in Adaptive Cruise Control, which intelligently regulates speed and distance to the vehicle in front.
All of this has already been successfully implemented, but as things currently stand a data-capture vehicle has to be sent out expressly to record every single change to a location – a relatively slow process. Phase Two will see the sensors in production vehicles come into play. “The project for the future is a dynamic, ‘self-healing’ map which remains completely up-to-date at all times,” explains Rabel.
The car will remain a “robust” machine
As part of this process, BMW will be providing anonymised sensor data and cooperating with Israeli technology company Mobileye, an Intel subsidiary that is a global leader in vision-based advanced driver-assistance systems. The idea is for BMW to provide real-time, camera-based information on the driving environment. The data is then aggregated at the back end and used to update the highly precise maps. Crowdsourcing on the road will have huge advantages – as soon as the critical mass of vehicles with on-board sensors is reached, it will be possible to keep the map material up-to-date at all times. In other words, the map will achieve real-time capability.
Büttner insists, however, that in spite of the recent advances, the car of the future should still not be equated with a computer on wheels. The car must remain a “robust” machine that is absolutely reliable and safe. For starters, it has to be able to perform all its essential functions without connectivity and “to come to a safe stop in any situation.” Some observers, he adds, believe that highly automated driving has already completed its test phase. When asked what’s next on the agenda for BMW, Büttner replies: “We’re now going to drive many millions of kilometres to ensure that the quality and safety of the functionality are guaranteed.”
When a computer recently beat the best Chinese player at Go, the most challenging board game in the world, it was another of those defining moments in which we can sense the future in the here and now. For many people, the idea of intelligent cars conjures up a familiar feeling of foreboding and uncertainty. Why should cars be able to drive autonomously anyway? Can’t a good human driver react better – by acting intuitively? To say nothing of the sheer pleasure you get out of being in complete control of a vehicle. Why should we hand over such an enjoyable activity to computers? Steering wheels will, however, be around for a long time to come. Cars will assist us on routes that we’re not interested in experiencing as drivers. The first generation of autonomous vehicles will allow us to not always have to drive on motorways. A later generation will guide us through our commute in a way that allows us to make more efficient use of our time than we do today.
A more human-friendly future
Automated driving is particularly appealing to mobility services in towns and cities. Urban life could benefit enormously from more intelligent cars. Brad Templeton, mobility expert at Silicon Valley think tank Singularity University, recently described how parking in cities could be better organised. In current practice, everyone wants to park within walking distance of the shops. “This means that shopping malls are surrounded by huge parking lots that are only fully utilised at peak times, such as the holidays.” So this is a waste of space. Intelligent vehicles could change this by dropping off passengers at their destination and parking further away or perhaps recharging themselves while the passengers shop. This would mean more convenience, but also a use of space that was more human-friendly rather than car-friendly. Templeton also envisages a future in which private cars will no longer have to be banned from inner cities – “and cars will be much more readily accepted in cities than they are today.” It will be some time before all of this happens, but the knowledge and the tools for a future with autonomous vehicles are available today. And the next levels are already on the horizon.
The levels of automated driving
Level 1: The driver is in charge. The vehicle is merely equipped with a number of assistance systems, such as Active Cruise Control.
Level 2: The vehicle takes charge, but the driver has to monitor traffic and be prepared to intervene at any time.
Level 3: The driver can rest, read or watch a film during the trip, but must still be able to take control of the vehicle immediately if the vehicle warns of a problem.
Level 4: The driver can leave the driving seat or sleep. The car will stop itself in critical situations or if a problem arises.
Level 5: The vehicle can drive even without a driver on board – as in fully automated taxis, for example.