HD Mapping & Data Ownership

Article By : Junko Yoshida

The world now knows that Tesla’s Elon Musk thinks that high-precision GPS maps for self-driving cars are a “really bad idea.”

During the company’s Autonomy Day in April, Musk made it abundantly clear that too much dependency on HD Maps can turn an autonomous vehicle (AV) into a “system that becomes extremely brittle,” making it more difficult to adapt.

The rest of the automotive industry, however, pretty much believes that AV could use an HD map as, at least, a backup system.

“HD Maps are all about adding intelligence to improve the performance and safety of automated vehicles,” said Phil Magney, founder and principal at VSI Labs. 

As Matt Preyss, Product Marketing Manager at HERE, explained, HD maps are not the familiar GPS helpers used by human drivers. The HD maps in question — loaded with geo-coded metadata — are specifically designed for machines, said Preyss.

The foremost purpose for AVs to use mapping data is, according to Magney, “to add confidence to the system. It takes a load off the computational problem of road parsing.” The layers in HD maps include “precision lane markings, boundaries, geometries, and 3D markers for localization,” he said. 

HD mapping for AVs, as it turns out, is becoming a very competitive market not just among traditional navigation map vendors, but also tech companies, car OEMs and startups. In one way or another, the field is jockeying for data ownership, where the high-stakes game plays today. 

EE Times got some help from AV experts at VSI Labs to break down pros and cons of HD mapping for the future of ADAS and AVs.

Why HD maps are needed
There are many reasons why AVs should leverage HD maps. Best of all, HD maps offer the most straightforward solution for either automated driving systems (ADAS) or AV to get better at “road parsing” – analyzing ground and aerial images for road segmentation.

Even for an ADAS car to do basic jobs like lane-keeping, HD maps are effective when lane markers have faded, covered by snow or obscured other weather conditions.

The combination of data from different sensors (computer vision, radars, sonars) should also be able to parse the road. But they have their limitations.

“A good example of sensor limitations comes from Tesla accidents,” explained Magney. “In the Mountain View accident, a likely contributing factor was road surface changes. The dark asphalt surface next to the light concrete surface may have caused the computer to interpret this as a lane line leading to the improper trajectory.”

Magney noted, “If Autopilot had a detailed lane model (an HD Map) this accident could have been prevented as the system knows where it should and should not be.”

Presumably, cameras are still the best, most cost-effective sensors, but they would struggle with road parsing, Magney cautioned. The problem is that it is computationally expensive, he said. “HD mapping assets alleviate the computational load and add confidence to the environmental model.”

What about lidars, then? Shouldn’t the addition of lidars be able to improve parsing?

Not so fast, said Matthew Linder, senior AV software engineer at VSI Labs. Based on lots of experience with lidars at VSI, he acknowledged that lidar is best for 3D perception and environmental modeling, and it reliably detects unknown objects and free space. “However, lidar offers limited performance in poor weather,” Linder warned. “For example, lidar fails to detect lane markings if they are completely covered by snow. Lidar is also still very expensive. Lidars (or any sensor for that matter) cannot see through occlusions, like other vehicles, trucks, hills, around a bend or building.” In his opinion, “The map can extend as far you need, and does not have issues with occlusion.”

When asked about HERE’s HD Live Map, Preyss claimed, “We can support any localization solution developed by an OEM. We believe a fusion of GPS/IMU, HD maps and other sensors is the ideal solution for SAE L3 through 5 systems.”

Nonethelss, “OEMs will have different strategies with different price-points, roll-outs and technologies,” he acknowledged. Noting that the automotive industry is “largely taking an incremental approach to automation,” Preyss said, “We need to make sure our solution can support their various strategies, not only for today, but for the long-term.”

Why HD Maps could be AV foes
Musk might not be entirely wrong about his assessment of HD mapping.

First, HD mapping can’t be useful unless an AV can locate itself on the map with high precision. Further, the AV must always be able to maintain its position accurately.

Second, HD maps constantly need to be updated. Without updates, an AV needs a strategy to cope with inaccurate map data.

Third, HD maps are not free.

Fourth, presumably with trained networks that are trained as well as Tesla claims, who needs HD Maps? Of course, right now, this blithe dismissal is mere hypothesis.

The first and foremost challenge, however, is the question of the vehicle locating itself precisely on the map.  

Danny Kim, senior partner and advisor at VSI Labs, said that one method for precision localizing is “enhancing a GPS with Real Time Kinematics (RTK) or Precise Point Positioning (PPP). RTK/PPP services are “often provided by Positioning Engine software companies like Trimble (service called RTX), GMV (magicPPP), Hexagon/NovAtel (Terrastar) and others.” He added that sometimes a state transportation department will provide public free services through a Continuously Operating Reference Station (CORS) Network or some private CORS charge paid services.

If not those RTK/PPP services, a common approach is using sensors. For example, it “compares a lidar 3D point cloud from a sensor scan to an existing 3D point cloud,” according to Kim.  “So, you create a base map with lidar and then match them with real-time lidar sensing data.” The challenge, however, is the cost of lidars. “As it is expensive using lidar to create large region base maps not to mention of adopting lidar in production vehicles, this is mostly used for development phase of localization algorithms, or some robotaxi implementations now,” Kim noted.

The third approach is “landmark-based localization,” according to Kim, by using lidar and/or camera sensors. “Here, these sensors must detect and classify landmarks. Landmark-based localization is basically comparing those landmarks/road furniture attributes in HD maps with real-time perception.”

Magney sees the need for constantly updating HD as “a huge challenge” yet to be solved. This solution is, however, “a premise of Mobileye REM,” he explained, because Mobileye can crowdsource data collection from millions of cars installed with Mobileye cameras. “Many others are also working on ways to aggregate, manage and redistribute crowdsourced data,” he explained.

Mobileye’s REM in action (Source: Mobileye)

Finally, how much more will it cost to plug HD mapping into ADAS cars or AVs? VSI Labs’ Kim told us, “Honestly we really don’t know the answer.” Most HD map vendors aren’t talking price at this point.

“About six months ago, Chinese map maker Alibaba’s Autonavi started charging OEMS RMB 100 (US$15) per car per year for licensing in April 2019,” Kim noted. The company also said it plans to reduce this price by at least 2% per year. Also mapping/surveying vehicles are very expensive. Chinese media has reported that a single data-collecting autonomous vehicle from AutoNavi Maps costs more than US$1 million.

Many in the automotive industry were surprised to hear about Tesla’s approach – under which Musk is betting the farm on perfecting his neural networks. He has claimed that everything else, from HD mapping to additional sensors like lidars, are a distraction and unnecessary cost.

So, is Musk right?

VSI Labs’ Linder believes that this hypothesis could be true if such a neural network were to emerge. “It does not exist today.  However, even if it does exist, say you have a sensor failure, the maps still provide road parsing redundancy to pull over safely in conditions like that,” he stressed. 

But what about Tesla’s claims that a HD map would be superfluous even on snow covered roads?

Kim said, “What they basically say is that if human annotators can draw lane lines on snowy roads reasonably well by looking at those images, artificial neural nets can be trained and perform reasonably well.” However, he added, “This is still unproven. Tesla’s Andrej Karpathy admitted that they haven’t specifically run a data engine with real world data of such situations. That means Tesla fleets need to send lots of data and Tesla needs iterative processes of training their neural networks with those real-world data.”

Competitive HD mapping market
Even if Tesla sees no need for HD maps, the rest of the AV industry is still looking for precision maps for robocars. So, who are the key players in HD maps?

It turns out that they are “pretty regional,” according to Kim, “especially in Asia.”

Korea has MnSOFT, China has NavInfo, Baidu and Alibaba-owned Autonavi.

DMP (Dynamic Map Platform) of Japan is a regional coalition with many stakeholders (co-invested) from the government, OEMs, tier ones and map vendors. Kim explained, “Their mobile mapping vehicles collect much of HD base map data and create semantic maps in a cooperative manner before each map vendor post-processes the data and add lots more competitive attributes (such as localization assets).”

Meanwhile, Europe and North America are HERE and TomTom, along with USHR (now acquired by DMP).

Kim summed up: “All these guys (except DMP) are traditional navigation map makers and understand Geographical Information System-based map generation and maintenance at scale — at least at regional level.”

On one hand, base road-geometric maps and HD lane models (mostly for highways) have been built or are being built by the current players. On the other hand, “localization attributes in the HD maps for landmark-based localization are still being collected by many start-ups,” explained Kim.

The key here, he stressed, is “whether such map attributes can be generated at scale.”

In his view, “Harvesting/crowdsourcing such data with more affordable sensors like camera is almost the only way to scale and that’s why Mobileye has been so dominant.” He added, “Now startups like Deepmap, Civil Maps, Carmera, Mapper, Netradyne, Lvl5, Atlatec are worth noting.” The key to separating the successful startups from the riffraff is “how much they can automate those map generation and maintenance processes,” he explained.

Data ownership issues
Mobileye is providing a full mapping and localization stack using its own cameras and tightly coupled AI-based perception algorithms. “Perception and localization go hand in hand,” Kim noted.

In contrast, “Tesla doesn’t collect base mapping data. They collect real-word data to train their perception and prediction algorithms,” Kim explained.

Mobileye is in a league of its own. “Mobileye is not just about collecting data, but they also process them in the cloud and sell localization algorithms as well.”

Notably, however, “Mobileye doesn’t own the data unless they make a full vehicle system like one in Israel. It is usually OEMs who own the data, because they collect data with their vehicles via Mobilye EyeQ4 cameras.” Kim pointed out, “This data ownership model is one of the key debating areas — where in practice Mobileye wouldn’t be able to own the whole mapping and localization program.”

Kim predicts that data ownership model is “where OEM would fight against full stack mapping and localization programs.”

He explained, “Localization is core IP of OEMs and they can’t really give it up to third-party platform suppliers or just map data provider.”

The challenge here, said Kim, is that “localizing your vehicle against localization assets in fresh HD map data requires in-vehicle connectivity, data coverage at scale, and dynamic map data management workflow.” At this point, no OEMs, except for small OEMs like Tesla, has yet to pull that off.

HERE’s strategy
VSI Labs’ Kim observed, “HERE is supposed to integrate REM data into their base maps (as part of partnership) so their German automaker clients’ vehicles can consume for localization.”

At the same time HERE in their R&D program is utilizing customized cameras to collect localization asset, Kim noted. “HERE is not selling those camera-based assets as map data yet but the purpose is to show how they (or their German clients) can harvest-aggregate/process-consume such camera-based localization assets in the map for now,” he speculated.

HERE’s Preyss acknowledged that HERE tests its map data against various localization solutions like a camera-based solutions and a lidar point cloud based solution. This is to “ensure our data supports all localization approaches,” he noted.

“The data is formatted so that it can be used in an any localization solution.” But he made it clear that HERE does not provide localization software. Instead, the company offers “just data to support it.” Preyss explained, “The localization algorithm/solution is up to the OEM in terms of how they want to approach it (doing it in-house or outsourcing). What we provide is the environment data to help the vehicle to localize, we don’t build localization software.”

Preyss summed up: HERE’s role in the future is in “aggregating sensor data, making updates and distributing those updates accordingly, based on our customer engagements.”