Augmented Reality in Vehicles


Having the ability to obtain and process important information while driving a vehicle is critical to safety. Most vehicles today are equipped with a global positioning system (GPS) to help make navigation easier. Technology such as a heads-up display (HUD) can display the GPS data on the windshield so that the driver can keep their eyes on the road.

This paper explores the innovation of expanding on current technology to build an augmented reality (AR) system to help improve driver and pedestrian safety in and around the roadways. Research has already been performed to combine many different technologies to make driving safer, but there are still accidents the cost of human lives every year. The goal of this innovation is to provide crucial information to the driver as it is needed so that the number of accidents can be reduced.


Systems that utilized deep learning, or machine learning, are already in use to recognize and process information quicker than humans (Abdi & Meddeb, 2018). Despite the advancements, over the last several years in computing power, we are still a way off from a fully autonomous vehicle. The deep learning techniques could be used in an AR system to help relay critical information to a driver (Abdi & Meddeb, 2018). Sensors such as ultrasonic and night vision could be used to relay information to the driver iv an AR system, so they are aware of unseen issues ahead. Providing crucial information to the driver when conditions are not ideal can help avoid accidents while on the road.

Improving visibility is the first step in making vehicles safer. Systems such as automatic braking have already been invented and are in use in cars today. AR systems could be used to help predict if somebody was going to step into the roadway or anticipate if a car was going to make a lane change or stop suddenly. The idea is not to create a fully autonomous vehicle, but rather have the computer systems provide critical information to the driver so they are able to make the decision.

One of the main decisions that need to be made when driving revolves around navigation and the majority of drivers today use a GPS to help make those decisions. An AR system could enhance the navigation experience for drivers. Drivers would be able to see information such as which exit to take overplayed on the roadway giving the driver clear indication on how to navigate. This would help also reduce the risk of accidents due to inattentive driving while looking at a GPS or a map.

However, there are risks to utilizing an AR system. Accidents have happened while people have been using AR in the past. While playing Pokémon Go, a young man fell onto an electric railroad track and obtained serious injury throughout his body (Kate Gemma, Kai Yuen, & Khan, 2018). Ultimately he required amputation of one of his legs due to the injury (Kate Gemma et al., 2018). Other causes of injury have also been reported over the years. Cases such as this demonstrate that technology can be distracting when it is not used properly. An AR vehicle system would need to be thoroughly tested to make sure it would not be considered distracting to the driver.


The overall goal of having an AR system in a vehicle is to improve safety for the occupants of the vehicle and pedestrians alike. By providing the driver with real-time information about the surrounding conditions, such as a pedestrian walking into the roadway, the driver would have more time to react and avoid the accident. A secondary goal of the AR system is to aid in navigation. Navigating an unfamiliar city can be hazardous to the driver and the people around them. An AR system could be used to overlay the correct directions over the real-world streets the driver is able to see out of the windshield.

Supporting forces

One supporting force is the current state of the liquid crystal on silicon (LCoS) display technology. In the past few years, LCoS panels have achieved a resolution of 4K2K, and research is underway for 8K4K resolution panels (Huang, Engle, Chen, & Wu, 2018). The panels also have a sub-millisecond response time for intensity modulation (Huang et al., 2018). By increasing the resolution of the panels, an AR system would appear more realistic and aid in user adoption.

Along with the LCoS panels, technology has increased around the various types of sensors that can be added to vehicles. New types of visual sensors have been developed that can be used to aid an AR system in correctly detecting obstacles (Abdi & Meddeb, 2018). The idea of merging technology with the human driver is being referred to as “cooperative driving” (Abdi & Meddeb, 2018). Having a cooperative driving system can be seen as a pathway to fully-autonomies driving vehicles.

Challenging forces

One challenging force would be user acceptance of an AR system within a vehicle. Older drivers could easily read and interpret information for a standard vehicle dashboard but demonstrated difficulty when needing to read a dashboard and follow navigation directions (Kim & Dey, 2016). Younger drivers did not exhibit the same difficulty when asked to perform similar tasks (Kim & Dey, 2016). However, the majority of the safely advantages will come from older drivers utilizing an AR system to help avoid accidents.


Like most research dealing with new technology, there are a lot of unanswered questions. To help answer these questions, experts in both the technology and phycology of humans should be consulted to understand better the feasibility of an AR system being adopted. After examining different types of methods, it was determined to use the Delphi method would be suitable to gain insight from various experts (Haughey, nd). The Delphi method is used to gather the thoughts on a question of different experts anonymously (Haughey, nd). These thoughts are collected, combined, and then share back with the group of experts (Haughey, nd). This process is repeated until a consensus is reached by the various experts to answer the proposed question (Haughey, nd).

The Delphi method was selected because it can be used over an extended period of time and does not require the experts to be in the same physical space. Without a time constraint, experts have the ability to research and think about their answers to the proposed question. By not requiring people to be in the same physical location, experts from around would be able to participate in the process. These factors should increase the accuracy of the predictions made by experts.


A short summary video can be found here.


Abdi, L., & Meddeb, A. (2018). Driver information system: A combination of augmented reality, deep learning and vehicular Ad-hoc networks. Multimedia Tools and Applications, 77(12), 14673-14703. doi:10.1007/s11042-017-5054-6

Haughey, D. (nd). DELPHI TECHNIQUE A STEP-BY-STEP GUIDE. Retrieved from

Huang, Y., Engle, L., Chen, R., & Wu, S.-T. (2018). Liquid-Crystal-on-Silicon for Augmented Reality Displays. Applied Sciences, 8(12). doi:

Kate Gemma, R., Kai Yuen, W., & Khan, M. (2018). Augmented reality game-related injury. BMJ Case Reports, 11(1). doi:10.1136/bcr-2017-224012

Kim, S., & Dey, A. K. (2016). Augmenting human senses to improve the user experience in cars: applying augmented reality and haptics approaches to reduce cognitive distances. Multimedia Tools and Applications, 75(16), 9587-9607. doi:10.1007/s11042-015-2712-4