Automotive Holographic Head‐Up Displays

Driver's access to information about navigation and vehicle data through in‐car displays and personal devices distract the driver from safe vehicle management. The discrepancy between road safety and infotainment must be addressed to develop safely operated modern vehicles. Head‐up displays (HUDs) aim to introduce a seamless uptake of visual information for the driver while securely operating a vehicle. HUDs projected on the windshield provide the driver with visual navigation and vehicle data within the comfort of the driver's personal eye box through a customizable extended display space. Windshield HUDs do not require the driver to shift the gaze away from the road to attain road information. This article presents a review of technological advances and future perspectives in holographic HUDs by analyzing the optoelectronics devices and the user experience of the driver. The review elucidates holographic displays and full augmented reality in 3D with depth perception when projecting the visual information on the road within the driver's gaze. Design factors, functionality, and the integration of personalized machine learning technologies into holographic HUDs are discussed. Application examples of the display technologies regarding road safety and security are presented. An outlook is provided to reflect on display trends and autonomous driving.


Introduction to Head-Up Displays (HUDs)
This section introduces the history of HUDs and discusses 2D and 3D HUDs. Commercial HUDs of industrial manufacturers are discussed in terms of efficiency, usability, safety, and security.
introduced to HUDs. [17,18] In contrast to the aircraft technology, most commercial automotive HUDs show ghost images focused at a short distance from the windshield. The mismatch between the focal distance of the displayed image and the real objects has significant implications. [19,20] The eye lens requires a change in accommodation between the displayed image and the road even if the displayed image is in the line of sight. [21] The displayed virtual objects or informative signs cannot be fixated in place of the real objects. [22] These technical challenges limit applications in an extended reality experience. The most important challenges in the implementation of HUDs to reach a real applicability in AR are i) the multifocal display, large viewing area without compromising the field of view (FOV), ii) the optimal positioning on the windscreen, and iii) minimal invasiveness in the driving behavior by accurately pointing out hazards on the road.
The manufacturing cost of a HUD based on microelectromechanical systems (e.g., micromirrors) is constantly decreasing ($ . [23] The built-in HUDs can provide a wide range of information including vehicle data and night vision with the option to turn the HUD off and automatically on to alert the driver. [24] Current studies are focused on language processing and gesture control to provide driver specific information using sensors and cameras. [25] As compared to head-down displays that require an information uptake in 1.8 s, HUDs can attain information uptake in 1.0 s, which can decrease mental fatigue and prevent gaze shift from the road. [26] Automotive manufacturers have developed built-in HUDs with different operating mechanisms. Peugeot proposed a portable projector system located on the dashboard of the vehicle. [27] The panel of translucent polycarbonate adjusts the brightness of the projected images with built-in sensors. This built-in HUD is like external HUDs, but it has the advantage of being fully connected with the operation system of the vehicle and can maintain a safe distance between vehicles by taking and processing sensor information in real time on the road. BMW proposed a full windshield HUD with mirrors capable of projecting high-contrast, easy-to-read images into the driver's line of sight on the windshield in full color. [28] Jaguar Land Rover developed a HUD with local dynamic depth-variable viewing angle and enlarged driver eye box by considering the driver's head movement. [29] This technology can project dynamic pop-ups for hazard warnings within the HUD, creating an image depth of 50 cm and an image size of 1.4 times the natural viewing experience. This review focuses on holographic techniques for AR applications in the automotive sector to increase road safety and driver's comfort. The article discusses hardware and software, including LiDAR, AR, in-eye panoramic holographic projections, point cloud data generation and processing, 3D computer-generated holography (CGH) custom elaboration, 3D holographic video displays, and the expansion of the field of view and the driver's eye box. [30] The review elucidates optical methods, the interactions of the holographic technology with an interactive urban environment, the impact on the driver and the surroundings, and an outlook of this technology in combination with the environment. It also provides an analysis of the placement of the content on the windshield in AR technologies according to the driver's understandability of the provided information, safety, and user satisfaction. Information layout concepts based on machine learning technology could subdivide the projection area and customize it. Certain objects could be prioritized and placed into an area of personal preference by the driver. The question of possible driver distraction with advanced infotainment options was addressed with support from statistical data. The article analyzes the system components, subsystems such as the effects on the driver and the environment. The article discusses the robustness of commercial products, proposed holographic devices, and the overall supersystem network operations. Possible driver distraction with advanced infotainment systems is also discussed. An outlook is provided on AR in consumer devices and autonomous driving with artificial intelligence (AI) capabilities.  the image and the perceived image in the driver's field of view, and the crucial component of the driver's personal eye box (Figure 1b). [31] Optical phenomena created by the human eyes need to be considered when designing an elaborated HUD: [32] • Accommodation • Motion parallax • Convergence • Binocular disparity • Linear perspective • Shading • Occlusion • Texture • Prior acquired knowledge Accommodation represents the adjustment of the focal length of the human eyes by controlling the ciliary muscle tension. [33] This process evaluates the rate of change of the human eye lens when the ciliary muscles contract to obtain a focused image of an observed object. Motion parallax compares relative motion of different elements in a 3D environment by using depth cues. [34] When the viewer moves, closer objects appear to move faster than the objects further away. Convergence measures the angular difference between the viewer's eyes and the viewing directions when observing the same fixated point on one object; the closer the object, the greater the convergence. [35] The difference in resulting images by both eyes is presented by binocular disparity (stereo). The further an object, the further apart the two images acquired with both eyes. Linear perspective is an appearing relative distance of acquired objects. [36] Any two straight parallel lines appear to converge at a distant point. Shading describes variations in light intensity that form a 3D spatial-relationship clue, which recreates object orientation and surface shape in the brain. [37] The occlusion effect indicates that objects behind an opaque object appear further away than two interposing objects. [38] Texture refers to the surface structure of objects recreating the 3D shape of the object together with the object distance from the viewer. [39] Importantly, prior knowledge considers familiar shapes and sizes of common structures static, and in motion can recreate their 3D shapes and their distances in the viewer's brain. [40] Some commercial 3D displays cover all the depth cues. HUDs can be classical and holographic. The classical HUDs utilize a projector instead of a spatial light modulator (SLM) (Figure 1a). The advantage of holographic HUDs is the depth information and the possibility to project objects as augmented and mixed reality in 3D in the driver's field of view. [41] Hence, holographic HUDs are paramount to explore the possibilities of augmented and mixed reality for road safety. For example, Philips offers an end-to-end 3D display enabling live capture of new 3D content, visualization, and animation of 3D objects. [42] SeeReal Technologies' holographic 3D display products offer selective accommodation of the observer's eye lenses and natural depth perception. [43] VividQ offers holographic displays for AR application with real depth perception and bright image quality of 500 000 cd m −2 without exceeding 1.64 W. [44] QinetiQ's printing by active tilting apparatus can subdivide the photosensitive area into tilted subareas. Various illuminations account for selected pixelated light pattern controlling the assembly of the final image produced from the subareas with over 5 bln px. [45] Envisics provides holographic HUDs with AR experiences catering specifically to the automotive market for advanced driver assistance systems. [46,47] BMW and Siemens VDO Automotive system developed a HUD with a brightness of 500 000 cd m −2 . The optical path was diverted with a reflector to a thin film transistor display, where the display images were generated at 65 000 px. [48] The HUD provided a homogeneous backlighting for the image-generating device with light emitting diode (LED) sources. [49] Lincoln in collaboration with Continental generated a HUD with a brightness of 15 000 cd m −2 , a dynamic dimming range of 5000-to-1, and temperature stability. The DLP3000-Q1 technology does not require light source polarization. The HUD is capable of projecting virtual images 15-20 m ahead of the driver. [50] The HUD consists of a 0.3 in. digital micromirror device (DMD) and a DLPC120 DMD controller. The field of view on the windshield is 10° × 2.5° and the resolution is 70 px deg −1 . [51] Mercedes-Benz and Nippon Seiki's collaboration resulted in a HUD generating an active projection image of 45 × 15 cm 2 projected at a distance of 10 m. A micromirror and a customized digital projector generated a virtual image. The implemented HUD is adjustable in terms of the projection location of the display light. This allows for adjusting the height of the HUD based on the observer's point of view without the movement of the virtual image in vertical direction. [52] Additionally, the compact HUD from Nippon Seiki enables generating images in the HUD with a plurality of display distances. This is achieved through the display that emits projection light to illustrate the virtual image at a predetermined distance. An image formation adjusting mirror receives the projection light and converts it into a plurality of first and second projection lights at different distances. [53] Nissan and Panasonic collaborated on a HUD with a see-through display. This HUD included a light source, an optical projection system, which was capable of displaying volumetric holograms. [54] Table 1 shows commercial 3D display technologies. Table 1 illustrates existing commercial HUDs that focus on improved resolution, a greater field of view for the driver, and increased display brightness. These are industrial requirements to improve the HUD technology, but the effect on the driver and overall road safety are not yet clear. Further human-centered studies need to be carried out to understand the industrial design effects on overall road safety and security. For example, the need to personalize content in HUDs for the driver is another design criterion that should be considered. An inclusive strategy for the driver should be developed to quantify the progress and improvement of each technological design requirement of current HUDs. Displays, which require additional features such as stereoglasses to bridge the accommodation and convergence effects, can cause fatigue to the spectator as the images are projected in 2D instead of their physical distance in 3D space. [55,56] Design proposals incorporate a balance between the risks and the advantages of windshield HUDs such as considering driver's abilities and the priority of tasks in every single context of the transportation. [58] A need for a human-centered approach in the design of windshield HUDs is paramount for safety and security on roads. [59] The driver's cognitive and sensory abilities need to be balanced with the driver's needs for windshield HUDs. A windshield HUD provides the driver with 3D information that merges with the road scene. [60] The focus point of the driver wanders within the field of view when primarily targeted forward sceneries are processed. The sensory capability of the driver allows for small pieces of the 3D information to be processed at once when presented visually to the driver. [61] Appropriate design of individual information customized for every situation, such as the position and size supported with colors, can guarantee rapid danger recognition and improved road safety.

Windshield Projections
The projected information on the windshield should follow strict guidelines in design to follow a user-centered approach for reducing hazards. [62] The principles of information presentation must enhance the driver's intuition and must enforce safety while driving. The projections should be displayed in specific areas on the windshield and must not require long attention and perception times to understand the meaning of the presented data. [63] Information density should be increased and critical safety awareness signs must be strategically placed on the windshield to i) reduce traffic blind moments, ii) minimize distraction, iii) cluster relevant and similar information, iv) enhance situation awareness, and v) highlight relevant input information for the driver (Figure 2a-c). An option is the layout of locating items by relevance such as contextual relation and spatial proximity to reduce the driver's effort in processing visual information.
An efficiency analysis of the industrial HUD system architecture with failure-mode analysis framework was performed to strategize the advantages and disadvantages of current commercial HUDs. Each system component was justified, including components such as the laser source and the SLM. The 3D holography applications across commercial products were analyzed through a critical assessment in terms of security, safety, color, uniformity, and luminance. Additionally, 2D windshield and 2D/3D in-eye projection-based holographic setups were benchmarked with commercial display technologies in terms of component arrangement in the optical system and efficiency. The assessment parameters presented for industrial HUD manufacturers apply to 2D windshield projections ( Table 1). The luminance and uniformity of the projections decrease in quality in 3D augmented reality projection mode. Hence, the need for display performance metrics for 2D and 3D HUDs should be established.  The highlighted area in red shows the space, where most driver attention is situated, to display crucial information. The area highlighted in orange shows secondary driver attention space for a calculated layout of the HUD to enhance road security. A method of defining requirements and boundaries was developed for the qualified viewing space which was based on the display quality performance parameters. [64] Such boundaries could include but are not limited to a set of physical location points for the driver's eyes, where the HUD luminance degrades to 50% from the peak intensity. Another boundary could be the projection distance for 3D holographic AR projections. 2D windshield HUD efficiency studies have been carried out to describe another crucial parameter: the cognitive capture effect. [65] This parameter evaluates the inefficient attention switching from the HUD projections to the driving task. In a simulation study of 48 users, 2D windshield HUDs showed fast response time in unanticipated road events such as speed limit detection. Additionally, 2D HUDs promote driving behavior such as smaller variances in lateral acceleration. [66] One study verified the driving behavior with less mean deviation from lane in an International Standard Organization (ISO) 26022 lane changing task. [67] Another metric for 2D windshield HUDs could be the relative ghost luminance (RGL). [68] A P-pol combiner film was developed to increase the projection image quality by reducing the RGL to 20% compared to wedge-system windshield projection with RGL of 40-60%. [69] Regarding the 3D AR HUD metrics, an optical seethrough HUD (AR DriveSim) was developed to examine physiological and cognitive effects of HUDs with context and focal distance switching. [70] Another criterion for introducing metrics for HUDs is inclusivity, where the superposition of text in projections was found to have a negative effect among elderly users. [71] Finally, ergonomic studies suggested that the placement and the amount of HUD projection content is crucial. [58] Fewer than 6 symbols should be displayed at one time and the main content should be placed in the center of the windshield. Gap analysis frameworks can allow for defining metrics for the component arrangement and replay field efficiency to meet the industry requirements for holographic HUDs.

Holographic HUDs
This section focuses on holographic HUDs for safety applications on public road. HUD setups including the manufacturers of optical components are evaluated. Holographic projection methods and associated computing algorithms are described to achieve 3D projections. Holographic HUDs can serve as a defensive technology to promote driver attention, reducing the eye strain of the driver, and provide shorter obstacle visualization times. [72] It is highly desirable to develop holographic HUD technologies for diverse user groups including elderly and disabled populations. [73] By means of human-machine interaction studies, optimized data architecture and differentiation can be analyzed and implemented in HUD to project the sensed objects on the road. A key development area in holographic projections is the obtained image (replay field). [74] An uneven replay field results from the overlap of neighboring pixel fields, creating an envelope that should be compensated. Computational approaches to reduce the uneven replay field and speckle can benefit the holographic displays. [75] Moreover, goal-oriented approach can be used to obtain single objects in the replay field; and hence, in the "eye box." This eye box measures 6 × 6 cm 2 with an overall brightness of 16 000 cd m −2 . [76] It is a precaution toward the ever-growing tendency of superimposing visual information over the real-world scene on the windscreen. The principle of this work is based on minimal interference with driver assistance systems in the driving behavior. This could be monitored via a smartphone application or on the holographic projector itself to reduce driver tasks while using sensors. A collimated optical system needs a predefined exit pupil. As conventional HUDs are fixed, retaining a small exit pupil is challenging. [62] With the introduction of the panoramic display, which can be projected directly in the eye box (90 × 90 mm 2 ) of the driver's personal preference, the requirement of the exit pupil of a diameter of 150 mm can be achieved. [11] Holographic projections can be integrated in automotive HUDs, which require a high image contrast for AR to enable a tunable projection distance for both the near and the far fields. Industrial HUD technologies are limited in projecting floating images on multiple layers, hindering applications in 3D AR. [77] CGH allows for displaying floating 3D objects for HUDs and head-mounted screens. Ultrahigh-definition (UHD) (4K) SLMs can enable increasing the field of view and display size in CGH. [78] Holography is a unique technology to develop virtual and augmented realities. [79] CGHs allow for projecting digital inference patterns through a SLM without the requirement of a real object. [80] Holographic HUDs can project 3D objects directly into the retina to achieve an AR experience.

Lasers and LED Sources
Light sources and their coherence properties in holographic displays have a direct impact on image sharpness and speckle noise. [81] Both temporal and spatial coherence can be affected by the intrinsic spectrum bandwidth, the size, and the optical path of the light source. Spectral bandwidth and source size are key parameters for choosing a light source. The spectral bandwidth should be narrow band (<1 nm) to reduce the blurriness of the replay field image. The blurriness increases with increased spectral bandwidth of the laser. [82]  key requirements for lasers in automotive HUDs. The diodepumped solid-state (DPSS) frequency-doubled Nd:YVO 4 laser (532 nm, 340 mW) showed cost-effectiveness and elimination of electromagnetic noise, which resulted in less speckle and noise in the replay field images ( Table 3). [83] The He-Ne laser (λ = 633 nm, 5 mW) with a collimated beam and long coherence length can also be utilized. [84,85] In terms of temporal coherence, laser diodes and DPSS lasers achieved increased sharpness in the replay fields compared to LEDs and microLEDs (mLEDs). [86] The spatial coherence value and the image sharpness values are directly proportional. [81] Hence, light sources with a low temporal coherence and high spatial coherence are suitable for application in holographic displays to generate sharp replay field results with minimal speckle noise. Such light sources could be superluminescent light emitting diodes, mLEDs, and LEDs with a broad spectrum and spatial filters ( Table 4). [81] LED backlight units have been incorporated into an optical setup, which allowed for emitting light in a narrow cone shape to collect the light around the driver's eye box area. [81] The implementation of 50 LEDs (23 W) resulted in a brightness of 13 398 nit in the driver's eye box with a total homogeneity of 97% white and 83% black measured over 25 points. [91] This process is desirable for specific optical setup configurations, such as when using Liquid Crystal Display (LCD) panels with lenticular lenses, which require a fixed angle of luminescence from the light source. The optical setup design was simulated in Optic-Studio (Zemax) to achieve an angular spot size of <1 arcmin to match the human eye resolution to >60 px deg −1 . This optical setup configuration resulted in high brightness and the suppression of stray light in the replay field.

SLMs
The phase modulation criterion instead of an intensity modulation is crucial for choosing a SLM. This is based on higher diffraction efficiencies of the phase-modulated devices than the amplitude-modulated. [72] The efficiency of the replay field projection increases when the SLM modulates the phase instead of the intensity. [98] The SLM offers i) at least a 2π modulation, ii) a fast response time, iii) high resolution, iv) large fill factors, and v) multilevel phase modulation for the maximum efficiency. [72] The SLM should have adequate resolution of at least 512 × 512 px to reproduce an accurate replay field. The higher the frame rate of the SLM, the less the neighboring pixels on the same hologram generated. Studies focused on antiparallelly aligned electrically controlled birefringence mode liquid crystal on silicon (LCoS) SLMs. [99] A LCoS SLM consists of two substrates: i) Indium Tin Oxide (ITO) glass as the electrode and ii) Complementary metal-oxide semiconductor (CMOS) backplane with a resolution of 1280 × 770 px. There are pretilted angles of ≈5° which are created by a coating on both substrates of the alignment layers. The two substrates are aligned in antiparallel directions resulting in a liquid crystal (LC) cell with 1.7 µm space in-between. The liquid crystal material (JTS-1) should fill the complete cell to a birefringence level of 0.212 and dielectric anisotropy of 6.7. [72] A LCoS binary SLM was developed for industrial displays. [100] This device, however, only showed a π modulation. A 4K SLM (Jasper Display) has a phase of 2π, which could facilitate optical field manipulation such as the creation of complex beam structures with a total conversion efficiency of 49%. [78] The LCoS technology offers reflective coating and 4K display with a panel resolution of 3840 × 2160 px. The SLM was operated at 633 nm with a He-Ne laser (5 mW). Linear polarizers allowed for controlling the axial polarization of the laser source. The polarizers were crucial to find the so-called liquid crystal switching angles. Generally, light polarized at 90° would be parallel to the axis of the liquid crystal and the SLM. A minimized zero-order and the clearest image appears when all polarizers were placed at 45° to align with the liquid crystal switching angle of the SLM. [78] Metasurfaces covered by ultrathin plasmonic structures can be utilized to control the phase of light. [101] There are different types of metasurfaces: i) coupled thermal antenna arrays for the manipulation of spontaneous or stimulated emission [102] and ii) ultrathin devices consisting of a monolayer of subwavelength (UV-vis to Near-Infrared (NIR)) plasmonic resonators for controlling the wavefront of electromagnetic waves. [   the geometric metasurfaces with arrays of plasmonic nanorods with spatially varying orientations showed properties of phase control based on the geometry of their phase profile. CGHs were created based on geometric metasurfaces that reached diffraction efficiencies of 80% at a wavelength of 825 nm and a broad bandwidth between 630 and 1050 nm. [101] Other investigators demonstrated multilayer subwavelength metal elements capable of forming a CG phase-only hologram in the IR region (10.6 µm). [104] Introducing metamaterials can lead to greater control over the propagation of light, which might result in more compact, efficient, and versatile optical components. The metamaterials implemented consisted of gold disks (75 nm), rectangular patches, and I-beams of various dimensions centered in a cuboid SiO 2 substrate (1 µm × 1 µm × 500 nm). Although gold is a suitable metal at the wavelength of operation (560-590 nm), SiO 2 substrate with aspect ratio (AR) (0.25) showed significantly more absorption than gold substrate (AR = 0.2). The replay field had an image of 150 × 150 px each consisting of 5 × 5 metamaterial elements. The metamaterial structures could be modified to manage polarization; hence, controlled anisotropy or even chirality may be incorporated to the material resulting in more degrees of control with minimal manufacturing complexity. [104]

CGH Image Generation
The ability to focus at different depths within the replay field results and layer various objects at different distances was created by a virtual Fresnel lens based on the Fresnel diffraction theory. [111] The holograms are computationally created, and are layered at different distances within the observer's replay field from near-eye and extending to the far field. [112] Positive lenses could be utilized as Fourier transformers for phase transformation. [113] A layered hologram based on the angular tilting concept and a graphic rendering approach generated a 3D hologram. In this process, a 3D object was sliced into cross-sectional layers, which were placed perpendicular to a chosen viewing direction. Additionally, depth cues, occlusion, and shading were incorporated due to the graphics rendering approach. Parallel computing algorithms enabled a computation time of 176 ms by using a graphics processing unit card. [114] In Figure 3a, the concave lens is at distance d away from the input, which is a complex transmission function U(x,y). The lens is illuminated by a collimated monochromatic laser light (λ = 633 nm) and amplitude 1. [115] Figure 3b,c illustrates different lens arrangements in the holography setup.
No lens is present in Figure 3b, and the light path is undiffracted. In Figure 3c, the light path is shown with a convex lens. Figure 3d,e shows a comparison between a conventional optical holographic display system and a Fourier transform optical system with a virtual lens. [116] The Huygens-Fresnel diffraction method can be applied on the basis of Maxwell's and the Rayleigh-Sommerfeld equations on the propagation of the electromagnetic field. [117] The propagation of an electromagnetic field M from a random point 0 r  to an arbitrary point r  can be described as the surface integral [118] where k = 2π/λ is the wavevector. There is a theory, described as the Huygen's wavelet theory, which elaborates on each point of light becoming a secondary emitter of spherical wavelets. [119] This theory leads the preservation of straight wavefronts, which cannot be sustained unless the waves pass through an aperture. All edges of the aperture will cause a lack of emitters leading to an overall distortion of the wavefront diffraction. [120] Two approximations result from the nontrivial integral of Equation (1): the Fraunhofer approximation for the far field, and the Fresnel approximation for a near or intermediate field. [121] The first approximation, the Fraunhofer field, occurs with [121] which is evaluated to a point approaching infinity. With a standard Taylor expansion, it can be demonstrated for a large distance z [122] , Equation (3) represents the Fourier transform of the field function at the diffraction plane, where k/z describes the relative size of the resulting hologram. Any utilized SLM can only define a retardation phase of the propagated field. This implies that a given phase at the diffractive plane should be capable of creating an intensity profile at the far field. However, there is no analytical solution to obtain the phase at the hologram plane creating an intensity profile in the Fourier domain. [123] The analytical solutions optimizing the process are known as phase retrieval algorithms such as the Gerchberg-Saxton (GS) method. [80] To obtain a solution in the intermediate field, the Fresnel approximation is required. [124] This presents a necessity for a further term in the Taylor expansion. An additional approximation shows a multiplicative quadratic term [122] , , d d The Fresnel diffraction is a convolution process which can be expressed as where M(u, v) is the initial field and h is the impulse response of the optical system, which is given by the quadratic term where h is referred as the Fresnel diffraction lens. To find the focal point with the used diffraction pattern, the variable z can be set as the focal distance. [121] Fresnel holography utilizes the encoding of an object. The encoding process is generated with Fresnel diffraction and interferences with a reference wave. Equation (5) performs a transformation of the convolution process into a Fourier transform, which allows numerical fast Fourier transform (FFT) algorithm implementation. [125]

Holographic Image Generation Algorithms
Fresnel virtual lens or a Gabor superlens algorithmic methods can be utilized to superpose 3D holographic replay fields by a summation process of their direct viewed replay field focused at infinity. [126] These methods utilize an advanced GS algorithm for phase retrieval to optimize the replay field results. The replay fields were focused at infinity to superimpose the summated results in the method in order to create floating replay field reconstructions. [127] To achieve this result, the focal length of the CGH was set to infinity and the projection (viewing) angle was kept constant. The field of view was limited based on the superposition method because this effect could have been only reached with a specific viewing angle (4.85° and 14.58° depending on the focal length of the lens). To reduce the zero order in the replay field results, the rotations of the two polarizers in the setup were modulated. A layering technique achieved 10-20 µm layers for 1024 × 1024 px resolution by introducing a sparse template holographic fringe pattern for 3D objects. [128] In another study, UHD 3840 × 2160 px resolution was attained based on a layering approach in real time with 24 frames s −1 . [116] This technology has the potential to be utilized in AR applications. [112] This algorithm-only approach was utilized to display replay field results with multiple depths by using a stochastic gradient decent (SGD) algorithm with complex loss function. [129] Previous algorithms calculated the loss function generally by comparing the obtained amplitude of the reconstructed image with the target images at different depth planes. [130] Hence, the optimization time is a crucial challenge. The SGD uses the complex loss function for optimization. This reduces the optimization time by comparing the complex loss instead of the amplitude-based multidepth loss. Hence, the optimization time of the algorithm is close to the singledepth optimization time. [129] Fourier and Fresnel methods have been used to generate CGHs. [131] The Fresnel method generates arbitrary large images with object depths. [132,133] The Fourier method utilizes the kinoform technique to project the replay field (projection area) at the focal plane of a lens. [134] A CGH can be formed through the phase retrieval, [135,136] the wavefront recording plane, [137] the multiview, [132] and the polygon-based [138] algorithms such as triangular of hexagonal. With the polygonbased method, full-parallax CGHs with four billion pixels can be obtained. [139] Phase-only CGHs provide high diffraction efficiency for the reconstructed replay field. Holograms are capable of producing 3D colored images with a modified GS algorithm. [140] By utilizing a Fresnel incoherent correlation holography, experimental and digital colored holographic images were obtained. [141] This cost-effective holography system could be applied in automotive HUDs. Multilevel phase modulators based on LCoS with reflective coating, 4K UHD, and a panel resolution of 3840 × 2160 px have a greater efficiency due to the absence of conjugate orders and residual zero orders. [142] The robustness of holographic projectors lies in the setup as only one lens is used without a polarizer. [143] Another advantage is the control over the output image due to the continuation of the holographic projection, which is an important criterion in selecting the optical setup used for applications in cars. A study proposed layered holographic projection methods to display UHD 3D images in direct-view floating mode. [144] A laser and a LED at 532 nm were used. Computational algorithms were created based on phase retrieval algorithms and a virtual Fresnel lens to project several holograms onto the LCoS display panel of a 4K SLM. Additional codes such as the Wirtinger derivatives and hardware in the loop were developed for generating multilayer 3D near-eye projections by adding multiple retrieved holograms with an independent Fresnel zone plate into a single hologram. [145] Using Wirtinger derivatives allows for posing the phase retrieval problem as a quadratic problem, which can be minimized with first-order optimization methods. [146] This method facilitates rendering 3D scenes. The HOLOEYE LETO-I LCoS with reflective coating phase-only spatial light modulator with a resolution of 1920 × 1080 px was utilized for the optimization method. The pixel pitch of the SLM was 6.4 µm, resulting in the active area of 12.28 × 6.91 mm 2 . A single optical fiber was coupled to laser diodes (446, 517, and 636 nm), in combination with collimating optics, to illuminate the SLM in the optical setup.  The laser power was controlled by a laser diode controller, and the output was linearly polarized. The SLM had a refresh rate of 60 Hz, which was illuminated by the laser source in a color field sequential manner. This method reached reconstruction quality of 40 dB. Others developed accelerated algorithms by combining coherent ray tracing algorithms with look-up tables (LUTs), [147] combining data compression with LUTs, [148] and by introducing customized computer clusters such as the HORN-6. [149] The developed method outperformed double phase encoding holograms by more than 10 dB across all datasets. [150] Others managed to obtain reconstructions with a He-Ne laser (633 nm, 5 mW) and a 4K SLM with reflective phase modulation. 3D holograms were directly observed floating as ghost images at a different focal distances and sizes. [151] Concave and convex lenses were varied at the output of the LCoS display panel in the direct-view mode to increase the field of view of the hologram up to the size of the beam splitter.
Every image can be represented as an intensity map of pixels. The most crucial parameter which can be varied currently is the phase. The phase controls the replay field projection by regulating the pixel-to-pixel interactions. [152] The phase can additionally control the gratings' interaction in the CGH. [153] This parameter is paramount in the FFT method when creating the CGH because it can be varied to optimize the replay field projections either manually or computationally. When the CGH is created with the FFT, one limiting factor is the bandwidth of this system containing the amount of gratings. [154] Therefore, the phase between 0 and 2π is applied to set the amount of grating within the limits of the bandwidth of the supersystem. A method was developed to optimize the CGH, which was based on the FFT and the GS methods. [121] This method was based on the phase retrieval method to optimize the replay field projection. The GS method was based on a chosen phase map, which was applied to the square root of the intensity map of pixels from the original image with an iterative approach. [155,156] The FFT was based on a discrete amount of pixels (points) to prevent a continuing source of pixels into the FT integral formula. [157] Each pixel had one grating in the original image f(u, v). The inverse digital Fourier transform (DFT) of the original image provided the total amount and the arrangement of the discrete gratings, which could be defined as the sampling of an image. To improve the replay field projection, the grating could be modified with a GS code. [84,158] The DFT and the FFT were different equations leading to the same result; hence, the FFT equation was used for the FT in the GS algorithm. In the DFT, the output determined N points, each consisting of N number of input points, which needed to be multiplied by a complex number. In mathematical terms, the problem can be displayed based on the complexity of the order O(N 2 ). In the general case, M × N input and output points were used. There is a constant factor (N 2 ) targeted to increase the algorithm's speed and performance. [113,159] The use of FFT accelerates the traditional computational process. [119] To reduce the size of the optical setup, algorithms could be implemented as functions for: i) image conversion into a hologram instead of a holographic plate, and ii) integration of a virtual Fresnel lenses to remove at least one lens in the optical setup. Furthermore, algorithms could be used to test optimization and focusing functions: i) GS function, ii) Fresnel function, and iii) scripts for generating multilayer holograms. [116,160]

Functions
The functions evaluate the mathematical expressions. These mathematical expressions are validated within the function's algorithms.
Conversion of 3D Objects into Holograms: Functions have been developed to implement the GS algorithm, which iteratively finds an optimum value for a multiphase hologram. [80] A Fourier transform was taken to approach the target image. The input variables in this function were the initial phase of the hologram, the target image intensity, and the total number of iterations. To start the function, an initialization of the mode is achieved by a for loop. [135] In the for loop script, a hologram is defined with an initial allocated phase. When the FT is applied to the hologram, a new phase of the final replay field result is retrieved. This indicates that the phase of the replay field image is extracted and combined with the target image. [156,161] Algorithmic Virtual Fresnel Lens and Gabor Superlens: A virtual lens creates a virtual propagator system that enables an experimental and theoretical study of beam propagation and light diffraction without physically moving any optical component. [162] Virtual lenses simplify the optical setup by decreasing the required number of lenses. An angular spectrum method that used Fresnel and Gabor superlens zoneplates was implemented on a phase-only SLM (PLUTO, HOLOEYE Photonics). The SLM had a reflective LCoS microdisplay (1920 × 1080 px). The pixel pitch of the SLM was 8 µm, with 256 grayscale levels. [163] The design of the algorithm included holograms capable of producing 3D images at different distances, where the effective plane of observation could be changed. This process was carried out by changing the encoded propagation instead of physically moving components in the optical setup. The input variables in the algorithm are the size of the grid in x-and y-axes, the physical period of the pixel pitch, the focal point of the target, and the wavelength of the optical beam. [125] A mesh of coordinates in the axes is generated and evaluated, thereafter, the Fresnel and Gabor functions are retrieved. [164] Additionally, an acceleration of the virtual Fresnel and Gabor superlens implementations can be achieved by a Graphics Processing Unit (GPU). [165] Holographic computations integrating at a standard graphics pipeline enables real-time (>90 Hz) data processing.

Scripts
A script is a set of commands developed to test the functions of the written algorithm (e.g., in MATLAB, Python). The following scripts describe each step of the algorithmic testing process.
Test of the Gerchberg-Saxton Function: The output of this script consists of an optimized hologram that reconstructs a target image at the far field. This script has the following steps: i) a general user interface is utilized to retrieve an image, ii) the image is converted to grayscale and normalized, iii) the GS algorithm is executed with a random phase, the target, and 10 cycles, iv) the retrieved hologram phase is normalized and saved on the hard drive, v) the hologram phase is utilized to retrieve the reconstruction through a FT, vi) the retrieved hologram reconstruction is stored on hard drive, and vii) the signalto-noise ratio is calculated.

Test of the Fresnel and Gabor Superlens Functions:
The compiled Fresnel and Gabor functions are run and tested in this section. These two scripts consist of the following steps: i) the Fresnel/ Gabor function is defined with a given SLM resolution, a SLM pixel pitch (a smaller pixel pitch of preferable, refer to Table 5 pixel pitch comparison), one or more light source wavelengths (if two light sources are considered, two-color spatial multiplexing in the field of view can be achieved), and the desired focal distance, ii) the retrieved phase is redefined in the range of −π to π, and iii) the resulting digital image is saved on the hard drive to be implemented in the holography setup. [125,166] Multilayer Holograms: To create multiple layers in the replay field, virtual Fresnel and Gabor superlens functions and a script are applied to define the focal distance for each holographic layer. [56] Furthermore, a single hologram or multiple holograms can be defined. [167] The lens maker's formula allows for recreating the experimental view of a physically applied Fresnel lens. [168] The multiple layers' script consists of the following steps. i) The parameters are defined, including the resolution of the CGH (1080 × 1080 px), the pixel pitch (8 µm), and the wavelength of the excitation source (671 nm). [169] The target complex amplitude distribution is defined as the amplitude ranging from 0 to 1. The optimal angle between the reference beam and the object beam was found to be 1.2°. [166] ii) With known focal lens length, the lens maker's formula is applied to retrieve the focal distances of the zone plates. iii) A for loop is utilized to combine through an addition of the fields of each hologram: 1) the images are retrieved, normalized, and raised according to the dimensions of the SLM (1080 × 1080 px), (2) the GS algorithm is applied with an random initial phase and the image target for a number of cycles (e.g., 10 iterations), 3) a Fresnel zone plate is created and added to the hologram, and (4) the retrieved hologram is added to the previous hologram to create a layer-by-layer effect. iv) The image hologram is stored in the hard drive. [112,170] Hologram Generation: The GS algorithm was the first method to optimize phase retrievals. [85] The algorithm operates by comparing phases of the several original images present in an image and their replay fields. In a CGH, the phase at the SLM and the phase in the replay field result are unknown during the computational GS process. The intensity of the SLM is programmed to 1 in a loop to determine the unknown parameter of the phase of both the original image and its replay field. [166] During this computational process, the GS algorithm allocates a random phase to each pixel of the original image. The FFT is run to obtain the phase parameter in the replay field result. [116] This process is repeated several times, where each time the retrieved phase is allocated by the algorithm to each pixel of the original image, and a different parameter for the phase of the replay field is generated. [171] The GS algorithm represents an iterative process of the FT with specific boundary conditions at the input and output planes. The function at the input plane is retrieved when the target image at the output condition plane is assigned. In the case of an error between the reconstructed image in the replay field result and the original target image, the phase-only function is obtained. [107] Additionally, multiphase retrieval algorithms, which contain several phase-only functions, are used to reconstruct the replay field as the target, where specific Fresnel holograms include the information of the original object. [155,172] Image recreation systems could be achieved through monocular cues and binocular disparity. [173,174] Monocular cues, which were generated with 2D imaging, were visible with one eye, where the occlusion effect used layering to emphasize the object overlap in the fore-and backgrounds. [173] Pictorial depth cues were due to shading and perspective rendering. A dynamic version of the monocular cues was represented by the motion parallax, where objects at greater distance from the observer (>10 m) had less lateral movement than that of closer objects to the observer (<3 m). This method was achieved by the rotation or translation of a scene in 2D with parallax effect in the rendering process. [175,176] Original images could also be recreated through the binocular disparity (stereopsis), which defined a visual process for both eyes when a scene is viewed from slightly different angles. [174,177] This process recreated close to real-life holographic projections due to the meshing process of 2D scenes in one coherent depth-perceived visual response. Different layers could create holograms at different depths. [178] A modified GS algorithm was generated to create CGHs with depth by using a layering technique, which superimposed up to four holographic replay field layers. [163] In other studies, by superimposing layers with an optimized algorithm, the hologram generation time was reduced by 4 times for application in dynamic 3D holographic displays and HUDs. [170,179] The GS algorithm generating phase-only CGHs has fast calculation ability due to an iterative Fourier transform process. [161] The underlying principle of the iterative Fourier transform algorithm is Fraunhofer diffraction, which limits the projection distance to the focal plane of the lens. [82] Hence, this algorithm is limited to the 2D plane as for 3D images, the light propagation takes place in the near-field diffraction with the angular spectrum, the virtual Fresnel or Gabor superlenses, and the Fresnel diffraction. [180] An angular spectrum method was developed to improve the image quality of multiplane holographic projections. [8] The benefits of the angular spectrum method lie within the computation  of diffraction pattern with a high numerical aperture and working in both paraxial fields and nonparaxial fields. [181] The Fresnel diffraction method is limited to the paraxial fields. [182] The angular spectrum method however suffers from speckle noise and image diffraction distance disturbances. [8] A three-step hybrid algorithm was developed for multibeam steering. This algorithm generated reconstructed output fields with flexible power ratios. In addition, the algorithm balanced multiple indicators such as diffraction efficiency, crosstalk power, and the reconstructed root mean squared error. [183] This result was achieved with a three-step combination of the iterative Fourier transform algorithm. A hybrid algorithm was used for error reduction due to convergence abilities and fast computation speed. Compared to other algorithms, the iterative Fourier transform algorithm was susceptible to initial states and was not capable of solving a multitude of tasks. [116] Hence, the iterative Fourier transform algorithms were enhanced with a global optimization process with fitness functions to enhance search direction in order to tackle the multiple objectives problem. [184] However, the speed of the overall algorithm reduced with the introduction of the global optimization algorithm. [183] Both hybrid algorithms in combination generated a hologram, which had continuous phase due to the iterative FT algorithm and balanced multi-indicators quantifying the hologram to 256 phase levels. The hybrid algorithm was a two-step iterative process, where the first one erased the background and in the second iteration enhanced the image depth of the chosen object by focusing on the signal-concentrated area. [183] The depth of both iteration steps could be enhanced with the root mean squared error function. [185] Another algorithm was developed based on the hybrid iterative FT algorithm with one additional step. The image plane was separated into the background area, the signal area, and the additional noise area. Each of the area was constrained by different amplitudes (Figure 4a,b). [8] An iterative FT algorithm was produced to create holograms that diffract the light into arbitrary 2D intensity profiles. [82] The intensity distributions outperformed the frequently used algorithms by one and two orders of magnitude in accuracy and roughness, respectively. The central technique used in the algorithm was mixed-region amplitude freedom. It consisted of a single mixing parameter controlling the relative distribution of optical power in two subsets: the signal region and the noise region of the output plane. Phase freedom was allowed everywhere in the output plane of the replay field, but amplitude freedom was permitted solely in the noise region to emphasize the accuracy (Figure 4c). [184] This algorithm controlled the intensity in a bounded 2D subset of the focal plane with greater accuracy (>10%) than the GS algorithm. This single plane intensity profile algorithm could only be applied to create 2D arbitrary optical traps (Figure 4d). [186] Figure 4d illustrates the creation of 6 holograms able to diffract light into the 6 chosen 2D intensity profiles. To create the mentioned holograms, a mixed-region amplitude freedom (MRAF) algorithm was utilized. [187] Adv. Mater. 2022, 34, 2110463   Figure 4. Algorithmic holograms. a) Multiconstrain angular spectrum algorithm showing the signal (α), the background (β), and the noise (γ). b) Intensity profiles with odd-shaped beams achieved with two SLMs. Reproduced with permission. [189] Copyright 2014, Springer Nature. c) The holographic double-plane HUD demonstration at near and far planes: 9.0 m on the left, near plane: 1.5 m on the right. d) Target intensity profiles to analyze the performance of the MRAF algorithm. The field of view (i-iv) is a 200 × 200 px and for (v) and (vi) 400 × 400 px. The grayscale represents intensity, where black corresponds to the regions of zero intensity. Reproduced with permission. [186] Copyright 2008, Optica Publishing Group. e) Intensity profiles for the MRAF algorithm based on the test target profiles from (d). Reproduced with permission. [186] Copyright 2008, Optica Publishing Group. f) Numerical and optical reconstruction results with Fourier domain segmentation method when viewing from (i, iv) left, (ii, v) center, and (iii, vi) right viewpoints. Reproduced with permission. [190] Copyright 2019, Optica Publishing Group.
Algorithms were elaborated to spatially segment the CGH into multiple hologels, which captured the parallax views by perspective projections from the corresponding viewpoints. [188] These were based on stereograms such as spatially multiplexed 2D parallax views of a 3D scene. Spatial segmenting was achieved through computer graphics rendering techniques. These algorithms had the disadvantage of being view-dependent. The depth performance decreased during optical reconstruction process and needed multiviewpoint rendering. [170,175] An algorithm was created to calculate photorealistic 3D CGHs with Fourier domain segmentation without the paraxial approximation, which was present in the stereogram-based algorithms. [163] Segmented angular spectrum layer-based algorithm calculated partitioned elements in the Fourier domain. This segmentation could reduce aliasing errors by limiting the bandwidth of the transfer function and by extending the limited region of the conventional angular spectrum. This algorithm was compatible with computer graphics rendering techniques, which provided a smooth motion parallax with occlusion effect in the replay field results (Figure 4e). Figure 4e shows the intensity profiles of the 2D objects from Figure 4d, proving that the MRAF algorithm was able to improve the results by two orders of magnitude on roughness compared to the GS algorithm and by roughly one order of magnitude on the accuracy of the results. [186] The numerical simulations Figure 4f(i-iii) and the optical experiments Figure 4f(iv-vi) demonstrated photorealistic 3D images with depth information. [190] The speckle noise appears due to the random and erratic phase distribution as significant intensity fluctuations occur in the reconstructed image. [164] The phase of the image directly influences the intensity of the points in between the sampling range as the amplitude may not be controlled on finite sampling points in the hologram generation process. [191] The output light field is determined by the interpolation of the sampled points, which are based on the finite size of the generated hologram. [8,183,192] To reduce the speckle noise, random-free phase (linear, quadratic, constant, and conical) is referred to an initial phase within the iteration algorithm. However, the speckle noise cannot be removed completely.
Achieving high accuracy replay field results requires the ability to arbitrarily manipulate the spatial full field information of a light beam. [184] This was accomplished by manipulating the amplitude and phase of the incoming light beam with two phase-only SLMs and without any phase iterative algorithm. This method generated multiple collinear orbital angular momentum beams, Laguerre-Gaussian beams, Bessel beams, and arbitrary beams with odd-shaped intensities. [189,193] Figure 5a,b shows the working mechanism of arbitrary manipulation of amplitude and phase of a light beam with two cascaded SLMs. The polarization direction of the input light was set to 45° with respect to the x-direction, which was the working direction of the polarization-dependent SLM1. After the polarizer of 45° direction has been passed by the light beam, the light reached SLM2. In both systems, the light propagates from a collimator laser (1550 nm) (Figure 5a,b). The collimated light passes through the polarizer (Pol1) and a half-wave plate (HWP1). The crucial components in the optical setup are two phase-only SLMs. As the phase and amplitude might change upon light beam propagation between the two SLMs, lenses are added to develop two imaging systems with unit magnification. The lenses facilitate the cancellation of spurious phase delays caused by the light propagation between the two SLMs in the optical assembly. Figure 5a shows a one-lens imaging system. The distance between the two SLMs is 60 cm. The focal length of the lens was 15 cm. In this system, the plane of the SLM1 images the plane of SLM2 with unit magnification. This optical arrangement can retrieve both amplitude and phase of the SLM1 plane at the position of the SLM2 plane. Figure 5b illustrates a 4f optical system with two lenses placed between the two SLMs separated by 60 cm and focal length of 15 cm. This optical arrangement also images the plane of SLM1 to the plane of SLM2 with unit magnification. In the optical assembly in Figure 5a(ii), no additional radial phase distribution was used as opposed to the optical system in Figure 5a(i). Figure 5b(i,ii) illustrates the mechanism of the arbitrary full-field manipulation of a light beam technique used to independently control the phase and amplitude. The symmetrical version of the iterative FT algorithm with phase and amplitude freedom and flexible weightings of the fitness function in the Genetic algorithm were utilized in combination. Figure 5b(iii) shows the basis for the hybrid algorithm. When a phase-only hologram was uploaded onto any position of the LCoS chip, multiple or a single waveband was reflected and switched from the shown input fiber port to an arbitrary multiple or single output port. The optical signals were angularly dispersed in the direction of the y-axis. [183] Figure 5c(i,ii) shows a multicasting hybrid algorithm principle, where a Gaussian beam travels from the fiber coupling microlens array through a grating. The optical signals with multiple wavelength channels were observed to angularly disperse in the y-axis direction. The optical signals were projected onto the LCoS chip that was placed at the back focal plane of a collimating lens and a cylindrical lens. This flexible power weighting allows for optimization of multicasting holograms, ensuring that intensity distributions match with the port location. [184] The circular symmetry of the zone plate algorithm calculated the CGH at high speed. A computer graphics technique evaluated the circular symmetry of the zone plates as fringe patterns formed by interferences of spherical waves originating from a single point cloud and a reference plane wave. When running Nishitsuji and co-workers' algorithm on Intel Core i7-4790K Central Processing Unit (CPU) (4.0 GHz), the computational time for one CGH was 12 ms. [194] This speed can cope with real-time reconstruction and interactive information presentation for multiple viewers.

Application Examples of AR and Machine Learning (ML) in HUDs
This section presents current and future application examples of 3D holographic HUDs. The possibilities of 3D HUDs to achieve augmented and mixed reality projections are discussed. The role of AR in HUD applications is to increase driving efficiency by reducing the time spent by the driver to look at the built-in display or the windshield for navigation purposes. AR can serve as natural medium for the driver to increase road obstacle alertness and reduce driving stress levels with virtual objects projected into the driver's field of view.
Additionally, this section discusses the outlook in machine learning algorithms for gesture recognition and automatic obstacle detection.
AR enriches the 3D world by superimposing computer-generated virtual objects in real time into the environment. These 3D virtual objects can serve as additive information to warn, Adv. Mater. 2022, 34, 2110463 Figure 5. Schematics of different optical system arrangements. a) Experimental setup with two SLMs. (i) One-lens imaging system. (ii) 4f lens imaging system: half-wave plate (HWP), collimator (Col), polarizer (Pol). Reproduced with permission. [189] Copyright 2014, Springer Nature. b) Principle of the arbitrary full-field manipulation of a light beam with two phase-only SLMs, which control both the amplitude and the phase. (i) Phase-only manipulation.
(ii) Independent manipulation of the phase and the amplitude. Reproduced with permission. [189] Copyright 2014, Springer Nature. c) Hybrid algorithm illustrated with a Gaussian beam traveling from a fiber coupling microlens array through a grating. Reproduced with permission. [184] Copyright 2020, IEEE.
inform, or entertain the user. Current AR technologies do not require stationary displays as carriers, instead they can be embedded into the real world with a holographic optical system. Spatially embedded virtual objects contain or transfer data about associated and known real objects, locations, or events. With AR technologies, data have the potential to generate personalized projection layout based on the driver's needs and preferences; hence, the urge of gazing at in-car displays could be reduced. Enhanced driver perception can be achieved with the combination of objects or events at their inherent location, thus condensing the displayed information. A study was conducted with 20 participants within a high-fidelity driving simulator to compare the fully embedded gesture recognition system to a traditional head-down display interface. [195] A reduced collision probability of 90% was found on a motorway compared to the traditional HUD system. A multimodal HUD interface with gesture recognition was proposed to enhance human responses in situations of danger. [196] The gesture control was based on cell division for increased accuracy (Figure 6a).
AI fast deep learning algorithms can detect hidden road objects and holographically project 3D images into the driver's gaze. Machine learning algorithms can predict complex road scenarios and elaborate a safe travel plan for the vehicle with a single convolutional neural network based on the open source Caffe deep learning library. [197] AR can allow for the incorporation of driver and safety-related information into the automotive domain such as lane highlighting and gesture control (Figure 6b,c). However, an abundance of information offered to the driver within a modern automotive environment requires the driver to multitask, overwhelming the cognitive load that increases the probability of collision. [195]

Application Examples of Virtual Shadow Casting HUDs
This section discusses examples of 3D HUD applications, which include virtual shadow warning systems that can be observed by both the driver and other road participants such as pedestrians, bicyclists, and other vehicle drivers. Virtual shadow warning systems may increase situation awareness and reduce traffic accidents. Stereovision-based algorithms have been demonstrated in experimental outdoor studies under several weather conditions such as rainy, foggy, and snowy during both day and night. [198] Additionally, virtual warning systems are based on computer vision, which also provides vehicle detection, distance estimation, and a realtime data stream from mounted cameras by using monocular vision. The real-time data transfer can be implemented for automotive HUDs. The time effectiveness of the data stream is based on processing of 25-28 fps when using one monocular camera on a Core i5 2.7 GHz Personal Computer (PC) with 8 GB of Random Access Memory (RAM). HUDs can be utilized for AR pedestrian warning systems by using in-vehicle volumetric HUDs that provide spatial virtual shadows. A user interface was proposed for cross traffic alert systems by using Adv. Mater. 2022, 34, 2110463 Figure 6. Gesture control recognition schematics. a) Right-handed finger gestures for system automation for HUDs with AR content. Reproduced with permission. [196] Copyright 2020, Oxford University Press. b) Road obstacle detection with machine learning algorithms. c) 3D Augmented reality HUD navigating through public roads.
an AR HUD compatible with both the driver's cognitive process and physical reality of driving environment (Figure 7a). [17] This design involved casting virtual shadows of approaching obstacles through an AR HUD (Figure 7b). [199,200] The calculated minimum distance for AR holograms for urban traffic is 5 m (18 km h −1 , 1 s driving time) with a field of view of 40°, and this distance for highway traffic is 80 m (144 km h −1 , 2 s security distance time) with a field of view of 20°. [201] Current near-eye HUDs achieve a field of view of 45° but limit the driver's eye box to 7 × 7 mm 2 with a resolution of 1280 × 1024 px. Recently, an eye box size of 25 × 36 mm 2 was achieved in the far-field HUD setting with a resolution of 3840 × 2160 px. These inclusive studies serve as a basis for the development of virtual shadow warning systems in the transportation sector. Existing pedestrian collision warning systems incorporate either visual, auditory, or both alarms to inform the driver about upcoming obstacles. However, these methods do not give insights into the exact location of the obstacles, which are crucial for the driver to respond accordingly. Volumetric HUDs capable of providing spatial information based on conformal graphics integrated into the HUD can result in increased safety and security in roads. [200,202] The design idea can be leveraged to create reliable and safer AR interfaces for drivers.

Panoramic Holographic Projections
This section discusses the experimental strategies to increase the driver's field of view for the 3D holographic projections. Increasing the field of view results in situation awareness without causing driver distraction. A wide field of view in LCD displays originates from the diffused light sources in the back of the LCD panel, which randomizes the phase of propagating waves. The holographic setup using only coherent light sources such as lasers or LEDs has a limited field of view based on the SLM display size and pixel period in the holographic setup.
A conventional LCoS SLM is 10 × 20 mm 2 , has a pixel period of 8 µm and a corresponding diffraction angle of 1.9°. [204] Several studies introduced techniques to overcome the limited field of view by introducing multiple SLMs in different arrangements. [170,181,194,[204][205][206] Figure 8a-c shows the relation between a single SLM or multiple SLMs in different configurations. Figure 8a depicts the concept of capturing the entire field of view through the entire bandwidth of the SLM. When the viewer moves away from the designated field of view, the bandwidth for reconstructing the objects decreases. This reduces the sharpness of the image in addition to making some parts of the object invisible. This dilemma is caused by the narrow diffraction angle and the size of a single SLM. Figure 8b shows a holography configuration that has multiple SLMs. A planar arrangement of the three SLMs does not increase the field of view for the viewer as only the SLM in the middle is contributing to the entire reconstruction of the object. To increase the field of view, a curved arrangement of SLMs is elaborated (Figure 8c). In a circular configuration, the field of view increases significantly compared to the planar one. This configuration allows for implementing 3D holographic displays as the reconstruction appears binocularly, where the viewer can rotate around the reconstructed object within the field of view of 24°. [204] Figure 8d shows that the beam splitter aligns the active areas of the SLMs (without the frames) for a continuous field of view. A cone mirror can be utilized to direct the light into all SLMs. A beam splitter was placed between the SLMs to create a   [203] Copyright 2018, Springer Nature. b) Visualization of the operability of the current industrial HUD in a driving situation. Reproduced with permission. [205] Copyright 2018, Springer Nature. gap illusion. Commercial SLMs are mounted in frames, which discontinue the field of view. A cone mirror is utilized to direct light into all SLMs. The SLMs are slightly tilted to prevent the optical components from blocking the view as the reconstructed objects appear slightly above the setup. In volumetric displays, each volume element in a 3D scenery emits visible light. This ability to project volumetric autostereoscopic images has application in automotive HUDs. [207] To expand the limited field of view by a few degrees, which is limited by the use on one SLM, a curved array of twelve SLMs was developed to reduce the spatial bandwidth of SLMs and to obtain a greater number of data points compared to a single SLM (Figure 8e). [181] From the theory of holographic stereograms, it is vital to restrict the angular spectrum of the object wave. [208] Holograms were generated to form triangle-based mesh-modeled 3D surface objects. [209] The angular spectrum equals the object wave, which passes through a viewing window within holographic stereograms. [210] The mathematical representation of the light field radiated from a unit triangular facet is shown with surface diffusiveness, the so-called occlusion. [136] Figure 9a shows two hands composed of 2436 triangles. To project a triangle meshmodeled 3D object, an incident He-Ne laser was used to modulate a phase-only SLM (Epson L3P06) with pixel size of 12 µm. The 3D object was optically Fourier transformed by a singlet with a focal length of 50 mm.
A Charge-Coupled Device (CCD) camera (Kodak Megaplus ES 1.0, pixel size = 9 µm) was used to capture the image light fields at different defocus points. Figure 9b-e illustrates the geometry and mesh analysis of the chosen object with reference frames. The surface diffusiveness factor was set to 5 and the phase distribution was designed to generate shade effects. The amplitude (Figure 9f(i)) and phase (Figure 9f(ii)) distributions of the angular spectrum are presented. The size of the generated hologram was 501 × 501 px. [209] Figure 9f(iii,iv) presents the simulated results with focus on the right and focus on the left hands, respectively. Figure 9f(v,vi) presents the experimental results. [209] Figure 9f illustrates numerical (ii-iv) and experimental (v, vi) reconstructions validating a 3D model, which confirms the monocular cues (defocus effect). [211] The angular spectrum equals to the object wave, which passes through a viewing window in a generated holographic stereogram. Figure 10 shows the 3D object arrangement and the definition of central directions corresponding to the central frequencies of the angular spectrum in the local view. In the dynamic holographic stereogram, every central direction was rotated at the interval of an angle equivalent to 0.705°, generating 36 possible views when 12 SLMs were adopted. Figure 10a illustrates the object arrangements and the 36 viewing angles with a combined field of view of 39.7°. Figure 10b shows the numerical simulation results from the viewing directions (left, center, right). Figure 10c presents the experimental results from 4 different viewing windows, each having a different field of view with visible zero order in the middle of the image. [181] A conical surface with anisotropic diffusion characteristics for incoming light rays was created to project a 3D   [204] Copyright 2011, Optica Publishing Group. d) Optical setup using multiple SLMs in circular configuration with a beam splitter to align the active areas of the SLMs without discontinuation. Reproduced with permission. [204] Copyright 2011, Optica Publishing Group. e) Dynamic holographic stereogram achieved by a curved array of several SLMs. Reproduced with permission. [181] Copyright 2008, Optica Publishing Group.
scene with the help of 288 projectors spaced at 1.25° (7.4 mm) pitch. [212] Dynamic holographic stereogram with several SLMs had a field of view of 22.8°. To expand this number, a method was developed to project floating 3D images on a flat and empty tabletop surface by utilizing static components (Figure 11a). [206] This method allowed multiple viewers to observe the projected 3D image from any angle, not limiting the field of view without the need of 3D glasses. [206] This technique provided full-color animations with interactive contents to be projected for mixed reality applications. The illumination process was reproduced by a conical screen and multiple projectors, providing a horizontal parallax in circular (Figure 11b). The viewer's eye observes the slit-like parts of each image projected at any position of the annular viewing area of the table. Hence, the eye collects fractional slit-like images from different projectors and assembles an appropriate image from the perspective of the eye at each viewpoint. [213] With the circularly arranged projectors and the conical anisotropic rear-projection screen, 288 multiperspective images were used to reconstruct the 360° 3D images, providing parallax without glasses. 3D models were taken as a base to project 3D objects (h = 5 cm, w = 10 cm).
Adv. Mater. 2022, 34, 2110463 Figure 9. Triangle-based mesh-modeled 3D surface objects. a) Projection image of mesh-modeled two hands directed toward the viewer. b) Triangle mesh-modeled 3D object of two hands. c) Triangle mesh surface-polyhedron objects in the global coordinate system. d) Local coordinate system of the kth unit triangle. e) Relationship between the global and the local coordinate systems. f) CGH of the 3D mesh model: (i) amplitude and (ii) phase distributions of the angular spectrum, respectively, (iii, iv) simulated results with focus on the right and focus on the left hands, respectively, (v, vi) experimental results. Reproduced with permission. [209] Copyright 2008, Optica Publishing Group. A 360° tabletop electronic holographic display was developed. [214] This system had enlarged 3.2 in. holographic images and a 45° oblique viewing angle, which was not visually distorted. This system was created through four synchronized high-speed digital micromirror displays. These were optically calibrated to comprise a two-by-two multiple-vision panel, facilitating the holographic image enlargement together with time-division multiplexing of the holographic image content. The optical layout of the designed system was composed of parabolic mirrors and an aspheric lens to deliver the enlarged optical images. Figure 12a,b shows the effect of the inserted aspheric lens between the parabolic mirrors with a ray tracing analysis. An integrated center of rotational axis, an aspheric lens (L8), resulted in a focused light beam.
A multiprojection of lenticular displays was developed to construct 50 dense views. [215] This was achieved with one lenticular lens pitch covering 5.333 subpixels, a screen resolution of 1920 × 1080 px, and a maximum clear displayed depth of 24.8 cm. This technology generated 3D images by multiple view flat-panel displays, which were superimposed on a single screen using a multiprojection system. The eye boxes of the flat-panel 3D display were produced in the pupils of the projection lenses and imaged to the viewing space with a screen lens. Sixteen flat-panel 3D displays with 16 views were used to generate a multiview display counting 256 views. The resolution was 256 × 192 px and the screen size was 26.2 cm, showing a full-color 3D object (5 × 10 cm 2 ) rendered from 3D models. [216] The horizontal interval of the eye boxes was 1.3 mm. This could have a positive effect for accommodating the HUD within the car as the lenticular arrays have the potential to project the laser light within a certain angular range for the viewer. The benefits for implementing such lenses could accommodate the HUD for a better fit in the car. Additionally, the driver would only see the HUD object in a certain area of the head position to prevent distraction.

Holographic Video Displays
This section introduces video projection capabilities for holographic HUDs. Video projections allow for perceiving road obstacles in full depth and within a 360° view. Video projections enable the driver to assess the hidden road obstacles and react accordingly. Holography allows for creating multidimensional images at different depths as overlayered objects for 3D AR applications. [178] Algorithms can add an infinite number of layers to facilitate the AR experience in the replay field in full color. [217] This can be achieved without visual fatigue or glasses. A 3D femtosecond laser technology was fabricated with a high-aspect ratio of up to 15 nanopillars (w = 390 nm, l = 780 nm) meta-atom with different heights (3.40-4.45 µm) to reduce shadowing effects. [218] The results achieved a hologram (2.5 × 2.5 mm 2 ) consisting of 2000 × 2000 px. In terms of holographic video projection, 3D motion pictures were projected to 2.5 mm 3 with one SLM and two parabolic mirrors. [194] A hologram generation time of 12 ms was achieved by employing 4 CPU cores. [194] High-resolution images (25 frames s −1 ) were taken from objects (football, basketball) to create videos. An orbital angular momentum diffuser array with random phase function sampled each image frame (Figure 13a). [218] A highresolution (10 000 × 10 000 px) complex-amplitude orbital angular momentum multiplexed hologram was generated.
Adv. Mater. 2022, 34, 2110463 Figure 11. Construction of the 360° 3D images. a) Light-field reproduction of 3D images using the conical screen technique with multiple projectors. b) 3D objects projected from 3D models at the tabletop using the conical screen technique with multiple projectors. Scale bar: 5 cm. Reproduced with permission. [206] Copyright 2016, Optica Publishing Group.
The numerical results and optical results are shown as a holographic video display (Figure 13b,c). Each video contained 100 image frames at 260 px per 25.4 mm. [218] By duplicating and adding phase shifts to the optical path, including eye tracking to compensate for the low diffraction angle, and to increase the eye box and size of HUDs, large arrays of SLMs have been assembled. [219] Industrial devices for spatial light modulation such as microelectromechanical systems, bulk-wave acousto-optic and liquid crystal modulators present challenges for integration in holographic video projections. [220] A major obstacle is the high cost in the mass manufacturing of the SLMs. Low-cost SLMs were fabricated with aggregated temporal bandwidth of more than 50 bln px s −1 . [221] The SLM assembly exploited the guidedwave phenomenon. This phenomenon specializes in anisotropic mode conversion to eliminate the zero-order undiffracted light spot and tunable wavelength filtering to generate simultaneous and superimposed modulation of color signals (Figure 14a-c). [221] Figure 14a shows a holographic stereogram created with a single laser light (red) anisotropic waveguide modulator with an overall display of 35 × 20 mm 2 . Figure 14b shows the single channel and Figure 14c illustrates a superimposed (red, green, blue (RGB)) stereogram. The subsystem components of the SLM consisted of an anisotropic leaky-mode coupler, which had a proton-exchanged channel waveguide on a LiNbO 3 substrate with a transducer within one single end. [222] This waveguide is only able to hide the light in one polarization due to its anisotropic behavior. The transducer generated surface acoustic waves when excited by a radio frequency signal, which propagated collinearly with the light trapped in the anisotropic waveguide. [223] Large outputs creating persistence of vision were generated with anisotropic mode couplers. This allowed the development of holographic video projections in an analog device. The viewing angle, the frame rate, the image extent, and vertical resolution can be interchanged within the bandwidth. [221] Currently, a frame rate of 9523 frames s −1 and 2 28 different holographic frames are achieved through high-speed dynamic laser beam modulation and space channel metasurfaces. [224,225] For real-time video projections in HUDs, the continuous depth sensation and accuracy in the replay field have a profound impact. An optical setup was utilized in combination with a deep-learning-based CGH pipeline consisting of 4000 pairs of RGB-depth images. [226] The optical setup was capable of synthesizing a photorealistic color 3D hologram from a single RGBdepth image in real time. Figure 14d presents a 2D target image for testing the spatial resolution of the optical setup of 1920 × 1080 px at 60 Hz. Figure 14e shows the convolutional-neuralnetwork (CNN)-predicted hologram with antialiasing double phase method.
To incorporate holographic video projection in automotive HUDs, the requirements for safety, security, and comfort should be assessed. There is a trade-off between the 3D display size and the field of view of the driver resulting from the spatiotemporal resolution of industrial displays. To bridge this trade-off, a technique of combining digitally designed holographic optical elements with digital holographic projection has been developed. [227] An enlarged holographic image (10 × 10 cm 2 ) with 77 bln px of wavefront information was achieved. The advantage is that the display size and the field of view for the driver can be designed independently; hence, the enhancement of the display size can accelerate the adoption of holographic video projection in automotive HUDs. Another crucial parameter for the adoption of automotive holographic video HUDs is the hologram generation time. By combining double random phase encoding techniques with optimized random phases, fast encryption and decryption were achieved. [228] Color phase-only holograms with a LCoS SLM was generated in HUDs. To address the limited field of view of the driver, a holographic wide angle display (306 × 161 mm 2 ) at a 700 mm distance was generated. [229] This setup included a single Fourier lens imaging system, a 4K SLM, a frequency filter utilizing complex coding, and nonparaxial diffraction.

Conclusions and Future Directions
Customizable 2D windshield displays have been developed to enhance the information intake of the driver. [112] However, the impact of AR on the driver while operating a vehicle on public roads is not fully understood. [230] Full AR in 3D with depth perception has been created to project the visual information on the road within the driver's gaze in AR applications. [231] Layered architectures can add virtual objects matching in size and distance with real-life road objects into the driver's field of view. [160,170] To display road information, LiDAR sensing can leverage both image and 3D point cloud information, but also can provide accurate moving object detection and grid detection for localization and mapping. [3,19] LiDAR can be incorporated in future 3D point cloud data systems and into accurate navigation maps. [232] New opportunities in HUDs range from design factors to functionality to the integration of new content connected with the urban environment into the driver's space. Further analysis of the placement of the content on the windshield and as AR projections in the driver's field could include behavioral studies, neurodiversity studies, and system engineering studies. The content could be projected according to the driver's understandability of the provide personalized information, overall safety, and user satisfaction. Ultrafast switching holographic AR video displays may project 360° road objects into the driver's field of view by using high-bandwidth metasurface holograms. [233] The physical mechanism of a 64-level complexamplitude-based orbital angular momentum multiplexing holography technique with 3D metasurfaces can independently control phase and amplitude of the transmitted light. [218] Current industrial modulators used for holographic video HUDs have limitations such as i) high cost, ii) low diffraction angles, Adv. Mater. 2022, 34, 2110463 Figure 13. Replay field of results showing the physical mechanism of complex-amplitude-based orbital angular momentum multiplexing holographic applications in video displays. a) Algorithm of a complex-amplitude orbital angular momentum multiplexed hologram utilizing image frames of "football" and "basketball" videos at two image planes. R 1 and R n represent different random phase functions. b) High-resolution (10 000 × 10 000 px) complex-amplitude orbital angular momentum multiplexed holograms. c) Optical results of the holographic video displays. Reproduced with permission. [218] Copyright 2020, Springer Nature.
iii) suboptimal scalability for mass manufacturing, iv) relatively low bandwidth, v) persistence of quantization noise, vi) unwanted diffracted orders, and vii) the zero-order undiffracted light spot. The cost and optical system complexity of holographic HUDs originate from the compensation for the issues regarding the zero-order spot.
Layered 3D holographic projections could be optimized through both the computational and the experimental approaches. The accuracy of the replay field images could be enhanced with a time-efficient algorithm to spatially slice the hologram into holographic elements (hologels) by using multiple viewpoint rendering and providing motion parallax with an occlusion effect. [170,185] The slices can provide accurate depth cues with computer graphics rendering techniques for producing high-quality 3D images at a minimal computing load. The optimization of the computational algorithms is a promising solution as layering techniques show limitations in replay field size. Existing layering algorithms can be integrated with the dynamic 3D holography with high frame rates to produce video projections in the visible range. [225,234] To achieve applications of holographic video HUDs in vehicles, an important step is the secure information management. For example, 3D hologram projection techniques can be combined with optical information security. An optical security technique can be realized with real-time encryption-decryption in combination with experimental optical projection methods to generate color holographic videos. [235] Personal driver data can be integrated into point cloud storage methods to share vehicle and driver data with the smart urban environment. This, however, would require faster algorithm processing methods than GPU systems and accuracy of the replay field video projections should be enhanced. Legislative requirements must be met for the resolution of the field of view of driver for 2D windshield projections. AR HUDs can be integrated into in-car display systems with secure requirements such as wide field of view to detect obstacles and estimate the driving situation. Future legislative requirements should be developed for safe automotive video and AR HUDs to prevent the driver not to suffer from fatigue due to changing views. Different optical component arrangements such as the incorporation of several SLMs and additional convex lenses allow for increasing the field of view for the driver. A panoramic field of view of 100° and an Figure 14. Photorealistic 3D holograms. a) Monochrome holographic stereograms made with a single channel anisotropic waveguide modulator in a display (35 × 20 mm 2 ). (i) Intended output projection, (ii) zero-order reflection, (iii) unwanted conjugate image projection, (iv) higher order image projection and quantization noise, and (v) diffracted higher order resulting from modular pixel structure. Reproduced with permission. [221] Copyright 2013, Springer Nature. b) Holographic stereogram reproduced with red laser light. Reproduced with permission. [221] Copyright 2013, Springer Nature. c) Holographic stereogram with different colors created by superimposed modulation of red, green, and blue lights. Reproduced with permission. [221] Copyright 2Springer Nature. d) A 2D target image for testing the spatial resolution of the optical setup. Reproduced with permission. [226] Copyright 2021, Springer Nature. e) A photograph of the CNN-predicted hologram with antialiasing double phase method. The insets (yellow and blue) show the magnified regions. Reproduced with permission. [226] Copyright 2021, Springer Nature.
all-round view with holographic video projection of 360° field of view are achievable. This is a crucial step toward a full obstacle assessment and could be incorporated into legislative requirements.
Holographic HUDs can aid in providing safer and inclusive transportation. The implementation of liquid-crystal lenses with varying focal distances, the so-called "zoomable HUDs" are promising directions to address multilayered AR systems in HUDs. [77,236] Furthermore, the integration of parallel processing for algorithms could address remaining challenges of the hologram generation process in terms of computation time. [237] Positive measures can be implemented for equal opportunities on public roads for people with visual and movement impairments toward inclusive transportation. [238] Finally, ML could be utilized in HUD for intelligent collision avoiding, visual enhancements, and support for people with neurodiverse conditions. [239] HUDs integrated in the transportation sector with various features including ML, virtual shadow casting systems, and metamaterials could realize customizable, transformative image projection capabilities to enhance road safety.