Head-Mounted Displays
SCI-FI GADGETS OR MUST-HAVE TECHNOLOGY?
In 1977, George Lucas’s sci-fi thriller Star Wars provided viewers with some of the greatest movie action ever, played out in space battle scenes between Imperial TIE fighters and the Rebel Alliance’s X-Wing fighters. One advanced technology available to the Rebel pilots was a targeting sighting device attached to their flight helmets. Pilots would lock their target sight on an enemy fighter and fire weaponry, blowing the enemy away in dramatic explosions (in spite of the real fact that except in movies, sound does not travel in the vacuum of space).
As these futuristic Star Wars gadgets wowed audiences around the world, the U.S. Army was developing the real thing for the AH-64 Apache attack helicopter. However, the system developed for the Apache was more than just a target sighting device, such simple targeting sights having been used by the Army on versions of the AH-1 Cobra helicopter and by the U.S. Navy and Marine Corps in their Phantom fighters since the early 1970s.
The Apache visual device, known as the Integrated Helmet and Display Sighting System (IHADSS), was the first fully integrated head/helmet-mounted display (HMD) fielded by the Army. The IHADSS HMD consists of a fully functional flight helmet to which is mounted a monocular optical device that can present to the pilot’s eye combinations of aircraft symbology (e.g., heading, torque, altitude, etc.), a targeting crosshair, and pilotage imagery that originated from a forward-looking infrared (FLIR) sensor mounted on the nose of the aircraft. The IHADSS has been successfully fielded since the early 1980s.
The phrase helmet-mounted display and its acronym HMD have taken on a wider meaning in the past decade. There is a general trend to recognize that all HMDs do not have to be part of, or attached to, a helmet. While this is the wide-ranging scenario in military applications, it is possible to have HMD designs that are more simply headmounted via a headband or harness. So, while this article is focused on military HMD designs that are inherently helmet-mounted, it is to be recognized that head-mounted designs and applications exist.
A functional HMD has three basic components, with a fourth required if head aiming is needed:
• A mounting platform or sub-system.
• An image source.
• An optical system that delivers the imagery from the source to the eye(s).
• An optional head/helmet tracking system.
The mounting platform in a military application is usually a helmet. In integrated HMDs, the helmet is designed as part of the HMD. However, while performing the additional function as the supporting platform for the HMD, the helmet must still provide all of the basic protection that is the original purpose of the helmet. In less demanding environments, the HMD may be attached to the head via a simple headband or harness.
The image source is the device that displays symbology and/or forms an image of the outside world that is being viewed on the HMD. In the AH-64 IHADSS HMD, the image source is a miniature, 1-inch, cathoderay- tube (CRT). More recently, miniature versions of liquid-crystal-displays (LCDs) and electroluminescent (EL) (typically 1-inch in diagonal size and sometimes referred to as microdisplays) displays are the display of choice due to their low power requirement, small volume and decreased weight, as compared to the CRT.
Once the image source reproduces the scene of the outside world, it needs to be optically provided to the eye. This is achieved through the use of various combinations of lenses and reflecting elements. In some designs, the viewing optics is a simple magnifier. In other designs, this optical system can be very complex. This is especially true when the image source is located on the side or top of the helmet/head and has to be delivered to the eye(s). The miniature CRT (with its cable that can be seen exiting at the rear of the HMD) fits up into a barrel-looking, Lshaped housing that contains the lenses that bring the image on the image source up the pilot’s eye. The last element in the IHADSS delivery optics is outside of the housing and is commonly referred to as a “combiner.” While its more technical name is “beamsplitter,” it gets its more familiar name because it combines the image coming from the CRT with the view of the outside world.
The Apache IHADSS HMD has a monocular optical design with pilotage imagery and symbology being presented only to the right eye. This monocular approach was driven by the weight associated with the CRT image source. With the use of miniature displays (e.g., LCD and EL), head supported weight is no longer as limiting a factor and binocular designs have become popular. While twoeyed designs do offer advantages in some applications, it is not a foregone conclusion that two eyes are better than one.
The optional component of a basic HMD is a head tracker. In our example of the IHADSS HMD on the AH-64, the image that is presented on the CRT originates from the FLIR sensor that is mounted on the nose of the helicopter. This FLIR sensor provides the pilotage imagery used to fly the helicopter. In order to point the sensor in the direction of interest, it is necessary to be able to track where the pilot is looking (head position). The head tracking system can vary in technology, but generally consists of two components, one attached to the helmet/head and one attached to the helicopter. In the AH-64, four infrared light-emitting diodes (LEDs) are embedded in the helmet (two on each side), and these small energy sources are monitored by an array of sensors mounted behind the pilot’s seat. As the pilot moves his head, the positions of the LEDs (and hence the head) are used to send signals to the gimbal-mounted sensor, directing its line-ofsight direction.
An externally mounted sensor, as in the AH-64, is not always required or feasible. Night vision goggles (NVGs), the most widely fielded HMD in aviation, have image intensifier (I2) tube sensors mounted directly on the helmet, as an integrated sensor/ display package, eliminating the need for head tracking. While FLIR and low-lightlevel sensors in ground vehicles may require head tracking, obviously HMDs on individual warfighters would preclude the need for this component.
PROLIFERATING ON THE BATTLEFIELD
Over the last two decades, military branches in almost every nation in the world have sought to employ HMDs as a viable technology for presenting heads-up information to its warfighter in the air, on land and at sea. While aviation was not the first to explore HMD use, it was the first to widely adopt it. More recently, armor and infantry applications have emerged and have been expanding rapidly. HMDs even may be flying in space, as NASA has conducted research into implementing HMDs into its next generation of space suits. The HMD would be used to enhance manned extravehicular activities (EVAs) utilizing a space suit and portable life support backpack.
The U.S. Army has led in the fielding of HMDs. In addition to the monocular IHADSS HMD used by Apache pilots, the aviation community has depended heavily on the use of NVGs, a helmet-mounted sensor and display system that is based on the principle of image intensification.
HMD applications in ground vehicles include the fielding of the NOMAD Augmented Vision System (Microvision Inc.) for the Stryker Brigade in Operation Iraqi Freedom. This is a helmet-mounted device that allows the vehicle commander to stand (down) in his hatch and retain a view of the outside world, hence maintaining situation awareness. Similar NOMAD displays have been designed for use in maintenance, repair and overhaul applications. Being able to present vehicle and equipment repair checklists, parts lists, and schematics and diagrams in a head-up format right at the repair site can increase efficiency and reduce downtime.
The NOMAD class of displays uses a miniature LCD that provides 800 by 600 pixels of resolution. Its manufacturer’s specifications include: a brightness (luminance) output of up to 1,000 foot-Lamberts (making it readable under sunlight conditions), 32 shades of grey (an indication of ability to display contrast), weight of less than 200 grams (7 ounces), and an operating temperature range of 32 degrees F to 113 degrees F). The individual infantry soldier is not being ignored in the development of HMD systems. While continuously being fine-tuned, both the Land Warrior (now integrated into the Future Force Warrior Program) and the Mounted Warrior Programs include HMDs in their technology suite. Monocular in design, the HMD will be worn over the dominant eye and provide command and control information, as well as situational awareness. Potential military applications are not limited to the battlefield. As any video gamer knows, HMDs are perfect for simulating environments, making them perfect for training applications. The potential of threedimensional (3-D) fully-immersive military training is most attractive. The advantages of virtual environment (VE) or virtual reality (VR) training include high fidelity, full flexibility, decreased costs, and improved safety.
ENABLING TECHNOLOGIES
New HMD designs, especially those that are small and lightweight enough to be worn by the individual warfighter, have been made possible because of two major enabling technologies: miniature displays and wearable computers. Without the development of miniature flat-panel display technologies, the required dependence on miniature CRTs (but still being heavy and power hungry) would have severely limited, if not totally prevented, the development of HMDs for the individual warfighter. Fortunately, the last decade has been an explosion in the manufacturing capability of miniature displays (typically 1 inch in diagonal size and less than an ounce in weight).
LCD, EL and organic LED (OLED) technologies have dominated newer HMD designs. These miniature displays can provide high brightness, high-contrast images without the penalty of excessive weight and power consumption associated with CRT technology. However, while these newer displays overcome the obstacles to individual warfighter HMD use, each display type has its own drawbacks. For example, LCDs can suffer from off-axis viewing problems (although great progress has been made in this area), presence of pixel defects, and higher cost. In addition, LCDs differ from ELs and OLEDs in that while ELs and OLEDs are self emissive displays (they produce their own light), most LCDs require a backlight. The liquid crystals in LCDs act as tiny shutters that open and close, modulating the light from the backlight. This need for a backlight slightly increases the power requirements for LCDs. However, currently LCDs are the most mature miniature display technology and have the greatest portion of the market.
The second enabling technology for newer HMDs is that of computers. HMDs require computers to create, manage and direct the images produced on the displays. Such computers must be wearable, in other words, being lightweight and having low power requirements. A wearable computer is defined as a small portable computer that is designed to be worn on the body during use. Wearable computers can be either integrated into the user’s clothing or can be attached to the body. U.S. Army programs such as Land Warrior, which will eventually be merged into the Future Force Warrior system, have greatly accelerated wearable computer technology.
Neither of these enabling technologies alone would have been sufficient to support the development of HMDs needed for the ground soldier. It is the combination of the two that has led to successful applications.
BENEFITS AND LIMITATIONS
Technology for technology’s sake has no place on the battlefield. If a device or system cannot provide the warfighter with advantages, it is of little use and will be left behind.
The purported benefits of HMDs include:
• Improved situation awareness
• Expanded night operational capability
• Improved performance of complex tasks
• Increased mission effectiveness
• Decreased workload
HMDs can display night sensor imagery, tactical information, moving map imagery and other information, all in a head-up configuration, while still allowing the warfighter to view the surrounding world.
However, HMD designs and implementation are not without problems and limitations. Virtually, every HMD, concept or fielded system, suffers from one or more deficiencies. These include high head-supported weight, center-of-mass off-sets, inadequate exit pupil, limited field-of-view, low luminance, limited resolution, fitting problems, incompatibility with other equipment and less than universal user acceptance.
Of the design issues with HMDs, none are more troublesome than those associated with the interfacing of the HMD hardware with the characteristics of the human user. It is with the human-machine interface that the greatest problems and mistakes have been encountered and continue to occur. The most prevalent of these is that when the human head is used as a platform, an inherent design characteristic of HMDs, the wide variation in head and face anthropometry makes HMD design a formidable task, requiring creative design approaches that are rich in flexibility and user adjustments.
REALITY
In the 30 years since the release of the first Star Wars movie, HMDs have moved from sci-fi props to real-world military systems. They offer the individual soldier enhanced capabilities and situation awareness. Ideally, HMDs, if designed for a specific application, can be an effective use of technology on the battlefield. ♦