52 March/April 2015 Published by the IEEE Computer Society 0272-1716/15/$31.00 © 2015 IEEE
Visual Computing Challenges
Legibility in Industrial AR:
Text Style, Color Coding, and Illuminance
Michele Gattullo, Antonio E. Uva, and Michele Fiorentino ■ Polytechnic Institute of Bari
Joseph L. Gabbard ■ Virginia Tech
The creation of leading-edge options for interaction between people and technol-ogy plays a key role in the Industrie 4.0 vision, which calls for intelligent resources that enable and support decentralized, reaction-driven, and flexible production systems. In this context, augmented reality (AR) is one of the most suitable solutions. However, it is still not ready to be effectively used in industry, and one crucial problem is the legibility of text seen through AR headworn displays (HWDs).
An overarching goal of our work has been to validate AR as an improvement over current practices in industrial maintenance and assembly operations, along the lines of Pierre FiteGeorgel’s work (see the sidebar “A Survey of Industrial Augmented Reality Applications”).1
Early in our exploratory phases, we performed a pilot test using video-based AR with a large screen display near the operator workbench and a combination of multiple fixed and mobile cameras (see Figure 1).
Participants performed similar operations via two modalities: paper manuals and AR instructions.
We found that the large screen AR instructions significantly reduced participants’ overall performance time and error rate. We assumed that the main additional benefits were due to hands-free operation.
We also posited that AR HWDs could further improve operator performance by reducing completion times and head and neck movements.
Maintenance operations are divided-attention tasks that require operators to repeatedly switch between instructions and their work area. Using traditional manuals or even a large-screen setup, operators must continuously move their heads and chain their lines of sight. On the contrary, with
AR HWDs, instructions and the work area are collocated because information is always presented within the operator’s field of view.
Recently, we have been examining HWD legibility in real industrial settings. The perception of a stimulus (such as text) is affected by both the contrasting luminance and the texture of the stimulus and the background. Thus, AR text discrimination, and therefore legibility, is likely affected by these two factors. In industrial contexts, other aspects may also affect the perceived contrast of AR text, including workspace background, text color, and ambient lighting.
To address this problem, a promising approach for industrial applications is to maximize text contrast in real time via software by adding outlines or billboard backgrounds around the text. To examine this hypothesis, we ran four user studies to examine the legibility of text seen through
AR HWDs using typical industrial backgrounds,
Augmented reality (AR) has numerous applications in industrial settings, but a crucial problem is the legibility of text seen through AR headworn displays. The authors test several variables affecting text legibility and derive guidelines with an emphasis on deriving guidelines to support AR interface designers. g2gat.indd 52 2/26/15 12:31 PM
IEEE Computer Graphics and Applications 53 color coding, and lighting levels. In this article, we briefly describe each study and discuss how we used the results of each study to inform the next.
We then synthesize our findings across these studies and make recommendations where possible to improve text legibility in industrial settings.
User Study Design and Experimental Task
For our legibility experiments, we used a mixeddesign approach similar to that used by coauthor
Joseph L. Gabbard and his colleagues to examine user performance in a text identification task.2 It can be argued that text is one of the most fundamental graphical elements in any user interface, if not the most important. Thus, the experiment we designed abstracted short reading tasks that are common in technical AR industry applications. Specifically, we used a low-level visual identification and search task, which avoided having to account for text semantics (for example, cognitively understanding textual content or meaning) or confounding the results with top-down contextual visual or cognitive processing. We simply evaluated how quickly and accurately users could visually discern English characters via the following four-step experimental task: 1. Scan a set of meaningless, short, random text strings. 2. Identify a target letter contained within Step 1 stimuli. 3. Count the number of times the target letter appears in a second random set of short text strings. 4. Indicate the number of occurrences counted in Step 3.
We asked users to perform this task with various text styles—plain text, outline, and billboard—and colors. The outline style uses a line that marks the outer contours of each letter. The billboard style uses a uniform color rectangle that established a
User portable camera
Engine (real scene)
Figure 1. Interactive video-based augmented reality instructions on a large screen. Augmented instructions significantly reduced participants’ overall performance time and error rate.
P ierre Fite-Georgel provided an interesting survey of industrial augmented reality (AR) applications.1 He suggested that this technology could be used in all phases of the product life cycle.
In the area of product design, AR has been used for many purposes, such as to augment a car mock-up with different light optics to evaluate its in-situ appearance, design piping systems, and plan factories. This tight integration of AR in the design workflow helps close the loop between real and virtual mock-ups, thus creating a more efficient development process.
In manufacturing, AR can be used as a replacement for paper-based assembly instruction manuals. The development overhead of an AR manual can be justified because product life cycles are increasingly shortened. AR can be used to support not only unskilled workers but also highly trained operators who use complex machinery. It can also help train new workers, especially when there are complex procedures to learn.