Sage Journals: Discover world-class research

Abstract

Industry 5.0 represents the next frontier in advancing human-machine interfaces through human-centric approaches. This paper outlines industrial regulations emphasizing operators’ cognitive and physiological capabilities in HMI design for control systems. Through process control cases, we demonstrate how eye-tracking captures operators’ engagement with interfaces and perception of design elements. Fixation, scan path, and saccadic measures effectively evaluate attention allocation and search efficiency. Pupil size, blink rate, and saccade amplitude support in assessing mental workload under changing HMI configurations. Findings remain preliminary; involving experienced operators and diverse scenarios could improve generalizability and promote practitioner engagement to develop human-centered HMIs for industrial applications.

Keywords

eye-tracking human-machine interaction process safety human-machine interface control room human-centered design

Human errors are attributed to around 70% of accidents in process industries (Bhavsar et al., 2017; Mannan, S., & Lees, 2012). Control room operators, who monitor plant operations and intervene during process upsets, are associated with most of these errors (Amazu et al., 2023). Although advanced digital tools, such as industrial sensors and control systems, provide operators with critical real-time data, the growing complexity of integrated processes and the need for continuous optimization increase demands on human-machine interaction (Wanasinghe et al., 2022). This often leads to mental overload in operators (Pan et al., 2017), as seen in the BP Toledo refinery incident, where an overwhelming alarm flood delayed critical responses and led to severe consequences (U.S. Chemical Safety Board, 2022). Operators’ performance in managing process control efficiently and safely depends heavily on the human-machine interface (HMI), especially its design and functionality (Bántay & Abonyi, 2024; Xu et al., 2018).

Industry has developed standards and HMI design guidelines for alarm systems, display design, and control room management (Ikuma et al., 2014). Subsequent revisions of the guidelines have proposed specific design recommendations to reduce cognitive load and support effective human-machine interaction (O’Hara & Fleger, 2020). The forthcoming section provides a brief overview of such guidelines. However, many leading technology providers evaluate HMI design primarily through iterative expert reviews, think-aloud protocols, subjective measures, and usability ratings (Tharanathan et al., 2012). These methods may overlook implicit, non-verbal indicators of cognition and behavior, such as how operators perceive, process, and respond to information under stress. This limits the ability to objectively verify whether the HMI truly meets design requirements aimed at reducing cognitive load. Therefore, a more comprehensive evaluation of HMI design is necessary, one that considers various dimensions, including task performance, workload, and situational awareness (Xu et al., 2018). Researchers particularly emphasize the need to integrate cognitive and behavioral characteristics when studying complex human-machine systems (Naderpour et al., 2015; Nazir et al., 2021). Process safety experts also stress the importance of methodologies that bridge this gap, enabling operators to process information efficiently and respond effectively (Bullemer & Nimmo, 1994; Shin, 2014). The emergence of Industry 5.0, which prioritizes human-centered approaches, further reinforces this notion. Together, these efforts highlight the need for user-centered HMI design solutions that enhance operator capabilities.

As sensor technology advances, eye-tracking has emerged as a promising approach for evaluating operators’ responses while interacting with HMIs (Lin et al., 2003). In control operations, eye-metrics could offer potential for developing performance indicators (Manca et al., 2012; Shahab et al., 2021), improving training design in VR-based settings (Kluge et al., 2014; Srinivasan et al., 2022), and enabling human digital twinning (Balaji et al., 2023). Although these methods deliver cost-effective solutions and demonstrate usefulness (Pavlas et al., 2012), most studies rely on participants who are not professional board operators, which limits the applicability of findings to real-world settings.

This paper provides an overview of specific industry standards and design recommendations that emphasize incorporating operators’ cognitive and physiological capabilities into HMI design. It also outlines design initiatives by major technological powerhouses and provides actionable perspectives for human factors and ergonomics practitioners on how eye-tracking metrics reveal operators’ engagement with interfaces and their perception of on-screen elements and support systems. Representative case studies from process control operations illustrate these applications. By involving actual operators in future research, these preliminary works can be strengthened to support the development of next-generation HMIs that are operator-centric and better aligned with industry needs.

Overview of Industry Guidelines on HMI Design

Advances in computer science have enabled industries to integrate computer-powered video consoles as a central interface in Distributed Control Systems (DCSs), transforming how operators interact with critical systems. However, deficiencies in alarm management and graphic display design have contributed to industrial accident severity. For example, in the Brenham accident in 1992, the graphical display did not present pressure and flow trends in an interpretable format, delaying recognition of a pressure drop and causing an overfill and ignition. In the Gramercy incident in 1996, multiple alarms occurred simultaneously, preventing the operator from evaluating each alarm effectively and delaying leak detection, which led to a substantial gasoline release. Similar incidents at Winchester in 2000 and Kingman in 2004 demonstrate failures in alarm evaluation and display interpretation (National Transportation Safety Board, 2005).

Regulatory bodies and industrial organizations provide high-level HMI design recommendations, including display formats, text styles, code labels, and color coding, to enhance readability, clarity, and salience. These features help reduce operator search time in both normal and abnormal situations. Figure 1 illustrates some of the high-level HMI design guidelines. We reference this information from the PETRONAS Technical Standards 14.00.02 (PTS 14.00.02, 2014), and the Fundamentals of Industrial Control (Coggan, 2005). Both guidelines emphasize developing HMI interfaces that support clear, consistent, and ergonomically sound operator experiences. They recommend thoughtful use of color (e.g., avoiding dark hues and using green, yellow, and red for alarms), appropriately sized text and symbols, and minimal clutter so that operators can quickly spot and interpret critical information. In this context, the Human-System Interface Design Review Guidelines (NUREG-0700, Revision 3) provide comprehensive documentation of high-level HMI design principles (O’Hara & Fleger, 2020).

Figure 1.

Illustration of some of the recommended high-level HMI design guidelines from the PETRONAS Technical Standard and the Instrument Society of America guidelines.

Nevertheless, an important part of HMI design is how users interact with the system, how tasks are performed, and how much workload can affect plant monitoring and control tasks (O’Hara & Brown, 2002). The International Society of Automation (ISA) developed ISA 18.2, which accounts for the interaction between the operator and the interface elements (American National Standards Institute ANSI & ISA, 2009). It is also important to identify interface features that demand excessive attention or cause distractions, which could divert cognitive resources from critical tasks and reduce overall performance (O’Hara & Brown, 2001). Therefore, designers must integrate design principles that address operators’ cognitive and physical needs, ensuring the HMI system supports effective decision-making and minimizes distractions and overload. Although we did not find universal best practices for interface design, we summarize key recommendations that emphasize accounting for operators’ cognitive and physiological capabilities to enhance HMI design and effectiveness in control systems. We sourced these design recommendations from the third revision of NUREG-0700 (O’Hara & Fleger, 2020).

1. Design HMIs to align with cognitive and physiological capabilities while ensuring tasks are engaging and maintaining a balanced workload that supports attention and high performance.

2. Present information clearly and quickly to ensure operators can easily recognize and understand the system status. Evaluate display formats and information density to ensure important details are readily perceived.

3. Assess the impact of new graphics on cognitive load and situation awareness while minimizing visual fatigue. Ensure that any new visual elements do not hinder operator performance.

4. Train operators in task management strategies for high-workload situations and develop test scenarios that assess operator performance and workload under varying conditions.

5. Adopt ecological interface design to align interfaces with how humans naturally perceive and process information.

To address these recommendations, eye-tracking could serve as a practical and objective tool for designers and engineers. It could reveal how users interact with the interface in real time, what draws their attention, what they overlook, and where cognitive bottlenecks occur. These insights could facilitate targeted improvements and help achieve HMI design enhancements following the recommended principles. A more detailed perspective on the application of eye-tracking metrics in human-centered HMI design, along with representative examples, appears in the later sections of this review. Prior to that, we present a brief outline of HMI design initiatives proposed by leading technological powerhouses.

Overview of HMI Design Initiatives by Technological Powerhouses

Technological companies actively develop future-ready HMI designs that are more interactive, user-friendly, and align with the industrial goal of reducing operator search time across all conditions. In fact, these advanced HMI solutions, deployed in real plant environments, achieve measurable improvements in control room operations (ABB, 2025). Figure 2 depicts HMI design initiatives from five of the world’s leading automation and digitalization powerhouses. For instance, Siemens provides “SIMATIC HMIs” (Siemens, 2024) with dimmable, multilingual displays and glove-compatible multitouch, to make operations smoother in diverse environments. Honeywell’s Experion Panel PC (Honeywell, 2021) focuses on quick, localized control through flexible operation modes that could let operators manage tasks efficiently. ABB’s “Compact HMI” (ABB, 2024) develops reusable graphic libraries and “one-click” data access, coupled with alarm shelving, so operators can swiftly identify and address issues. Rockwell Automation proposes “Optix Panel and PanelView” (Rockwell, 2025), which offers improved interface usability through features such as color correction, built-in PDF viewers, and advanced remote-access capabilities, to ensure seamless decision support. Lastly, “Magelis” (Schneider, 2025) performs refinements in HMI design with high-resolution displays and precision industrial interfaces, creating a crisp and intuitive user experience. These solutions address the common goal of reducing search time and cognitive load by tailoring interfaces to real-world operator needs.

Figure 2.

Overview of advanced HMI initiatives developed by leading automation and digitalization providers.

Nevertheless, the HMI design effectiveness is generally evaluated by iterative expert reviews, think-aloud protocols, subjective measures, and usability ratings (Tharanathan et al., 2012). This evaluation approach may overlook implicit indicators of operators’ underlying cognitive state, limiting the ability to objectively confirm that the HMI truly meets design recommendations intended to lower cognitive load. Here, eye-tracking measures could offer additional benefits to the designers, strengthening the design evaluation process. Eye-tracking could provide a more objective way to evaluate how effectively these design choices work in practice. Practitioners can determine whether users notice, interpret, and act on key interface elements as intended. This could make eye-tracking a powerful tool for aligning interface design with real human interaction, mainly offering a deeper understanding of how operators perceive and process critical information on the screen. The next frontier in HMI design lies in developing human-centered interfaces that truly reflect operator perception and interaction (Mourtzis et al., 2023). This aspect is also central in the five design recommendations stated earlier. In the following sections, we use scientific studies as representative cases to show how insights from eye-tracking metrics shape human-centered HMI design.

Role of Eye-Tracking in Human-Centered HMI Design

Eye-tracking has proven valuable in human-computer interactions (Majaranta & Bulling, 2014), especially for understanding visual attention and cognitive processes (Glöckner & Herbold, 2011; Wade & Tatler, 2005). In Figure 3, we illustrate how eye-tracking is being applied at three progressively detailed levels of HMI design in process control rooms. Within the broadest scope, the “Global Interaction Analysis” captures an overall view of how operators engage with the interface, focusing on process tracing to identify broad usage patterns. Moving to a more focused tier, the “Contextual Support Assessment” examines how additional displays and operator support systems influence mental workload and attention allocation across users of different ages and experience levels, thereby accounting for individual differences. Lastly, the “Element-Level Interface Optimization” examines specific on-screen design refinements and their direct impact on cognitive load. This tiered approach ensures that eye-tracking metrics provide insights on both high-level system improvements and precise interface enhancements.

Figure 3.

Hierarchical integration of eye-tracking measures for optimizing HMI design.

A thoughtfully designed interface helps keep the operator’s mental workload at an appropriate level (Parasuraman et al., 2000), while enabling rapid and efficient screen scanning (Niemelä & Saarinen, 2000). Eye movements offer a valuable window into how users interact with interfaces, revealing whether an interface supports efficient task performance or creates usability challenges (Ahlstrom & Friedman-Berg, 2006; Toker et al., 2013). In the following subsections, we discuss the use of eye-tracking in the three-tier HMI design. A summary table outlines key eye-tracking metrics that are referenced throughout the following discussions. Each metric is briefly defined, with its relevance to cognitive workload, attention, or usability in HMI design highlighted (Table 1).

Table 1.

Summary of Key Eye-Tracking Metrics in the Context of HMI in Control Rooms.

Eye-tracking metric	Definition	Implication
Fixation count	The total number of fixations recorded during a task period within an area of interest (AOI)	A higher fixation count may reflect greater information processing during search (Loftus, 1972). It also highlights important AOIs and indicates how effectively the HMI supports visual search (Kovesdi et al., 2018)
Percentage of time spent looking at AOI	Proportion of total gaze duration spent within a defined area	Higher values could indicate which specific interface elements receive greater relative visual attention (Ikuma et al., 2014)
Dwell duration	Duration of all visits within an AOI, including fixations and saccades	Helpful in mapping the operator’s cognitive state. Longer dwell times on correct AOI reflect sustained attention, while shorter dwell times may suggest attention lapses or misallocation (Sharma et al., 2016)
Fixation duration	Duration of each individual fixation within an AOI	Longer fixation durations could indicate that the user sustains attention and exerts greater cognitive effort to process information (Kotval & Goldberg, 1998)
Saccade count	The total number of saccades recorded during a task or period	Helpful in determining search efficiency, with fewer counts indicating a more focused and efficient search (Goldberg & Wichansky, 2003)
Saccade duration	The time taken to move between fixations to complete a saccade (Sharma et al., 2016)	Serves as an indicator of cognitive processing, with saccade duration increasing under higher cognitive load (Das et al., 2018)
Scan path length	The total distance covered by eye movements across the interface	Indicator of visual search efficiency; shorter values indicate greater efficiency (Kovesdi et al., 2018)
Scan path duration	Total time taken by the eyes to complete a sequence of fixations and saccades across a visual scene	Indicator of visual search efficiency; shorter values indicate greater efficiency (Kovesdi et al., 2018)
Scan transition matrix density	It measures the frequency of eye movements between different AOIs on an interface. Lower density means fewer repeated visits to the same AOI and vice versa	Lower density could indicate efficient and focused visual search, while higher density values reveal frequent revisits, suggesting less efficient scanning (Kovesdi et al., 2018)
Saccade amplitude	The angular distance the eye moves during a saccade	Indicates cognitive processing; higher values reflect broader attention distribution (Liversedge & Findlay, 2000). It also negatively correlates with workload (Li et al., 2018)
Fixation/saccade ratio	The proportion of processing time versus searching time (Kotval & Goldberg, 1998)	Indicates the relationship between cognitive processing and visual attention
Blink rate	The number of blinks per second	Higher blink rate could indicate the release of cognitive resources (Gao et al., 2013)
Pupil size	The variation of pupil diameter	Increase in pupil size signals cognitive processing, attention, mental imagery, and emotional arousal, and correlates positively with workload (Bhavsar et al., 2016)

(I) Global Interaction Analysis

Eye-tracking metrics, particularly fixation counts, offer valuable insights into how operators interact with HMIs by highlighting which AOIs capture their attention (Salehi et al., 2018). For example, during a drilling operation where detecting a drill break and kick was critical, heat maps of operator gaze were recorded for both expert and novice drillers (see Figure 4) (Naqvi et al., 2020). The expert’s gaze, as shown in Figure 4(a), was predominantly focused on the Drilling Parameters (DP) on the display monitor, ensuring that key information, such as the Rate of Penetration (ROP) and Return Flow (RF), was closely monitored to detect abnormalities quickly. In contrast, the novice exhibited a scattered fixation pattern with insufficient attention to ROP and an overemphasis on less critical areas like the Manifold Panel, resulting in a detection delay around 5 times longer than that of the expert. These findings suggest that process tracing through eye-tracking can inform HMI configuration improvements, such as repositioning less important panels further and using subtle animations to highlight key parameters. This, in turn, could help novices to perform critical, time-sensitive tasks. The effectiveness of fixation counts in usability testing and enhancing operators’ situational awareness has also been demonstrated in other studies (Sanfilippo, 2017).

Figure 4.

Use of heat maps of fixation counts to evaluate the efficacy of HMI design through operator interaction with AOIs (features of this figure are adopted from the original version published by Naqvi et al. (2020)).

Another eye-tracking metric, the percentage of time spent looking at AOI, reveals that under high workload, operators spent less time on the main display and shifted attention toward the faceplate and alarm bar (Ikuma et al., 2014). Unlike fixation counts, which only indicate how often an operator looks at an AOI, the percentage of time spent looking at an AOI accounts for total viewing time on that area, providing insight into the overall attention it receives. Findings by Ikuma et al. (2014) suggest that the percentage of time spent looking at AOI on key interface elements offers a practical and objective metric to assess and refine HMI performance and its usability.

In the same regard as investigating attention on interface elements, dwell duration could be considered a more informative eye-tracking metric that captures not only where operators look, but also how long and how often they engage with important areas. Dwell duration captures repeated visits and sustained attention, where prolonged dwell duration may indicate that an operator is struggling with a decision. A high percentage of time spent on an AOI alone cannot distinguish whether the attention is sustained at a single glance or distributed across multiple glances. This distinction can be captured through dwell duration.

To put this into context, we provide an example depicted in Figure 5, where participants are tasked with increasing the feed flow to the distillation column in response to a flow reduction from a continuous stirred-tank reactor (CSTR). Appropriate attention to the control valve (V201), gauges for feed flow (F105), and temperature (T104 and T106), alongside the alarm summary and trend pane, is important for this task. The arrangement of these AOIs in HMI is particularly strategic for achieving the correct execution of tasks as soon as possible. The dwell duration plot in Figure 5 shows the prolonged focus on F105, its trend, and the alarm summary pane after the alarm at 26 seconds enabled quick recognition of the abnormal state. Afterward, frequent gaze shifts between the V201 slider and the F105 trend during control actions ensured precise monitoring and adjustment until the process stabilized (Sharma et al., 2016). This confirms that dwell duration provides a more refined understanding of operator attention, capturing not only where but also how long and in what pattern critical AOIs hold focus, making it a valuable metric for optimizing HMI design to enhance situational awareness and response efficiency.

Figure 5.

Eye-tracking dwell duration analysis during disturbance response, highlighting operator attention shifts across critical interface elements (features of this figure are adopted from the original version published by Sharma et al. (2016)).

These are some examples, while other studies have demonstrated the effectiveness of more advanced and sensitive metrics derived from eye-tracking for evaluating HMI design. For illustration, association and salience metrics derived from eye-tracking data provide distinct insights into HMI design effectiveness. The association metric quantifies the alignment between scenario-relevant AOIs and system elements, ensuring that the critical information is positioned for optimal visibility. Higher association values indicate a more substantial alignment between interface elements and task-relevant data. The metric is calculated based on the angle between the scenario and AOI vectors, where a smaller angle corresponds to a value approaching unity, signifying an improved association (Shahab et al., 2021).

The salience metric identifies interface components that are frequently referenced across multiple scenarios, aiding layout optimization. It measures the relative distance of an AOI from the “origin” in an asymmetric plot, where lower values indicate higher relevance (Shahab et al., 2021). For instance, the trend panel, which displays real-time data trends, serves as a consistently referenced AOI owing to its role in tracking variable changes. A salience metric approaching zero suggests that an interface element is integral across operational contexts. Refer to Shahab et al. (2021) for further details on the equations and calculations.

(II) Contextual Support Assessment

As mentioned earlier, the contextual support assessment examines how supplementary displays or software-based support systems influence HMI usability. From this perspective, a case study on the Chemical and Volume Control System (CVCS) used eye-tracking to compare two progressively advanced HMI configurations against a conventional HMI, evaluating their effectiveness in improving visual search efficiency, attention, and mental workload. HMI-1 is the conventional configuration. HMI-2 integrates a hybrid control interface with instrumentation and process control, and computer-based procedures that show upcoming steps, along with overview HMI displays. HMI-3 combines a hybrid board with a dedicated computerized operator support system (COSS) (Kovesdi et al., 2018).

Figure 6 presents the percentage change in eye-tracking metrics to evaluate the visual search efficiency of new HMIs, using HMI-1 as the baseline for comparison. HMI-2 significantly reduced visual search efficiency, as evidenced by longer scan path length and duration, higher fixation and saccade counts, and higher scan transition matrix density. HMI-3 demonstrated better visual search efficiency than HMI-2 but remained less efficient than HMI-1, as indicated by higher relative values of these eye-tracking metrics.

Figure 6.

Percentage change in eye-tracking metrics indicating visual search efficiency for HMI-2 and HMI-3, relative to the baseline HMI-1. (Stats in this figure are adopted from the original version published by Kovesdi et al. (2018)).

Moving beyond these visual search efficiency findings, both HMI-2 and HMI-3 were found to reduce cognitive effort compared to HMI-1. Specifically, pupil size decreased by approximately 4% and 30%, fixation duration dropped by 39% and 24%, while blink rates increased by about 1.4 and 1.25 times, for HMI-2 and HMI-3, respectively (Kovesdi et al., 2018). In terms of attentional allocation, participants showed significant attention to the AOI displaying the operator support system in HMI-3, as reflected by higher fixation counts and longer dwell durations on the AOI. This indicates that while using HMI-3, operators relied more on the support system than on their search efforts, and this reliance probably reduced their cognitive load. Other studies support this argument, showing that when operator decision support systems were added to process control HMIs, participants fixated significantly more on the support system’s suggestions, as revealed by eye gaze heat maps (Abbas et al., 2025).

In a similar context, one study compared three overview display configurations: (a) Heat Map: a grid of color-coded tiles showing process deviations; (b) Surface Chart: a topographical view where deviations form peaks and valleys; and (c) Visual Thesaurus: a gauge-based interface using bars, balls, and dials (Shi et al., 2021). The Surface Chart display yielded higher click accuracy, faster response times, and significantly greater fixation duration, count, and fixation/saccade ratio. Since both the fixation duration and fixation/saccade ratio reflect more cognitive processing and less visual searching, this indicates that the Surface Chart supports efficient attention to process information.

The authors in another study analyzed how auxiliary input devices in HMI systems affect cognitive load and situational awareness (Shi & Rothrock, 2022). Using the touchscreen interface, participants showed shorter fixation durations and counts, which indicated lower cognitive workload and improved situational awareness. Their lower fixation/saccade ratios suggest they spent more time searching than processing the information. Older adults had longer saccade durations, indicating more cognitive processing with touch screens, but higher saccade amplitudes, suggesting lower cognitive workload when handling abnormal situations. However, their fixation behavior mirrored that of younger adults. Hence, this presents preliminary evidence that saccadic measures may be more sensitive than fixation metrics to mental workload changes induced by variations in auxiliary HMI inputs among older adults (Shi & Rothrock, 2022). Nonetheless, this should be interpreted as exploratory, requiring further research to draw a definitive conclusion. Other studies compared gaze-based control with keyboard input for switching between multiple control screens (Smith et al., 2015).

(III) Element-Level Interface Optimization

For element-level interface optimization, we illustrate a case study where researchers redesigned the Chemical and Volume Control System (RCV) interface elements using cognitive psychology principles to enhance intuitiveness and operator-friendliness (Chen et al., 2019; Yan et al., 2017). Figure 7 shows some of the design reconsiderations introduced by the redesigned version, which include the following:

(1) More salient solid lines depicting active flow that can quickly provide operators with the information they need.

(2) Consistent grouping of task-relevant parameters and variables to ease the readability of information.

(3) Closely arrange the valve information to be compared to the process data.

(4) Removal of extra information that could easily distract the operator.

Figure 7.

Hypothetical redesigning of RCV process-computer interface using cognitive psychology principles (features of this figure are adopted from the original version published by Yan et al. (2017)).

Based on the NASA-TLX index, researchers showed that the redesigned RCV interface reduced mental demand by 14%, demonstrating its improved usability (Yan et al., 2017). Notably, eye-tracking measures revealed that operators using the redesigned interface exhibited significantly shorter fixation durations, smaller pupil sizes, and increased blink rates. These eye-based metrics not only align with the NASA-TLX findings but also highlight their potential as a useful real-time indicator of mental workload arising from element-level interface designs.

Other Relevant Research

This paper explores the effectiveness of eye-tracking metrics in designing and evaluating HMIs, providing representative examples from studies. It is worth acknowledging that numerous other studies have applied these metrics to assess variations in cognitive workload due to task difficulty. For instance, Iqbal et al. (2018) observed more dispersed eye movements, reflected in increased gaze entropy, under higher cognitive demand. During process disturbances, studies reported more prolonged fixation and saccade durations, indicating their potential for monitoring mental effort (Das et al., 2018). Huang et al. (2019) found that task difficulty was associated with more frequent fixations and longer dwell times. Studies also observed larger pupil sizes during mentally demanding tasks and smaller sizes during tasks with lower cognitive demand (Bhavsar et al., 2016; Noah & Rothrock, 2015; Wu et al., 2020). Ha et al. (2016) derived fixation-based metrics to infer operators’ thought processes during nuclear control room tasks. However, this review did not address these studies in the discussion, as it focused specifically on the application of eye-tracking metrics to human-centered HMI design rather than their secondary use in inferring operator states due to task complexity.

Limitations for Industrial Application and Future Directions

While eye-tracking metrics demonstrate strong potential for evaluating HMI design, certain findings highlight the need for contextual sensitivity during interpretation. For example, studies have reported both increases and decreases in saccade amplitude and velocity under varying task demands and performance levels (Abbas et al., 2025; Fan et al., 2022; Shi & Rothrock, 2022), reflecting variability based on task complexity or individual strategies. Similarly, although pupil size is often used as an indicator of operator state, some studies have found it insensitive to certain conditions (Lin et al., 2003). Others have linked pupil size to attention shifts and arousal caused by error, indicating it may not always reflect increased cognitive processing (Gao et al., 2013). Additionally, studies also reported non-linear pupil behavior with memory load (rising initially but decreasing when memory becomes overloaded) (Granholm et al., 1996; Peavler, 1974). These mixed findings do not undermine the utility of eye-tracking measures but present an important caveat that successful industrial implementation requires careful calibration and adaptation to context.

One limitation in the discussed studies is that most demonstrate the efficacy of eye-tracking metrics in various HMI design systems involving non-industrial participants, which may not fully reflect the responses of industrial operators (Abbas et al., 2025; Shahab et al., 2022). Validating these metrics with actual board operators remains essential to establish their practical relevance and reliability. Accordingly, future work could extend these preliminary efforts by involving real industrial operators.

For example, the study by Sharma et al. (2016) can be extended, where dwell patterns of experienced board operators during task execution could reveal expert cognitive strategies, offering a means of understanding decision-making processes that verbal reports cannot easily represent. Furthermore, analyzing visual attention across AOIs during process events may help identify decision-making heuristics of experts that inform evidence-based benchmarks or training guidelines for novices (Ikuma et al., 2014; Naqvi et al., 2020). Likewise, the approach proposed by Shahab et al. (2021) can be applied to quantify the alignment (association metric) and importance (salience metric) of interface elements based on actual operator behavior. This enables designers to tailor HMI layouts to real-world usage. This ensures critical information is optimally placed and frequently attended components receive appropriate emphasis, enhancing both intuitiveness and operational relevance.

Furthermore, real-world process control operations encompass a broad spectrum of operator profiles, including individuals with 30–40 years of experience (Koskinen et al., 2024). Eye-tracking metrics have demonstrated sensitivity to individuals of varying ages, as shown by Shi & Rothrock (2022), who reported that saccade durations and amplitudes significantly differ in older adults. Applying their approach to experienced board operators would further test their validity in capturing expertise grounded in years of plant-specific operational knowledge and ensure applicability to complex industrial settings.

Conclusion

Eye-tracking metrics offer valuable insights for enhancing HMI design by capturing operator perception of design elements, whether it be overall process tracing or element-level optimizations. We present findings grounded in scientific research demonstrating that fixation-related, scan path, and saccadic measures effectively evaluate attention allocation and search efficiency. Pupil size, blink rate, and saccade amplitude provide additional support in assessing mental workload variations influenced by changes in HMI configurations. However, various findings discussed in this paper should be interpreted as exploratory, and for meaningful impact on industries, studies must involve real board operators to ensure insights translate to actual work environments. As Industry 5.0 prioritizes human-centered innovation, leveraging insights from eye-tracking metrics could help design HMI systems that empower operators rather than overwhelm them.

Footnotes

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research,authorship,and/or publication of this article.

Funding

The authors greatly acknowledge the financial support and professional assistance provided by the Centre of Advanced Process Safety (CAPS),Universiti Teknologi PETRONAS,Malaysia,and the European Union’s Horizon 2020 research and innovation program under Marie Sklodowska-Curie grant agreement No 823904 (Project: Enhancing Human Performance in Complex Socio-Technical SystEms,ENHANCE).

Asher Ahmed Malik is a PhD researcher specializing in process safety and loss prevention,and human factors and ergonomics. He completed his Master’s in 2019 at Universiti Teknologi PETRONAS,Malaysia,focusing on CFD-based consequence assessment of fire,explosion,and gas dispersion hazards. His current research aims to enhance human performance in control rooms by integrating psychophysiological modalities with AI algorithms to predict operators’ behavioral attributes and provide objective assessments for improved training efficiency and human reliability. ORCID iD:

Azizul Buang is a senior lecturer in the Department of Chemical Engineering at Universiti Teknologi PETRONAS (UTP),where he teaches process safety courses at both undergraduate and postgraduate levels. Before joining UTP,Dr Azizul worked as a process engineer. His research interests include process and occupational safety as well as wastewater treatment technology. He has also served as a committee member of the National Management Committee for IECEx (Explosive Atmosphere) Malaysia.

Salman Nazir is a Professor in the Department of Maritime Operations at the University of South-Eastern Norway,where he leads the Training and Assessment Research Group (TARG). He is also an adjunct Professor at Nord University Business School. Prof. Nazir has focused his research on human factors,training methods,virtual reality training simulators,performance assessment,learning processes,complex socio-technical systems,safety,crisis management,and situational awareness. ORCID iD:

Risza Rusli is an Associate Professor at the University of Doha for Science & Technology. She has actively involved in the research and development of measurement tools for inherently safer design,consequence analysis for carbon capture technology by applying CFD and risk-based tools considering human factors for the application of chemical processing,oil and gas industries. She has provided professional courses related to Process Safety and QRA consultancies for oil and gas companies in Malaysia.

Annette Kluge is a Full Professor of Work,Organizational,and Business Psychology at Ruhr University Bochum,Germany. She obtained her Diploma in Work and Organizational Psychology from the Technical University of Aachen and her doctorate in ergonomics and vocational training from the University of Kassel,Germany,in 1994. Her expertise includes human factors and ergonomics,training science,skill acquisition and retention,safety management,organizational learning from errors,and organizational forgetting. ORCID iD:

Mardhati Zainal Abidin is an Assistant Professor in the Department of Chemical Engineering at the University of Doha for Science & Technology. She is an Associate Member of the Institution of Chemical Engineers (IChemE),a Professional Technologist with the Malaysia Board of Technologists (MBOT),and a Graduate Engineer with the Board of Engineers Malaysia (BEM). Her research interests include inherently safer design,consequence modeling,quantitative risk assessment,nanomaterial risk,and process modeling and simulation.

References

ABB (2024). Compact Product Suite. Retrieved March 27, 2025, from https://search.abb.com/library/Download.aspx?DocumentID=3BSE063717

ABB (2025). State-of-the-art control room improves productivity at Victrex polymer plant in UK. Retrieved March 27, 2025, from. https://new.abb.com/control-systems/industry-specific-solutions/chemical/advanced-control-room-improves-productivity-at-victrex-polymer-manufacturer

Abbas

A. N.

Amazu

C. W.

Mietkiewicz

Briwa

Perez

A. A.

Baldissone

Demichela

Chasparis

G. C.

Kelleher

J. D.

Leva

M. C.

(2025). Analyzing operator states and the impact of AI-enhanced decision support in control rooms: A human-in-the-loop specialized reinforcement learning framework for intervention strategies. International Journal of Human-Computer Interaction, 41(12), 7218–7252. https://doi.org/10.1080/10447318.2024.2391605

Ahlstrom

Friedman-Berg

F. J.

(2006). Using eye movement activity as a correlate of cognitive workload. International Journal of Industrial Ergonomics, 36(7), 623–636. https://doi.org/10.1016/j.ergon.2006.04.002

Amazu

C. W.

Briwa

Demichela

Fissore

Baldissone

Leva

M. C.

(2023). Analysing “‘Human-in-the-Loop’” for advances in process safety: A design of experiment in a simulated process control room (pp. 2780–2787). Esrel. https://doi.org/10.3850/978-981-18-8071-1_p481-cd

American National Standards Institute (ANSI) & ISA . (2009). ANSI/ISA-18.2: Management of alarm systems for the process industries. American National Standards Institute (ANSI), 14–79.

Balaji

Shahab

M. A.

Srinivasan

(2023). ACT-R based human digital twin to enhance operators’ performance in process industries. Frontiers in Human Neuroscience, 17(February), 1–12. https://doi.org/10.3389/fnhum.2023.1038060

Bántay

Abonyi

(2024). Machine learning-supported designing of human-machine interfaces. APPLIED SCIENCES-BASEL, 14(4). https://doi.org/10.3390/app14041564

Bhavsar

Srinivasan

(2016). Pupillometry based real-time monitoring of operator’s cognitive workload to prevent human error during abnormal situations. Industrial & Engineering Chemistry Research, 55(12), 3372–3382. https://doi.org/10.1021/acs.iecr.5b03685

10.

Bhavsar

Srinivasan

(2017). Quantifying situation awareness of control room operators using eye-gaze behavior. Computers & Chemical Engineering, 106, 191–201. https://doi.org/10.1016/j.compchemeng.2017.06.004

11.

Bullemer

P. T.

Nimmo

(1994). Understanding and supporting abnormal situation management in industrial process control environments: A new approach to training. Proceedings of the IEEE International Conference on Systems, Man and Cybernetics, 1, 391–396. https://doi.org/10.1109/ICSMC.1994.399870

12.

Chen

Yan

Tran

C. C.

(2019). Comprehensive evaluation method for user interface design in nuclear power plant based on mental workload. Nuclear Engineering and Technology, 51(2), 453–462. https://doi.org/10.1016/j.net.2018.10.010

13.

Coggan

D. A.

(2005). Fundamentals of Industrial Control: Practical Guides For Measurement And Control (2nd ed., p. 893). International Society of Automation.

14.

Das

Iqbal

M. U.

Bhavsar

Srinivasan

(2018). Toward preventing accidents in process industries by inferring the cognitive state of control room operators through eye tracking. ACS Sustainable Chemistry & Engineering, 6(2), 2517–2528. https://doi.org/10.1021/acssuschemeng.7b03971

15.

Fan

Liang

Cao

Pang

Zhang

(2022). Effects of noise exposure and mental workload on physiological responses during task execution. International Journal of Environmental Research and Public Health, 19(Issue 19), 12434. https://doi.org/10.3390/ijerph191912434

16.

Gao

Wang

Song

Dong

(2013). Mental workload measurement for emergency operating procedures in digital nuclear power plants. Ergonomics, 56(7), 1070–1085. https://doi.org/10.1080/00140139.2013.790483

17.

Glöckner

Herbold

A.-K.

(2011). An eye-tracking study on information processing in risky decisions: Evidence for compensatory strategies based on automatic processes. Journal of Behavioral Decision Making, 24(1), 71–98. https://doi.org/10.1002/bdm.684

18.

Goldberg

J. H.

Wichansky

A. M.

(2003). In Hyönä

Radach

Deubel

H. B. T.-T. M. E.

(Eds.), Chapter 23 - eye tracking in usability evaluation: A practitioner’s guide (pp. 493–516). North-Holland. https://doi.org/10.1016/B978-044451020-4/50027-X

19.

Granholm

Asarnow

R. F.

Sarkin

A. J.

Dykes

K. L.

(1996). Pupillary responses index cognitive resource limitations. Psychophysiology, 33(4), 457–461. https://doi.org/10.1111/j.1469-8986.1996.tb01071.x

20.

J. S.

Byon

Y.-J.

Baek

Seong

P. H.

(2016). Method for inference of operators’ thoughts from eye movement data in nuclear power plants. Nuclear Engineering and Technology, 48(1), 129–143. https://doi.org/10.1016/j.net.2015.09.004

21.

Honeywell . (2021). Experion PPC meets your operational requirements: Intuitive design for maximum performance (Accessed Online on 27/3/2025). https://prod-edam.honeywell.com/content/dam/honeywell-edam/pmt/hps/products/pmc/modular-systems/experion-panel-pc/pmt-hps-experion-panel-pc-brochure−2019.pdf?download=false

22.

Huang

Hildebrandt

Lau

(2019). Comparing eye-gaze metrics of mental workload in monitoring process plants. In Harris

(Ed.), Engineering psychology and cognitive ergonomics (pp. 55–65). Springer International Publishing.

23.

Ikuma

L. H.

Harvey

Taylor

C. F.

Handal

(2014). A guide for assessing control room operator performance using speed and accuracy, perceived workload, situation awareness, and eye tracking. Journal of Loss Prevention in the Process Industries, 32, 454–465. https://doi.org/10.1016/j.jlp.2014.11.001

24.

Iqbal

M. U.

Srinivasan

(2018). Towards obviating human errors in real-time through eye tracking Computer Aided Chemical Engineering (43, pp. 1189–1194). Elsevier Masson SAS. https://doi.org/10.1016/B978-0-444-64235-6.50207-2

25.

Kluge

Nazir

Manca

(2014). Advanced applications in process control and training needs of Field and control room operators. IIE Transactions on Occupational Ergonomics and Human Factors, 2(3–4), 121–136. https://doi.org/10.1080/21577323.2014.920437

26.

Koskinen

H. M. K.

Savioja

Mannonen

Aikala

(2024). The process is under control! Understanding the building blocks of user experience in operator work. Nordic Conference on Human-Computer Interaction, 1–12. https://doi.org/10.1145/3679318.3685394

27.

Kotval

X. P.

Goldberg

J. H.

(1998). Eye movements and interface component grouping: An evaluation method. Proceedings of the Human Factors and Ergonomics Society - Annual Meeting, 42(5), 486–490. https://doi.org/10.1177/154193129804200509

28.

Kovesdi

Spielman

Leblanc

Rice

(2018). Application of eye tracking for measurement and evaluation in human factors studies in control room. https://www.osti.gov/servlets/purl/1375330

29.

W.-C.

Kearney

Braithwaite

Lin

J. J. H.

(2018). How much is too much on monitoring tasks? Visual scan patterns of single air traffic controller performing multiple remote tower operations. International Journal of Industrial Ergonomics, 67, 135–144. https://doi.org/10.1016/j.ergon.2018.05.005

30.

Lin

Zhang

W. J.

Watson

L. G.

(2003). Using eye movement parameters for evaluating human–machine interface frameworks under normal control operation and fault detection situations. International Journal of Human-Computer Studies, 59(6), 837–873. https://doi.org/10.1016/S1071-5819(03)00122-8

31.

Liversedge

S. P.

Findlay

J. M.

(2000). Saccadic eye movements and cognition. Trends in Cognitive Sciences, 4(1), 6–14. https://doi.org/10.1016/S1364-6613(99)01418-7

32.

Loftus

G. R.

(1972). Eye fixations and recognition memory for pictures. Cognitive Psychology, 3(4), 525–551. https://doi.org/10.1016/0010-0285(72)90021-7

33.

Majaranta

Bulling

(2014). Eye tracking and eye-based human – computer interaction. Advances in physiological computing (pp. 39–65). https://doi.org/10.1007/978-1-4471-6392-3

34.

Manca

Nazir

Lucernoni

Colombo

(2012). Performance indicators for the assessment of industrial operators. In Bogle

I. D. L.

Fairweather

M. B. T.-C. A. C. E.

(Eds.), (30, pp. 1422–1426). Elsevier. 22 European Symposium on Computer Aided Process Engineering. https://doi.org/10.1016/B978-0-444-59520-1.50143-3

35.

Mannan

Lees

F. P.

(2012). Lee’s loss prevention in the process industries: Hazard identification, assessment, and control (4th ed.). https://shop.elsevier.com/books/lees-loss-prevention-in-the-process-industries/lees/978-0-12-397189-0

36.

Mourtzis

Angelopoulos

Panopoulos

(2023). The future of the human–machine interface (HMI) in society 5.0. Future Internet. https://doi.org/10.3390/fi15050162.15Issue 5162.

37.

Naderpour

Nazir

(2015). The role of situation awareness in accidents of large-scale technological systems. Process Safety and Environmental Protection, 97, 13–24. https://doi.org/10.1016/j.psep.2015.06.002

38.

Naqvi

S. A. M.

Raza

Ghazal

Salehi

Kang

Teodoriu

(2020). Simulation-based training to enhance process safety in offshore energy operations: Process tracing through eye-tracking. Process Safety and Environmental Protection, 138, 220–235. https://doi.org/10.1016/j.psep.2020.03.016

39.

National Transportation Safety Board, Supervisory Control and Data Acquisition ( SCADA ) in Liquid Pipelines . (2005). Safety Study NTSB/SS-05/02.

40.

Nazir

Ahram

T. Z.

Karwowski

(Eds.), (2021). Advances in human factors in training, education, and learning sciences Proceedings of the AHFE 2021 virtual conference on human factors in training, education, and learning sciences, july 25-29, 2021, USA (p. 399): Springer. https://doi.org/10.1007/978-3-030-80000-0

41.

Niemelä

Saarinen

(2000). Visual search for grouped versus ungrouped icons in a computer interface. Human Factors, 42(4), 630–635. https://doi.org/10.1518/001872000779697999

42.

Noah

Rothrock

(2015). Using eye tracking for live measures of workload in a refinery control room process monitoring task. https://www.asmconsortium.net/Documents/NoahandRothrock_2015_Usingeyetrackingforlivemeasuresofworkloadinarefinerycontrolroomprocessmonitoringtask.pdf

43.

O’Hara

Brown

(2001). Human-system interface management: Human factors review guidance (BNL Technical Report W6546-T6A-1-3/01) .

44.

O’Hara

Brown

(2002). The Effects of Interface Management Tasks on Crew Performance and Safety in Complex, Computer-Based Systems (NUREG/CR-6690). https://www.nrc.gov/docs/ML0225/ML022520381.pdf

45.

O’Hara

J. M.

Fleger

(2020). Human-system interface design review guidelines. (NUREG-0700 Revision 3) (Issue July). https://www.nrc.gov/docs/ML0217/ML021700337.pdf

46.

Pan

Lin

(2017). A review of cognitive models in human reliability analysis. Quality and Reliability Engineering International, 33(7), 1299–1316. https://doi.org/10.1002/qre.2111

47.

Parasuraman

Sheridan

T. B.

Wickens

C. D.

(2000). A model for types and levels of human interaction with automation. IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans: A Publication of the IEEE Systems, Man, and Cybernetics Society, 30(3), 286–297. https://doi.org/10.1109/3468.844354

48.

Pavlas

Lum

Salas

(2012). How to build a low-cost eye-tracking system. Ergonomics in Design: The Quarterly of Human Factors Applications, 20(1), 18–23. https://doi.org/10.1177/1064804611428928

49.

Peavler

W. S.

(1974). Pupil size, information overload, and performance differences. Psychophysiology, 11(5), 559–566. https://doi.org/10.1111/j.1469-8986.1974.tb01114.x

50.

PTS 14.00.02

: Human-machine interface in a control room. (2014). PETRONAS.

51.

Rockwell

Automation.

(2025) Human machine interface visualization for smart manufacturing. Retrieved March 27, 2025, from. https://www.rockwellautomation.com/en-us/products/hardware/allen-bradley/hmi.html

52.

Salehi

Kiran

Jeon

Kang

Cokely

E. T.

Ybarra

(2018). Developing a cross-disciplinary, scenario-based training approach integrated with eye tracking data collection to enhance situational awareness in offshore oil and gas operations. Journal of Loss Prevention in the Process Industries, 56, 78–94. https://doi.org/10.1016/j.jlp.2018.08.009

53.

Sanfilippo

(2017). A multi-sensor fusion framework for improving situational awareness in demanding maritime training. Reliability Engineering & System Safety, 161(February 2016), 12–24. https://doi.org/10.1016/j.ress.2016.12.015

54.

Schneider

Electric.

(2025). Advanced HMI Panels. Retrieved March 27, 2025, from https://www.se.com/my/en/product-category/2100-hmi-terminals-and-industrial-pc/.

55.

Shahab

M. A.

Iqbal

M. U.

Srinivasan

(2021). Metrics for objectively assessing operator training using eye gaze patterns. Process Safety and Environmental Protection, 156, 508–520. https://doi.org/10.1016/j.psep.2021.10.043

56.

Shahab

M. A.

Iqbal

M. U.

Srinivasan

(2022). HMM-based models of control room operator’s cognition during process abnormalities. 2. Application to operator training. Journal of Loss Prevention in the Process Industries, 76(July 2021). Article, 104749. https://doi.org/10.1016/j.jlp.2022.104749

57.

Sharma

Bhavsar

Srinivasan

(2016). Eye gaze movement studies of control room operators: A novel approach to improve process safety. Computers & Chemical Engineering, 85, 43–57. https://doi.org/10.1016/j.compchemeng.2015.09.012

58.

Shi

Noah

Rothrock

Kim

J. H.

(2021). Potential benefits of eye tracking within process control monitoring tasks. Human Factors and Ergonomics in Manufacturing & Service Industries, 31(3), 316–326. https://doi.org/10.1002/hfm.20892

59.

Shi

Rothrock

(2022). Validating an abnormal situation prediction model for smart manufacturing in the oil refining industry. Applied Ergonomics, 101, Article 103697. https://doi.org/10.1016/j.apergo.2022.103697

60.

Shin

I. J.

(2014). Loss prevention at the startup stage in process safety management: From distributed cognition perspective with an accident case study. Journal of Loss Prevention in the Process Industries, 27(1), 99–113. https://doi.org/10.1016/j.jlp.2013.11.003

61.

Siemens . (2024). SIMATIC HMI/ PC-based Automation: Human Machine Interface Systems/PC-based Automation - Catalog ST 80/ST PC. (accessed online on 27/3/2025).https://www.siemens.com/industrial-communication/mall

62.

Smith

M. A.

Mead

P. R.

Squire

P. N.

Coons

R. L.

Mead

A. S.

(2015). Look, don’t touch: Evaluation of a gaze-based interface control system. Ergonomics in Design: The Quarterly of Human Factors Applications, 23(2), 10–15. https://doi.org/10.1177/1064804613516113

63.

Srinivasan

Iqbal

M. U.

Shahab

M. A.

Srinivasan

(2022). Review of virtual reality (VR) applications to enhance chemical safety: From students to plant operators. ACS Chemical Health & Safety, 29(3), 246–262. https://doi.org/10.1021/acs.chas.2c00006

64.

Tharanathan

Bullemer

Laberge

Reising

D. V.

Mclain

(2012). Impact of functional and schematic overview displays on console operators’ situation awareness. Journal of Cognitive Engineering and Decision Making, 6(2), 141–164. https://doi.org/10.1177/1555343412440694

65.

Toker

Conati

Steichen

Carenini

(2013). Individual user characteristics and information visualization: Connecting the dots through eye tracking. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 295–304. https://doi.org/10.1145/2470654.2470696

66.

U.S. Chemical Safety Board . (2022). Chemical safety board. https://www.csb.gov/us-chemical-safety-board-issues-final-report-into-fatal-2022-fire-at-bp-husky-refinery-near-toledo-ohio/

67.

Wade

Tatler

(2005). The Moving Tablet of the Eye: The origins of modern eye movement research. Oxford University Press. https://doi.org/10.1093/acprof:oso/9780198566175.001.0001

68.

Wanasinghe

T. R.

Galagedarage Don

Arunthavanathan

Gosine

R. G.

(2022). Industry 4.0 based process data analytics platform. In Khan

Pasman

-M

M. B. T.

Yang

C. P. S.

(Eds.), Methods to Assess and Manage Process Safety in Digitalized Process System (6, pp. 101–137). Elsevier. https://doi.org/10.1016/bs.mcps.2022.04.008

69.

Liu

Jia

Tran

C. C.

Yan

(2020). Using artificial neural networks for predicting mental workload in nuclear power plants based on eye tracking. Nuclear Technology, 206(1), 94–106. https://doi.org/10.1080/00295450.2019.1620055

70.

Anders

Pruttianan

France

Lau

Adams

J. A.

Weinger

M. B.

(2018). Human performance measures for the evaluation of process control human-system interfaces in high-fidelity simulations. Applied Ergonomics, 73, 151–165. https://doi.org/10.1016/j.apergo.2018.06.008

71.

Yan

Tran

C. C.

Chen

Tan

Habiyaremye

J. L.

(2017). Effect of user interface layout on the operators’ mental workload in emergency operating procedures in nuclear power plants. Nuclear Engineering and Design, 322, 266–276. https://doi.org/10.1016/j.nucengdes.2017.07.012

Effectiveness of Eye-Tracking Metrics in Human-Centric Design of Human-Machine Interface: Cases on Process Control Operations

Abstract

Keywords

Overview of Industry Guidelines on HMI Design

Overview of HMI Design Initiatives by Technological Powerhouses

Role of Eye-Tracking in Human-Centered HMI Design

(I) Global Interaction Analysis

(II) Contextual Support Assessment

(III) Element-Level Interface Optimization

Other Relevant Research

Limitations for Industrial Application and Future Directions

Conclusion

Footnotes

Declaration of conflicting interests

Funding

References