Abstract
The integration of cognitive systems engineering (CSE) principles within the traditional systems engineering (SE) process is a critical area of research. CSE emphasizes the goal of providing “the designer with a realistic model of how the human functions cognitively” (Hollnagel & Woods, 1983, p. 586). Over the past three decades, numerous design methodologies and frameworks—such as naturalistic decision making (Klein, 2008), joint cognitive systems (Woods & Hollnagel, 2006), hierarchical task analysis (Annett, 2003), cognitive work analysis (CWA; Rasmussen, Pejtersen, & Goodstein, 1994; Vicente, 1999), and contextual design (Beyer & Holtzblatt, 1998)—were developed to provide a principled approach to provide designers models of human cognitive function to inform system design. Nonetheless, traditional SE frameworks, such as the V-model, utilized in industry (Haskins, 2011; National Aeronautics and Space Administration [NASA], 2007; U.S. Air Force Human Systems Integration Office, 2009) still lack a formalized approach to incorporating these insights beyond simply identifying what data or information is needed to be included on the graphical user interface for a human operator (Elm et al., 2008). At the core of traditional SE, requirements are defined by customer and user needs from relevant stakeholders at the onset of the project. Oftentimes, the cognitive requirements derived by traditional SE methods (e.g., stakeholder interviews, observations) are ill-defined, assumed, or ignored, leaving a high degree of uncertainty in the system design to successfully support cognitive functions (Militello, Dominguez, Lintern, & Klein, 2009). There exist deficiencies that limit the synchronization between traditional SE and CSE methods. Traditional SE methods lack the following:
Moreover, CSE methods lack the following:
Design requirements are the medium through which a systems design is realized and vetted in practice. Therefore, we demonstrate, as part of the first stage of the SE process, CSE requirements that can be derived by applying the CWA methodology and posited in a compatible SE requirements format. Specifically, we leverage the first two phases of CWA—work domain analysis (WDA) and control task analysis (ConTA)—to define and ground the research effort and to derive a set of decision support system (DSS) requirements for future human spaceflight extravehicular activity (EVA) operations, respectively.
Currently, NASA aims to send humans beyond low Earth orbit, and the EVA work domain serves as an evolutionary work domain for the application of the proposed CSE requirements derivation process. The intent of this work is to provide an example of how a CSE practitioner can generate requirements from analyzing a complex work domain as it exists today to inform and shape how it will exist in the future. An inherent hypothesis in this research is that the requirements definition process needs to be separated from the actual representative prototype design. Requirements do not provide design solutions (Zave & Jackson, 1997). By explicitly establishing a set of CSE-inspired requirements, CSE insight can be more readily carried throughout the SE process in the form of requirements. The exact physical representation of the resultant support system should be thought of as a hypothesis derived from the CSE requirements. Only once requirements are established can the CSE requirements be referenced and tested to verify and validate the representative design.
Background and Literature Review
CSE Requirements Definition Review
The proposition that traditional SE processes and CSE insight should be more seamlessly integrated is not new. Pew and Mavor (2007) detailed a comprehensive assessment of the opportunities to integrate CSE perspectives with traditional SE processes. The Committee on Human-System Design Support for Changing Technology emphasized that the “definition of user requirements should begin when the system is first being conceived, and those requirements should continue to provide important evaluation criteria right up to the time the system is placed in use” (p. 296). In response to the committee report, the CSE is ideally suited to be the key breakthrough in the development of good system requirements because of its focus on overall system goals and the associated cognitive work (including coordination) that needs to be accomplished by the people using the system to achieve those goals. (p. 255)
Systems engineering V-model adapted from Elm et al. (2008) overlaid with cognitive systems engineering integration points. This article focuses on Integration Point 1: high-level requirements.
However, few examples confirm that CSE is ideally suited to be the key breakthrough in the development of good high-level system requirements (Elm, Potter, Gualtieri, Roth, & Easter, 2003; Gualtieri, Elm, Potter, & Roth, 2001; Potter, Gualtieri, & Elm, 2007).
The CSE literature includes examples of design efforts that span the possible range of the requirements definition process from omission (Bisantz & Burns, 2009; Jenkins, Stanton, Salmon, & Walker, 2009) to formal articulation of requirements in a structured format (Elm et al., 2003; Gualtieri et al., 2001; Potter et al., 2007; Potter, Roth, Woods, & Elm, 1998). Klein, Orasanu, Calderwood, and Zsambok (1993) emphasized “rapid prototyping as a way of
Much of the CSE literature references requirements only tangentially as an important component of informing system design. When practitioners do emphasize requirements, they are presented in a variety of forms, such as information requirements (Burns, Bryant, & Chalmers, 2005; Jamieson, 2003; McGeorge et al., 2015; Roth & Mumaw, 1995), usability requirements (Pew & Mavor, 2007), cognitive support requirements (Sanderson, 2003), situation awareness requirements (Nehme, Scott, Cumming, & Furusho, 2006), and display design requirements (Bisantz et al., 2003; C. A. Miller & Vicente, 2001) with limited definition, consistency, and specificity. Furthermore, while various methods have attempted to demonstrate and link CSE requirements to specific CSE model components (Cattermole, Horberry, & Hassall, 2016; Cummings, 2004; Cummings, Tappan, & Mikkelsen, 2012; Lintern, 2006; Nehme et al., 2006), the extent to which these studies emphasize the actual requirements derivation process and situate their development within traditional SE processes is limited. Although examples are deemed valuable within the CSE community for their specific domain application, if they cannot be translated to the SE community of system developers, customers and other engineers, CSE insight and influence will remain limited (McIlroy & Stanton, 2012).
One approach to integrate CSE requirements with the SE process was proposed by Elm et al. (2003) using an applied CWA framework. The applied CWA process involved first constructing a functional abstraction network (FAN) model that captured the essential work domain concepts and relationships that defined the problem space. Subsequently, a layered set of requirements were derived from FAN elements that specified cognitive work, information relationship, representation design, and presentation design requirements. The applied CWA framework initially appealed to us; however, three key deficiencies of the FAN method lead us to use a modified applied CWA:
Specifically, we adapted the applied CWA to leverage the more commonly utilized models from traditional CWA (Vicente, 1999). We first performed a WDA that involved the construction of an information flow model and abstraction hierarchies (AHs). Second, a ConTA was performed that resulted in a contextual activity template and decision ladders. Finally, we incorporated cognitive work requirements (CWRs) and information relationship requirements (IRRs), as originally defined by Elm et al. (2003), in a traceable way by anchoring those requirements to specific stages of the decision ladder model. We hypothesize that CWR and IRR statements articulate the CSE perspectives critical to instill in the larger SE design process. CWR and IRR content and format convey the characteristics expected by the SE community practitioners. Traditional requirements specification characteristics include being necessary, correct, unambiguous, feasible, achievable, prioritizable, quantifiable, measureable/verifiable, traceable, and results oriented (Turk, 2006).
Given the aforementioned state of the CSE requirements derivation process, we demonstrate one potential avenue for generating CSE-inspired requirements utilizing the first two phases of CWA. However, before we begin that discussion, we review the research activities in the EVA work domain, which has limited formal examination by the CSE community.
EVA Work Domain Challenges and Research Activities
EVA is a mission-critical capability with a proven track record in spacecraft and payload inspection, repair, and construction (Wilde, McBarron, Manatt, McMann, & Fullerton, 2002). EVA is defined as “any space operation or activity performed outside the protective environment of a spacecraft therefore requiring supplemental or independent life support equipment for the astronaut” (McBarron, 1994, p. 5). The successful execution of EVA requires significant effort undertaken by astronauts and the personnel that support all aspects of EVA from Earth. These support personnel are known as EVA flight controllers, and they strive to manage the flight crew, the EVA timeline, and all associated hardware (M. J. Miller, McGuire, & Feigh, 2015). Ground personnel oversee all preplanning, day-of, and post-EVA operations and support off-nominal events such as hardware malfunctions, unscheduled task deviations, and crew health variations. The EVA work domain can be considered a complex sociotechnical system and has the characteristics shown in Table 1.
Characteristics of the EVA Work Domain.
The EVA work domain is a particularly important system to analyze because of NASA’s shifting efforts toward deep-space exploration. NASA is employing a flexible pathways approach to guide the future of human spaceflight (Augustine et al., 2009). The near-term potential destinations include the moon, near-earth objects, and Mars. While each destination presents a unique set of technological challenges, one new constraint will remain: time-delayed communication associated with the vast distances that the communication relays must travel to reach mission control, located on Earth. For the moon, time delay is on the order of seconds, but for near-earth objects and eventually Mars, time delays on the order of minutes to tens of minutes will be commonplace. Time delay will impose a fundamental limitation on how ground support personnel can affect and influence EVA during real-time operations. For the entire history of the EVA work domain, astronauts have relied on active participation from ground support during real-time operations, and as a by-product, many of the work functions required to execute EVA safely and efficiently reside within ground support personnel. The question therefore becomes, how might we transition and integrate ground support capabilities with in-flight crew to envision and enable future, deep-space EVA operations?
The majority of research within the EVA community to date has been dedicated to the technological and operational challenges associated with the tools and hardware related to the suited astronaut. These challenges include (a) reducing the time and resources required to prepare an astronaut for EVA through skill-based training, as opposed to task-specific training (Bell & Coan, 2012; Ney, Looper, & Parazynski, 2006; Thuot & Harbaugh, 1995), and (b) enhancing overall capabilities of the spacesuit (Abercromby, Gernhardt, Litaker, & Martin, 2010; Jaramillo, Angermiller, Morency, & Rajululu, 2008; Reid et al., 2014; Thuot & Harbaugh, 1995). Additional efforts have assessed astronaut task execution efficiency (Looper & Ney, 2005, 2006; Marquez, 2010). Various EVA timelines were decomposed into content categories to quantitatively describe the time spent in each category to identify aspects of an EVA that warrant efficiency improvements—that is, how can we reduce crew time spent during, or increase crew efficiency on, particularly long or challenging EVA tasks? However, astronaut task execution constitutes only one component of the larger volume of work required to execute EVA. More detail regarding the roles and responsibilities of, and the information flow among, operators within present-day
A further complication of the envisioned EVA work domain is the limited literature that exists describing the impact of time-delayed communication on task performance. Only recently has research begun to investigate effective communication protocols and modes of communication within time-delayed environments (Fischer & Mosier, 2014; Fischer, Mosier, & Orasanu, 2013; Frank et al., 2013; Frank et al., 2015; Stetson et al., 2015). Some of the prime challenges associated with time-delayed communication include confusion of communication sequence, communication interruption, and an impaired ability to provide relevant information (Love & Reagan, 2013). Recently, various mitigation tools and techniques were tested in a human-in-the-loop study that simulated notional human spaceflight scenarios within a time-delayed operational environment to understand the utility of decision aid systems (Frank et al., 2013; Frank et al., 2015). The mitigation tool capabilities included the ability to visualize spacecraft telemetry and issue commands from procedure displays, track procedure execution status across time delay, and automatically isolate faults and recommend procedures based on vehicle configuration. Results from the study indicated that the support tools both increased and decreased operator workload, depending on the specific operator and test condition. In another human-in-the-loop simulation, Fischer and Mosier (2014) found that teams took significantly longer to repair system failures when operating under time-delayed communication and that adapting to the communication medium itself is key to establish shared task understanding (Fischer & Mosier, 2014).
Numerous other human-in-the-loop simulations specifically focused on EVA operations have attempted to characterize the necessary capabilities for EVA operators in a variety of operationally relevant environments, such as the moon, moons of Mars, and Mars surface (Abercromby, Chappell, & Gernhardt, 2013; Chappell, Abercromby, & Gernhardt, 2013; Rader, Reagan, Janoiko, & Johnson, 2013; Reagan, Janoiko, Johnson, Chappell, & Abercromby, 2012). However, there is currently still a lack of research dedicated to understanding how to appropriately transition inherent EVA work functions from the large team of ground support personnel to a few astronauts operating in deep space. The consensus among the current body of EVA literature is that in the envisioned EVA work domain, astronauts will need to be more self-reliant and that self-reliance will be manifested via enhanced training and advanced DSSs (Office of the Chief Technologist, 2015). EVA research efforts currently lack a comprehensive consideration of the EVA work domain that includes ground-based support personnel to shape support system design. Appropriately scoping and bounding the EVA work domain is crucial to identify the present work constraints and the necessary work functions that will need to be divested to astronauts during future time-delayed EVA operations.
Finally, the EVA work domain is ideally situated early in the SE design phase for the application of the CWA framework. NASA is presently undergoing early concept of operations specification and high-level requirements specification. The current EVA work domain can be leveraged, due to our unprecedented access to subject matter experts (SMEs) and document materials within the domain, to guide the envisioned world problem of future EVA operations by situating support system design understanding, captured by WDA and ConTA, within the field of practice the design is intended to support. By situating the support system design efforts within a particular field of work, such as EVA, a more refined understanding of the operational environment can be leveraged to influence DSS design (Woods & Dekker, 2000).
WDA: Phase 1 of CWA
A WDA is the first phase of CWA and was used to establish a broad understanding of the EVA work domain and to identify constraints that exist within the work domain (Naikar, Hopcroft, & Moylan, 2005; Vicente, 1999). The result of the WDA was an aggregated abstraction model, known as an AH, which provided a graphical depiction of the structure and functioning of the work domain. This study utilized the traditional five abstraction levels (Burns et al., 2005; Rasmussen, 1985). In addition, this study constructed an information flow model as part of the WDA to identify and relate the unique operators who work collectively to perform EVA (Cummings, 2004). For a more general description of human spaceflight operations and mission control structure, refer to the works of Patterson, Woods, and Watts-Perotti (1999) and Watts et al. (1996).
The WDA presented here is the first comprehensive CSE examination of the EVA work domain. The WDA purposefully takes a broad perspective to ensure the inclusion of all relevant work domain elements. We utilized the WDA to capture and structure the current state of EVA operations and prioritize research efforts for the ConTA. The remainder of this section describes the WDA process undertaken, the models generated, and the resulting DSS design considerations. The second half of this study focuses the ConTA phase and is discussed in the subsequent section.
WDA Methods and Considerations
We performed an iterative information-gathering and model (AH and information flow model) development process. The data sources used for the WDA were varied and extensive (see Table 2). These data sources were chosen according to their accessibility and intimate association to the EVA work domain (Naikar et al., 2005; Vicente, 1999). Given the highly procedural and detailed nature of the EVA operations, we leveraged detailed console manuals that delineated the specific expectations of the work to be performed by all members of the EVA community. We recognize that detailed operational documentation may not be available for many other work domains, in which case more direct observations and interviews may be required to articulate the same level of work detail.
Data Collection for EVA Work Domain Analysis.
First, the data sources were examined to identify the primary operators and construct a layout of operators within the EVA work domain, characterized by EVA operator work responsibilities and geospatial locations. As a result, an information flow model of the EVA work domain was generated. Second, two AHs were developed: one that depicted the EVA environment and one that incorporated the work of EVA operators and their engineered systems. The AHs were generated through use of the guiding prompts and keywords provided by Naikar et al. (2005, p. 33) on each data source. Finally, two certified EVA flight controllers reviewed the AHs to assess the content and validity of each AH element. The purpose of the WDA was to extract a comprehensive understanding of present-day operations and to identify the primary components of the AHs and information flow model to guide subsequent CWA analysis phases, as opposed to performing a strategies analysis or attempting to directly extract design requirements. Furthermore, at this stage of CWA, we did not presuppose any capabilities or expectations of future DSSs. The intent of the WDA was to collectively understand the constraints and demands of EVA operators to express the potential purposes of a DSS within the work domain. Rather than presupposing design solutions directly from the WDA, we focused on “the nature of the work and how it might be accomplished as a prelude to thinking about how that work might be supported” (Lintern, 2012, p. 80).
WDA: Results
EVA Work Domain Layout
The EVA work domain is shaped by two main considerations: (a) the geospatial distribution of EVA operators and systems and (b) the potential habitable environments of the flight crew. Figure 2 shows the primary operators involved during present-day EVA operations. The dashed-line boundaries signify geospatial separation (physical barriers), and the solid block denotes geospatial and temporal separation (i.e., time-delayed communication). Astronauts, known as the extravehicular (EV) crew, are located outside their spacecraft and are inextricably constrained by the confines of the spacesuit during EVA operations. EV crew are also typically accompanied by assets such as robotic aids. The hull of the spacecraft encapsulates the operators within the spacecraft, known as the intravehicular (IV) crew, who exist locally with the EV crew. The IV crew are typically unsuited astronauts who rely on the internal environment of the spacecraft for life support needs and protection from the external environment. The external environment surrounding the in-space crew imposes severe physical constraints that result in sophisticated engineered systems necessary for EV and IV crew to simply survive in EVA environments, let alone perform their expected tasks. Finally, the EVA work domain includes the mission control center (MCC), located on Earth. A subset of personnel within mission control known as EVA flight controllers are dedicated to EVA support (M. J. Miller et al., 2015).
Extravehicular activity (EVA) work domain information flow model. Adapted from M. J. Miller, McGuire, and Feigh (2015). Vector icons can be found at http://www.flaticon.com/. EV = extravehicular; IV = intravehicular; CAPCOM = capsule communicator.
EVA ground personnel are divided into three primary console positions: systems, task, and airlock systems within the multipurpose support room who report directly to the EVA front room controller. The arrows shown in Figure 2 identify the primary communication pathways utilized during EVA for EVA-specific support personnel. EVA multipurpose support room consoles speak directly with the EVA front room controller, who then communicates with colocated personnel in the front room, such as the flight director. Controllers located within the same room also speak directly with one another to exchange information. Finally, messages are condensed and transferred to the crew via the capsule communicator and ground IV. The vehicle and assets, such as communication and power systems, required to support EVA operators are managed by the IV operator and teams of other personnel also located in MCC. Due to the real-time communication environment of
The information flow model isolated and identified potential SMEs for refined data collection as well as useful observation opportunities. An additional value of building the information flow model was to establish the current state of the domain so that we could clearly demarcate the transitions that will need to be made to support future (time-delayed) EVA operations. In particular, we leveraged the flow model to identify the IV operator as a potential end user of the DSS design in the envisioned EVA work domain. The AHs discussed in the subsequent sections were built to highlight the work domain constraints within which the EVA operators shown in Figure 2 must contend during EVA operations.
EVA Work Domain AHs
Given the complex, distributed operations involved with performing EVA, the AH shown in Figure 3 was limited to show functions and constraints present during the execution of an EVA, as opposed to incorporating all the preparatory and post-EVA operations. The question that still remains at this stage of analysis is, what capabilities should an envisioned DSS have to support EVA operations? Broadly speaking, all EVAs require a common goal or set of objectives that unify all members of the work domain to justify the inherent risks to the crews’ lives. In an effort to remain comprehensive in assessing the work domain, the AHs were built to describe the constraints and considerations in the work domain that contribute to the execution of EVA rather than situate their descriptions on any specific EVA objectives.
Abstraction hierarchy of the extravehicular activity (EVA) work domain that includes EVA-specific mission control, intravehicular, and extravehicular operators. MCC = mission control center.
Figure 3 shows the AH pertinent to EVA operations among the EV, IV, and EVA ground support personnel. Almost all operational ownership over each abstraction element articulated in Figure 3 is currently retained within mission control. However, as with many complex work domains, the members and units each have “control over a collection of resources but does not have direct control over the entire work domain” (Burns et al., 2005, p. 605). For example, MCC operators do not currently have any direct control capability over the EV spacesuits. Conversely, while the EV crew have control over their spacesuits, they have limited capability to view the telemetry of their spacesuits. In addition to the physical and operational constraints, we specifically modeled the EVA environment as shown in Figure 4. All members of the EVA work domain must contend with the environmental constraints imposed by both local environments of the EV/IV and MCC, as well as the environment that spans the separation between the operators. While the environment of both the EV/IV crew and MCC are different, each exhibits and imposes similar natural phenomena that can have serious implications for EVA operations, particularly as related to communication capabilities. Each element articulated in Figures 3 and 4 captures the unique constraints persistent within the work domain. An important contribution of the AHs to this research project was to identify these fundamental elements to structure work domain understanding. The following descriptions summarize the insights derived from each AH level.
Abstraction hierarchy of the extravehicular activity environment. EM = electromagnetic.
EVA AH Decomposition
Functional purposes
At the root of all EVA is a clear, prescribed set of objectives. Aside from a few
Abstract functions
Crew safety and EVA objectives are influenced by the balance of mass, energy, and resources. The spacecraft and external agents are closed systems and are constrained in the physical sense that every single system at the crew’s disposal has operational envelopes. Flight rules catalog these critical criteria in the form of
Generalized functions
The generalized function level provides a decomposition of the main functions required to execute EVA. Many functions at the generalized function level rely on the processes of generating, receiving, and processing signals due to the distributed nature of operators. In raw form, transmitted signals include telemetry data from sensors, as well as audio, video, and text information from the crew. In addition to generating and receiving transmissions, operators must contend with the archiving and management of that data. Present-day examples of this process includes audio transmission that are archived into handwritten and digital flight notes. Processes of inventory management constrain EVA execution by imposing storage, access, and maintenance limitations enforced by the tools and hardware utilized during execution. Processes of translation, orientation, and stabilization require geospatial considerations, such as fields of view, keep-out zones, attachment points, and translation paths. The processes associated with egress and ingress impose a host of physical integration constraints with the spacecraft airlock in addition to health considerations such as decompression sickness. Finally, anomaly response and resolution within EVA operations consist of diagnosis-related constraints, such as the ability to access internal hardware and recover relevant engineering design specifications. The dashed-line element serves as a placeholder of activities performed before and after the execution portion of EVA that includes all the planning, training, and maintenance functions deemed outside the scope of this study.
Physical functions
The physical function level encapsulates the functional capabilities of the individual astronaut, spacesuit, spacecraft, tools, and hardware. Currently, limited diagnosis and anomaly response capabilities exist for the EV crew, and mission control is relied upon to monitor these physical capabilities. In addition, mission control must convey modifications to the crew because the ground has in some cases no direct manipulation capability, such as the spacesuit hardware. Constraints included in the EVA domain also comprise the unique capabilities of the astronauts themselves, in terms of physical endurance and skill sets as well as mental and workload capacities. Additional constraints to consider include internal environmental constraints such as crew comfort and hardware capabilities.
Physical form
At the physical form level, constraints result from the geographical distribution of assets, resources, and signal characteristics. The characteristics and content of the transmissions are the only forms of interactions that extend to all EVA operators within the domain, and communication constraints in terms of bandwidth and coverage impose critical limitations on EVA. For example, during present-day EVA operations, if the crew cannot establish audio communication with MCC within 30 minutes of loss of signal, the flight rules dictate the EVA to be terminated. The opportunity for EV and IV crew to interact also exists at the physical form level through interfaces between the spacecraft and spacesuit/hardware.
EVA Environment AH
The EVA environment was modeled similar to the natural environment found in naval operations (Burns et al., 2005) because of the profound impact that the environment has on EVA operations. The functional purpose level was not included, because as a natural system, functional purposes are not necessary. The following discussion describes the remaining four abstraction levels.
Abstract function
The measures by which the environment operates follow the forces of nature in the forms of the conservation equations (i.e., conservation of mass, momentum, and energy). The conserved physical quantities impose limitations on the operational capabilities of the engineered systems utilized by the EV and IV operators throughout the EVA.
Generalized function
The generalized function level is divided into physical processes that are associated with various environmental properties. These processes include planetary processes such as orbital mechanics as well as solar processes. Orbital mechanics imposes the temporal separation (time-delayed communication) constraint depicted in Figure 4 and is a constantly varying phenomenon. Whether in the vacuum of space or on a planetary surface, the EVA work domain is influenced by the presence of electromagnetic radiation from both a crew health standpoint and a signal transmission perspective. Engineered systems also interact with the environment by introducing additional radiation fields as well as potentially harmful contaminants.
Physical function
The elements included in the physical function level pertain to the operational environments of the operators’ systems. Once beyond low Earth orbit and the protection of the Earth’s magnetosphere, solar and extrasolar processes become strong physical constraints on crew health and signal transmission capabilities. Earth atmospheric processes also affect mission control’s ability to communicate with the crew. EV crew must also physically interact with their environment, which in some cases can be engineered surfaces such as the
Physical form
At the physical form level, the model delineates elements that are pertinent for EVA operators to collect and monitor. Elements such as terrain layout, radiation levels, and foreign objects and debris are examples of the environmental measures that are useful to EVA operators.
WDA: Guidance for DSS Development
By performing the WDA, we uncovered the inherent constraints that are present within the EVA work domain. Two perspectives in particular were uncovered during the WDA: (a) The processes of signal generation and retrieval are critical to convey EVA state information among the EVA operators, particularly since EV and IV operators are reliant on the operational and engineering knowledge that currently resides within MCC. The content of those transmissions and the methods by which that information is managed require further examination for successful EVA operations under time-delayed communication. (b) The reallocation of generalized work functions shown in Figure 3 among the EVA operators is required for future EVA operations, and DSS development could enable that transition. The transfer of work capabilities from mission control to crew, particularly IV crew, will likely be required, and the individual elements displayed in Figures 3 and 4 must, at a minimum, be supported to ensure successful future EVA operations. At this point, we did not presuppose the appropriate level of function allocation for the generalized functions identified in Figure 3, because future concepts of operations have not been finalized by NASA. However, we did identify two high-priority work functions worth further investigation: (a) life support system monitoring and (b) timeline tracking and alteration. These two functions were selected due to their importance to maintain EVA safety and productivity, time criticality, and the limited capability that MCC will have on providing timely, relevant, and actionable information due to time-delayed communication.
A fundamental trade-off exists at the functional purpose level where EVA objectives are traded off against the safety of the crew. While the exact EVA objective may vary, maintaining crew safety remains constant. Crew life support is associated to the physical hardware and resources that exist locally with the IV and EV crew. Considering an envisioned EVA work domain where time delay will reduce MCC’s ability to rapidly influence crew work, a DSS has the potential to help the IV operator synthesize the vast amount of spacesuit and systems information for safe timeline execution and alteration.
The IV operator has not only the unique placement within the work domain to monitor the physical resources locally but also the infrastructure and bandwidth within the spacecraft/vehicle to manage those resources in addition to interacting with mission control. While it is true that future EVA life support hardware capabilities will likely be more advanced than those of the present day, the fact that these systems must be closely supervised and integrated with real-time EVA decision-making activities will remain, especially for time-critical situations where crew safety is threatened. Furthermore, preliminary studies have already indicated the importance of the IV crew as a critical operational component of future EVA operational concepts (Abercromby et al., 2013). Therefore, by performing the WDA, we empirically derived the motivation to design a DSS specifically for the IV crew with the intended capability to enable them to locally monitor life support systems and EVA timeline progress. Effective management and presentation of aforementioned work functions based on a DSS also have the potential to ease the burden of information overload on the crew.
In summary, the WDA synthesized the EVA work domain to identify and prioritize potential avenues for DSS development. At the onset of a design project, there is a tendency to make assumptions about the purposes and potential end users of the envisioned system. We demonstrated that by leveraging the WDA, the AHs articulated the elemental functions and constraints that influence and shape EVA execution. From this synthesis process, we identified a potential DSS design pathway to provide tactical decision support for an IV operator tasked with maintaining crew and vehicle safety while accomplishing EVA objectives. The emphasis is made on a tactical support because the dynamics of an EVA can change suddenly, which will force EV crew members to respond appropriately, in the absence of immediate mission control input (due to the time delay in communication). Primary support functionality of an EVA DSS was identified from the generalized function level as two complementary components: (a) life support system monitoring and (b) timeline tracking and alteration. The current EVA literature emphasizes the hardware that EV operators may need in the future. This study identified the critical role of the IV operator and established the specific types of support he or she will likely need. Given this initial analysis of the EVA work domain, the ConTA discussed in the next section refined our operational understanding and translated SME knowledge for DSS requirement specification.
Conducting ConTA: Phase 2 of CWA
The WDA phase decomposed the work domain into constituent elements and provided a holistic view of the constraints that shape EVA execution, independent of any particular EVA objective or technology. The ConTA was performed to further refine EVA understanding by investigating specific generalized work functions identified in the WDA. Two models—a contextual activity template (Naikar, Moylan, & Pearce, 2006) and decision ladders aggregated with details from multiple CSE sources (Bisantz & Burns, 2009; Jenkins, Stanton, Salmon, & Walker, 2011; Rasmussen, 1986; Vicente, 1999; Wickens, 1984)—were built to articulate SME insight and translate it into DSS requirements.
The contextual activity template shown in Figure 5 decomposes the work functions along the vertical axis and operational phases along the horizontal axis. The circles/whiskers and dashed boxes delineate the typical and potential associations that each work function has with each operational phase, respectively (Stanton et al., 2013). From this template, the execution of work functions can be mapped to the prototypical operational phases of the EVA work domain to convey the variety of cognitive demands on domain operators (Naikar et al., 2006).
Extravehicular activity (EVA) execution phase contextual activity template.
Additionally, Rasmussen’s decision ladder (Rasmussen, 1986) was updated as indicated by the annotated decision ladder shown in Figure 6 by more recent CSE literature (Bisantz & Burns, 2009; Jenkins et al., 2011; Vicente, 1999). Two decision ladders were constructed for two generalized work functions: (a) life support systems monitoring and (b) timeline tracking and alteration. We also incorporated the descriptions of each step of the decision ladder from multiple sources and utilized those descriptions, shown in Table 3, to facilitate the SME interviews. Presenting a variety of decision ladder–stage descriptions promoted consistent data collection and stimulated knowledge elicitation during interview sessions. We present a subset of the decision ladder describing life support system monitoring to depict the knowledge synthesis and requirements derivation process. Furthermore, we present a subset of derived requirements for discussion purposes.
Annotated decision ladder template.
Detailed Descriptions of Each Step in the Decision Ladder.
ConTA Methods and Considerations
The development of each of ConTA model was facilitated by a series of observation sessions and semistructured interviews with certified EVA flight controllers (EVA front room controller, systems, and task positions, as shown in Figure 2). Based on initial observation sessions of two
Control Task Analysis Data Collection Activities.
ConTA: Results
EVA Contextual Activity Template
The EVA work domain, as shown in Figure 5, is decomposed into an allocation matrix that associates EVA phases of operation with EVA generalized work functions identified from Figure 3. Along the horizontal axis, the EVA work domain is composed of 11 distinct phases of operation that span the entire execution portion of EVA. The
The astronauts then enter the egress phase of operations to exit the spacecraft with their tools and hardware. Once egress is complete, the crew then cycle between the operational phases of being
The EVA work functions generated from the WDA are shown along the vertical axis and describe the set of activities to be performed to conduct an EVA. Two work functions in particular—(a) process of life support monitoring and (b) process of timeline tracking and alteration—are imperative to successful EVA execution and are expected work functions throughout each phase of the EVA. Each minute of an EVA is planned to a prescriptive timeline that directs task execution; however, EVA execution is a dynamic process that rarely maintains nominal timeline operations. In fact, out of the 391 EVAs performed up to July 27, 2016, 110 (28%) experienced significant incidents, such as systems issues, operational incidents, and inadvertent releases that caused a timeline deviation (Packham & Stockton, 2016). An EVA timeline must be constantly assessed throughout the duration of and transition between phases specified in Figure 5 for validity due to potential fluctuations in task execution and life support system performance. Underlying the execution of EVA is the notion that every task is performed within affordances provided by the life support systems. Operationally, knowledge of the life system performance across the various operational phases is paramount to maintain crew safety, and it significantly influences potential timeline alterations.
In summary, the activity template mapped EVA work functions to the various operational phases to refine operational understanding and potential avenues for study. Life support system monitoring and timeline tracking and alteration in particular spanned across all relevant EVA operational phases. The decision ladder depicted in the following section provides further insight into the processes involved in life support system monitoring and provides the basis to link those processes to DSS design requirements.
EVA Life Support System Decision Ladder
DSS requirements were generated from the decision ladder model in two forms: CWRs and IRRs. Given the identified work functions, a decision ladder was used to derive the cognitive demands in the form of states of knowledge (SoKs) required to achieve that function. Each stage of the decision ladder communicates the various decision gates to be traversed to meet a specified goal, such as ensuring crew safety from a life support system perspective. IRRs were generated from information associations required to address their respective CWRs.
We show a portion of the derivation process and a resultant subset of requirements for an envisioned EVA DSS in Figure 7. The remaining requirements are deemed beyond the scope of this article and will be available in a future publication. The decision ladder for life support system monitoring was used as a primary data source to extract functional DSS design requirements at the CWR and IRR levels. For this study, only CWRs and IRRs were generated in an effort to demarcate the requirements generation process from the DSS artifact design process. Expanding these requirements with representation design requirements and, eventually, prototype design artifacts is an area of future research.
This schematic of the requirements derivation process includes a representative sample of the translation from the subject matter expert’s (SME’s) states of knowledge (SoKs) to the resultant decision support system (DSS) requirements. Each SoK manifested in at least one cognitive work requirement (CWR) and one information relationship requirement (IRR). EV = extravehicular; EVA = extravehicular activity; LSS = life support system.
Figure 7 shows a partial decision ladder with the primary goal of maintaining a safe life support system configuration to enable EVA operations. Starting at the bottom left of the decision ladder, we highlight important SME insights captured in each knowledge state of the decision ladder in the following subsections. Rather than focus on the data-processing activities themselves, we emphasized the expected SoKs from the perspective of current-day SMEs. SoKs are emphasized because these elements represent current EVA work domain characteristics likely extensible to future EVA operations. The data-processing activities are not explicitly included, because those processes are inherently tied to modern-day tools and technologies. The goal at this stage in the CWA process is to not prescribe how a DSS might work but rather establish the purposes and requirements for a DSS to support the specified SoKs. The remainder of this section describes the analysis and synthesis stages of life support system management and provides exemplars to articulate the requirement derivation process. We address each SoK in turn as we traverse a portion of the decision ladder in the subsequent sections.
SoK: Alert
The alert state consists of two main sources: (a) suit telemetry and (b) the EV crew members. The suit telemetry consists of data streams from sensors inside the space suit that provide insight into the operational status of the various subsystems and level of consumables. In raw form, the telemetry of the current
SoK: Sets of observations
The goal at this stage in the decision ladder is to gather information to generate sets of observations regarding the life support system. Sets of observations include consumable estimations, data accuracy considerations, fault tree generation, and timeline task associations. The core set of consumable values needed to maintain the life support system includes oxygen, secondary oxygen, battery, and water. The consumables are viewed in multiple forms (time series, averaged, and filtered), which are used collectively to estimate a total operational time remaining or affordance for each consumable. Both the spacesuit and the console systems independently synthesize raw suit telemetry data to generate an estimate of the state of the life support system. Due to the restrictions or limitations in information transmission, the raw data may be incomplete or corrupt, prompting ground operators to assess the validity of the presented data before any conclusions can be made. In addition to consumable calculations, spacesuit telemetry data convey information regarding the functions of the various subsystems. In the presence of a spacesuit hardware alert, operational fault trees are commonly generated to establish a diagnosis path to isolate and confirm the source of the alert. This process requires an intimate knowledge of the subsystem components and their interdependencies. Additionally, temporal timeline awareness must be applied to understand the telemetry values. Understanding the tasks performed as it relates to life support data is a critical SME SoK to support. As a result, the associated CWR states that the DSS
SoK: System state
At this stage, the set of observations is synthesized to estimate the consumable and physical state values of the suit. Deconflicting data trends and identifying confirming cues play an important role in diagnosing the system state. Knowledge of the redundancy levels for each subsystem is also incorporated into state understanding. Again, life support monitoring is integral to the successful execution of the timeline, and so timeline tracking and alteration are important considerations with life support system state estimation. Anticipatory knowledge of upcoming tasks is also incorporated into system understanding to forecast potential affordances or issues.
A critical link within the EVA work domain was identified at this stage of the analysis: Life support systems provide the global limiting constraints on timeline progress and execution potential (barring any unforeseen systems failures). As highlighted, the synthesis of timeline data with life support system data is critical to supporting the overall EVA execution. The implication of this insight is that to formally link life support system constraints to EVA operations, explicit awareness of the timeline task data must be incorporated into the DSS. Currently, the timeline exists in static paper-based formats. Therefore, present-day EVA operators (EVA TASK and SYSTEM controllers in Figure 2) meticulously link timeline task details and EVA telemetry data manually, relying on practice and expertise alone to manage associations and implications.
SoK: Ambiguity
Many sources of ambiguity exist within the data synthesis and interpretation process related to the spacesuit. Ambiguity is typically assuaged by generating a comprehensive list of “what if” scenarios, many of which are formulated a priori to EVA execution. The scenarios incorporate potential implications to the physical subsystem performance, as well as contingency procedures in an attempt to remedy or “safe” the system configuration. Of course, a priori consideration of failures is limited by the imagination of the operators to conceive system deviations. Another source of uncertainty resides with the metabolic rates of the crew members and how those rates can influence the life support system. The goal is to ascertain how much timeline margin the life support system can provide; however, that margin is highly predicated on how the crew members are performing the EVA in real time. Furthermore, ambiguity is generated from the unknown unknowns with regard to potential hazards and system sensors. As it currently stands, EVA practitioners must extract operational knowledge from a limited set of sensors, with uncertainty in their measurements and with uncertainty in their measurements’ validity. The CWR requirement highlights the need for the DSS to estimate the feasible distributions of tasks, given the timeline execution performance as it relates to the life support systems. The specific information required for such a calculation is associated to the measure of carrying capacity of the limiting consumables.
SoK: Goals
The highest-priority goal is to keep the crew alive. If there is any indication that crew life is imminently threatened, the timeline is assessed, and all resources and attention are dedicated to ensuring crew safety. Aside from immediate life-threatening events, the EVA goals shift to increase timeline productivity, which manifest as potential timeline alterations become feasible or necessary. In traditional SE requirements documents, requirements or groups of requirements are often summarized by primary or secondary functional objectives (NASA, 2007). A similar approach can be implemented with the process depicted in Figure 7, where we transformed the specific SME goals to expressed functional objectives of the DSS. In doing so, we provide the motivation and purpose that the subsequent CWR and IRR statements strive to achieve.
EVA ConTA: Derived Requirements
The CWRs convey a set of functional DSS requirements for the envisioned world of EVA execution, specifically related to life support system monitoring. They capture “what the [EVA flight controller] operator was thinking” (Elm et al., 2003, p. 376) during EVA execution to reflect the cognitive activity required to achieve the overall goal of life support system monitoring. For example, the DSS should handle the alerting processes of assessing state variables throughout the various operational phases and consumable levels in comparison to expected or historical values, in the forms of trending as well as in absolute terms of critical upper and lower bounds. Furthermore, the DSS should also have the capability to recognize the current state of the life support system as it relates to the geospatial and temporal location of the EV crew members established by the timeline. For system state estimation, the DSS should track the completed and remaining EVA task priorities and estimate the affordances that the life support system can provide in the presence of timeline alterations. Both life support margins (i.e., relative to the critical limits of consumables/system variables) and timeline task margins (i.e., relative to previously performed or accepted time limits) should also be estimated according to EVA progression for timeline forecasting. Finally, the DSS should estimate the appropriate distribution of task-relevant assets, such as tools, crew, and hardware, for current or altered timeline projections.
The IRRs convey the set of information relations required to perform the activities conveyed at the cognitive work level to establish meaningful context (Elm et al., 2003). For example, the DSS should calculate the deviations of consumables data with respect to a priori expected values. Simply tracking the current life support system values independently does not yield operationally relevant information. Additionally, the DSS should estimate a measure of geospatial accessibility to regions such as worksites or storage depots. The carrying capacity of the life support system variables should also be incorporated into the timeline-tracking activities to link the potential affordances to potential timeline alterations.
The specific phrasing of the CWR and IRR statements should accomplish three primary objectives: (a) to capture the intent of the associated SME’s SoKs, (b) to fulfill the basic definitions of CWR and IRR statements as defined by Elm et al. (2003), and (c) to meet the minimum standards of SE requirements quality as defined by Turk (2006). Iterative model development and requirements specification with SME critique offers the opportunity to judge and refine CSE insight in relation to these specific objectives.
Discussion of DSS Requirements Drawn From CWA
The potential avenues for DSS development efforts within the EVA work domain are vast, as indicated by the 13 generalized functions identified in Figure 3. This article demonstrates how the CWA framework can provide a systematic process to scope, guide, and establish the DSS design purpose, which can be linked to a preliminary set of DSS requirements, thereby providing a detailed example of the requirements creation process. Construction of the CWA models incrementally revealed the constraints pertinent to EVA operations. First, the WDA identified the work and structure of constraints present within the EVA domain. The WDA grounded the development efforts of the ConTA and was valuable in structuring and prioritizing what envisioned purposes were appropriate to pursue for DSS development. Second, the ConTA elicited details related to the cognitive demands–specific work goals imposed on EVA domain operators and provided a mapping of activities to phases of operation. Particularly, when considering future systems within an envisioned work domain, DSS design must be grounded in “predictions on relevant empirical results abstracted from observations in context” (Dekker, Woods, & Mooij, 2002, p. 29). The constraints of future EVA operations (i.e., communication time-delayed operations) will be an extension of the constraints and work already present in the current domain; therefore, we must appropriately acknowledge the constraints that already exist to understand their implications for future operations.
While providing official “proof” that the aforementioned process meets the intended goals of formally synchronizing the CSE and SE requirements derivation process is beyond the scope of a single article, we can argue the following reasons why this process is helpful: (a) It delivers DSS requirements at an appropriate level of abstraction to convey the cognitive purposes/goals with expressed relations to the raw data required by the work domain; (b) it explicitly considers the work inherent to the work domain from a joint human-systems integration perspective; and (c) the process is traceable and tractable. As opposed to the FAN model, where the linkages expand organically in an ill-defined manner, we demonstrate the structured and repeatable application of decision ladders to analyze specific work functions identified in the WDA, which remains tractable. Additional requirements, such as representation design requirements and presentation design concepts, can be readily incorporated and associated to their parent CWR and IRR statements in the detailed requirements definition and design phases. In the event that requirement additions or modifications are warranted, the SoK statements provide suitable anchor points to incorporate updates to the set of requirements.
Table 5 shows an assessment of the derived CWR/IRR statements in regard to the characteristics of traditional SE requirements. The stated requirements meet the characteristics of being necessary, correct, and unambiguous because they were explicitly derived from SME knowledge of the work domain. Articulating requirements from each SME SoK enables the resultant requirements to be weighted and prioritized against one another, in addition to being traceable to their source CSE model elements. The CSE practitioner plays a key role in the translation of SoK statements to formal requirement statements. Ambiguity in requirements is minimized by iteratively critiquing the requirements statements with domain experts. Finally, the requirements are considered results oriented because the requirements are linked to the intended support of DSS design. Two characteristics, at this early point in the SE design process, are open to further assessment. Because we have neither translated the requirements into a representational prototype nor tested a prototype in a relevant operational environment, we cannot currently comment on the qualities of feasibility or measurability.
Assessment of Cognitive Systems Engineering–Inspired Requirements Characteristics Versus Traditional Systems Engineering Requirements Characteristics.
Finally, the insights gained from this work were formally recognized by the EVA management community as a valuable contribution to the early functional requirements derivation process currently being conducted. Prior to this research, the set of SE requirements for future EVA operations emphasized the physical/hardware-related components of the EV spacesuit and lacked a more comprehensive perspective of EVA work domain elements identified in this study. Particularly, we successfully articulated to the SE practitioners the utility and importance of considering the IV operator work and the development efforts and considerations that will need to be included in the overall EVA systems development process (for news coverage of the 2016 NASA@Work Mars EVA Gap Challenge, see http://www.nasa.gov/feature/nasawork-february-2016-monthly-winners).
Nevertheless, the identification of constraints present within a particular work domain is not necessarily a sufficient condition to generate useful DSS design (McIlroy & Stanton, 2015). When a system is being designed, requirements are the source from which interface design is built and assessed. However, few studies utilizing CSE methods emphasize the requirements specification process, and when they do, their efforts make no attempt to extend their requirements to audiences beyond the CSE community. This study supplements previous CWA applications by depicting how the CWA framework can not only extract meaningful insight from a complex work domain but also relate those CSE perspectives to the system design process by more explicitly articulating high-level design requirements. Rather than stating underspecified requirements or attempting to directly derive the interface representation, this study provides a case-based example of establishing cognitive purpose via the CWRs and articulating the set of information relationships required to address the aforementioned cognitive demands from which a DSS should be derived.
The ultimate goal within the CSE community is to bridge the gap between analysis (cognitive or otherwise) and useful DSS design. The process demonstrated in the article provides a tractable way to reason about the requirements of the DSS design. The design process is an iterative process, and its success depends on a deep understanding of the specifics of the user’s work (Deal & Hoffman, 2010). In particular, for envisioned systems, the nature of the user’s work will become more available the further along in the design process a practitioner progresses until the envisioned world becomes reality. First establishing the cognitive work and required information relationships is paramount before one can hypothesize the representational artifacts of DSS design. By decomposing the requirements definition process, we can more readily accommodate the iterative representation design process while maintaining linkage to the fundamental elements inherent to the work domain under investigation, which remains a needed area of future research (Lintern, 2005; Read, Salmon, & Lenne, 2015).
Conclusions
In this article, the requirements of a DSS for use in the EVA work domain were derived from CWA. We implemented an incremental analysis process that first established the EVA work domain and the associated elements, using an information flow model and AHs as part of the initial WDA. Second, we refined our EVA work domain understanding by delineating the operational phases and their associated work functions via a ConTA. Finally, we leveraged a decision ladder to convey and map SME SoKs associated with EVA life support system monitoring to a structured requirements format for DSS development. Specifically, at each analysis phase, the various CWA models were used to guide analysis efforts to arrive at a set of cognitive work and IRRs for DSS development.
The CWA framework provided a useful, systematic, tractable pathway to derive purposes and goals that an EVA DSS should strive to manage in the envisioned work domain. In addition, DSS requirements were purposefully framed to be compatible with the traditional SE requirements definition currently underway within the NASA EVA engineering community. Future work aims to extend the stated requirements into representational design requirements for DSS prototype development as well as human-in-the-loop testing of the DSS prototype situated within the envisioned work domain.