Artificial Intelligence in the cockpit

In today's world, change has become the norm, not the exception. We are faced, not with a peer competitor who is well known and well understood, but, instead, with adversaries whose location and capabilities are highly variable. Our forces are being called to fight in places where the terrain and the pre-existing infrastructure vary greatly. And they must be prepared to perform a wide spectrum of missions - from peacekeeping and humanitarian activities to full-scale, traditional warfare, with the resulting rapidly shifting rules of engagement. The technologies critical to the U.S. for new weapons systems have become increasingly available in the global marketplace and thus are accessible to our potential adversaries as well.
The arrival of the first CV-22 osprey on Sept. 18, 2000 ushered in a new era in aviation. It can take off and land like a helicopter, but once in the air its engines rotate and it becomes a turboprop airplane (2001) Congressional Testimony). The MV-22 is the highest priority for Marine Corps aviation.... The V-22 will fly twice as fast, several times further and with a heavier payload than the helicopters it replaces. The plan is for the Osprey eventually to be used by all branches of the military. It also has navigational beacons and radios, radar altimeters and an internal intercom/radio system for communications among the crew and troops onboard.
AFSOC -- which has headquarters at Hurlburt, Field, Fla. -- is confident in the belief that the CV-22 will enable the command to do what it can't do now ((1998) Helicopter News). That is, conduct long-range, high-speed, vertical lift missions in an aircraft capable of getting troops into and out of an area in one night.
CIE is Crew Intent Estimation and it is to ensure that the crew is always in charge of the Rotorcraft Pilots Associate (RPA) function - they're always working towards the same goal and never pursuing old or counterproductive behaviors (1999) Helicopter News). In essence, CIM is the 'personality' of the RPA, whereby all functions are unified under CIM. It appears to be watching the crew, tracking their behavior, and providing coordinating information to planners and assessors attempting to support the crew, without having to ask! In this way, the intelligent systems are always up to date. The CIE is an essential component of the RPA’s crew-vehicle cockpit interface, known as the Cockpit Information Manager (CIM) function.
Intent interpretation is the process of identifying patterns of human operator behavior in order to form a causal explanation for observed actions in terms of the current objectives of the operator. The outputs of this process are utilized by intelligent subsystems to support changing operator goals in a dynamic environment.
Employment of the CIE in the associate allows the numerous complex-aiding functions of the RPA to remain lock step with the crew as the mission unfolds. CIE interpretation behavior, goal-processing activities, knowledge representation approach, and external communication mechanisms with other intelligent RPA subsystems are described with emphasis on CIE’s essential role in the coordination of RPA functions to accurately follow the crews lead and quickly regain coordination when the crews intentions change. Of particular interest is the RPA teams significant accomplishment in the area of combining prescriptive, automated task coordination with descriptive, intent understanding for producing intelligent associate behavior. An Associate is defined (Rouse, Geddes, and Curry, 1990) as an intelligent computer aiding system that has:
· A shared sense of purpose with the operator
· A shared understanding of the world through knowledge representation
· Intelligence to be helpful and coordinate tasks,
· And authority to act on the operators behalf, but not without consent.

Pilots Associate systems are fundamentally a suite of intelligent functions designed to assist the pilot, or crew, in executing the mission using advanced aircraft, avionics, and weapons systems.
One of the fundamental rules of associate system behavior is the pilot is always in charge. (Rouse, Geddes, and Curry, 1987). This rule remains consistent with military operations, where the mission commander has ultimate authority over mission assets, and each cockpit has a senior authority.
Similar to a human co-pilot, RPA acts as a decision aiding entity aware of mission objectives, knows how to execute tasks, and will flexibly and dynamically aid the crew in support of the mission objectives. As an automated crewmember the RPA must act in kind. In order to support these advanced behavior requirements, the RPA must be able to accurately answer the question: What is the crew attempting to accomplish? With moment to moment tracking of the crew’s intentions, the CIE transmits the dynamic desires of the crew to RPA planners and assessors so that they can support the crew proactively, just as a well-trained human associate would. The CIE determines what the crew is doing through the only means it has: look at the numerous switch activations, stick and collective movements, sensor usage, etc. that are occurring in the cockpit and figure out what is going on. It is CIE’s job to determine what these actions mean right now, in the current situation. This explanation is formed based on current understanding of intent, situations in the external world, and knowledge about acceptable behavior in the cockpit.Understanding crew intentions is really only one function of CIE that ensures human-centered RPA behavior. CIE also dynamically identifies conflicts between the current crew intentions and RPA activities as posted by RPA’s internal Task Network Architecture (TNA) scheduler. The TNA is the mechanism that coordinates RPA aiding activities. CIE can identify potentially counterproductive RPA behaviors and transmit an indication of conflict and also type of conflict so that the RPA Task Network can possibly identify a way to complete its scheduled tasks without being counterproductive to the crew’s objectives. This design is the first time in associate systems development where an intent estimation system is used for both identification of dynamic crew intentions and for coordination of task-network based intelligent support functions (Geddes (1985).
The key to representing intentions in the RPA is the Plan-Goal Graph (PGG). The PGG is a graphical decomposition of the purposes of the RPA like scout and attack helicopter operations in terms of the goals to be pursued and the plans employed to satisfy goals (e.g., avoid obstacles, successfully execute target, etc.). The Plan-Goal Graph is a directed acyclic graph that is both an abstraction class hierarchy and a compositional class hierarchy representing the purposes of the agents in the RPA system. Plans represent activities that can be performed by agents or groups of agents to achieve goals. The most concrete and primitive nodes are actions of agents, where an action is a direct operation by the crew or an RPA agent. The PGG decomposition expresses a strict set of relationships. Goals are decomposed into task children, who are successively decomposed into either subgoal children or actions. The parent-child relationship between any two nodes of the PGG is subject to constraints that reflect the underlying causal principles of the system and the normal behavior of agents within it. The PGG is a class structure; that is, each node in the PGG represents a class of goal, task or action that can be set during the intent interpretation process (Hammer, and Small (1995).
So how does the RPA know when to do what? Or what is a Crew Intent Estimator (CIE), and how does it support associate behavior in the RPA? A fundamental behavior characteristic of an associate system is that even though the system can carry out several tasks automatically for the crew, ultimate authority for action is the purview of the human crew.
There are two primary goals of CIE in the CIM:
· Allow CIE to output information about the goals of the crew and identified plans for pursuing the goals, thereby providing the RPA with context specific information about what the crew wants the RPA to do, moment by moment.
· Allow CIE to identify conflicts between the current crew intentions and RPA activities (from the Task Network) that would indicate counterproductive RPA so that the RPA and crew are always working together towards common goals.
In order to support the coordination goal, a secondary, but nonetheless important role of CIE is to integrate the CIE intent model with the existing RPA Task Network Architecture (TNA). Recall that the TNA is a task-decomposition network that coordinates all other RPA functions by specifying what sequences of tasks the other functions should execute to support the crews desire for their outputs (Andes, R.C.) (1996). This secondary role is designed to support bonafide crew centered aiding and coordination from within the RPA by combining task thread activity (associate activity) with crew intent (crew activity). The TNA and CIE freely exchange task nodes and CIE plan and goal nodes so that the RPA will take direction from crew intent and the crew will be aware of RPA’s tasks simultaneously (Andes, 1996). In allowing the RPA to begin data transmission when a visual detection of a threat is made can lead to counterproductive, or even worse, fatal behaviors. (TNA and PGG) come at the problem from different sides of the human-computer interface: The TNA provides task-analytic representations of both crew and RPA tasks, while the PGG provides a crew-intent based representation of human and RPA tasks. More specifically, the TNA is prescriptive in determining the currently active tasks (and next to execute) while the PGG is descriptive in determining what thread of intentionally the crew is pursuing. This required a third objective for CIE:
Provide an intelligent communication mechanism between the two knowledge representation schemes that will provide the following advantages over using one or the other exclusively: thereby providing CIE with task-thread specific information about what tasks the RPA believes the crew is/should be doing from the TNA’s perspective. By satisfying all three objectives, the CIE provides context-sensitive, accurate estimations of dynamic crew intentions while simultaneously coordinating a suite of intelligent agent functions based on the crew’s lead without explicit interaction.
CIE is fundamentally an abductive intent interpreter the CIE determines what the crew is doing through the only means it has: look at the numerous switch activations, stick and collective movements, sensor usage, etc. that are happening in the cockpit and figure out what is going on using the PGG. (Andes, R.C., Jr. (1987). CIE infers what the crew is doing by considering the actions of the crew in terms of their mission goals and plans for achieving them. As an example a severe move of the stick could mean that the pilot is doing a planned, on-route maneuver, or could indicate an actions-on-contact situation, depending on the presence and suspected lethality of a threat. Basically, every action issued by the crew is interpreted by CIE by matching the action into the PGG. The PGG is constructed for the domain by mapping all possible cockpit actions into the PGG decomposition as leaf, or bottom level nodes. When the crew issues and action, CIE attempts to explain that action within the current context. Working from the bottom of the graph up to the top, CIE attempts to find the best explanation for that action under the circumstances. This explanation is formed based on current understanding of intent, situations in the external world, and knowledge about acceptable behavior in the cockpit. The plans and goals associated with that explanation is sent out by CIM to planners and assessors so that they can remain lock-step with the crew towards reaching the mission goals. In this way, maintenance of current intentions, tracking of consistent actions within the current intent space, transient behaviors, and drastic changes in intent (e.g., switch from reconnaissance to defensive actions, etc.) can be tracked. Details about crew intentions are available to all RPA consumers. One of the primary mechanisms of tracking crew intentions is in the identification and maintenance of goal-based behaviors in the cockpit. The detection of goal-based behavior is used by higher level functions in the RPA to assist in the coordination of co-operating system functions. CIE maintains active instances of identified, monitors goals satisfaction conditions, and notifies interested RPA consumers of goal-state changes.
In order to conduct this function, CIE goals contain state conditions based on interesting attributes in the operational world. States define the system status that must be achieved in order for a goal to be satisfied. A goal's state is represented as a list of constraints. A constraint is a mathematical statement that consists of an operator and its necessary operands. Goals are processed during each CIE execution cycle. Every time CIE is involved it activates a method to satisfy and update the state of all active goals affected by changes to objects in the context model. When a goal's state is achieved, CIE responds according to the recurrence type of the goal. By employing the different goal types, CIE maintains an estimate of the crew’s objectives.
Successful semantic integration of the PGG with the TNA provides the necessary checks and balances to orchestrate crew behavior with RPA support. In this way, the RPA achieves associate status --- knowing what to do without being explicitly told by the crew. The communication is two way: The TNA will communicate upcoming tasks (active and executing) from the active task flows to the PGG, while OPAL will communicate newly activated plans and goals to the TNA. The former flow provides the crew with information regarding what the RPA will do next, while the latter provides the RPA information on what tasks the crew is currently pursuing. The link is two-way for completeness in the system design.
In conclusion the purpose for having the CIE in RPA is to ensure that the crew is always in charge of the RPA’s functions – they’re always working towards the same goals and never pursuing old or counterproductive behaviors. In essence, CIM is the functional liaison between the crew and RPA, whereby all functions are unified under CIM. It appears to be watching the crew, tracking their behavior, and providing coordinating information to planners and assessors attempting to support the crew, without having to ask. In this way, the intelligent systems are always up to date and lock-step with the crew. Success of this function lies in its ability to coordinate the RPA with the crew.
The CIE is an abductive intent interpreter, where tracking of context is the most subtle, but essential mechanism for coordinating all of the intelligent RPA functions. The two mechanisms goal-processing and the integration of prescriptive and descriptive reasoning -- represent significant enhancements in the field of intent interpretation. This design provides tighter integration of supporting associate functions in the RPA, while providing the ability of the system to check its own behaviors against the changing intentions of the human crew. It provides an introspective mechanism to the RPA to guard against counterproductive, and potentially catastrophic behavior by the associate by detecting and / or reducing behavior conflicts between the crew and the RPA.
The Osprey promises to save lives by virtue of its superior capabilities, but this promise cannot be realized without the assumption of risk (2001) Helicopter News). Modern-day rotorcraft is tremendously complex and utilizes a vast array of advanced technological systems. Most of these systems are automated, but they need to be administered properly by the pilots (2000) Helicopter News).
With an internal air filtering system for both the cabin and cockpit, the CV-22 will protect special operations forces from nuclear, biological and chemical agents. The CV-22 is the only known aircraft with this capability.