Flight Test Activity

This post presents a conceptual model of flight test activity to provide a common framework for organizing test management and conduct.

First, a heirarchy of scopes and a set of decisions associated with flight test activity.

A couple definitions, drawn from abstractions used in discrete event modeling and simulation:

• Event

  • A time-bound object associated with a single time

  • Examples of events: state transition, threshold transition, phase transition, indication of unexpected behavior

• Activity

  • A time-bound object with a beginning, a middle, and an end, bounded by start and stop times

  • Activity start and activity stop are also examples of events.

  • Examples of activities: run, mission/session, campaign, project, program, portfolio 

The smallest useful scope of test activity is a “Run.”

Anything less than a “run” doesn’t serve as an organizing unit of action that provides value for meeting test objectives.

From there, it is often useful to label collections at different scopes with different names. The graphic shows 5 levels above run.

It’s important to note that:

1. the collections can be as few as one element, and there’s no upper limit conceptually, though too many elements may make organizing activities unweildly

2. the collections don’t assume a linear sequence of its elements, which may executed in parallel and with arbitrary spacing

Short descriptions of each scope:

• “Run”

  • Beginning: Setup and Configuration Check

  • Middle: Test Maneuver

  • End: Recovery

• “Operation” is a collection of runs and equivalent to a flight test mission, a simulator session, or a propulsion test sequence

  • Start: Brief and Pre-Operation Actions

  • Middle: Run Execution

  • End: Post-Operation Actions and Debrief

• “Campaign” is a collection of operations, typically organized around a type of testing like low-speed taxi, first flight, flutter, loads envelope expansion, etc

  • Start: Campaign Kick-Off Meeting

  • Middle: Test Operations

  • End: Campaign Outbrief or Report

• “Project” is a collection of campaigns, typically organized around the system under test like a suite of software upgrades, a new airframe, a new subsystem integration

  • Start: Project Kick-Off Meeting

  • Middle: Project Management

  • End: System Delivery

• “Program” is a collection of projects, typically an ongoing activity organized around family of systems and their accompanying support components like “F-16,” “AMRAAM,” “787”

  • Start: Program Definition, Design, Plan

  • Middle: Implementation, Evaluation, Monitoring

  • End: Program Closeout and Resource Disposition

• “Portfolio” is a collection of programs, typically organized around a set of stakeholders like “domestic programs,” “commercial transport,” “IRAD”

  • Start: Portfolio Definition

  • Middle: Portfolio Management

  • End: Portfolio Closeout

What would you call a collection of portfolios? I’ve seen “Group” but that strikes me as too generic. Most flight test organizations don’t have that level of scope, anyways.

It’s tempting to conceptualize continuation decisions as a simple GO/NO-GO choice, as made popular in accounts of NASA launch status checks from movies and TV shows.

Complexity enters when there’s more than one “event” in play, more than just a “launch".”

An activity has a Before-During-After timeline. “During” is further decomposed into Beginning-Middle-End.

The consequences of decision timing can be very high in terms of safety, schedule, and budget, so explicitly defining terms can reap large benefits.

GO/NO-GO still have a place in the pantheon, but a more precise definition, and the addition of one more concept, will provide the basis for the different decisions available.

I believe this is a comprehensive list.

The 3 states below must be continuously assessed before, during, and even after the activity.

• GO: All resources required for an activity are available and meet continuation criteria. PROCEED with activity.

• NO-GO: Any resource required for an activity is unavailable or does not meet continuation criteria. ABANDON activity.

• NOT-YET: One or more resources for an activity is not currently available or does not currently meet continuation criteria, but there is reason to believe the state will transition back to GO.

The continuation decisions are phrased as directive statements for brevity to indicate test team action.

The timing of the state change affects which continuation decision is made. The figure provides a visual of the timeline. In roughly chronological order, the decisions and their definitions:

• CANCEL: A NO-GO decision to REMOVE the activity from the timeline

• CONTINUE: A GO decision to PROCEED with the activity AS BRIEFED

• CLEARED…: A GO decision to TRANSITION from BEFORE to the BEGINNING or from BEGINNING to the MIDDLE of the activity

• RECOVER: A GO decision to TRANSITION from the MIDDLE to the END of the activity

• SKIP: A NOT-YET decision to TRANSITION directly to the END of the activity before transitioning to the middle of the activity, without completing the BEGINNING or MIDDLE of the activity; often followed by an assessment of whether to attempt a repeat of the activity

• HOLD: A NOT-YET decision to maintain current conditions and to standby for an update; may or may not be directly associated with the activity

• ABORT: A NO-GO decision to CEASE activity and urgently JUMP to an abort procedure. Often triggered by meeting one or more safety-related abort criteria

• CEASE TEST: A NO-GO decision to transition to the END of an activity EARLY before the middle of the activity is fully complete without urgency; often triggered by not meeting technical constraints or efficiency expectations

• KNOCK-IT-OFF: A NO-GO decision that the activity is no longer relevant and to transition to safe condition

• KILL: A NO-GO decision to disable the system under test

• PAUSE: A SIM & TRAINING decision to pause the scenario to discuss or change the activity

• RESUME: A SIM & TRAINING decision to resume the scenario as discussed