
%
%

\documentclass[letterpaper,twocolumn]{esapub}
\usepackage{alltt,bar,graphicx, times, url, xspace}
\usepackage{natbib}
\input{johann/tricks.sty}
\input{karen/tricks.sty}
\input{timm/tricks.sty}


\newcommand{\Section}[1]{\section{#1}}
\newcommand{\SubSection}[1]{\subsection{#1}}


%------------------------------------------------------------------------- 
\begin{document}

\title{
Parametric Analysis of
Antares Re-Entry Guidance
Algorithms using Advanced
Test Generation and Data
Analysis%
%\thanks{The research described in this paper
%was carried out at the NASA-Ames Research Center and
%West Virginia University under
%contracts with NASA.
%Reference herein to any specific commercial product, process, or service
%by trademark, manufacturer, or otherwise, does not constitute or imply
%its endorsement by the United States Government.
%}
}

\input{karen/name_affil}
\input{johann/name_affil}
\input{timm/name_affil}
\input{tony/name_affil}

\maketitle

\keywords{}

\begin{abstract}

Large complex aerospace systems are generally validated in regions local to anticipated
operating points rather than through characterization of the entire feasible operational
envelope of the system.  This is due to the large parameter space, and complex, highly coupled
nonlinear nature of the different systems that contribute to the performance of the aerospace system.
We have addressed the factors deterring such an analysis by applying a combination of technologies
to the area of flight envelop assessment.  We utilize n-factor (2,3) 
combinatorial parameter variations to limit the number of cases, but still 
explore important interactions in the parameter space in a systematic fashion.
The data generated is automatically analyzed through a combination of
unsupervised learning using a Bayesian multivariate clustering technique
(AutoBayes) and supervised learning of critical parameter ranges using
the machine-learning tool \TAR, a treatment learner.
Covariance analysis with scatter plots and likelihood contours are 
used to visualize correlations between simulation parameters and simulation 
results, a task that requires tool support, especially for large and complex models. 
We present results of simulation experiments for skip re-entry return scenarios.

\end{abstract}

\section{Introduction}

The any-time lunar return requirement for the Constellation Orion capsule drives algorithm 
development of skip re-entry guidance algorithms as well as hardware items such 
as configuration and sizing of the reaction control system.  A skip re-entry maneuver 
involves an initial aero-braking maneuver and then a skip out of the earth's 
atmosphere. At apogee of the skip maneuver, control is applied such that the 
capsule de-orbits and descends to a targeted landing zone.  For a successful 
landing, a significant number of requirements must be met (e.g, 
acceptable G-loads, fuel consumption, and on-target landing). 

The performance of the capsule depends on a large number of parameters and a major 
task is to establish safe ranges for those parameters. 
Exhaustive exploration of all parameter combinations
is infeasible for such a complex system, so, traditionally, parameters are randomly
sampled from a defined distribution for a statistically significant
number of runs (traditional Monte Carlo testing).  Vast amounts of
data can be generated that way, and manual inspection of this data is usually
confined to gross features of the solution (such as absolute compliance
with requirements).  Anomalous or unexpected data can easily be overlooked.

The creation of these practical and
 functional models is a resource-intensive activity, especially in
 terms of human intelligence.  Researchers  have recognized that ``designers must
 be able to examine various design alternatives quickly and easily
 among myriad and diverse {\em configuration possibilities}'' \cite{gray06}.
The number of configuration possibilities within a model, such as the re-entry system
described above, can be dauntingly large. A model with only 20 binary choices already has $2^{20}>1,000,000$
possible configurations, far beyond the capability of human comprehension.

Worse yet, when the models can be executed early
in the development life cycle, analysts face another problem.  For
example, software engineering for space systems is rarely a green field
design. Often, pre-existing physics models are available. Therefore,
early in the life cycle, system prototypes can interact with physics
models to develop usage policies for the software.  Given the ready
availability of super-computers, and even inexpensive LINUX clusters,
analysts may have to analyze gigabytes of data generated automatically
from simulators.


Accordingly, since 2000~\cite{me00e}, we have explored
sampling those configurations at random, running the resulting model,
scoring the output with some oracle, then using  data mining techniques to
find the configuration options that  most improve model output,
Here, we try a new combination of methods and tools to study the configuration
parameters on a software controller for   spacecraft re-entry:
\bi
\item
An n-factor combinatorial test vector generation algorithm to target
tests toward regions of the parameter space where interactions among parameters are key
to performance (Section 3).
\item
The \TAR minimal contrast set learner~\cite{me03c}; 
is a supervised learning method that returns the minimal
deltas between desired outcomes (hitting the target) and all the other outcomes (Section 4.1).
\item
EM clustering algorithms that are autogenerated by the \AB program
synthesis tool.
Clustering is an unsupervised learning method
that obtains the most probable estimates
for class frequency and the governing parameters (Section 4.3).
\ei

It was found that the combination of methods yielded more information
than any method used in isolation.
The operation of each algorithm can be intelligently
informed and usefully constrained by using the output of the other.
Combinatorial test exposes interactions between parameters, \TAR focuses 
the analysis on a small number of variables while {\AB} reveals
structures missed by \TAR. 

The rest of this paper discusses the problem domain and how it was analyzed
with combinatorial test, \TAR and \AB.
We show that data mining
in combination with {\em functional requirements}
can be used to determine best parameter ranges for such tasks as
landing a spacecraft skip re-entry scenario within a narrowly defined target zone.

\input{karen/sim_descrip}

\input{tony/combinatorial}

\input{timm/boretar}

\input{johann/autobayes}

\input{timm/conc}

\bibliographystyle{aa}
\bibliography{johann/refs,karen/refs,timm/refs,tony/refs}

\end{document}

