
Software
engineering is modeling~\cite{schmidt06}.  
Software engineers build
models, be they paper-based or executable,
at every phase of the life cycle.


The creation of practical and
 functional models is a resource-intensive activity, especially in
 terms of human intelligence.  Researchers  have recognized that ``designers must
 be able to examine various design alternatives quickly and easily
 among myriad and diverse {\em configuration possibilities}'' \cite{gray06}.
%Finding an appropriate and effective balance between human intelligence and automation can be a recondite task.  
The number of configuration possibilities within a model can be
dauntingly large. A model with 20 binary choices has $2^{20}>1,000,000$
possible configurations, far beyond the capability of human
comprehension.  

Worse yet, when the models can be executed early
in the development life cycle, analysts face another problem.  For
example, software engineering for space systems is rarely a green field
design. Often, pre-existing physics models are available. Therefore,
early in the life cycle, system prototypes can interact with physics
models to develop usage policies for the software.  Given the ready
availability of super-computers, and even inexpensive LINUX clusters,
analysts may have to analyze gigabytes of data generated automatically
from simulators.


Accordingly, since 2000~\cite{me00e}, we have explored
sampling those configurations at random, running the resulting model,
scoring the output with some oracle, then using  data mining to
find the configuration options that  most improve model output,
Here, we try a new combination of methods and tools to study the configuration
parameters on a software controller for   spacecraft re-entry:
\bi
\item
the \TAR minimal contrast set learner~\cite{me03c}; 
Contrast set learning is a supervised learning method that returns the minimal
deltas between desired outcomes (hitting the target) and all the other outcomes.
\item
EM clustering algorithms that are autogenerated by the \AB program
synthesis tool
\cite{buntine-fischer-pressburger-99-kdd,fischer-schumann-03-jfp,gray-etal-02-nips};
Clustering is an unsupervised learning method
that obtains the most probable estimates
for class frequency and the governing parameters
(e.g., for Gaussian mixture models mean $\mu$ and standard deviation
$\sigma^2$ for each class).
\ei

What we found is that the combination of methods yielded more information
than either method used
in isolation.
The operation of each algorithm can be intelligently
informed and usefully constrained by using the output of the other.
\TAR focuses the analysis on a small number of variables while {\AB} reveals
structures missed by \TAR. 

The rest of this paper discusses the problem domain and how it was analyzed
with \TAR and \AB. The authors acknowledge that this work is not a standard
PROMISE paper. 
Here, we strive to make data mining a routine
part of any model-based software engineering development.
For example, we don't apply data miners to generate
predictors for {\em non-functional requirements} such as learning defect predictors
from static code measures. 
Rather, we show that data mining
can used to study {\em functional requirements}
such as how software settings
effect how
close we can land
a spacecraft to a target zone.







