\begin{figure*}[!t]
\centerline{\includegraphics[width=6in]{figures/mat_den_SD.pdf}}
\caption{A visualization tool for scientific data. From~\cite{treinish98}.}
\label{fig:treinish}
\end{figure*}
\Section{Discussion}

The major effect apparent in Figure~\ref{fig:inp_par}, 
that \TAR misses an important region,  is quite  prominent. So is \TAR a 
sub-optimal
tool for this domain? 

We would argue not. 
Figure~\ref{fig:inp_par} was only rendered {\em because} \TAR's treatments concluded
that that particular pair of variables was important. Without \TAR, in the worst case,
an analyst would have to
examine $24^2/2-24=264$ plots. Further, if the important control effect was higher-dimensional,
then an exponentially larger number of 
plots would have to be examined. 
In fact, the historical genesis of TAR was when one of the authors took home for the
weekend 2,230 plots (printed 10 to a page) with the stated aim of ``flicking through them all looking
for what factors matter''. Very quickly, the cognitive overload of that task became apparent and the
plots were discarded.

To be sure, there are many impressive visual environments
for exploring multi-dimensional tools. For example,
\fig{treinish}~\cite{treinish98} shows a tool developed at IBM
that augments a standard
three-dimensional display with visual cues relating to higher-dimensional data; e.g.,  supplementary
dimensions are shown bottom right; blue circles around the axle show circular motion information;
and 
a color key, shown bottom left, indicates how colors in  the display relate to density information.
Such visualizations help analysts explore visual information, but they present their own
challenges. For example, note how the tool shown in \fig{treinish}
contains numerous controls that allow analysts to change various visualization
options. 
As the visualization environment grows more
sophisticated, some users find they have traded a data browsing problem with the new problem
of exploring the full range of the effects of all the controls.

Rather than declaring some method (e.g., visualizations, \AB, \TAR)
to be ``better'' than another, a more insightful question is: how to combine these
tools to take advantages of their strengths.
For example,
visualization environments are exciting and motivating to use.
Potentially, they allow users to uncover important insights that might be missed
by a fully automatic analysis.
Therefore, we should
always offer them to our users.
On the other hand,
visualization environments (and \AB) suffer from the curse of dimensionality.
\fig{treinish} is an impressive display of five dimensions (x,y,z, angular velocity,
density) but would not scale to, say, the 24 dimensions of the flight guidance system.
Similarly, 
\AB synthesizes functions, which can exhibit slow convergence and
numerical stability problems when dealing with high-dimensional data.
\TAR quickly prunes dimensions and defines regions of  where the input variables
have most impact. If used as a pre-processor
to visualizations or \AB, then the former would have few dimensions to display 
and the latter could scale to problems with higher dimensions. Further, a visualization environment
could augment its current displays with some distance cue to the critical regions
defined by \TAR.



Not only could \TAR be used as a pre-processor to \AB,
it could also be used as an  \AB post-processor.
\TAR uses \BORE and, in the language of the \AB clustering algorithm,
\BORE is simple categorical binary clustering tool that decides an example is either
``best'' or ``rest''. 
If dependent variables were classified via \AB' subtler clustering methods,
then \TAR-running-after-\AB could find nuances missed by the current implementation.

Similarly, not only could \AB improve \TAR's knowledge of the dependent variables,
but \AB could synthesize attributes that improve \TAR' understanding of the
independent variables.
Recall from Figure~\ref{fig:inp_par} that there exists a linear region of interest
with a negative slope flowing from top-left to bottom-right. In a manner analogous
to principal component analysis, \AB could synthesize new attributes (e.g.,
a straight line representing that region) and these could augment/replace, attributes
in the data sets past to \TAR.

Better yet, \AB could be used to augment our data miners with more
elaborate domain knowledge about variable distributions. This study
assumed that the distributions
 mixtures  were extremely simple.
However, if additional domain knowledge exists
about the specifics of the classes, data analysis can be made more effective.
For example, we might want to specify Class~1 as the ``landing on target''
class, where longitude and latitude of the landing spot is within narrow
margins of the designated landing area.
In a Bayesian framework like \AB, such extra domain modeling is 
very simple
using (conjugate) priors.
Figure~\ref{fig:priors} shows the modifications (with respect to the
specification in Figure~\ref{tab:mog-spec}) that are necessary to define
a conjugate prior on the mean for each class: we know
that the (unknown) mean $\mu$ is Gaussian distributed around a known $\mu_0$
(``target area'') with a known standard deviation $\sigma_0$ and a 
confidence factor $\kappa_0$. A similar specification can be written for
priors on {\texttt sigma}. 
Although this change in the specification is only minor, the resulting
algorithm is substantially different---another reason to use the automatic
code generation tool \AB.

\begin{figure}
{\footnotesize \addtolength{\baselineskip}{-0.40mm}
\begin{alltt}
\ul{const} \ul{double} mu_0(1..D, 1..C) \ul{as} 'expected means';
\ul{const} \ul{double} kappa_0(1..D, 1..C)  \ul{as} 'confidence'
      \ul{with} 1 < kappa_0(_,_);

\ul{const} \ul{double} sigma_0(1..D, 1..C); 
\ul{double} mu(1..D, 1..C);
mu(_i,_j) ~ gauss(mu_0(_i,_j), 
            sigma(_i,_j)*kappa_0(_i,_j);
\ldots
\ul{max} pr(\{mu, sigma, x\} | phi) 
    \ul{wrt} \{phi, mu, sigma\};
\end{alltt}
\addtolength{\baselineskip}{0.40mm} } \caption{Additions to AutoBayes 
specification for Gaussian mixture model with priors (additional
domain knowledge).}
\label{fig:priors}
\end{figure}



\begin{figure}[!t]
\centerline{\includegraphics[width=3in]{timm/old/all1.pdf}}
\caption{A data mining workbench. }
\label{fig:all1}
\end{figure}


\section{Conclusions}

In this paper, we have explored a combination two learners (\AB
and \TAR) to explore the internal state space of some flight guidance
software. The combination of these two technologies revealed features
that would have been invisible to either, if used in isolation. Further,
the experiment suggests some novel ways that these technologies could
usefully augment each other. Currently, we are working towards automating the  integrated toolkit shown in \fig{all1}.

More generally, given the growing importance of model-based reasoning in
software engineering, the ability to use data miners to find and constrain
the most important parts of our software models, should prove to be a technique
of growing importance in the years to come.
