\chapter{Conclusions}\label{chap:conclusions}
This thesis addresses the question of whether ITL can be 
successfully used as a metaheuristic search technique
for model-based development.
To address this question we investigated
models from two stages of the software life cycle.
\Chap{casestudies} experimented with requirements models,
often used in the earlier life stages of the software
life cycle.
\Chap{properties} experimented with digital
logic circuit models, often used during the testing stage,
\ie late stage, of the software life cycle.
This chapter we will discuss conclusions
from our work with requirements models,
\tion{surferCon}, and conclusions
from our work with the SPY framework, \tion{spyCon}.

\section{Conclusion from Early Life Cycle Models}\label{sect:surferCon}
We did a comprehensive study of three requirements models
to investigate some preference characteristics of
ITL and our new discretization method, extreme sampling.
We found
\begin{itemize} \itemsep +4pt
\item repeated success with \holo model and extended success to \aero and \cob models
\item \bore is currently the best version of extreme sampling
\setstretch{1.0}
\item larger batch sizes, up to 500, increase performance on requirements models
\item the good/bad ratio should be in the range 15\% to 25\%, or possibly higher
\item \bore has high stability and low variance
\item \bore clearly outperforms diagonal striping on
  \begin{itemize} \itemsep -4pt
  \item solution quality
  \item speed of convergence
  \end{itemize}
\item \bore outperforms simulated annealing on speed of
  convergence, while finding similar quality solutions
\end{itemize}

\section{Conclusions from Late Life Cycle Models}\label{sect:spyCon}
We used five models written in the SPY language,
two by hand and three translated from Simulink,
to investigate the capabilities of the SPY
framework.
We found that 
\begin{itemize} \itemsep +4pt
\item SPY could find property violations in temporal models with real-valued inputs
\item SPY is able to find range restrictions that constrain model behavior
\setstretch{1.0}
  \begin{itemize} \itemsep -2pt
  \item SPY found these restrictions quickly, often only 2-5 iterations
  \item SPY could tell the difference between relevant and irrelevant parameters
  \end{itemize}
\item the search bias of the underlying learner can lead to over restrictive bounds
\item discrete worth functions reduce learning efficiency.  Worth may need to
  \begin{itemize} \itemsep -2pt
  \item be augmented with additional heuristics, like path coverage
  \item include hints as to how close intermediate variables were to correct values
  \end{itemize}
\item translation framework preserves model semantics
\end{itemize}

The availability of analytic techniques for one of
our models allows us to say conclusively that SPY may find
overly restrictive bounds on input variables,
but that the bounds were always correct.
While this is a result of the search bias
of the underlying learner (maximizing true positives
without penalizing false negatives),
an improved search strategy in SPY may correct this.

The discrete nature of behavior mode identification
caused significant problems in one of our models.
Optimizing low cardinality discrete functions, particularly boolean
functions, will be a problem for any stochastic search.
Addressing this problem will involve new methodologies
for developing objective functions, rather than
a change to the search strategy.

The only area where domain knowledge or apriori mathematical
knowledge was needed was in picking the original bounds
on the parameters.

%%% Local Variables: 
%%% mode: latex
%%% TeX-master: "../thesis"
%%% End: 
