Through the course of this work, we have presented a tool that uses AI search in order to produce project recommendations and estimates. The two major conclusions that we are able to draw from the empirical studies that preceeded are:
\be
\item 
Lack of data, whether for present or for future projects, is not a show-stopper in terms of our ability to explore possible policies to use. Rather, this can be exploited in order to produce recommendations that can reduce estimates substantially (see \fig{nasabraillep2}, \fig{exlarge}, \fig{exmedium}, and \fig{exsmall}).
\item
With a such a system, methods used in software engineering and development (like those presented in \chap{hohin} and \chap{bfc}) can be evaluted and tested. Doing so, we found that drastic change within a project is not advisable, and that optimizing for all models at once will yeild some compromises.
\ee

As for the summary of the empirical results that were presented in the seperate chapters:
\bi
\item
Chapter 3: Within the space of $Project$ options that most reduce $Estimation$ median and variance, the
predictions made by our process models are remarkably similar to those made by 
conventional methods (see \fig{deltas} and \fig{starlc}). 
\item 
Chapter 4: $Estimation$ median values can be greatly reduced (see \fig{nasabraillep2}). In comparisons with other effort estimation tools, the reduction can be quite dramatic. In the best case our tools found
$Project$ ranges that yields estimates that were 5\% of estimates found by other means. In addition, $Estimation$ variance can be reduced by only varying the $Project$ values and leaving the $Model$ values free (see \fig{ALLall} to \fig{OSP2tactical}). Also, varying a small amount of model values makes a big difference. Also, advanced \textit{Automated Software Engineering} tools are essential for software projects, and become more necessary as projects mature and near the end of their development cycle. They are also necessary for future projects (see \tion{aseexp}). These tools are also only recommended to be applied at there highest values.
\item 
Chapter 5: Two case studies are presented. For the first JPL case study, several results were observed. One result was that it confirmed our findings in \chap{ase} concerning the  importance of ASE tools. Another result was that the most constrained projects (ground 2000, flight 1970 and 1980) had the least reductions, while the most unconstrained project (flight 1990) exhibited the largest reductions compared to the initial estimates of that project template. This is a result that is also seen in \chap{ase}, as well as presented in a workshop paper by Menzies et. al~\cite{me07g}, where it is observed that ``if you fix everything, you have nothing else to fix''.
\item
Chapter 6: The study here showed that conservative change and policies that abide by project limits were a better, more consistent choice for project managers. Even though drastic change did seem to do better in some cases, the difference was either not significant enough or the strategy itself had serious side effects on other aspects of the project. 
\item
Chapter 7: BFC is a jack of all trades, achieving reductions accross all the models with moderation compared to the focused ``pick any two'' practices like FC for example for speeding project delivery.
\ei
%
%The first result suggests that it may be highly advantageous to use STAR.
%Projects designed around STAR's recommendations will be cheaper,
%delivered sooner and have fewer bugs.
%
%The second result is of much practical importance  since it means
%we do not require calibration data to tune the $Model$ variables. 
%If process models can be deployed without calibration, then they
%can be used with much greater ease and {\em without} the requirement for an expensive and time-consuming period of data collection. Also the fact that we only need a small subset of values set to make a large difference in project estimates is a compelling proposition for managers, allowing them to impart control easily and in a flexible manner.
%
%The third effect shows the importance of advanced ASE tool in the present and future of software engineering. This also shows the important role that technology will have in software engineering.
%
%The forth result is showing that (a)~this method can find and remove the major sources of uncertainty in a project;
%(b)~in the reduced space, it is possible that the estimates
%in the resulting constrained space will be close
%to estimates found via tuning on historical data.
%In the above discussion section, we commented that this result
%has precedent in the AI planning and  theorem proving literature.
%
%The last two results shows the flexibility of such a search based scheme, where using STAR we were able to predict future trends in software (even without the consensus of experts), as well analyze certain software management practices and assess them.

Finally, we comment on the external validity of these results.
Compared to many other process models\footnote{See
 Software Process journal,  issue on Software Process Simulation,
vol. 7, No. 3-4,  2002.} this combination of  effort/schedule/threat/defect models is relatively simple.
As model complexity grows, then the space of possible $Estimates$ can grow exponentially and STAR's
controlling effect may disappear. 
Therefore it is clear that   we can not claim that, for {\em all}
process models, that
$Estimate$ variance can be controlled by just constraining $Project$, not
$Model$, variance.

Nevertheless, data collection for the purposes of model calibration remains
as  a
expensive, tedious, and often incomplete process.
Our results suggest that such data collection may be, for some process models,
an optional activity (caveat: provided that a process model exists that
specifics the general relationships between concepts in a domain).
In this work, we managed to establish the following strategy:
\bi
\item finding the fewest number of variables that most effect model output;
\item constrain them;
\item check for stable conclusions in the constrained space.
\ei
If these results from STAR generalize to more complex models,
then is should be possible to make reasonably accurate
predictions without local calibration data.

Note that if such stability is {\em absent} in more complex models,
and those models are being used in domains with data collection problems,
then we would argue that that is a reason to abstain from such complexity,
and use COCOMO-style models instead.

\section{Future work}
This work has been used as a proof of concept for further work that is currently being undertaken by several other graduate students at the WVU LCSEE dept. These include the following:
\bi
\item In this work, we use simulated annealing. In other work ~\cite{menzies09} williams explores another stochastic search engine, SEESAW, to compare the recommendations of his version of STAR to DELPHI recommendations by experts.
\item Green \cite{green09} is using our design to assess the impact of value-based software engineering on standard process control based on \cite{huang05}.
\item Orrego~\cite{orrego09} is using our design to assess the value of reuse within the context of reducing effort, defects, schedule, and threats.
\ei
Aside from the above, other future work include deriving several other publications from this work.
