\documentclass{article}
\usepackage{url}
\usepackage{timtricks}
\newcommand{\W}{$\mathcal{W}$}

\begin{document}
\section{Conclusion}

\noindent {\em To attain knowledge, add things every day.\newline
To attain wisdom, remove things every day.\newline
                        -- Lao-tse}
\newline

Managers must make management decisions about changes to software projects.
Currently, they have only very limited guidance from the SE literature on how
to make the fewest number of most effective changes to their projects.
Hence, we have spent several years exploring methods for guiding managers
towards better choices.

Originally, our work focused on model based methods.
Models have many advantages such as representing and visualizing expert domain knowledge.
Also, models let us
extrapolate from past observations 
to new situations that may not have been seen previously.
As demonstrated
by the
recent increase in conferences devoted to the construction and exploration of
models (e.g.
MODELS\footnote{\url{http://ecs.victoria.ac.nz/Events/MODELS2011}} and SBSE\footnote{\url{http://www.ssbse.org/2011/}}),
there is much current interest in model-based software engineering, 

Since models seemed so useful, in previous work,
we have  employed model-based methods to find changes for software projects.
This paper reports the surprising result that a small extension to standard instance-based
methods (a greedy search over the neighborhood of some query, divided into a $best$ and $rest$
region) out-performs numerous model-based methods. Specifically, when compared to model-based
methods, the {\W}2 instance-based
method:
\bi
\item Is faster to run.
\item Is simpler to code;
\item Is easier to maintain (just add more cases);
\item Is faster to adapt to data sets from new domains;
\item Finds equivalent or better ways to improve projects;
\item Scales to large problems (since it runs in log-linear time)
\ei
Following on from this report,  there many issues that could motivate future work:
\bi
\item Are model-based methods worse for noisier data?
\item Is ``data cleansing'' recommended for regression, but deprecated for  instance-based methods?
\item How best to reduce spread, thus increase the confidence a user has in the results?
\item Why does {\W}2 performs as well as more elaborate model-based methods?
\ei
As to this last point,
we conjecture that the kinds of process data we can collect from projects are a {\em shallow source}
of knowledge. With such {\em shallow sources}:
\bi
\item
Very simple methods can plum their depths;
\item 
More elaborate methods may do no better than the very simple.
\ei
As evidence for this conjecture, we note that our data sets are often very small (e.g. the 
15 rows of KEMERER or the 18 rows of TELECOM). Such  small data sources may only hold
very limited, and very shallow, structures. If so, then sophisticated AI search algorithms
may find little more than the simple greedy search of {\W}2. 


This work has focused on software process data. Nevertheless,
instance-based methods (such as the {\W}2 algorithm used
in this study) can be effectively used in many other fields.
For (very long) lists of application areas of instance-based reasoning, see
Kolodner~\cite{kol93}.
Aamodt~\cite{aamod94}, 
Lenz~\cite{lenz98}, Shepperd~\cite{She02} and  the proceedings of the International Conference
on Case-Based Reasoning\footnote{\url{http://www.iccbr.org/iccbr10/html/program.html}}.
While our results {\em do not} show that {\em all} instance-based methods are better than
{\em all} instance-based methods, 
they do motivate more investigations of case-based methods.
Before researchers elaborate their model-based methods,
it may be both theoretically insightful (as well as pragmatically useful) to build
an instance-based version of their method. Based on our experience, we predict
that such an instance-based method would be simple to build and, at the very least,
provide a baseline against which it is possible to demonstrate the value of
more elaborate systems.
\end{document} 
