The debate between case-based reasoning and model-based methods can be conducted on at least two levels:
\be
\item
At one level, it is an engineering-based discussion that assesses these approaches on criteria
like ease of implementation, runtime speed, and the observed output performance. 
\item
At another level of assessment, we can assess case-based vs model-based in terms of their cognitive
implications.
\ee
Since most of this paper is about level (1), the rest of this section discusses
level (2).

Platonic model-based reasoning is meant to seek out universal  truths. For example, Newton's agenda was to find a set of equations  (e.g. $F=ma$) that can be applied universally on earth, as well as to  well as distant planets and stars. He  succeeded.  In 1846,  rival astronomers John Adams (in England) and Urbain Leverrier (in France) raced to find  a previously unseen planet that was disturbing the orbit of Uranus. Neptune was first sighted by Adams, then Leverrier, after both men pointed their telescopes
at the precise point in the sky indicated by Newton's equations.

\begin{figure}[!t]
\center
{\scriptsize
\begin{tabular}{l|r@{:~}l|}\cline{2-3}
upper:   &acap&analysts capability\\
in theory&pcap&programmers capability\\
$\beta < 0$&aexp&application experience\\
&modp&modern programming practices\\
&tool&use of software tools\\
         &vexp&virtual machine experience\\
         &lexp&language experience\\\cline{1-3}
middle   &sced&schedule constraint\\\cline{1-3}
   &data&data base size\\
lower: &turn&turnaround time\\
in theory&virt&machine volatility\\
$\beta > 0$&stor&main memory constraint\\
&time&time constraint for cpu\\
         &rely&required software reliability\\
         &cplx&process complexity\\\cline{2-3}
\end{tabular}}


\includegraphics[width=3.3in]{figs/ranges.pdf}
\caption{COCOMO 1 effort multipliers, and the sorted coefficients found by linear regression from
twenty 66\% sub-samples (selected at random) from the NASA93 PROMISE data set; from~\cite{me05a}.
Prior to learning, training data was {\em linearized}
in the manner recommended by Boehm ($x$ was changed to $log(x)$; for details, see~\cite{me05a}).
During learning, a greedy back-select removed attributes with no impact on the estimates: hence,
some of the attributes have less than 20 results. 
After learning, the coefficients were unlinearized.
}\label{fig:coc1}
\end{figure}
We dream of the day that our SE models will achieve the 
same universality of Newton's equations. To date, we have not been successful.
Researchers like Boehm developed parametric models that predict development effort for software.
In Boehm's COCOMO parametric model (the 1981 version~\cite{boehm81}):
\begin{equation}\label{eq:one}
\small Effort = a * Loc^b * \prod_i\beta_ix_i
\end{equation}
where $x_i$ are one of the {\em effort multipliers} shown in \fig{coc1} (at top) and $\beta_i$ is a
coefficient that controls the influence of $x_i$. 

Such learning combines expert intuition with automatic reasoning. Expert intuitions define the general
form of the parametric model, while automated data mining fills in the details of that model.
For example, the goal of data mining over parametric models is to take local data and learn appropriate values for the tunable attributes. In the above model, those tunable attributes are $(a,b,\beta_i)$.

Based on linear regression over historical data~\cite{boehm81,boehm00b}, Boehm offers values to $(a,b,\beta_i)$ to three significant figures. Previously~\cite{me05a}, we have reported that such precision is somewhat optimistic since $\beta_i$ has a very large variance. The plot at the bottom of \fig{coc1} shows the $\beta_i$ values learned from twenty 66\% samples (selected at random) of the NASA93 data set from the PROMISE repository. While some of the coefficients are stable (e.g. the white circles of $loc$ remains stable around 1.1), the coefficients of other attributes are highly unstable:
\bi
\item The $(max - min)$ range of some of the coefficients is very large; e.g. the upside down black triangles of $stor$ ranges from $-2 \le \beta_i \le 8$.
\item Consequently, nine of the coefficients in \fig{coc1} jump from negative to positive.
\ei
We have seen instability in other datasets, including the COC81 data used by Boehm to derive \eq{one}~\cite{me05a}. This is an troubling observation. It seems that while Newton's equations let us precisely locate Neptune, Boehm's equations cannot point us exactly at which project attributes will lead to lower effort.

Parametric modeling assumes that (i)~one parametric form (e.g. \eq{one}) is universal across 
multiple domains and (ii)~that form is tuned to the local situation by adjusting some tuning attributes. 
An opposite approach to  parametric models is case-based reasoning (CBR). In CBR, there are no universally-applicable parametric models. Rather, every conclusion is dependent on the particulars of the task at hand. CBR is based on a theory of {\em reconstructive memory}. According to this theory,
humans do not remember things as they actually happened. Rather, ``remembering'' is an inference
process, characterized by Bartlett as:
\begin{quote}
{\em ... a blend of information contained in specific traces encoded at the time it occurred, plus 
{\em (retrieval time)} inferences based on knowledge, expectations, beliefs, and attitudes derived from other sources~\cite{bartlett32}.}
\end{quote}
Bartlett's work was ignored when first published (1932) but today it is highly influential; e.g.
experts in psychology \& law caution reconstructive memory means that {\em leading questions} can significantly alter a report given by a human witness~\cite{loftus03}.

In AI research, Janet Kolodner~\cite{kolodner83} used reconstructive memory to characterize expert explanations.  To support her claim, she offered a set of transcripts of experts explaining some effect.
Her reading of those transcripts was that the experts do not use {\em verbatim recalling}  when discussing the past. Rather, they {\em reconstruct} an account of their expertise, on the fly, in response to a particular query. CBR inference is usually characterized~\cite{aamod94} in four steps:
\be
\item \textit{Retrieve}: Find the most similar cases to the target problem. 
\item \textit{Reuse}: Adapt our actions conducted for the past cases to solve the new problem.
\item \textit{Revise}: Revise the proposed solution for the new problem and verify it against the case base.
\item \textit{Retain}: Retain the parts of current experience in the case base for future problem solving.
\ee
\begin{figure}[!h]
\begin{center}
         \includegraphics[scale=0.45]{figs/cbrcycle1.png}
         \caption{Four steps of CBR, from \protect\url{http://www.peerscience.com/intro_cbr.htm}.}
         \label{fig:cbr}
 \end{center}
\end{figure}

Having verified the results from our chosen adapted action on the new case, the new case is added to the available case base. The last step allows CBR to effectively learn from new experiences. In this manner, a CBR system  is able to automatically maintain itself.

In terms of cognitive theory, CBR challenges notions of reasoning as model-building.
The mantra of CBR is ``don't think, remember''. That is, when faced with some new situation:
\bi
\item
Do not reason it out using some underlying model (e.g. Newton's equations or Boehm's parametric models).  
\item
Rather, respond to a new situation via an on-demand survey of past experiences \cite{schank83}. 
\ei
CBR challenges the premise of the PROMISE conference series. Currently, this conference bills itself as ``Predictive Models in Software Engineering''. This title assumes that model building is the best way to analyze software engineering. However, if {\em model-heavy} methods like COCOMO do worse than {\em model-lite} CBR methods, then we would need to rethink the premise of PROMISE.

(Note that we call CBR {\em model-lite}, but not {\em model-free}. For more on this distinction, see the {\em Discussion } section, below.)
