We've demonstrated several improvements to our {\W} algorithm with {\W}2. Namely:
\bi
\item Optimization - {\W}2 runs faster, can be applied to larger datasets, and doesn't sacrifice performance.
\item Explanation - {\W} and {\W}2 are simple implementations of Contrast Set Learning, an easy to explain, intuitive learning process.
\item Certification - {\W}2 performs as well or better than a multitude of software quality optimization techniques. Including parametric modelling techniques (SEESAW), drastic project changes, and multiple data sets based on various case descriptions.
\item Application - {\W}2 can be applied to find locally learned recommendations that offer potentially unconsidered avenues for software quality optimization. {\W}2 can find alternatives to potentially dangerous situations such as the Brooks' Law example and drastic project changes.
\ei

In comparing the merits of a model-lite, case-based approach to a parametric one, advocates of reconstructive memory such as Barlett~\cite{bartlett32}, Kolodner~\cite{kolodner83}, or Shank~\cite{shank77} argue that {\em we make it up as we go along}. In case-based reasoning (CBR), inference repeats every time there is a new query. Our reading of the papers at this conference is that, except for a few papers that deal with reasoning-by-analogy (e.g.~\cite{azzeh08}), most of this community avoids the model-lite approach of CBR.

Proponents of parametric models argue that there exist {\em domain-independent models} which can be {\em tuned} to local details.
In
this approach, reasoning can take the form of a data miner learning
values for tune-able attributes of a parametric  model like \eq{one}. In this way, learning can happen once
and users can use the tuned model for all future queries.  
%While few exceptions~\cite{weyuker07,tosun09,me09b,me10a,me10b},
%the literature does not support the existence of 
%domain-independent SE models:
%\bi
%\item Zimmerman et al.~\cite{zimmermann09}  studied 629 pairs of projects. In only 4\% of these pairs
%could a defect 
%prediction model learned from one project be applied to another.
%\item
%A recent survey by Kitchenham et al.~\cite{kitch07}  reviewed claims that effort estimation data can be usefully
%gathered in one company, then applied in another.
%They found the existing evidence inconclusive and even contradictory. 
%\ei
%In order to check if the above papers are isolated reports, or part of a more general pattern, we 
%reviewed all presentations made at PROMISE.
%Since 2005, at the PROMISE conference: 
%\bi
%\item
%There have been 68 presentations, 48 of which either tried a new analysis on old data or made 
%reports  that a new method worked for one particular project. 
%\item
%Nine papers raised questions about the validity of prior results (e.g.~\cite{me09f});
%\item Four papers argued that SE model generality was unlikely and/or impossible (e.g.~\cite{briand06});
%\item Only rarely (7/68) did researchers report generalizations from one project to other projects: 
%\bi
%\item Four papers reported a software quality predictor learned from one project that was usefully 
%applied to a new project (e.g.~\cite{tosun09,weyuker07});
%\item Three other papers made a partial case that such generality was possible (e.g.~\cite{boehm09}).
%\ei
%\ei

Unfortunately, these supposedly domain-independent models (like COCOMO) suffer from massive internal variance (see \fig{coc1}).
Previously, we have tried to manage internal variance of  this problem with SEESAW: an AI algorithm that sought stable conclusions
across the space of possible tunings within a parametric model. While a successful prototype, SEESAW has
disadvantages:
\bi
\item Dependency on  a particular parametric model
\item A requirement that all the data be in a format acceptable to that model
\item Too many arbitrary internal design decisions
\item Slow runtimes
\item A code base that proved too large to maintain, modify, and add support for more models
\ei
With a result supporting CBR, this paper finds little to recommend from SEESAW over the {\W}2 case-based reasoning tool. Standard CBR applies a query $q$ to find relevant examples from a set of cases $C$ using the retrieve-reuse-revise-retain loop of \fig{cbr}. {\W}2 extends standard CBR by learning an adaption of $q$, called $q'$, that retrieves {\em better} quality examples. Based on the analysis of~\cite{me10c} and this paper, we recommend {\W}2 on several grounds:
\bi
\item
{\W}2 finds similar, or better, results than SEESAW (see \fig{win-loss-tie}).
\item
{\W}2  is simpler to code:
200 lines of AWK as opposed to the 5000 lines
of LISP code used in SEESAW.
\item
{\W}2 is
faster to run: the above experiments took seconds for {\W}2,
but hours for SEESAW.
\item
{\W}2 is simpler to maintain since, in CBR, ``maintenance'' means nothing more than  ``add more cases''.
\item
{\W}2 makes no use of an underlying
model and is therefore free from the assumptions
of parametric modeling. Hence it can be applied to more data sets. For example,
SEESAW requires data to be in the COCOMO format but {\W} has been applied to numerous
data sets in other formats~\cite{me10c}.
\ei
Having said that, there is one situation where we'd recommend SEESAW over {\W}. Like all CBR systems,
{\W}2 needs cases. If there is {\em no} local data, then
SEESAW would be the preferred (only) option. 

Firstly, there is insufficient evidence in this paper to make the conclusion that CBR {\em always} beats model-heavy methods like parametric models. Neverthless,  these results clearly motivate further exploration and comparison between the value of CBR and model-heavy techniques. For example, at our lab we are exploring very fast clustering methods to support scaling CBR to very large data sets.

Secondly, there are at least two kinds of ``models.'' In the traditional model-heavy definition, models are specific {\em products} that can be applied to multiple domains. In the CBR model-lite definition, a model is a {\em process} that generates many products, each of which is customized to the particulars of a local domain. In this paper and ~\cite{me01c} we have seen the following advantages of CBR: easy implementation, fast runtimes, easy maintenance, able to be applied to more data, and out-performance of model-heavy methods.
