Imitation {\em is} the sincerest form of flattery. As humans we tend to emulate our superiors in order to learn from them. For example, a writer may read their favorite author for inspiration, an artist may draw influence from the the works of the Renaissance, and a programmer may subscribe to the blog of a famous startup-turned-empire. From them we can gleam what separates the best of us from the rest of us, so that we might repeat their successes.

Our drive to imitate translates not only to personal traits, but software projects as well. Certainly a software project manager also seeks to imitate past sucesses and avoid prior failures. However, just as the artist studies another artist in a similar medium, the manager seeks guidance from projects in a similar ecosystem. One could generalize that from these similar projects, a general strategy is to examine the {\em best} performers, and adopt the attributes that are most unique to the {\em best} rather than the {\em rest}. We call this strategy {\em contrast-set learning}.

But how should we use this apriori knowledge to infer about new instances? Should we extrapolate from old data to build a parametric model; e.g. using  a Bayes net~\cite{fenton07a}, or the linear equations of COCOMO~\cite{boehm81,boehm00b}? Or is it best to reason directly from data, without an intervening parametric model,
using case-based reasoning (CBR)~\cite{shepperd97}?
 
This is a difficult question to answer, unless we restrict ourselves to a particular context.  In this paper, we adopt the context of {\em software quality optimization}; i.e.  adjusting a software
project such that we improve quality attributes such as  the  {\em defects} (number of delivered defects),  the {\em months} (calendar time to delivery) and the {\em effort} (staff time, in person months, required for that delivery).  This {\em quality optimization} task is different to {\em effort estimation}.  Effort estimators just predict measures on the {\em current} project while quality optimizers seek {\em changes} that most improve a project.

Quality optimization is a non-linear problem. Improving any one goal can  harm the others. For example:
\bi
\item If management rushes projects to completion, they decrease {\em months} but can increase {\em defects}.
\item Projects that adopt elaborate quality assurance procedures can reduce {\em defects} but 
at the cost of increased {\em effort}.
\ei
A quality optimizer must therefore trade-off between reducing months {\em and} defects {\em and} effort.

Prior work ~\cite{10promisew} compared two quality optimizers:
\be
\item
SEESAW is an AI algorithm that explores parametric models of software development, based on COCOMO.
\item
{\W} is a case-based reasoning algorithm that does the same task as SEESAW, without
using  a parametric model.
\ee
SEESAW was first introduced in~\cite{me07f} and has been applied to numerous domains~\cite{orrego09,me09f,me08c,me09e,me09a,me09i}. {\W} was first introduced in~\cite{me10c} but that report includes no comparisons with other quality optimizers. More recent work with W presented a simple quality optimization algorithm and compared it to SEESAW~\cite{10promisew}. From our comparisons between SEESAW and {\W} in~\cite{10promisew} we demonstrated: 
\bi
\item
{\W} finds similar or better optimizations.
\item
{\W}  is simpler to code:
200 lines of AWK as opposed to the 5000 lines
of LISP code used in SEESAW.
\item
{\W} is
faster to run: the following experiments took minutes for {\W},
but hours for SEESAW.
\item
{\W} is simpler to maintain since, in CBR, ``maintenance'' means nothing more than  ``add more cases''.
\item {\W} makes no use of an underlying model and is therefore free of all the assumptions of parametric modeling. Hence it can be quickly applied to more data sets. For example, SEESAW requires data to be in the COCOMO format but {\W} has been applied to numerous data sets in other formats~\cite{me10c}.
\ei

%this work extends prior results as follows:
%	optimization -> gives us stabilization
%	more extensive certification
%	explanation (slightly better story)

In this paper we extend our work with {\W} and SEESAW to further the discussion on model-based vs case-based reasoning methodologies. We relate {\W} and SEESAW's implementation to the general algorithm of contrast-set learning, and how this accessible approach is able to extract distinguishing attributes between two populations.

We extend {\W} into {\W}2, a optimization of the {\W} algorithm. Our discussion of {\W}2 offers the the following improvements over prior work:
\bi
\item {\em Optimization} - {\W}2 removes {\W}'s $O(n^2)$ nearest neighbor calculation and replaces it with a simple linear-time filter of the historical data. Despite this simplification, {\W}2 performs as well as {\W} (Figure ~\ref{fig:bestoverlap-vs-knn}).
\item {\em More Extensive Certification} - Prior work benchmarked the model-lite approach of {\W} with model-dependent approach of SEESAW. A direct comparision between {\W} and SEESAW required the exclusive use of the COCOMO model format. In this paper we have extended {\W}2's performance evaluation into new, arbitrary datasets in order to gauge {\W}2's usefulness in a wide variety of instutitions and applications.
\item {\em Better Explanation} - {\W}2 is even simpler to explain than {\W}. We have provided a visual example of {\W}2 that demonstrates the three main steps of {\W}2: Relevancy Filtering, Contrast Set Learning, and Holdout Set Performance Estimation.
\item {\em Direct Application} - When a software project veers away from perceived estimates, drastic changes may be needed to get it back on track. We compare the impact of several drastic management decisions, such as reducing functionality to cut development effort, to alternative decisions learned with {\W}2.
\ei

We conclude that, for the task of quality optimization, {\W}2 represents an improvement over {\W}(Figure ~\ref{fig:bestoverlap-vs-knn}). We also recommend {\W}2's case-based reasoning over SEESAW's parametric modeling (Figure ~\ref{fig:w2-seesaw-comparison}). For instances of drastic software changes, {\W}2 offers alternatives that can be equally effective in improving software quality (Figures~\ref{fig:drastic-ground},~\ref{fig:drastic-flight},~\ref{fig:drastic-osp},~\ref{fig:drastic-osp2}). Also, across multiple datasets, {\W}2's contrast-set learning demonstrates improvements in software quality regardless of the underlying data description (Figure~\ref{fig:other-reduce-values}).

While we offer no conclusion on the general merits of case-based reasoning compared to parametric modeling, we offer that {\W}2, contrast-set learning, and case-based reasoning are important tools for software quality optimization.
