
%------------------------------------------------------------------------- 

\section{Related Work}
\section{Does Explanation Always Reduce Performance?}


\PARstart{D}{iscretization}

Other problems with prec and accuracy. neg/pos bad

explanation not as an algorithm thing but a user-specific thing.
leake. hayes.
\subsection{KDD Cycle}

\PARstart{T}he specific technical contribution of this paper
is a new high water mark in learning how to explaining runaway projects.
The more general contribution of this paper is an overview
of effective methods for
knowledge
discovery in databases (KDD) in software engineering. 
An analyst in the $21^{st}$ century can access a plethora of data
mining methods such as those in the WEKA toolkit of \fig{weka}.  Potentially, data mining can discover important
structures in historical data that can be used to better control
future projects.  However, before that, the analysts must understand
how to use the tools.  


The KDD process has been characterised by
Fayyad~\cite{fayyad97},
as the cycle of
\fig{kddcycle}. Note that data mining appears in that cycle, along with numerous other tasks:
\bi
\item
data preparation, 
\item
data selection, 
\item
data cleaning, 
\item
incorporating appropriate prior 
knowledge, 
\item
and proper interpretation of the results .
\ei
In this view, data mining is just one step within  KDD; i.e.
the 
application of specific algorithms for extracting patterns from data.
Fayyad cautions that
the 
blind application of data mining methods (sometimes called ``data dredging'' in 
the statistical literature) without the rest of the KDD cycle
can be a dangerous activity 
easily leading to the discovery of meaningless patterns.

\begin{figure*}[!t]
\begin{center}
\includegraphics[width=6in]{kddcycle.pdf}\\
\end{center}
\caption{The KDD (Knowledge Discovery in Databases) cycle, from 
\protect\cite{fayyad97}.}\label{fig:kddcycle}
\end{figure*}

KDD is normally discussed in the context of very large
databases~\cite{provost99survey,hulten02mining}. Yet its processes apply, even to
small data sets like \fig{data}.  For example, this paper uses KDD to find a
model that is more explaniable that prior results, as well as
performing better. That model was found via: \bi \item {\em Data
preperation} to {\em discretize} the columns; \item {\em Data
selection} to {\em prune} irrelevant columns; \item As recommended
by \fig{kddcycle}, we explored \fig{data} via   13 cycles.  In
each cycle, a data mining was called. The results of each cycle was
used to select the activity in the next cycle.  \ei 

It is important
that industrial practioners understand the KDD process.  
In our work with commercial software engineers, we 
see a common misconception-  that all they need to do is
run the data through a learner.
We 
find ourselves repeatedly advising them that running the learners is the least
complex of their tasks. Rather, most of their effort will be devoted
to
the rest of the KDD process; i.e.
pre-processing the data before running the learners,
then
intepreting the results.

On the other hand, it is important that these practioner also
understand that executing the KDD process is not always a complex task.
For example, in this case study, the data preperation and data
selection operators were not particularly complex.



\section{Conclusion}
quirky. supposedly state-of-the-art methods that usually work best may not work best on a particular data sets. e.g. fayyad irani,

ripper

when geenrating explainations, brevity is useful. discretize to reduce range. wrap to remove columns

An important feature of this study is that is it reproducible using 
open source data mining
 toolkits (e.g. the WEKA tool shown in  \fig{weka})
and the data shown in \fig{data}.
Reproducability 
is an important methodological principle in other disciplines since it allows a community to confirm,
refute, or even improve prior results. 
In our view, in the field of software engineering,
there are all too few examples of reproduced, and extended, results\footnote{
Exception: see  the reports of the PROMISE
workshop
\url{http://promisedata.org/repository/papers.html}}. 
This report begain when the second and third authors
published their data~\cite{takagi05} and defined a research challeng: how to
better explain the results of their learning to developers~\cite{abe06}.
The first author found a way to meet that challenge using the
methods described in this paper.


