\Section{Background}
%
%The two branches above differ from each other by the level of program comprehension they demand. Hardest
%case analysis require deeper understanding of  the problem domain, the specific problem, implemented solution 
%along with the assembled documents and the source code. Hard case analysis should carried out by full Fagan inspections or its variations,
%where there is a certain need for program comprehension. This can be achieved through training programs 
%for inspection teams, which are shown to improve inspection performances \cite{Rifkin:1994p567, Rifkin:2000p568}.
%Furthermore, Dunsmore states that the levels of comprehension are unclear among 
%different inspection approaches, however using checklists, scenarios or visualization tools help inspection team focus on parts of software, 
%removing the need for full comprehension \cite{Dunsmore:1998p566}. We argue that LHF is a powerful tool for this purpose. It suggests
%that discovering simple statistical patterns in software can be as effective as and faster than HCA with 
%minimum comprehension. Then, we should ask if (a) statistical patterns can be generalized for any kind of software and (b) if they can be 
%applicable in practice.  
%
Previous research reports much evidence that
software products confirm tightly to simple and regular
statistical models,
For example,
Veldhuizen shows that library reuse characteristics in three unix based systems (i.e. Linux, SunOS and MacOS X), can be explained by Zipf's Law, that is most frequently used library routines \cite{Veldhuizen:2005p565}
are inversely proportional to their frequency ranks. As shown in   Figure \ref{fig:zipf},
the distribution is highly regular.
 
\begin{figure}[!t]
\begin{center}
\begin{tabular}{ccc}
\includegraphics[width=3in]{zipf.pdf}
\end{tabular}
\end{center}
\caption{Distribution of reuse frequencies in three unix based systems. From \cite{Veldhuizen:2005p565}.
}\label{fig:zipf}
\end{figure}

\begin{figure}
\scriptsize
\begin{center}
\begin{tabular}{rccc}
&\multicolumn{3}{c}{$100 * (pred - actual)/actual$}\\\cline{2-4}
&50\% & 65\% & 75\% \\
& percentile & percentile & percentile \\\hline
mode=embedded&-9&26&60\\
project=X&-6&16&46\\
all&-4&12&31\\
year=1975&-3&19&39\\
mode=semi-detached&-3&10&22\\
ground.systems&-3&11&29\\
center=5&-3&20&50\\
mission.planning&-1&25&50\\
project=gro&-1&9&19\\
center=2&0&11&21\\
year=1980&4&29&58\\
avionics.monitoring&6&32&56\\\hline
median&-3&19&39
\end{tabular}
\end{center}
\caption{158 effort estimation methods applied to 12 subsets of the COCO81 data.
}\label{fig:coc}
\end{figure}

\fig{coc} shows a summary of our prior work on effort estimation~\cite{me06d} using the COCOMO
features. These features lack detailed knowledge of the system under development. Rather,
they are just two dozen ultra-high-level descriptors of (e.g.) develeper experience, platform
violatily, software process maturity, etc. In \fig{coc}, different effort estimation methods
are applied to the COC81 data\footnote{Available from \url{http://promisedata.org}.} 
used by  Boehm to develop the original COCOMO effort estimation model~\cite{boehm81}.
Twenty times, ten test instances were selected at random and effort models were built from the remaining
models using 158 different estimation 
methods\footnote{For a description of those methods, see~\cite{jalali07}}. The resulting predictions
were compared to the actual estimation times using relative error 
$\left(\frac{pred-actual}{actual}\right)$.
COC81's data can be divided into the 12 (overlapping) subsets shown left-hand-side of \fig{coc}.
The right-hand-columns of \fig{coc} show  MRE at the median, 65\%, and 75\% percentiles. The median
predictions are within  within 3\% of the actual. Such a close result would be impossible
if software did not conform to very tight statistical regularities.


\fig{pareto} shows
Koru and Liu's analysis of  two large-scale open source projects (K-office and Mozilla).
As shown in those figures, these different systems confirm to nearly an identical
Pareto distribution of change-prone classes: 80\% of changes occur in 20\%
of classes~\cite{GunesKoru:2007p478}.
The same 80:20 changes:fault distribution has been observed by
 Ostrand, Weyuker and Bell in 
very large
scale telecommunication projects from AT\&T~\cite{Bell:2006p517, Ostrand:2004p514, Ostrand:2002p512, Ostrand:2007p519}. Furthermore, the defect trends in Eclipse also follows similar patterns, where they are explained better by a Weibull distribution \cite{Zhang:2008p1052} .
 
 
\begin{figure}
\begin{center}
\begin{tabular}{ccc}
\includegraphics[width=3in]{pareto.pdf}
\end{tabular}
\end{center}
\caption{Distribution of change prone class percentages in two open source projects. From \cite{GunesKoru:2007p478}.
}\label{fig:pareto}
\end{figure}

\begin{figure}
\begin{center}
\begin{tabular}{ccc}
\includegraphics[width=3in]{eclipse.pdf}
\end{tabular}
\end{center}
\caption{Distribution of faulty modules in Eclipse. From \cite{Zhang:2008p1052}.
}\label{fig:pareto1}
\end{figure}

\section{Regularities in Defect Predictors}
 Figures~\ref{fig:zipf}, \ref{fig:coc}, 
\ref{fig:pareto} and 
\ref{fig:pareto1} show that statistical regularities
simplifies predictions of  certain kinds of properties.
Previously~\cite{me07b}, we have exploited those regularities
with data miners that learn defect predictors from static
code attribtues.
Those predictors were learned
either from projects previously developed in the same environment or
from a continually expanding base of current project's artifacts. 

To do so,
tables of examples are formed where one column has a boolean value
for ``defects detected'' and the other columns describe software
features such as lines of code; number of unique
symbols~\cite{halstead77};  or max. number of possible
execution pathways~\cite{mccabe76}.  
Each row in the table holds data
from one ``module'',  the  smallest unit of functionality. Depending on
the language, these may be called ``functions'', ``methods'', or
``procedures''. The data mining task is to find
combinations of  features that predict for the value in the defects
column.  

The value of static code features as defect predictors has
been widely debated.
Some researchers vehemently oppose them~\cite{fenton00b,shepperd94},
while many more endorse their use~\cite{%
journals/ese/KhoshgoftaarS03,%
conf/ictai/TangK04,%
me07b,%
nagappan05,%
nikora03,%
porter90,%
srinivasan95,%
tiang95%
}. Standard verification and validation (V\&V) textbooks
\cite{rakitin01} advise using static code complexity attributes to
decide which modules are worthy of manual inspections. 
The authors are aware of
several large government software
contractors that won't review software modules {\em unless} tools like
the McCabe static source code analyzer predicts that they exhibit
high code complexity measures.

Nevertheless, static code attributes can never be a full
characterization of a program module. Fenton offers
an insightful example where {\em the same} functionality is achieved
using {\em different} programming language constructs resulting in
{\em different} static measurements for that module~\cite{fenton97}.
Fenton uses this example to argue the uselessness of static code
attributes for fault prediction.

Using NASA data, our fault prediction models find defect
predictors~\cite{me07b} with a probability of detection ($pd$) and
probability of false alarm ($pf$) of $mean(pd,pf)=(71\%,25\%)$.
These values should be compared to
baselines in data mining and industrial practice.
Raffo (personnel communication) found that industrial reviews find \mbox{$pd=TR(35, 50,
65)\%$}\footnote{ $TR(a,b,c)$ is a triangular distribution with
min/mode/max of $a,b,c$.} of a systems errors' (for full Fagan
inspections~\cite{fagan76}) to \mbox{$pd=TR(13, 21, 30)\%$} for
less-structured inspections.
Similar values were reported at  a IEEE Metrics 2002
panel. That
panel declined to endorse claims by Fagan~\cite{fagan86} and
Schull~\cite{shull00a} regarding the efficacy of their inspection or
directed inspection methods. Rather, it concluded that manual
software reviews can find ${\approx}60\%$ of defects~\cite{shu02};

That is, contrary to the pessimism of Fenton,
our $(pd,pf)=(71,25)\%$ results are 
better than currently
used industrial methods such as the $pd{\approx}60\%$ reported
at the 2002 IEEE Metrics panel or the $median(pd)=21..50$ reported
by Raffo.
Better yet, automated defect predictors can be generated
with a fraction of the effort of alternative methods,
even for very large
systems~\cite{nagappan05}. Other methods such as formal methods or
manual
code reviews may be more labor-intensive. Depending on the review method,
$8$ to $20$ lines of code (LOC) per minute can be inspected. This
effort repeats for all members of the review team (typically,
four or six~\cite{me02f}).

%Question (b) can be answered by analyzing the cost effectiveness of above studies. For example, Arisholm and Briand argues that a defect predictor 
%is useful in practice if $X\%$ of the code to be inspected contains $Y\%$ of defects and $X \textless Y$ \cite{Arisholm:2006p516}. Above studies satisfy Arisholm and Briand's rule, 
%since the majority of target classes reside in a minor piece of code in all studies.  
%
% which is a strong argument
%for the widespread exploration of data mining for learning defect predictors.
%
%For example,
%in this paper we focus on defect prediction. Authors' prior work includes extensive experimentation with straightforward
%data mining application to defect prediction \cite{me07e,me07b,me04g,me04c,me03q,me03k,me02l,me02g,me02e,jiang07, turhan07a, turhan07b}.. A recent TSE publication by one of the authors shows that a simple learner, Naive Bayes, outperforms many
%others for predicting defects in NASA software \cite{me07b}. The predictor exhibits $mean(pd, pf)=(71\%,25\%)$ performance, which is significantly better than the reported 
%industrial averages of $pd=TR(35,50,65)\%$ for full Fagan inspections, $pd=TR(13,21,30)\%$, for less structured inspections,  $mean(60\%)$ as
%concluded in  IEEE Metrics 2002 panel. We will demonstrate the generalization of that research on Turkish white-good's controller software and an open
%source anti-virus software. We will also show that this pattern does not only hold for local software repositories, but also for imported repositories.
%Further, it will turn out that these patterns can be discovered from only a handful of examples rather than hundreds or thousands. We have published similar results in our previous research \cite{me07i}, however their validation on open source projects and extended discussions are provided for the first time in this paper. 
%
%
