
\section{Discussion}

The results here are quite unequivocal -- activity is a strong predictor for software defects, and this effect can be detected
with a simple model such as linear regression. Hence, we need to explain why this effect has not been reported before.
We conjecture that the use of a small set of activity-centric static metrics
is \emph{too simple} and \emph{too novel} a concept to be reported previously. 
%Also, the popularity effect may have been \emph{obscured by other effects}.

\subsection{Too Simple?}
We can broadly classify object-oriented software quality research as (a)~studies with more focus on prediction models than the metrics, and (b)~studies with more focus on metrics validation than the models (as in this study). It is no surprise that the former kind of studies did not explicitly investigate the concept of activity, as they usually operate within existing sets of common metrics in order to choose the best model among many. The literature offers many complex methods for data mining such as support-vector machines, random forests, and tabu search to tune the parameters of a genetic algorithm (\ie,~\cite{taghi05a,taghi05,taghi00a,lessmann09}). In this era of increasing learner complexity, something as easy as linear regression on a small set of static code measures aiming especially on activity
may have been discounted before being explored rigorously. Therefore, our first explanation is that the use of activity as a concept \emph{is so simple} that it escaped the attention of this type of research. 

Nevertheless, we cannot ignore the latter type of studies, in which the focus has usually been validating object-oriented metrics as predictors of defects through correlational methods (\ie,~\cite{basili96, briand99, cartwright00, elemam01, aggarwal09}). Briand \& W{\"u}st provide an extensive survey of empirical studies of quality in object-oriented systems, and observe that the majority of studies falls in this category~\cite{briand02}. However, they also state that only half of the studies employ multivariate prediction models, and the other half just reports univariate relations between object-oriented metrics and defects. Further, only half of the studies with a prediction model, conduct a proper performance analysis through cross-validation. After this filtering, remaining work contains hard-to-compare empirical studies, where the size and the number of data sets are so small that the combined results are conflicting and do not reveal a common trend possibly due to varying contexts of the studies.  

Another aspect of related studies is that they consider certain subgroups of object-oriented metrics relating to concepts such as coupling, cohesion, inheritance and polymorphism, and size~\cite{briand02}. Briand \& W{\"u}st report that the significance of the relation between different subgroups of metrics and defects are mostly inconclusive, and only a number of size and coupling measures are consistent. We have further run a smaller-scale review of major studies conducted with the guidelines of the original survey~\cite{basili96, Briand00a , briand99, cartwright00, elemam01a, briandMW02, arisholm06, aggarwal06, aggarwal08, aggarwal09}. Similar to Briand \& W{\"u}st, we observed that the table of metrics vs. different systems used to assess those metrics were sparsely populated. 
%
%Further, we noticed that 
%the number of cases where a metric was not selected for a predictive model always exceeds the number of cases that it was included, 
%\ugh{except for size metrics}, for which there are many other studies confirming their clear relation with defects (\ugh{see}~\cite{elemam01}). \raj{May need to expand on what we mean by size metrics. Harrison98a is a better reference for a relationship between size and defects. Supplement with Fenton00a, Koru07a, Moser08a -- but all of these are based on small data sets, but should be ok for our purpose. El Emam is better associated with the confounding effect in literature, so better to avoid.}

\subsection{Too Novel?}
The starting point for this research was the observation in the Helix data sets that most classes stabilize very early in their life cycle while a very small number of active classes garner the most attention by developers~\cite{Vasa10a,vasa:09a}. As discussed above in Section~\ref{sec:pop}, this is not the standard picture of the life cycle of a class. To us, this observation was so unique that it prompted the question ``does the amount a class is used by developers predict for system defects'' (\ie, this study). However, without that initial surprising observation, we would not have conducted the study reported in this paper. 

Compared to other studies (\eg, studies surveyed in~\cite{briand02}),
the size and number of data sets used in our study is extensive and
reveals a clear %picture of activity 
benefit of using activity-centric
metrics in the context of
open-source object-oriented systems.
In contrast to our concept of activity, 
Turhan~\etal~\cite{turhan08aa} investigate popularity.
%the concept of popularity, rather indirectly compared to this study. 
Their approach is to augment standard static code metrics within a
call graph-based ranking framework, which is inspired by the PageRank
algorithm~\cite{brin98}. Rather than constructing learners with a
standard set of metrics that value each module equally, Turhan~\etal
first rank the modules using the dependency graph information and
weigh the information learned from ``popular'' modules more. Their
approach reduced the false alarm rates significantly. However, this
technique is an indirect way of utilizing activity, and does not
include explicit activity metrics that are used in this study.
Similarly, Zimmermann~\etal include {\em eigenvector centrality}, a measure of {\em closeness centrality} of network nodes similar to PageRank, in their analysis of complexity vs. network metrics for predicting defects from software dependencies~\cite{zimmerman08}. Though, eigenvector centrality is found to be correlated with defects for the Windows system they have evaluated, this metric did not stand out among other network or complexity based metrics to allow a discussion on ``activity'' (see the next section below for a possible cause). Finally,  Kpodjodo~\etal monitored their proposed, again PageRank inspired, {\em Class Rank} metric among several versions of a single system and found moderate evidence in favor~\cite{kpodjedo09}. In this paper, we handle activity as a concept rather than %trying to capture it with 
rely on
a single measure, and we achieve near optimal results compared to moderate improvements of similar work. 

\subsection{Hidden?}
Thirdly, it is possible that \emph{activity was buried} under other effects. When we look at the attributes we have used in previous studies (\eg,~\cite{me07b}), we can see some overlap between those attributes and those of \fig{metrics}. Miller~\cite{miller02}, Witten \& Frank~\cite{witten05}, and Wagner~\cite{wagner07aa} offer a theoretical analysis discussing how an excess of attributes containing multiple strong predictors for the target class can confuse learning. For example, both Wagner and Miller note that in a  model comprising $N$ variables, any noise in  variable $N_i$ adds to the noise in the output variables. 

We have also observed supporting evidence for this explanation in our small scale quality-in-object-oriented-systems review. In all cases, where both a univariate and a multivariate analysis is utilized, it is common for metrics verified by the univariate model not to be included in the multivariate model for the same data~\cite{aggarwal06, aggarwal08, aggarwal09, basili96, briand99 ,elemam01a}. El Emam~\etal use this phenomenon to control for the confounding effects of size on metrics believed to serve as suitable predictors for defects~\cite{elemam01}. Similarly, the multivariate model metrics may include those that are not verified by the univariate model~\cite{Briand00a, arisholm06, elemam01a}, for which Guyon~\etal provide simple examples showing that the prediction power can be significantly increased when features are used together rather than individually~\cite{guyon03}. Hence, even though some measures exist in a data set, noise from the other variables may have drowned out their effect.  

