
\section{Introduction}

How is it possible that we can predict software development effort
with any degree of accuracy (e.g. 
within 30\% of actuals, 70\% of the time~\cite{chulani99})?
The space of possible projects is vast, especially compared
to the small number of examples found in effort estimation data sets. 
For example, the effort estimation datasets used in
Mendes et al.~\cite{Mendes2003}, Auer et al.~\cite{Auer2006},
Baker~\cite{baker07}, this study, and Li et al.~\cite{Li2009} have
median size (13,15,31,33,52), respectively. We find it higly unlikely
that so few examples can cover the trends seen in modern software engineering-
especially if those examples were 
evenly spread across the space
of possible projects.

Clearly, effort estimation succeeds since
projects {\em clump together} in tiny regions of the total space.
Estimation works by reasoning about similarities found in those tiny clumps.
This paper asks the question {\em what is the best way to find those clumps}?




One method, supported by this paper, we shall call  ``{\em locality(N)}''.
This method
assumes that
effort estiamtion data  forms a  complex multo-dimensional space that can only
be  usefully divided {\em using multiple features}.
If $locality(N)$ is true, then it  would be {\em useless} to divide data by:
\bi
\item
Just the {\em organization} that the data comes from;
\item
Or just the {\em kind of software}being developed (e.g. embedded, desktop-app, etc);
\item
Or just the {\em physical location}  of the different developers;
\item
Or just the {\em application domain} (MIS\footnote{MIS=management information system}, 
GNC\footnote{GNC= guidance, navigation, and control}, etc);
\item
Or just the {\em targetted hardware platform};
\item
Or any other seperation {\em based on a single feature}.  \ei In the
field of effort estimation, $locality(N)$ is {\em not} a common
assumption.  Many
publications~\cite{stukes98,stukes91,stukes99,ferens98,habib98,lum02,boehm00a}
including several of our own~\cite{menzies05a,Menzies2006} either
explictedly or tacitedly assume ``$locality(1)$''; i.e.  clumps of
similiar projects can be discovered {\em using a single  feature}.  For
example, some authors claim that, for a project in a specific
organization, software effort models work better when calibrated with
local data collected within that same organization. Proponets of such
a {\em within company} (WC) approach assume that it is best to divide
the training data according to the value of a single feature called
something like ``{\em companyName}''.
 


Confirming (or refuting) $locality(1)$
is an open and urgent question for effort estimation.
If $locality(1)$ is true, then any lessons learned
from one organization may never apply to another.
If so,
then our ability to
make general conclusions about software engineering
be confined to small,
highly specialized,
sub-groups (e.g. just one company).

If, on the other hand, $locality(N)$ were true, then 
that would mean that
relevant data for effort estiamtion does not come just from
small sub-groups within one organizaton. Rather,  useful data could be collected
from many projects including cross company (CC) sources. 
This would simplify the construction of effort models for new projects:
just search other contexts for the right data 
for the new project.
Also, it could lead to conclusions about software engineering
that are general to many  development contexts.

We begin by making two predictions that would support $locality(N)$ and
would contradict $locality(1)$:
\begin{quote}{\em
\begin{fore}
Effort models built from training data divided on a single  feature
will perform no better (and, perhaps, even worse) than those that divide the data
using multiple features..
\end{fore}
\begin{fore}
Consider project data that has  been divided
according to the value of one l feature.
If training data is selected from {\em within} and {\em cross} those divisions,
then 
it would be $equally$ as probable to find useful data {\em within} as {\em cross} those
divisions.
\end{fore}
}
\end{quote}
Recent research offers much support for
{\bf PREDICTION~1}.
In a study
of 90 effort estimation methods (ten pre-processors and  nine
learners),
we found that the best methods
were those that divided the training data according to multiple features~\cite{keung11}.
That result is detailed in our {\em Related Work}
section.

Turning now to {\bf PREDICTION~2},
 this paper tests if  divisions
based on {\em single} or {\em multiple} features improve effort estimation.  
We will allow an {\em instance selection} algorithm to find which instances are best
for training. That algorithm will be given access to all the training data, or
just the data divided via a single feature.
It  will shown that:
\begin{quote} {\em 
The probablity of selecting
training data from within or across divisions based on single features \underline{is the same}.}
\end{quote}
This result, plus the results of Keung et al., are strong support for $locality(N)$
since they confirm both {\bf PREDICTION~1} and {\bf PREDICTION~2}.
These results have numerous practical implications:
\bi
\item Effort estimation is not dependent on some artifical, and useless,
 division of training data such as 
the source organization; the kind of application; or any other single feature
division of the data. 
Given the complex multi-dimensional nature of the software creation process, 
divisions of training data based on (say) geographical dimensions may be less important than other factors. 
The most similar software to what you are writing now may not be in the next office. 
Rather, it may be in an office on the other side of the world 
(and automatic {\em instance
selection algorithms} can  find that data).
\item It is useful to build effort estimation models from data taken from multiple contexts.
Hence:
\bi
\item Building estimation models is less expensive
since generating a local models need not wait for an elaborate
(and expensive and time-consuming) collection process from local data.
\item
There are effects in software engineering that trasncent our current divisions
of data (e.g. data from company1 or company2). This is a very exciting result since it
promises future generalizable SE results that apply to many organizations.
\ei
\item It offers a strong business case for collecting SE data in some sharable repository.
Such repositories can now be trusted to provide, at least some, relevant historical examples
for building effort estimators for some new project.
\ei
Before continuing, we 
offer one terminilogy clarification.
Other papers such as
Kitchenham  et al.~\cite{Kitchenham2007} and
Zimmermann et al.~\cite{Zimmermann2009} 
discuss the impact of dividing data according to a single
feature (either 
the {\em organization} or {\em application} name).
This paper explores those single-feature division {\em as well as} other single feature divisions
such as the {\em kind} of software being developed; the {\em physical location}
of the developers within
one organization; the  {\em application domain}; and the {\em targetted hardware platform}. 
So whereas (e.g.) Kitchenham et al. explore cross-vs-within organizational data sources,
we explore cross-vs-within data ``{\em sources}''\footnote{ekrem: dont say company unless
u mean kitch. say sources instead}  where the data is divided by the values
of any single feature.
As shown below,
we find no examples where single feature division improves a state-of-the-art
effort estimation method.

