\Section{Introduction}

It is surprisingly  difficult
to find relevant data within a single organization to fully
specify all the internal parameters inside a complete software process model.
For example, after 26 years of trying, Barry Boehm's team 
resaerchers from the Univestity
of Southern California collected less
than 200 sample projects for their COCOMO effort estimation database~\cite{me08c}.

There are many reasons for this, not the least being
 the business sensitivity associated
   with the data.
Software projects are notoriously
difficult to control:  recall the 1995
report
of the Standish group that described a \$250 billion
dollar American software industry where 31\% of projects
were canceled and 53\% of projects incurred costs exceeding
189\% of the original estimate~\cite{standish95}.
Understandably, corporations are reluctant to expose their poor performance
to public scrutiny. 

Despite this data shortage, remarkably effective predictors for software products have
been generated. In previous work we have built:
\bi
\item
{\em Software effort estimators} using only very high-level knowledge of the code
being developed~\cite{me05d} (typically, just some software process details).
Yet these  estimators offer predictions that are remarkably close to  actual development times~\cite{me06d}.
\item
 {\em Software defect predictors} using only static code features.
Fenton (amongst others) argues persauively that such features
are very poor charactizations of
the inner complexities of software modules~\cite{fenton97}.
Yet these seeingly naive defect predictors out-perform current industrial
best-practices
~\cite{me07b}.
\ei
The success of such simple  models seems
 highly unlikely. Organizations can work in different domains, have
different process, and define/measure defects and other aspects of
their product and process in different ways. 
Worse, all to often, organizations do not precisely define their
processes, products, measurements, etc. 
Nevertheless, it is true that very simple models suffice for generating 
approximately correct predictions for (say) software development time~\cite{me06d}, 
the location of software defects~\cite{me07b}.

One candidate explanation for the strange predictability in software development
is that:
\bi
\item
 Despite all the seemingly random factors
influencing  software construction \ldots
\item \ldots
  the net result follows very tight statistical
patterns. 
\ei
Other researchers have argued for a similar results \cite{Bell:2006p517, Ostrand:2004p514, Ostrand:2002p512, Ostrand:2007p519, GunesKoru:2007p478}
but here we offer new evidence. Specifically, we document the {\em early plateau}
effect seen with learning defect predictors and show that it also holds in a completely
novel domain:
\bi
\item
The performance of a data miner improves as the size of the training set grows.
\item
At some point, the performance {\em plateaus} and further training data does not improve that performance.
\item
Previously, we showed in one domain (NASA aerospace software)
that for defect prediction, plateaus occur remarkably early (after just 30 examples).
\item
In this paper, we show that this early plateau also occurs in 
Turkish whitegoods control software and two other open source software.
\ei
Observing an effect in one domain (NASA) might be a coincidence.
However, after observing  exactly the same effect in  two more
 unrelated domains (Turkish whitegoods \& open-source),
we can no longer dismiss the effect as quirks in one domain. Rather, we assert that:
\bi
\item The regularities we observe in software are very regular indeed;
\item We can depend on those regularities to generate effective defect predictors
using {\em minimal information from projects}.
\ei
This second point is a strong endorsement for our approach since, as discussed above,
it can be very difficult to access details and extensive project data from real world software development.

The rest of this paper is structured as follows. We give examples of statistical patterns from other research in Section 2. Then we introduce defect predictors and observed regularities in defect prediction research in Section 3. We discuss the external validity of these regularities in Section 4. Section 5 extends these regularities to open source domain. Discussions of our observations are given in Section 6 and we conclude our research in Section 7. 

Note that \fig{promise} and \fig{under}
 results of Section 3 have appeared previously~\cite{me07b,me08d,me08e}.
The rest of this paper is new work.

