\begin{kasten}
    \section*{ \hspace{0.1cm} {\color{red} \underline{METHODOLOGY}}}
    \large{
    In our research, we have used subsets of three commonly-used datasets in software effort estimation research: Nasa93, the original 
Cocomo81 \cite{Boehm1981}, and Desharnais\cite{Desharnais1989}.
\vspace{3 mm}
We will denote the subsets of Nasa93 as Nasa93c1, Nasa93c2 and Nasa93c5 that contain projects from development centers 1, 2 and 5 respectively.
In a similar fashion, subsets of Cocomo81 will be denoted as Coc81o, Coc81e and Coc81s (for
organic, embedded, and semidetached).
Finally, the
Desharnais dataset is split into three different subsets:
DesL1, DesL2 and DesL3 (languages 1, 2 and 3 respectively).
Since each of these subsets have certain common criteria (the development center, development mode, or development language), 
each subset will be treated as a separate {\em within} dataset.
All of the datasets used in this research are available in PROMISE data repository \cite{Boetticher2007}.

\vspace{3 mm}
For each of the three main datasets (Nasa93, Cocomo81 and Desharnais) in our research, we have conducted {\em within} and {\em cross} 
experiments.
Each subset became a {\em within} dataset that contains projects sharing the particular characteristics
of a single development firm.

\vspace{3 mm}
To understand the {\em within} and {\em cross} data formation, assume that a dataset $X$ with its three subsets $X_1$, $X_2$ and $X_3$ 
is under consideration.
For {\em within} experiments, relevancy filtering described is applied on each one 
of $X_1$, $X_2$ and $X_3$ separately and the median of the filtered project instances in the training set is stored as the effort estimate 
for the test instance.
For the {\em cross} experiments, one of $X_1$, $X_2$ or $X_3$ is chosen as the test set and the combination of the remaining two forms 
the {\em cross} dataset for training.
This time, the relevancy filtering is applied on the {\em cross} dataset, and the estimations for projects in the test set are stored.
Each of the {\em within} and {\em cross} experiments are repeated twenty times in order to remove any bias that would otherwise 
be brought by a particular test and training set combination. 

\vspace{3 mm}
In order to compare the performance of {\em within} and {\em cross} datasets,
we have used two measures: the magnitude of relative error (MRE) and win-tie-loss values generated by a statistical rank-sum test.
%\begin{equation} \small
%MRE = \frac{|actual_{i} - predicted_{i}|}{actual_{i}}
%\label{EquationMRE} 
%\end{equation}
Using a Mann-Whitney test (95\%), we checked how often one treatment won/lost/tied with the others. Here, a ``tie''
means
that they are not
statistically significant different.
If statistically different,  then the method with a lower median MRE score gets one more ``win'' and the other method gets one more ``loss''.
    }
\end{kasten}

\begin{kasten}
\section*{ \hspace{0.1cm} {\color{red} \underline{RESULTS-Without Relevancy Filtering}}}

 \begin{minipage}{17cm}
      \scriptsize \include{big-ass-chart}
    \end{minipage}
    
\vspace{3 mm}
Every odd and even line is a pair of experiments.
In each pair, there is a {\em within} and a {\em cross} experiment. 
In {\em cross} experiment, a linear regression model is built on {\em cross} data and tested on the {\em within} data. 
In {\em within} experiment, the test instance is selected with leave-one-out, 
and a linear regression model is built on the remaining instances and tested on the selected test instance.
A ``1'' denotes which item in the pair won, lost or tied.
	\vspace{-.5em}
\end{kasten}
