Each classifier will be run many times to insure a significant number of trials are run. For the Within Company experiments, each trial will start by randomizing the datasets to eliminate any artifacts caused by data ordering. Next, the dataset will be broken into $n$ separate parts, each of an equal size. In series, one is used as the testing dataset, and the remainder are used as the training dataset. This is referred to as a n-way cross validation\cite{nway}. N-way cross validation is important to the experimental procedure because it verifies that the classification algorithms are able to generalize the knowledge learned, and perform well on previously unseen data. I ran 5 random orderings of the datasets, and a 5-way cross validation. This will equate to 25 trials per dataset.

In Cross Company experiments, each trial will again start by randomizing all datasets. In series, each dataset is a testing dataset, and the remaining datasets are combined to form the training dataset. When performing a Cross Company experiment, it is not possible to do the standard n-way validation of splitting the data into $n$ equal sized bins because no part of the testing dataset can exist in the training dataset, or the purpose of Cross Company would be violated. The Cross Company experiments are also subjected to 5 random orderings of the datasets. The random orderings, coupled with the modified n-way cross validation will result in $5 * N$ trials, where $N$ is equal to the number of datasets being explored.
