The next several sections continue by exploring some possibilities for future research we believe could shed more light on the relationships between quality metrics and defect detection.

\subsection{Increasing Pd, lowering Pf}
Despite the fact that some datasets have been difficult to extract meaningful results from, many have yielded promising data. Among these datasets, there are those that have Pd and Pf values that we would prefer were higher and lower respectively. It could very well be that these datasets cannot be processed any better by {\W} and their less than optimal Pd and Pf values are simply reflecting the reality of the situation. It is also possible that there are certain operating parameters within {\W} that might need optimized specifically for certain datasets. Some parameters which could be considered are, for example, the number of lives until {\W} quits, the number of bins used for discretization, beam size, number of samples, as well as possible adjustments to our scoring parameters.

\subsection{Stubborn datasets}
As mentioned in section 4.3, we ran into problems with some datasets that {\W} simply could not extract meaningful data from. For example, as can be seen from our results tables, there is no run training on forrest-a and testing on forrest-b. {\W} simply wouldn't return any correct output when we attempted to run these files. It stands to reason that if we understood the cause of this phenomenon, we could attempt to compensate for whatever is happening. There are a few possible explanations we believe are worth examining further:

\bi
\item
A dataset might simply contain too few historical instances to make running it through {\W} a feasible operation. We believe this might be the case with at least one of the datasets containing a single project version. The missing forrest run mentioned above most likely failed due to this issue.
\item
A dataset might contain certain metrics with value ranges that require a different number of bins to be properly processed.
\ei

\subsection{Training and testing over different datasets}
We believe it would be of great interest to further experiment with {\W} and using separate datasets in the training and testing procedures. Since our results seem to indicate there exists a certain level of dependence between the dataset in question and which metrics are relevant to defect detection in that dataset, it stands to reason that using a different dataset for training might result in bad recommendations. On the other hand we must consider the possibility that our results show this dependence between relevant metrics and dataset used because we only trained and tested on the same datasets. We believe that training and testing on different datasets could reveal previously overlooked relationships between quality metrics and defect prediction and should be a part of any future research done on {\W}.

\subsection{Training and testing against more versions}
Another avenue of research is whether or not the number of project versions within a dataset is relevant to defect prediction. Our results hint at the possibility of this being true, with our largest dataset, prop, performing beautifully in all cases. We also have to keep in mind that prop, besides having the most project versions, was also the largest dataset in terms of historical instances. It is a possibility that prop performs so well due to the number of instances it contains, or more specifically due to the actual metric values it contains. 

