\dbsp
Chapter \ref{results} shares and briefly explains the results of the experimental comparison of the discretization methods used with a Na\"{i}ve Bayes classifier.
\sect{whichtree} describes a comparison of possible implementations of the DiscTree algorithm 
\sect{comparison} describes the results of the experimental comparison between the selected version of the DiscTree algorithm and the other selected discretization methods.
\sect{resultssummary} summarizes the results in this section.
\section{DiscTree Variant Selection}
\label{whichtree}
\dbsp

When the concept of DiscTree was originally created, we proposed three possible cases for how it would function.  Specifically, there were two features that we questioned including in the final method to be compared.  They were: 
\bi

\item Allowing the DiscTree algorithm to perform its organization and substitution on all data in a data set; that is, performing substitutions for both nominal and numeric data.  We will refer to this as nominal attribute discretization.

\item Performing the Garbage Collection described in \sect{DiscTree}.

\ei 

In order to make the decision about which methodology to use, we created scripts for three possibilities:

\bi 

\item That both features would be included. Specifically, a script called disctree2 was implemented that contained both Garbage Collection and nominal attribute discretization.

\item That only the Garbage Collection feature would be included.  Specifically, a script called disctree3 was created that implemented Garbage Collection.  However, it only  would only perform discretization on numeric attributes.

\item That neither Garbage Collection nor nominal attribute discretization would be used.  We created a script called disctree4 that only allowed for numeric attributes to be discretized.  The discretization trees could grow infinitely large; however, they still had the constraint of requiring a minimum number of instances in or below a subtree to use the root node of that subtree as a substitution point.

\ei 

\ssp

\subsection{Accuracy Results}
\label{whichtree-acc}

\dbsp

The results of a $U$-test comparing the three method's accuracy across all the results from each data set are found in \fig{overall-acc-dtree}.  The results find that \textit{disctree3} ties \textit{disctree4} in terms of accuracy, but performs better than \textit{disctree2}.  This tends to suggest that adding nominal attribute discretization to disctree would cause it to perform worse.

\begin{figure} [!ht]
\begin{center}
\begin{tabular}{|c|ccc|c|} \cline{1-5}
\textit{key}& \textit{win}& \textit{loss}& \textit{ties}& \textit{win-loss}\\\cline{1-5}
 disctree3& 1& 0& 1& 1\\
 disctree4& 0& 0& 2& 0\\
 disctree2& 0& 1& 1& -1\\
\cline{1-5}
\end{tabular}
\end{center}
\caption{overall for acc}
\label{fig:overall-acc-dtree}
\end{figure}

In addition to the $U$-test results across all the data sets, we have isolated the results for each data set, sorted the resulting accuracy scores, and provided them in the plots found in \fig{a2dacccharts-dtree} to \fig{s2wacccharts-dtree}.  The \textit{x-axis} of each of these graphs is the number of data points generated as results for that data set, with each point having one plot and being associated with an accuracy score.  The \textit{y-axis} of each of the graphs is the accuracy percentage of the method for plotted point.  

While the methods perform similarly on many of the data sets, there are some stand out results that are worth noting:

\bi 

\item The script \textit{disctree3}, in green on the plots, clearly wins in four data sets (auto-mpg, hayes-roth, imports-85, and iris) while only clearly losing in one data set (flag).  In the case of the flag data set, its appearance at the end of each segment denotes it had lower accuracy than the other methods at several points in the data set.

\item The script \textit{disctree2}, in red on the plots, clearly wins in only two data sets (diabetes  and flag), while it loses in several (heart-h, hepatitis, mushroom, soybean, and wine).  

\item The script \textit{disctree4}, in blue on the plots, has no clear wins, and only one loss where has lower performance measures than either of the other scripts in the diabetes data set. 

\ei

It is clear from these results that if accuracy were the only measure of contention, that the clear winner and choice of methods should be \textit{disctree3}.  However, we have put forth five methods of comparison, and will review the results for each before making a decision.

\clearpage

\addtolength{\oddsidemargin}{-.5in}
\begin{figure}[p]
\begin{center}
\begin{tabular}{cc}
\includegraphics[scale=0.74]{audiology-acc-dtree.pdf}&
\includegraphics[scale=0.74]{auto-mpg-acc-dtree.pdf}\\
\includegraphics[scale=0.74]{breast-cancer-acc-dtree.pdf}&
\includegraphics[scale=0.74]{breast-cancer-wisconsin-acc-dtree.pdf}\\
\includegraphics[scale=0.74]{credit-a-acc-dtree.pdf}&
\includegraphics[scale=0.74]{diabetes-acc-dtree.pdf}\\
\end{tabular}
\caption{Plots of the Accuracy Scores, Sorted by Value}
\end{center}
\label{fig:a2dacccharts-dtree}
\end{figure}
\clearpage
\begin{figure}[p]
\begin{center}
\begin{tabular}{cc}
\includegraphics[scale=0.74]{ecoli-acc-dtree.pdf}&
\includegraphics[scale=0.74]{flag-acc-dtree.pdf}\\
\includegraphics[scale=0.74]{hayes-roth-acc-dtree.pdf}&
\includegraphics[scale=0.74]{heart-c-acc-dtree.pdf}\\
\includegraphics[scale=0.74]{heart-h-acc-dtree.pdf}&
\includegraphics[scale=0.74]{hepatitis-acc-dtree.pdf}\\
\end{tabular}
\caption{Plots of the Accuracy Scores, Sorted by Value}
\end{center}
\label{fig:e2hacccharts-dtree}
\end{figure}
\clearpage
\begin{figure}[p]
\begin{center}
\begin{tabular}{cc}
\includegraphics[scale=0.74]{imports-85-acc-dtree.pdf}&
\includegraphics[scale=0.74]{iris-acc-dtree.pdf}\\
\includegraphics[scale=0.74]{kr-vs-kp-acc-dtree.pdf}&
\includegraphics[scale=0.74]{letter-acc-dtree.pdf}\\
\includegraphics[scale=0.74]{mushroom-acc-dtree.pdf}&
\includegraphics[scale=0.74]{segment-acc-dtree.pdf}\\
\end{tabular}
\caption{Plots of the Accuracy Scores, Sorted by Value}
\end{center}
\label{fig:i2sacccharts-dtree}
\end{figure}
\clearpage
\begin{figure}[p]
\begin{center}
\begin{tabular}{cc}
\includegraphics[scale=0.74]{soybean-acc-dtree.pdf}&
\includegraphics[scale=0.74]{splice-acc-dtree.pdf}\\
\includegraphics[scale=0.74]{vowel-acc-dtree.pdf}&
\includegraphics[scale=0.74]{waveform-5000-acc-dtree.pdf}\\
\includegraphics[scale=0.74]{wdbc-acc-dtree.pdf}&
\includegraphics[scale=0.74]{wine-acc-dtree.pdf}\\
\end{tabular}
\caption{Plots of the Accuracy Scores, Sorted by Value}
\end{center}
\label{fig:s2wacccharts-dtree}
\end{figure}
\clearpage
\addtolength{\oddsidemargin}{.5in}

\ssp

\subsection{Balance Results}
\label{whichtree-bal}

\dbsp

The results of a $U$-test comparing the three method's balance across all the results from each data set are found in \fig{overall-bal-dtree}.  The results find that the methods all tie and the $U$-test finds no clear winner in regard to which disctree script provides the best balance result.

\begin{figure}[!ht]
\begin{center}
\begin{tabular}{|c|ccc|c|} \cline{1-5}
\textit{key}& \textit{win}& \textit{loss}& \textit{ties}& \textit{win-loss}\\\cline{1-5}
 disctree4& 0& 0& 2& 0\\
 disctree3& 0& 0& 2& 0\\
 disctree2& 0& 0& 2& 0\\
\cline{1-5}
\end{tabular}
\end{center}
\caption{overall for bal}
\label{fig:overall-bal-dtree}
\end{figure}

In addition to the $U$-test results across all the data sets, we have isolated the results for each data set, sorted the resulting accuracy scores, and provided them in the plots found in \fig{a2dbalcharts-dtree} to \fig{s2wbalcharts-dtree}.  The \textit{x-axis} of each of these graphs is the number of data points generated as results for that data set, with each point having one plot and being associated with one balance score.  The \textit{y-axis} of each of the graphs is the balance percentage achieved for the discretization method of plotted point.  

While the methods perform similarly on many of the data sets, there are some stand out results that are worth noting:

\bi 

\item The script \textit{disctree3}, green, clearly wins in six data sets (auto-mpg, ecoli, hayes-roth, hepatitis, imports-85, and iris) while only clearly losing in one data set (segment).  

\item The script \textit{disctree2}, red, clearly wins in four data sets (audiology, diabetes, flag, and vowel), while it loses just as often (heart-h, hepatitis, soybean, and wine).  

\item The script \textit{disctree4}, blue, has no clear wins, and has six losses (auto-mpg, diabetes, flag, imports-85, iris, vowel) where it achieves lesser balance scores than the other methods for many of the same data points.

\ei

While the $U$-test finds no clear winner between the methods, examining the data plots leads to the assertion that if balance were to be used to choose the DiscTree algorithm's representative method that \textit{disctree3}, with the highest number of wins and fewest losses, would be the ideal choice.

\clearpage

\addtolength{\oddsidemargin}{-.5in}
\begin{figure}[p]
\begin{center}
\begin{tabular}{cc}
\includegraphics[scale=0.74]{audiology-bal-dtree.pdf}&
\includegraphics[scale=0.74]{auto-mpg-bal-dtree.pdf}\\
\includegraphics[scale=0.74]{breast-cancer-bal-dtree.pdf}&
\includegraphics[scale=0.74]{breast-cancer-wisconsin-bal-dtree.pdf}\\
\includegraphics[scale=0.74]{credit-a-bal-dtree.pdf}&
\includegraphics[scale=0.74]{diabetes-bal-dtree.pdf}\\
\end{tabular}
\caption{Plots of Balance Scores, Sorted by Value}
\end{center}
\label{fig:a2dbalcharts-dtree}
\end{figure}
\clearpage
\begin{figure}[p]
\begin{center}
\begin{tabular}{cc}
\includegraphics[scale=0.74]{ecoli-bal-dtree.pdf}&
\includegraphics[scale=0.74]{flag-bal-dtree.pdf}\\
\includegraphics[scale=0.74]{hayes-roth-bal-dtree.pdf}&
\includegraphics[scale=0.74]{heart-c-bal-dtree.pdf}\\
\includegraphics[scale=0.74]{heart-h-bal-dtree.pdf}&
\includegraphics[scale=0.74]{hepatitis-bal-dtree.pdf}\\
\end{tabular}
\caption{Plots of Balance Scores, Sorted by Value}
\end{center}
\label{fig:e2hbalcharts-dtree}
\end{figure}
\clearpage
\begin{figure}[p]
\begin{center}
\begin{tabular}{cc}
\includegraphics[scale=0.74]{imports-85-bal-dtree.pdf}&
\includegraphics[scale=0.74]{iris-bal-dtree.pdf}\\
\includegraphics[scale=0.74]{kr-vs-kp-bal-dtree.pdf}&
\includegraphics[scale=0.74]{letter-bal-dtree.pdf}\\
\includegraphics[scale=0.74]{mushroom-bal-dtree.pdf}&
\includegraphics[scale=0.74]{segment-bal-dtree.pdf}\\
\end{tabular}
\caption{Plots of Balance Scores, Sorted by Value}
\end{center}
\label{fig:i2sbalcharts-dtree}
\end{figure}
\clearpage
\begin{figure}[p]
\begin{center}
\begin{tabular}{cc}
\includegraphics[scale=0.74]{soybean-bal-dtree.pdf}&
\includegraphics[scale=0.74]{splice-bal-dtree.pdf}\\
\includegraphics[scale=0.74]{vowel-bal-dtree.pdf}&
\includegraphics[scale=0.74]{waveform-5000-bal-dtree.pdf}\\
\includegraphics[scale=0.74]{wdbc-bal-dtree.pdf}&
\includegraphics[scale=0.74]{wine-bal-dtree.pdf}\\
\end{tabular}
\caption{Plots of Balance Scores, Sorted by Value}
\end{center}
\label{fig:s2wbalcharts-dtree}
\end{figure}
\clearpage
\addtolength{\oddsidemargin}{.5in}

\ssp

\subsection{Precision Results}
\label{whichtree-prec}

\dbsp

The results of a $U$-test comparing the three method's precision across all the results from each data set are found in \fig{overall-prec-dtree}.  The results find that the methods all tie and the $U$-test finds no clear winner in regard to which disctree script provides the best precision result.


\begin{figure}[!ht]
\begin{center}
\begin{tabular}{|c|ccc|c|} \cline{1-5}
\textit{key}& \textit{win}& \textit{loss}& \textit{ties}& \textit{win-loss}\\\cline{1-5}
 disctree4& 0& 0& 2& 0\\
 disctree3& 0& 0& 2& 0\\
 disctree2& 0& 0& 2& 0\\
\cline{1-5}
\end{tabular}
\end{center}
\caption{overall for prec}
\label{fig:overall-prec-dtree}
\end{figure}


In addition to the $U$-test results across all the data sets, we have isolated the results for each data set, sorted the resulting accuracy scores, and provided them in the plots found in \fig{a2dpreccharts-dtree} to \fig{s2wpreccharts-dtree}.  The \textit{x-axis} of each of these graphs is the number of data points generated as results for that data set, with each point having one plot and representing one precision score.  The \textit{y-axis} of each of the graphs is the precision percentage achieved for the discretization method of plotted point.  

While the methods perform similarly on many of the data sets, there are some stand out results that are worth noting:

\bi 

\item The script \textit{disctree3}, green, clearly wins in five data sets (auto-mpg, ecoli, hayes-roth, imports-85, and iris) while only clearly losing in one data set (flag).  

\item The script \textit{disctree2}, red, clearly wins in four data sets (audiology, diabetes, flag, and vowel), while it loses in six (ecoli, hepatitis, heart-h, mushroom, soybean, and wine).  

\item The script \textit{disctree4}, blue, had one clear win in the segment data set, and has four losses (diabetes, auto-mpg, hayes-roth, vowel) where it achieves lower precision scores than the other methods for many of the same data points.

\ei

While the $U$-test finds no clear winner between the methods, examining the data plots leads to the assertion that if precision were to be used to choose the DiscTree algorithm's representative method that \textit{disctree3}, with the highest number of wins and fewest losses, would be the ideal choice.

\clearpage

\addtolength{\oddsidemargin}{-.5in}
\begin{figure}[p]
\begin{center}
\begin{tabular}{cc}
\includegraphics[scale=0.74]{audiology-prec-dtree.pdf}&
\includegraphics[scale=0.74]{auto-mpg-prec-dtree.pdf}\\
\includegraphics[scale=0.74]{breast-cancer-prec-dtree.pdf}&
\includegraphics[scale=0.74]{breast-cancer-wisconsin-prec-dtree.pdf}\\
\includegraphics[scale=0.74]{credit-a-prec-dtree.pdf}&
\includegraphics[scale=0.74]{diabetes-prec-dtree.pdf}\\
\end{tabular}
\caption{Plots of Precision Scores, Sorted by Value}
\end{center}
\label{fig:a2dpreccharts-dtree}
\end{figure}
\clearpage
\begin{figure}[p]
\begin{center}
\begin{tabular}{cc}
\includegraphics[scale=0.74]{ecoli-prec-dtree.pdf}&
\includegraphics[scale=0.74]{flag-prec-dtree.pdf}\\
\includegraphics[scale=0.74]{hayes-roth-prec-dtree.pdf}&
\includegraphics[scale=0.74]{heart-c-prec-dtree.pdf}\\
\includegraphics[scale=0.74]{heart-h-prec-dtree.pdf}&
\includegraphics[scale=0.74]{hepatitis-prec-dtree.pdf}\\
\end{tabular}
\caption{Plots of Precision Scores, Sorted by Value}
\end{center}
\label{fig:e2hpreccharts-dtree}
\end{figure}
\clearpage
\begin{figure}[p]
\begin{center}
\begin{tabular}{cc}
\includegraphics[scale=0.74]{imports-85-prec-dtree.pdf}&
\includegraphics[scale=0.74]{iris-prec-dtree.pdf}\\
\includegraphics[scale=0.74]{kr-vs-kp-prec-dtree.pdf}&
\includegraphics[scale=0.74]{letter-prec-dtree.pdf}\\
\includegraphics[scale=0.74]{mushroom-prec-dtree.pdf}&
\includegraphics[scale=0.74]{segment-prec-dtree.pdf}\\
\end{tabular}
\caption{Plots of Precision Scores, Sorted by Value}
\end{center}
\label{fig:i2spreccharts-dtree}
\end{figure}
\clearpage
\begin{figure}[p]
\begin{center}
\begin{tabular}{cc}
\includegraphics[scale=0.74]{soybean-prec-dtree.pdf}&
\includegraphics[scale=0.74]{splice-prec-dtree.pdf}\\
\includegraphics[scale=0.74]{vowel-prec-dtree.pdf}&
\includegraphics[scale=0.74]{waveform-5000-prec-dtree.pdf}\\
\includegraphics[scale=0.74]{wdbc-prec-dtree.pdf}&
\includegraphics[scale=0.74]{wine-prec-dtree.pdf}\\
\end{tabular}
\caption{Plots of Precision Scores, Sorted by Value}
\end{center}
\label{fig:s2wpreccharts-dtree}
\end{figure}
\clearpage
\addtolength{\oddsidemargin}{.5in}

\ssp

\subsection{Probability of Detection Results}
\label{whichtree-pd}

\dbsp

The results of a $U$-test comparing the three method's probability of detection across all the results from each data set are found in \fig{overall-prec-dtree}.  The results find that the methods all tie and the $U$-test finds no clear winner in regard to which disctree script provides the best probability of detection result.

\begin{figure}[!ht]
\begin{center}
\begin{tabular}{|c|ccc|c|} \cline{1-5}
\textit{key}& \textit{win}& \textit{loss}& \textit{ties}& \textit{win-loss}\\\cline{1-5}
 disctree4& 0& 0& 2& 0\\
 disctree3& 0& 0& 2& 0\\
 disctree2& 0& 0& 2& 0\\
\cline{1-5}
\end{tabular}
\end{center}
\caption{overall for pd}
\label{fig:overall-pd-dtree}
\end{figure}

In addition to the $U$-test results across all the data sets, we have isolated the results for each data set, sorted the resulting accuracy scores, and provided them in the plots found in \fig{a2dpreccharts-dtree} to \fig{s2wpreccharts-dtree}.  The \textit{x-axis} of each of these graphs is the number of data points generated as results for that data set, with each point having one plot and representing one probability of detection score.  The \textit{y-axis} of each of the graphs is the probability of detection percentage achieved for the discretization method of plotted point.  

While the methods perform similarly on many of the data sets, there are some stand out results that are worth noting:

\bi 

\item The script \textit{disctree3}, green, clearly wins in seven data sets (auto-mpg, ecoli, hayes-roth, heart-h, hepatitis, imports-85, and iris) while only clearly losing in two data sets (audiology, segment).  

\item The script \textit{disctree2}, red, clearly wins in four data sets (audiology, diabetes, flag, and vowel), while it loses in seven (breast-cancer, ecoli, heart-h, hepatitis,  mushroom, soybean, and wine).  

\item The script \textit{disctree4}, blue, had one clear win in the wine data set, and has three losses (diabetes, flag, vowel) where it achieves lower probability of detection scores than the other methods for many of the same data points.

\ei

While the $U$-test finds no clear winner between the methods, examining the data plots leads to the assertion that if probability of detection were to be used to choose the DiscTree algorithm's representative method that \textit{disctree3}, with the highest number of wins and fewest losses, would be the ideal choice.

\clearpage

\addtolength{\oddsidemargin}{-.5in}
\begin{figure}[p]
\begin{center}
\begin{tabular}{cc}
\includegraphics[scale=0.74]{audiology-pd-dtree.pdf}&
\includegraphics[scale=0.74]{auto-mpg-pd-dtree.pdf}\\
\includegraphics[scale=0.74]{breast-cancer-pd-dtree.pdf}&
\includegraphics[scale=0.74]{breast-cancer-wisconsin-pd-dtree.pdf}\\
\includegraphics[scale=0.74]{credit-a-pd-dtree.pdf}&
\includegraphics[scale=0.74]{diabetes-pd-dtree.pdf}\\
\end{tabular}
\caption{Plots of Probability of Detection Scores, Sorted by Value}
\end{center}
\label{fig:a2dpdcharts-dtree}
\end{figure}
\clearpage
\begin{figure}[p]
\begin{center}
\begin{tabular}{cc}
\includegraphics[scale=0.74]{ecoli-pd-dtree.pdf}&
\includegraphics[scale=0.74]{flag-pd-dtree.pdf}\\
\includegraphics[scale=0.74]{hayes-roth-pd-dtree.pdf}&
\includegraphics[scale=0.74]{heart-c-pd-dtree.pdf}\\
\includegraphics[scale=0.74]{heart-h-pd-dtree.pdf}&
\includegraphics[scale=0.74]{hepatitis-pd-dtree.pdf}\\
\end{tabular}
\caption{Plots of Probability of Detection Scores, Sorted by Value}
\end{center}
\label{fig:e2hpdcharts-dtree}
\end{figure}
\clearpage
\begin{figure}[p]
\begin{center}
\begin{tabular}{cc}
\includegraphics[scale=0.74]{imports-85-pd-dtree.pdf}&
\includegraphics[scale=0.74]{iris-pd-dtree.pdf}\\
\includegraphics[scale=0.74]{kr-vs-kp-pd-dtree.pdf}&
\includegraphics[scale=0.74]{letter-pd-dtree.pdf}\\
\includegraphics[scale=0.74]{mushroom-pd-dtree.pdf}&
\includegraphics[scale=0.74]{segment-pd-dtree.pdf}\\
\end{tabular}
\caption{Plots of Probability of Detection Scores, Sorted by Value}
\end{center}
\label{fig:i2spdcharts-dtree}
\end{figure}
\clearpage
\begin{figure}[p]
\begin{center}
\begin{tabular}{cc}
\includegraphics[scale=0.74]{soybean-pd-dtree.pdf}&
\includegraphics[scale=0.74]{splice-pd-dtree.pdf}\\
\includegraphics[scale=0.74]{vowel-pd-dtree.pdf}&
\includegraphics[scale=0.74]{waveform-5000-pd-dtree.pdf}\\
\includegraphics[scale=0.74]{wdbc-pd-dtree.pdf}&
\includegraphics[scale=0.74]{wine-pd-dtree.pdf}\\
\end{tabular}
\caption{Plots of Probability of Detection Scores, Sorted by Value}
\end{center}
\label{fig:s2wpdcharts-dtree}
\end{figure}
\clearpage
\addtolength{\oddsidemargin}{.5in}

\ssp

\subsection{Probability of Not False Alarm}
\label{whichtree-npf}

\dbsp

The results of a $U$-test comparing the three method's probability of not false alarm across all the results from each data set are found in \fig{overall-npf-dtree}.  The results show \textit{disctree3} and \textit{disctree4} each with a win over \textit{disctree2}, but in a tie with each other.  The $U$-test thus tells us we should not choose \textit{disctree2}, the method that includes both garbage collection and nominal attribute discretization, if we seek to maximize the probability of not false alarm results and minimizes the probability of false alarm scores for the method.  It does not, however, tell us which of the other methods to chose.

\begin{figure} [!ht]
\begin{center}
\begin{tabular}{|c|ccc|c|} \cline{1-5}
\textit{key}& \textit{win}& \textit{loss}& \textit{ties}& \textit{win-loss}\\\cline{1-5}
 disctree4& 1& 0& 1& 1\\
 disctree3& 1& 0& 1& 1\\
 disctree2& 0& 2& 0& -2\\
\cline{1-5}
\end{tabular}
\end{center}
\caption{overall for npf}
\label{fig:overall-npf-dtree}
\end{figure}

In addition to the $U$-test results across all the data sets, we have isolated the results for each data set, sorted the resulting accuracy scores, and provided them in the plots found in \fig{a2dpreccharts-dtree} to \fig{s2wpreccharts-dtree}.  The \textit{x-axis} of each of these graphs is the number of data points generated as results for that data set, with each point having one plot and representing one probability of not false alarm score.  The \textit{y-axis} of each of the graphs is the probability of not false alarm percentage achieved for the discretization method of plotted point.  

While the methods perform similarly on many of the data sets, there are some stand out results that are worth noting:

\bi 

\item The script \textit{disctree3}, green, clearly wins in six data sets (auto-mpg,  hayes-roth, heart-h, hepatitis, imports-85, and iris) while only clearly losing in one data set (flag).  

\item The script \textit{disctree2}, red, clearly wins in two data sets (diabetes and flag), while it loses in six (audiology, heart-h, hepatitis,  mushroom, soybean, and wine).  

\item The script \textit{disctree4}, blue, has no clear wins but has two losses (diabetes and iris) where it achieves lower not probability of failure scores than the other methods for many of the same data points.

\ei

While the $U$-test finds no clear winner between the \textit{disctree3} and \textit{disctree4} methods, a review of the data plots leads us to assert that of the two methods, \textit{disctree3} would be a better choice because it wins more often in the data plots.

\clearpage

\addtolength{\oddsidemargin}{-.5in}
\begin{figure}[p]
\begin{center}
\begin{tabular}{cc}
\includegraphics[scale=0.74]{audiology-npf-dtree.pdf}&
\includegraphics[scale=0.74]{auto-mpg-npf-dtree.pdf}\\
\includegraphics[scale=0.74]{breast-cancer-npf-dtree.pdf}&
\includegraphics[scale=0.74]{breast-cancer-wisconsin-npf-dtree.pdf}\\
\includegraphics[scale=0.74]{credit-a-npf-dtree.pdf}&
\includegraphics[scale=0.74]{diabetes-npf-dtree.pdf}\\
\end{tabular}
\caption{Plots of Probability of not False Alarm Scores, Sorted by Value}
\end{center}
\label{fig:a2dnpfcharts-dtree}
\end{figure}
\clearpage
\begin{figure}[p]
\begin{center}
\begin{tabular}{cc}
\includegraphics[scale=0.74]{ecoli-npf-dtree.pdf}&
\includegraphics[scale=0.74]{flag-npf-dtree.pdf}\\
\includegraphics[scale=0.74]{hayes-roth-npf-dtree.pdf}&
\includegraphics[scale=0.74]{heart-c-npf-dtree.pdf}\\
\includegraphics[scale=0.74]{heart-h-npf-dtree.pdf}&
\includegraphics[scale=0.74]{hepatitis-npf-dtree.pdf}\\
\end{tabular}
\caption{Plots of Probability of not False Alarm Scores, Sorted by Value}
\end{center}
\label{fig:e2hnpfcharts-dtree}
\end{figure}
\clearpage
\begin{figure}[p]
\begin{center}
\begin{tabular}{cc}
\includegraphics[scale=0.74]{imports-85-npf-dtree.pdf}&
\includegraphics[scale=0.74]{iris-npf-dtree.pdf}\\
\includegraphics[scale=0.74]{kr-vs-kp-npf-dtree.pdf}&
\includegraphics[scale=0.74]{letter-npf-dtree.pdf}\\
\includegraphics[scale=0.74]{mushroom-npf-dtree.pdf}&
\includegraphics[scale=0.74]{segment-npf-dtree.pdf}\\
\end{tabular}
\caption{Plots of Probability of not False Alarm Scores, Sorted by Value}
\end{center}
\label{fig:i2snpfcharts-dtree}
\end{figure}
\clearpage
\begin{figure}[p]
\begin{center}
\begin{tabular}{cc}
\includegraphics[scale=0.74]{soybean-npf-dtree.pdf}&
\includegraphics[scale=0.74]{splice-npf-dtree.pdf}\\
\includegraphics[scale=0.74]{vowel-npf-dtree.pdf}&
\includegraphics[scale=0.74]{waveform-5000-npf-dtree.pdf}\\
\includegraphics[scale=0.74]{wdbc-npf-dtree.pdf}&
\includegraphics[scale=0.74]{wine-npf-dtree.pdf}\\
\end{tabular}
\caption{Plots of Probability of not False Alarm Scores, Sorted by Value}
\end{center}
\label{fig:s2wnpfcharts-dtree}
\end{figure}
\clearpage
\addtolength{\oddsidemargin}{.5in}

\ssp

\subsection{Decision Tree Method Selection}
\label{discmethodselect}

\dbsp

The results above show us that, overall, the \textit{disctree3} method clearly obtains the most wins.  In regard to accuracy, it is the only method to achieve a win in the $U$-test, and wins in several of the data plots.  In regard to balance, precision, and probability of detection, it performs as well as the other methods in regard to the $U$-test but clearly wins in more of the data plots.  Not Probability of Failure gives us two options in regards to methods that achieve $U$-test wins, but clearly \textit{disctree3} wins in more data plot comparisons.  Another interesting note from that measure is that clearly the method with nominal attribute discretization, \textit{disctree2} would lead to higher a probability of false alarm, which, when paired with its result of one loss and one tie in the $U$-test for accuracy, would lead to a decision that it is certainly not the best method for us to use.

Based on the results above, we selected the implementation that used the Garbage Collection but did not perform nominal attribute discretization.  This method, \textit{disctree3}, will be used as the DiscTree algorithm in the comparisons against other discretization methods.

\ssp

\section{Discretization Method Comparison}
\label{comparison}

\dbsp

\ssp

\subsection{Accuracy Results}
\label{comparison-acc}

\dbsp

The overall results for the accuracy comparison between the discretization methods discussed in \chap{discretization}, across all the data generated, can be found in \fig{overall-acc}

\begin{figure} [!ht]
\begin{center}
\begin{tabular}{|c|ccc|c|} \cline{1-5}
\textit{key}& \textit{win}& \textit{loss}& \textit{ties}& \textit{win-loss}\\\cline{1-5}
 fayyadIrani& 4& 0& 0& 4\\
        tbin& 1& 1& 2& 0\\
        pkid& 1& 1& 2& 0\\
   disctree3& 1& 1& 2& 0\\
         cat& 0& 4& 0& -4\\
\cline{1-5}
\end{tabular}
\end{center}
\caption{overall for acc}
\label{fig:overall-acc}
\end{figure}

The results of this overview shows that in comparison with all the other discretization methods, the entropy-minimization method, called fayyadIrani in this comparison, won over each of the other methods.  The tenbins (tbin), PKID (pkid), and DiscTree (disctree3) methods each performed comparably well, each winning a comparison, losing a comparison, and reaching a tie twice.  The pure data set, provided by the "cat" method, had the worst results, losing out to each other method.

These results provide a great general summary of the results.   However, each data set has also had the win/loss/tie record created for that data set.  Those results can be found in \fig{audiology-acc} to \fig{waveform-5000-acc}.  \fig{accwintable} displays the winning method(s) for each data set where all the discretization methods did not tie.  The winner or winners for each dataset are displayed, along with the number of degrees by which it wins.  The counts for total wins are found in \fig{accaccumwins}.  Clearly, the entropy-minimization heuristic, fayyadIrani, had more accuracy wins than any other method with ten, while DiscTree and tenbins each receiving one third of that number of wins.  Interestingly, there is no clear manner by which to describe the conditions under which the DiscTree algorithm wins, other than that the data sets where it is recorded as a winner tend to have fewer classes (less than 4).  However, it also loses to the entropy-minimization heuristic in similar data sets.  

Another interesting result is the sorted data plots for each data set.  These plots, found in \fig{a2dacccharts} to \fig{s2wacccharts}, show that in most of the data sets where the fayyadIrani method wins, there is so very little difference in the accuracy scores that the plotted points for fayyadIrani, generate on top of or very close to the scores for the other methods.  This should mean that, even when the entropy-minimization method wins, it is doing so by very little, unlike the DiscTree win in the auto-mpg data set, where the DiscTree accuracy plot quite clearly shows it winning across most of the data plot.

\begin{figure}[!ht]
\begin{center}
\begin{tabular}{|c|c|c|c|c|c|}\cline{1-6}
\textit{Data Set}& \textit{Instances}& \textit{Winner(s)}& \textit{Degree}& \textit{Numeric}& \textit{Classes}\\\hline
hayes-roth& 132& {cat/fayyadIrani}& 1(1)&  1& 3\\
hepatitis& 155&  {fayyadIrani}& 2(3)& 6& 2\\
flag& 194& {tbin/pkid/fayyadIrani/disctree3}& 1& 10& 4\\
imports-85& 205& {fayyadIrani}& 3(4)& 15& 5\\
heart-c& 303& {tbin}& 1(1)& 6&  5\\
auto-mpg& 398& {disctree3}& 3& 5&  3\\
wdbc& 569& {fayyadIrani}& 3(3)& 30&  2\\
credit-a& 690& {disctree3}& 1& 0& 2\\
diabetes& 768& {fayyadIrani}& 2(4)& 8& 2\\
vowel& 990& {tbin/fayyadIrani}& 1(3)& 10& 11\\
segment& 2310& {fayyadIrani/tbin}& 2(2)& 19& 7\\
waveform-5000& 5000& {fayyadIrani}& 2(4)& 40& 3\\
letter& 20000& {fayyadIrani}& 1(1)& 16& 26\\
\cline{1-6}
\end{tabular}
\caption{These data sets had a particular winner(s) for their Accuracy comparison. In all cases, degree measures the number of wins over the next closest method.  In the event that disctree3 did not win, the number in parenthesis represents its win difference from the lead method. }
\label{fig:accwintable}
\end{center}
\end{figure}

\begin{figure}[!ht]
\begin{center}
\begin{tabular}{|c|c|}\cline{1-2}
\textit{Method}& \textit{Wins}\\\cline{1-2}
fayyadIrani& 10\\
disctree3& 3\\
tbin& 3\\
pkid& 1\\
cat& 1\\
\cline{1-2}
\end{tabular}
\caption{Total Wins Per Method Based on Mann-Whitney $U$-Test Wins on each Data Set's Accuracy Scores}
\label{fig:accaccumwins}
\end{center}
\end{figure}

\clearpage

\addtolength{\oddsidemargin}{-.5in}
\begin{figure}[p]
\begin{center}
\begin{tabular}{cc}
\includegraphics[scale=0.74]{audiology-acc.pdf}&
\includegraphics[scale=0.74]{auto-mpg-acc.pdf}\\
\includegraphics[scale=0.74]{breast-cancer-acc.pdf}&
\includegraphics[scale=0.74]{breast-cancer-wisconsin-acc.pdf}\\
\includegraphics[scale=0.74]{credit-a-acc.pdf}&
\includegraphics[scale=0.74]{diabetes-acc.pdf}\\
\end{tabular}
\caption{Plots of Accuracy Scores, Sorted by Value}
\end{center}
\label{fig:a2dacccharts}
\end{figure}
\clearpage
\begin{figure}[p]
\begin{center}
\begin{tabular}{cc}
\includegraphics[scale=0.74]{ecoli-acc.pdf}&
\includegraphics[scale=0.74]{flag-acc.pdf}\\
\includegraphics[scale=0.74]{hayes-roth-acc.pdf}&
\includegraphics[scale=0.74]{heart-c-acc.pdf}\\
\includegraphics[scale=0.74]{heart-h-acc.pdf}&
\includegraphics[scale=0.74]{hepatitis-acc.pdf}\\
\end{tabular}
\caption{Plots of Accuracy Scores, Sorted by Value}
\end{center}
\label{fig:e2hacccharts}
\end{figure}
\clearpage
\begin{figure}[p]
\begin{center}
\begin{tabular}{cc}
\includegraphics[scale=0.74]{imports-85-acc.pdf}&
\includegraphics[scale=0.74]{iris-acc.pdf}\\
\includegraphics[scale=0.74]{kr-vs-kp-acc.pdf}&
\includegraphics[scale=0.74]{letter-acc.pdf}\\
\includegraphics[scale=0.74]{mushroom-acc.pdf}&
\includegraphics[scale=0.74]{segment-acc.pdf}\\
\end{tabular}
\caption{Plots of Accuracy Scores, Sorted by Value}
\end{center}
\label{fig:i2sacccharts}
\end{figure}
\clearpage
\begin{figure}[p]
\begin{center}
\begin{tabular}{cc}
\includegraphics[scale=0.74]{soybean-acc.pdf}&
\includegraphics[scale=0.74]{splice-acc.pdf}\\
\includegraphics[scale=0.74]{vowel-acc.pdf}&
\includegraphics[scale=0.74]{waveform-5000-acc.pdf}\\
\includegraphics[scale=0.74]{wdbc-acc.pdf}&
\includegraphics[scale=0.74]{wine-acc.pdf}\\
\end{tabular}
\caption{Plots of Accuracy Scores, Sorted by Value}
\end{center}
\label{fig:s2wacccharts}
\end{figure}
\clearpage
\addtolength{\oddsidemargin}{.5in}

\ssp

\subsection{Balance Results}
\label{comparison-bal}

\dbsp

The overall results for the probability of not false alarm comparison between the discretization methods discussed in \chap{discretization}, across all the data generated, can be found in \fig{overall-bal}

\begin{figure} [!ht]
\begin{center}
\begin{tabular}{|c|ccc|c|} \cline{1-5}
\textit{key}& \textit{win}& \textit{loss}& \textit{ties}& \textit{win-loss}\\\cline{1-5}
 fayyadIrani& 4& 0& 0& 4\\
        pkid& 2& 1& 1& 1\\
   disctree3& 2& 1& 1& 1\\
        tbin& 1& 3& 0& -2\\
         cat& 0& 4& 0& -4\\
\cline{1-5}
\end{tabular}
\end{center}
\caption{overall for bal}
\label{fig:overall-bal}
\end{figure}

The results of this overview shows that in comparison with all the other discretization methods, the entropy-minimization method, won over each of the other methods.  The PKID (pkid), and DiscTree (disctree3) methods each performed comparably well, each winning two comparisons, losing a comparison, and reaching a tie once.  The pure data set, provided by the "cat" method, had the worst results, losing out to each other method, while tenbins performed slightly better winning once against other methods but losing to the other three.

These results provide a general summary of the results.   However, each data set has also had the win/loss/tie record created for that data set.  Those results can be found in \fig{audiology-bal} to \fig{waveform-5000-bal}.  \fig{balwintable} displays the winning method(s) for each data set where all the discretization methods did not tie.  The winner or winners for each dataset are displayed, along with the number of degrees by which it wins.  The counts for total wins are found in \fig{balaccumwins}.  Clearly, the entropy-minimization heuristic, fayyadIrani, had more balance wins than any other method.  DiscTree won five times, more than each of the other methods.  Interestingly, there is no clear manner by which to describe the conditions under which the DiscTree algorithm wins, other than that the data sets where it is recorded as a winner tend to have fewer classes (less than 5).  However, it also loses to the entropy-minimization heuristic in similar data sets.  

Another interesting result is the sorted data plots for each data set.  These plots, found in \fig{a2dbalcharts} to \fig{s2wbalcharts}, show that in most of the data sets where the fayyadIrani method wins there is so very little difference in the balance scores that the plotted points for fayyadIrani generate on top of or very close to the scores for the other methods.  This should mean that, even when the entropy-minimization method wins, in most cases it is doing so by very little, unlike the DiscTree win in the auto-mpg data set or the fayyadIrani win in the imports-85 data set, where the respective method's plot quite clearly shows it winning across most of the data plot.

\begin{figure}[!ht]
\begin{center}
\begin{tabular}{|c|c|c|c|c|c|}\cline{1-6}
\textit{Data Set}& \textit{Instances}& \textit{Winner(s)}& \textit{Degree}& \textit{Numeric}& \textit{Classes}\\\hline
hayes-roth& 132& {cat/disctree3/fayyadIrani}& 1&  1& 3\\
hepatitis& 155&  {fayyadIrani/tbin}& 1(1)& 6& 2\\
flag& 194& {fayyadIrani}& 1(1)& 10& 4\\
imports-85& 205& {fayyadIrani/pkid}& 1(1)& 15& 5\\
heart-c& 303& {disctree3/tbin}& 1& 6&  5\\
ecoli& 336& {cat/fayyadIrani/pkid}& 2(2)& 7& 8\\
auto-mpg& 398& {disctree3}& 3& 5&  3\\
wdbc& 569& {fayyadIrani}& 3(3)& 30&  2\\
credit-a& 690& {disctree3}& 1& 0& 2\\
diabetes& 768& {fayyadIrani}& 3(3)& 8& 2\\
vowel& 990& {fayyadIrani/tbin}& 1(3)& 10& 11\\
segment& 2310& {fayyadIrani/tbin}& 2(2)& 19& 7\\
waveform-5000& 5000& {fayyadIrani}& 2(3)& 40& 3\\
letter& 20000& {disctree3/fayyadIrani/pkid}& 1& 16& 26\\
\cline{1-6}
\end{tabular}
\caption{These data sets had a particular winner(s) for their Balance comparison. In all cases, degree measures the number of wins over the next closest method.  In the event that disctree3 did not win, the number in parenthesis represents its win difference from the lead method. }
\label{fig:balwintable}
\end{center}
\end{figure}

\begin{figure}[!ht]
\begin{center}
\begin{tabular}{|c|c|}\cline{1-2}
\textit{Method}& \textit{Wins}\\\cline{1-2}
fayyadIrani& 11\\
disctree3& 5\\
tbin& 4\\
pkid& 3\\
cat& 2\\
\cline{1-2}
\end{tabular}
\caption{Total Wins Per Method Based on Mann-Whitney $U$-Test Wins on Each Data Set's Balance Scores}
\label{fig:balaccumwins}
\end{center}
\end{figure}

\clearpage

\addtolength{\oddsidemargin}{-.5in}
\begin{figure}[p]
\begin{center}
\begin{tabular}{cc}
\includegraphics[scale=0.74]{audiology-bal.pdf}&
\includegraphics[scale=0.74]{auto-mpg-bal.pdf}\\
\includegraphics[scale=0.74]{breast-cancer-bal.pdf}&
\includegraphics[scale=0.74]{breast-cancer-wisconsin-bal.pdf}\\
\includegraphics[scale=0.74]{credit-a-bal.pdf}&
\includegraphics[scale=0.74]{diabetes-bal.pdf}\\
\end{tabular}
\caption{Plots of Balance Scores, Sorted by Value}
\end{center}
\label{fig:a2dbalcharts}
\end{figure}
\clearpage
\begin{figure}[p]
\begin{center}
\begin{tabular}{cc}
\includegraphics[scale=0.74]{ecoli-bal.pdf}&
\includegraphics[scale=0.74]{flag-bal.pdf}\\
\includegraphics[scale=0.74]{hayes-roth-bal.pdf}&
\includegraphics[scale=0.74]{heart-c-bal.pdf}\\
\includegraphics[scale=0.74]{heart-h-bal.pdf}&
\includegraphics[scale=0.74]{hepatitis-bal.pdf}\\
\end{tabular}
\caption{Plots of Balance Scores, Sorted by Value}
\end{center}
\label{fig:e2hbalcharts}
\end{figure}
\clearpage
\begin{figure}[p]
\begin{center}
\begin{tabular}{cc}
\includegraphics[scale=0.74]{imports-85-bal.pdf}&
\includegraphics[scale=0.74]{iris-bal.pdf}\\
\includegraphics[scale=0.74]{kr-vs-kp-bal.pdf}&
\includegraphics[scale=0.74]{letter-bal.pdf}\\
\includegraphics[scale=0.74]{mushroom-bal.pdf}&
\includegraphics[scale=0.74]{segment-bal.pdf}\\
\end{tabular}
\caption{Plots of Balance Scores, Sorted by Value}
\end{center}
\label{fig:i2sbalcharts}
\end{figure}
\clearpage
\begin{figure}[p]
\begin{center}
\begin{tabular}{cc}
\includegraphics[scale=0.74]{soybean-bal.pdf}&
\includegraphics[scale=0.74]{splice-bal.pdf}\\
\includegraphics[scale=0.74]{vowel-bal.pdf}&
\includegraphics[scale=0.74]{waveform-5000-bal.pdf}\\
\includegraphics[scale=0.74]{wdbc-bal.pdf}&
\includegraphics[scale=0.74]{wine-bal.pdf}\\
\end{tabular}
\caption{Plots of Balance Scores, Sorted by Value}
\end{center}
\label{fig:s2wbalcharts}
\end{figure}
\clearpage
\addtolength{\oddsidemargin}{.5in}

\ssp

\subsection{Precision Results}
\label{comparison-prec}

\dbsp

The overall results for the precision comparison between the discretization methods discussed in \chap{discretization}, across all the data generated, can be found in \fig{overall-prec}

\begin{figure} [!ht]
\begin{center}
\begin{tabular}{|c|ccc|c|} \cline{1-5}
\textit{key}& \textit{win}& \textit{loss}& \textit{ties}& \textit{win-loss}\\\cline{1-5}
 fayyadIrani& 4& 0& 0& 4\\
        tbin& 1& 1& 2& 0\\
        pkid& 1& 1& 2& 0\\
   disctree3& 1& 1& 2& 0\\
         cat& 0& 4& 0& -4\\
\cline{1-5}
\end{tabular}
\end{center}
\caption{overall for prec}
\label{fig:overall-prec}
\end{figure}

The results of this overview shows that in comparison with all the other discretization methods, the entropy-minimization method, fayyadIrani, won over each of the other methods.  The tenbins (tbin), PKID (pkid), and DiscTree (disctree3) methods each performed comparably well, each winning a comparison, losing a comparison, and reaching a tie twice.  The pure data set, provided by the "cat" method, had the worst results, losing out to each other method.

These results provide a general summary of the results.   However, each data set has also had the win/loss/tie record created for that data set.  Those results can be found in \fig{audiology-prec} to \fig{waveform-5000-prec}.  \fig{precwintable} displays the method for each data set where all the discretization methods did not tie.  The winner or winners for each dataset are displayed, along with the number of degrees by which it wins.  The counts for total wins are found in \fig{precaccumwins}.  The entropy-minimization heuristic, fayyadIrani, had more precision wins than any other method, with DiscTree winning half as many times.  Interestingly, there is no clear manner by which to describe the conditions under which the DiscTree algorithm wins, other than that the data sets where it is recorded as a winner tend to have fewer classes (less than 5).  However, it also loses to the entropy-minimization heuristic in similar data sets.
  
Another interesting result is the sorted data plots for each data set.  These plots, found in \fig{a2dacccharts} to \fig{s2wacccharts}, show that in most of the data sets where the fayyadIrani method wins there is so very little difference in the precision scores that the plotted points for fayyadIrani generate on top of or very close to the scores for the other methods.  While in some cases (DiscTree with the auto-mpg data set, entropy in the ecoli data set), in most cases the plots are very, very similar and very little difference appears between the methods.

\begin{figure}[!ht]
\begin{center}
\begin{tabular}{|c|c|c|c|c|c|}\cline{1-6}
\textit{Data Set}& \textit{Instances}& \textit{Winner(s)}& \textit{Degree}& \textit{Numeric}& \textit{Classes}\\\hline
hayes-roth& 132& {cat/disctree3/fayyadIrani/tbin}& 1&  1& 3\\
flag& 194& {cat/disctree3/fayyadIrani/pkid}& 1& 10& 4\\
heart-c& 303& {disctree3}& 1& 6&  5\\
ecoli& 336& {fayyadIrani}& 1(2)& 7& 8\\
auto-mpg& 398& {disctree3}& 3& 5&  3\\
wdbc& 569& {fayyadIrani}& 3(3)& 30&  2\\
credit-a& 690& {disctree3/fayyadIrani/pkid}& 1& 0& 2\\
diabetes& 768& {fayyadIrani}& 4(4)& 8& 2\\
vowel& 990& {cat/fayyadIrani/tbin}& 2(2)& 10& 11\\
segment& 2310& {fayyadIrani/tbin}& 2(3)& 19& 7\\
waveform-5000& 5000& {fayyadIrani}& 1(2)& 40& 3\\
letter& 20000& {fayyadIrani}& 1(1)& 16& 26\\
\cline{1-6}
\end{tabular}
\caption{These data sets had a particular winner(s) for their Precision comparison. In all cases, degree measures the number of wins over the next closest method.  In the event that disctree3 did not win, the number in parenthesis represents its win difference from the lead method. }
\label{fig:precwintable}
\end{center}
\end{figure}

\begin{figure}[!ht]
\begin{center}
\begin{tabular}{|c|c|}\cline{1-2}
\textit{Method}& \textit{Wins}\\\cline{1-2}
fayyadIrani& 10\\
disctree3& 5\\
tbin& 3\\
pkid& 2\\
cat& 3\\
\cline{1-2}
\end{tabular}
\caption{Total Wins Per Method Based on Mann-Whitney $U$-Test Wins on Each Data Set's Precision Scores}
\label{fig:precaccumwins}
\end{center}
\end{figure}

\clearpage

\addtolength{\oddsidemargin}{-.5in}
\begin{figure}[p]
\begin{center}
\begin{tabular}{cc}
\includegraphics[scale=0.74]{audiology-prec.pdf}&
\includegraphics[scale=0.74]{auto-mpg-prec.pdf}\\
\includegraphics[scale=0.74]{breast-cancer-prec.pdf}&
\includegraphics[scale=0.74]{breast-cancer-wisconsin-prec.pdf}\\
\includegraphics[scale=0.74]{credit-a-prec.pdf}&
\includegraphics[scale=0.74]{diabetes-prec.pdf}\\
\end{tabular}
\caption{Plots of Precision Scores, Sorted by Value}
\end{center}
\label{fig:a2dpreccharts}
\end{figure}
\clearpage
\begin{figure}[p]
\begin{center}
\begin{tabular}{cc}
\includegraphics[scale=0.74]{ecoli-prec.pdf}&
\includegraphics[scale=0.74]{flag-prec.pdf}\\
\includegraphics[scale=0.74]{hayes-roth-prec.pdf}&
\includegraphics[scale=0.74]{heart-c-prec.pdf}\\
\includegraphics[scale=0.74]{heart-h-prec.pdf}&
\includegraphics[scale=0.74]{hepatitis-prec.pdf}\\
\end{tabular}
\caption{Plots of Precision Scores, Sorted by Value}
\end{center}
\label{fig:e2hpreccharts}
\end{figure}
\clearpage
\begin{figure}[p]
\begin{center}
\begin{tabular}{cc}
\includegraphics[scale=0.74]{imports-85-prec.pdf}&
\includegraphics[scale=0.74]{iris-prec.pdf}\\
\includegraphics[scale=0.74]{kr-vs-kp-prec.pdf}&
\includegraphics[scale=0.74]{letter-prec.pdf}\\
\includegraphics[scale=0.74]{mushroom-prec.pdf}&
\includegraphics[scale=0.74]{segment-prec.pdf}\\
\end{tabular}
\caption{Plots of Precision Scores, Sorted by Value}
\end{center}
\label{fig:i2spreccharts}
\end{figure}
\clearpage
\begin{figure}[p]
\begin{center}
\begin{tabular}{cc}
\includegraphics[scale=0.74]{soybean-prec.pdf}&
\includegraphics[scale=0.74]{splice-prec.pdf}\\
\includegraphics[scale=0.74]{vowel-prec.pdf}&
\includegraphics[scale=0.74]{waveform-5000-prec.pdf}\\
\includegraphics[scale=0.74]{wdbc-prec.pdf}&
\includegraphics[scale=0.74]{wine-prec.pdf}\\
\end{tabular}
\caption{Plots of Precision Scores, Sorted by Value}
\end{center}
\label{fig:s2wpreccharts}
\end{figure}
\clearpage
\addtolength{\oddsidemargin}{.5in}

\ssp

\subsection{Probability of Detection Results}
\label{comparison-pd}

\dbsp

The overall results for the probability of detection comparison between the discretization methods discussed in \chap{discretization}, across all the data generated, can be found in \fig{overall-pd}

\begin{figure} [!ht]
\begin{center}
\begin{tabular}{|c|ccc|c|} \cline{1-5}
\textit{key}& \textit{win}& \textit{loss}& \textit{ties}& \textit{win-loss}\\\cline{1-5}
 fayyadIrani& 4& 0& 0& 4\\
        pkid& 2& 1& 1& 1\\
   disctree3& 2& 1& 1& 1\\
        tbin& 0& 3& 1& -3\\
         cat& 0& 3& 1& -3\\
\cline{1-5}
\end{tabular}
\end{center}
\caption{overall for pd}
\label{fig:overall-pd}
\end{figure}

The results of this overview shows that in comparison with all the other discretization methods, the entropy-minimization method, called fayyadIrani in this comparison, won over each of the other methods.  The PKID (pkid), and DiscTree (disctree3) methods each performed comparably well, each winning two comparisons, losing a comparison, and reaching a tie once.  The pure data set, provided by the "cat" method, had the worst results, losing out to each other method.  The tenbins method (tbin) performed just as poorly.

These results provide a general summary of the results.   However, each data set has also had the win/loss/tie record created for that data set.  Those results can be found in \fig{audiology-pd} to \fig{waveform-5000-pd}.  \fig{pdwintable} displays the winning method(s) for each data set where all the discretization methods did not tie.  The winner or winners for each dataset are displayed, along with the number of degrees by which it wins.  The counts for total wins are found in \fig{pdaccumwins}.  The entropy-minimization heuristic, fayyadIrani, had more probability of detection wins than any other method, with DiscTree having the next highest number of wins.  Interestingly, there is no clear manner by which to describe the conditions under which the DiscTree algorithm wins, other than that the data sets where it is recorded as a winner tend to have fewer classes (typically less than 5).  However, it also loses to the entropy-minimization heuristic in similar data sets.  

Another interesting result is the sorted data plots for each data set.  These plots, found in \fig{a2dpdcharts} to \fig{s2wpdcharts}, show that in most of the data sets where the fayyadIrani method wins there is so very little difference in the probability of detection scores that the plotted points for fayyadIrani generate on top of or very close to the scores for the other methods.  The DiscTree algorithms still performs visibly well in the auto-mpg data set, and other methods visibly win in one or two other data sets, but in most cases there is very little difference between the plots.

\begin{figure}[!ht]
\begin{center}
\begin{tabular}{|c|c|c|c|c|c|}\cline{1-6}
\textit{Data Set}& \textit{Instances}& \textit{Winner(s)}& \textit{Degree}& \textit{Numeric}& \textit{Classes}\\\hline
hayes-roth& 132& {cat/fayyadIrani}& 1(1)&  1& 3\\
hepatitis& 155&  {fayyadIrani}& 1(1)& 6& 2\\
flag& 194& {cat/disctree3/fayyadIrani/pkid}& 1& 10& 4\\
%imports-85& 205& {fayyadIrani/pkid}& 1(1)& 15& 5\\
heart-c& 303& {disctree3}& 1& 6&  5\\
ecoli& 336& {cat}& 2(4)& 7& 8\\
auto-mpg& 398& {disctree3}& 3& 5&  3\\
wdbc& 569& {fayyadIrani}& 2(2)& 30&  2\\
credit-a& 690& {disctree3/fayyadIrani}& 1& 0& 2\\
diabetes& 768& {fayyadIrani}& 4(4)& 8& 2\\
vowel& 990& {fayyadIrani/tbin}& 1(3)& 10& 11\\
segment& 2310& {fayyadIrani}& 1(2)& 19& 7\\
waveform-5000& 5000& {fayyadIrani}& 2(4)& 40& 3\\
letter& 20000& {disctree3/fayyadIrani/pkid}& 1& 16& 26\\
\cline{1-6}
\end{tabular}
\caption{These data sets had a particular winner(s) for their Probability of Detection comparison. In all cases, degree measures the number of wins over the next closest method.  In the event that disctree3 did not win, the number in parenthesis represents its win difference from the lead method. }
\label{fig:pdwintable}
\end{center}
\end{figure}

\begin{figure}[!ht]
\begin{center}
\begin{tabular}{|c|c|}\cline{1-2}
\textit{Method}& \textit{Wins}\\\cline{1-2}
fayyadIrani& 10\\
disctree3& 5\\
tbin& 1\\
pkid& 2\\
cat& 3\\
\cline{1-2}
\end{tabular}
\caption{Total Wins Per Method Based on Mann-Whitney $U$-Test Wins on Each Data Set's Probability of Detection Scores}
\label{fig:pdaccumwins}
\end{center}
\end{figure}

\clearpage

\addtolength{\oddsidemargin}{-.5in}
\begin{figure}[p]
\begin{center}
\begin{tabular}{cc}
\includegraphics[scale=0.74]{audiology-pd.pdf}&
\includegraphics[scale=0.74]{auto-mpg-pd.pdf}\\
\includegraphics[scale=0.74]{breast-cancer-pd.pdf}&
\includegraphics[scale=0.74]{breast-cancer-wisconsin-pd.pdf}\\
\includegraphics[scale=0.74]{credit-a-pd.pdf}&
\includegraphics[scale=0.74]{diabetes-pd.pdf}\\
\end{tabular}
\caption{Plots of Probability of Detection Scores, Sorted by Value}
\end{center}
\label{fig:a2dpdcharts}
\end{figure}
\clearpage
\begin{figure}[p]
\begin{center}
\begin{tabular}{cc}
\includegraphics[scale=0.74]{ecoli-pd.pdf}&
\includegraphics[scale=0.74]{flag-pd.pdf}\\
\includegraphics[scale=0.74]{hayes-roth-pd.pdf}&
\includegraphics[scale=0.74]{heart-c-pd.pdf}\\
\includegraphics[scale=0.74]{heart-h-pd.pdf}&
\includegraphics[scale=0.74]{hepatitis-pd.pdf}\\
\end{tabular}
\caption{Plots of Probability of Detection Scores, Sorted by Value}
\end{center}
\label{fig:e2hpdcharts}
\end{figure}
\clearpage
\begin{figure}[p]
\begin{center}
\begin{tabular}{cc}
\includegraphics[scale=0.74]{imports-85-pd.pdf}&
\includegraphics[scale=0.74]{iris-pd.pdf}\\
\includegraphics[scale=0.74]{kr-vs-kp-pd.pdf}&
\includegraphics[scale=0.74]{letter-pd.pdf}\\
\includegraphics[scale=0.74]{mushroom-pd.pdf}&
\includegraphics[scale=0.74]{segment-pd.pdf}\\
\end{tabular}
\caption{Plots of Probability of Detection Scores, Sorted by Value}
\end{center}
\label{fig:i2spdcharts}
\end{figure}
\clearpage
\begin{figure}[p]
\begin{center}
\begin{tabular}{cc}
\includegraphics[scale=0.74]{soybean-pd.pdf}&
\includegraphics[scale=0.74]{splice-pd.pdf}\\
\includegraphics[scale=0.74]{vowel-pd.pdf}&
\includegraphics[scale=0.74]{waveform-5000-pd.pdf}\\
\includegraphics[scale=0.74]{wdbc-pd.pdf}&
\includegraphics[scale=0.74]{wine-pd.pdf}\\
\end{tabular}
\caption{Plots of Probability of Detection Scores, Sorted by Value}
\end{center}
\label{fig:s2wpdcharts}
\end{figure}
\clearpage
\addtolength{\oddsidemargin}{.5in}

\ssp

\subsection{Probability of Not False Alarm}
\label{comparison-npf}

\dbsp

The overall results for the probability of not false alarm comparison between the discretization methods discussed in \chap{discretization}, across all the data generated, can be found in \fig{overall-npf}

\begin{figure} [!ht]
\begin{center}
\begin{tabular}{|c|ccc|c|} \cline{1-5}
\textit{key}& \textit{win}& \textit{loss}& \textit{ties}& \textit{win-loss}\\\cline{1-5}
 fayyadIrani& 4& 0& 0& 4\\
        tbin& 3& 1& 0& 2\\
        pkid& 0& 2& 2& -2\\
   disctree3& 0& 2& 2& -2\\
         cat& 0& 2& 2& -2\\
\cline{1-5}
\end{tabular}
\end{center}
\caption{overall for npf}
\label{fig:overall-npf}
\end{figure}

The results of this overview shows that in comparison with all the other discretization methods, the entropy-minimization method won over each other methods. The tenbins algorithm performed next best, winning a comparisons against each method besides fayyadIrani and losing just to that method.  The pure data set, DiscTree, and pkid each performed poorly in this comparison, losing out to each other method.

\fig{overall-npf} provides a general summary of the results.   However, each data set has also had the win/loss/tie record created.  Those results can be found in \fig{audiology-npf} to \fig{waveform-5000-npf}.  \fig{npfwintable} displays the winning method(s) for each data set where all the discretization methods did not tie.  The winner or winners for each dataset are displayed, along with the number of degrees by which they win.  The counts for total wins are found in \fig{npfaccumwins}.  Again, the entropy-minimization heuristic had the most wins, while the DiscTree method won about half as often.  Interestingly, there is no clear manner by which to describe the conditions under which the DiscTree algorithm wins, other than that the data sets where it is recorded as a winner tend to have fewer classes (typically less than 5).  However, it also loses to the fayyadIrani method in similar data sets.  

Another interesting result is the sorted data plots for each data set.  These plots, found in \fig{a2dnpfcharts} to \fig{s2wnpfcharts}, show that in most of the data sets where the fayyadIrani method wins, there is so very little difference in the not probability of failure scores that the plotted points for fayyadIrani, generate on top of or very close to the scores for the other methods.

\begin{figure}[!ht]
\begin{center}
\begin{tabular}{|c|c|c|c|c|c|}\cline{1-6}
\textit{Data Set}& \textit{Instances}& \textit{Winner(s)}& \textit{Degree}& \textit{Numeric}& \textit{Classes}\\\hline
hayes-roth& 132& {cat}& 1(1)&  1& 3\\
hepatitis& 155&  {fayyadIrani}& 1(1)& 6& 2\\
flag& 194& {disctree3/fayyadIrani/pkid/tbin}& 1& 10& 4\\
imports-85& 205& {fayyadIrani}& 2(3)& 15& 5\\
heart-c& 303& {disctree3}& 1& 6&  5\\
%ecoli& 336& {cat/fayyadIrani/pkid}& 2(2)& 7& 8\\
auto-mpg& 398& {disctree3}& 3& 5&  3\\
wdbc& 569& {fayyadIrani}& 2(2)& 30&  2\\
credit-a& 690& {disctree3/fayyadIrani}& 1& 0& 2\\
diabetes& 768& {fayyadIrani}& 4(4)& 8& 2\\
vowel& 990& {cat/fayyadIrani/tbin}& 2(2)& 10& 11\\
segment& 2310& {fayyadIrani/tbin}& 2(2)& 19& 7\\
waveform-5000& 5000& {fayyadIrani}& 1(2)& 40& 3\\
letter& 20000& {disctree3/fayyadIrani/pkid}& 1& 16& 26\\
\cline{1-6}
\end{tabular}
\caption{These data sets had a particular winner(s) for their not Probability of Failure comparison. In all cases, degree measures the number of wins over the next closest method.  In the event that disctree3 did not win, the number in parenthesis represents its win difference from the lead method. }
\label{fig:npfwintable}
\end{center}
\end{figure}

\begin{figure}[!ht]
\begin{center}
\begin{tabular}{|c|c|}\cline{1-2}
\textit{Method}& \textit{Wins}\\\cline{1-2}
fayyadIrani& 10\\
disctree3& 5\\
tbin& 3\\
pkid& 2\\
cat& 2\\
\cline{1-2}
\end{tabular}
\caption{Total Wins Per Method Based on Mann-Whitney $U$-Test Wins on Each Data Set's not Probability of Failure Scores}
\label{fig:npfaccumwins}
\end{center}
\end{figure}

\clearpage

\addtolength{\oddsidemargin}{-.5in}
\begin{figure}[p]
\begin{center}
\begin{tabular}{cc}
\includegraphics[scale=0.74]{audiology-npf.pdf}&
\includegraphics[scale=0.74]{auto-mpg-npf.pdf}\\
\includegraphics[scale=0.74]{breast-cancer-npf.pdf}&
\includegraphics[scale=0.74]{breast-cancer-wisconsin-npf.pdf}\\
\includegraphics[scale=0.74]{credit-a-npf.pdf}&
\includegraphics[scale=0.74]{diabetes-npf.pdf}\\
\end{tabular}
\caption{Plots of Probability of not False Alarm Scores, Sorted by Value}
\end{center}
\label{fig:a2dnpfcharts}
\end{figure}
\clearpage
\begin{figure}[p]
\begin{center}
\begin{tabular}{cc}
\includegraphics[scale=0.74]{ecoli-npf.pdf}&
\includegraphics[scale=0.74]{flag-npf.pdf}\\
\includegraphics[scale=0.74]{hayes-roth-npf.pdf}&
\includegraphics[scale=0.74]{heart-c-npf.pdf}\\
\includegraphics[scale=0.74]{heart-h-npf.pdf}&
\includegraphics[scale=0.74]{hepatitis-npf.pdf}\\
\end{tabular}
\caption{Plots of Probability of not False Alarm Scores, Sorted by Value}
\end{center}
\label{fig:e2hnpfcharts}
\end{figure}
\clearpage
\begin{figure}[p]
\begin{center}
\begin{tabular}{cc}
\includegraphics[scale=0.74]{imports-85-npf.pdf}&
\includegraphics[scale=0.74]{iris-npf.pdf}\\
\includegraphics[scale=0.74]{kr-vs-kp-npf.pdf}&
\includegraphics[scale=0.74]{letter-npf.pdf}\\
\includegraphics[scale=0.74]{mushroom-npf.pdf}&
\includegraphics[scale=0.74]{segment-npf.pdf}\\
\end{tabular}
\caption{Plots of Probability of not False Alarm Scores, Sorted by Value}
\end{center}
\label{fig:i2snpfcharts}
\end{figure}
\clearpage
\begin{figure}[p]
\begin{center}
\begin{tabular}{cc}
\includegraphics[scale=0.74]{soybean-npf.pdf}&
\includegraphics[scale=0.74]{splice-npf.pdf}\\
\includegraphics[scale=0.74]{vowel-npf.pdf}&
\includegraphics[scale=0.74]{waveform-5000-npf.pdf}\\
\includegraphics[scale=0.74]{wdbc-npf.pdf}&
\includegraphics[scale=0.74]{wine-npf.pdf}\\
\end{tabular}
\caption{Plots of Probability of not False Alarm Scores, Sorted by Value}
\end{center}
\label{fig:s2wnpfcharts}
\end{figure}
\clearpage
\addtolength{\oddsidemargin}{.5in}

\ssp

\section{Summary}
\label{resultssummary}

\dbsp

Our results for the discretization method comparison are what could be called a mixed bag.  For each method used, the entropy-minimization heuristic won most often when just viewing the results of the Mann-Whitney $U$-test scores.  This is consistent with results from \cite{dou95, det02}, which both found the method proposed in \cite{fi92} to have the best effect on the Na\"{i}ve Bayes classifier.  

However, it is notable that for each method, most of the plots of the data points appear very similar, with small differences between the data points plotted for the fayyadIrani method and the other methods it was compared against.  

Also notable is the considerable win that the DiscTree algorithm had for each measure in the auto-mpg data set.  That data set, described in \fig{autoinfo}, consistently had DiscTree winning by a degree of 3.  DiscTree also performed well against the PKID method proposed in \cite{yw01}.  In each measure, DiscTree consistently had more wins than PKID ( and in most cases the other methods).  This is notable because of the methods compared, PKID is the method proposed most recently and provides the basis for WPKID, NDD, and WNDD that could not be tested in this paper.

\begin{figure}[!ht]
\begin{center}
\begin{tabular}{|c|c|c|c|c|c|}\cline{1-6}
\textit{Data Set}& \textit{Instances}& \textit{Attributes}& \textit{Numeric}& \textit{Nominal}& \textit{Classes}\\\hline
auto-mpg& 398& 7& 5& 2& 3\\
\cline{1-6}
\end{tabular}
\end{center}
\caption{Data Set Information for auto-mpg}
\label{fig:autoinfo}
\end{figure}

The results in this section will be reviewed in \chap{conclusion}.

\ssp
