\section{Existing Open Source Data Mining Tools}

As OURMINE is an open source data mining tool, it is important to explore other freely available options. In doing so, it is hoped that insight can be gained by finding similarities and likenesses between these options and the tool discussed herein. It should be noted, however, that while OURMINE is presented in this thesis as an entirely adequate environment for performing data ming tasks, it is not intended to replace any existing options. Instead, its purpose is to supply the software community with yet another powerful alternative. Thus in this chapter other popular tools will briefly be explored.

  \subsection{ADaM}

ADaM~\cite{adamtool}, or the Algorithm Development and Mining System, was developed at the University of Alabama. The tool provides over one hundred components including classifiers, %such as Naive Bayes~\cite{rish01}, Decision Trees~\cite{Mitchell97}, Bayes Networks~\cite{Heckerman96atutorial}, etc., 
clustering algorithms, and feature selectors. Along with standard data mining components, ADaM comes with tools to process images. These include programs for cropping, rotation, scaling and other image processing techniques standard in computer vision. Such components are provided as executables and Python modules. 

Executables packaged with ADaM are utilized via the command line by issuing the component name. For example, in order to train a\nbc~\cite{rish01} classifier using this option, first the command \begin{verbatim} ITSC_NaiveBayesTrain \end{verbatim} must be issued in order to learn characteristics of the classes of interest by using sample patterns. Then, the command \begin{verbatim} ITSC_NaiveBayesApply \end{verbatim} is used to read the characteristics provided by the training module to then classify the input data. Thus, all classifiers in ADaM are structured in this manner -- with training and application modules.

Python modules that come with the system are used in typical Python scripts. For example, below illustrates a Python script from the documentation provided with ADaM that performs a median filter on an image by computing the median pixel value in a neighborhood of that pixel in the image. 

\begin{verbatim}
import sys
sys.path.append('E:/projects/ADaM/build/')
import ADaM
inFile = "input.bin"
winSize = 7
outFile = "filtered.bin"
id1 = ADaM.ReadImage(inFile) # return a handle as input for the filter funciton
id2 = ADaM.MedianFilter(id1, winSize)
ADaM.WriteImage(outFile, id2)
ADaM.DeleteImage(id1)
ADaM.DeleteImage(id2) 
\end{verbatim}


  \subsection{Databionic ESOM Tools}

  The Databionic ESOM Tool~\cite{databionictool} takes a noteworthy approach in performing data mining tasks by using Emergent Self-Organizing Maps (ESOM)~\cite{ultsch05}. A self-organizing map utilizes theory from artificial neural networks to construct a discretized representation of training examples in a low-dimensional space. This representation is called a $map$. SOMs, therefore, remain a useful alternative in visualizing high-dimensional spaces. 
 
  \subsection{Gnome Data Mining Tools}
  
  Gnome Data Mining Tools~\cite{gnome-data-mine} is a collection of freely available tools that are packaged together to make a single collection of units that can be used for data mining tasks. The package requires that Python and Gnome be installed on the system used to run it. 
  
  \begin{figure}[h]
    \begin{center}
      \includegraphics[width=4in]{figures/rw-gdmdtree1.png}
    \end{center}
    \caption[Selecting Decision Tree options]{Selecting Decision Tree options. Obtained from $http://www.togaware.com/datamining/gdatamine/gdmdtree.html$.}\label{fig:rw-gdmdtree1}
  \end{figure}
   
  The application takes an approach that utilizes command-driven GUIs in order to run machine learning algorithms on input data. For instance, \fig{rw-gdmdtree1} shows the operation of a Decision Tree~\cite{Mitchell97} via a GUI that is executed by typing \begin{verbatim} gdmdtree vote \end{verbatim} where $vote$ is the name of the data set. Many features are available to the user through this interface, such as the ability to select the data set, the option to display the decision tree (as in \fig{rw-gdmdtree2}) and perform tree pruning, etc. 
  Using Gnome Data Mining Tools, visualization of the data can also be obtained through bar charts, and frequency distributions of data can be represented through bin charts. 
  
%\clearpage

\begin{figure}[h]
\begin{center}
\includegraphics[width=4in]{figures/rw-gdmdtree2.png}
\end{center}
\caption[Output of the decision tree learned]{Output of the decision tree learned. Obtained from $http://www.togaware.com/datamining/gdatamine/gdmdtree.html$.}\label{fig:rw-gdmdtree2}
\end{figure}

  \newpage
  \subsection{KNIME}

  The Konstanz Information Miner, KNIME~\cite{knimetool} is a popular visual-based modular data exploration environment. It incorporates over 100 processing nodes for tasks such as data preprocessing, modeling, data mining and analysis, and supports integration to many other existing tools. Being based on the Eclipse platform, KNIME allows extensibility by constructing custom nodes within the environment that can be used in both production and research settings.

\begin{figure}[h]
\begin{center}
\includegraphics[width=4in]{figures/rw-knime1.png}
\end{center}
\caption[Connecting nodes in KNIME]{Connecting nodes in KNIME. Here, a data set is read through the $File Reader$ node, which is then colored, clustered, and analyzed. Obtained from $http://www.knime.org/documentation/getting_started$}\label{fig:rw-knime1}
\end{figure}

  KNIME works by allowing the user to build personalized workflows from either preexisting or custom nodes. For example, \fig{rw-knime1} shows a workflow created in order to parse a data set from an ASCII file, add color to it, and cluster the data using the K-means algorithm. The outcome is a scatter plot showing the results of the clustered data. Thus, workflows are created in this manner; icons are dragged to a {\em Workflow Editor} window, and then connected to complete a sequence of steps to be executed. 

\begin{figure}
\begin{center}
\includegraphics[width=4in]{figures/rw-knime2.png}
\end{center}
\caption[Configuring a node in KNIME]{Configuring a node in KNIME. Obtained from $http://www.knime.org/documentation/getting_started$}\label{fig:rw-knime2}
\end{figure}

  After nodes are connected, they must be configured and executed. Configuring nodes is accomplished by right-clicking a node, and then selecting desired characteristics for that node. \fig{rw-knime2} shows this configuration by selecting the file (data set) for the {\em File Reader} node, which marks the first step in this workflow. Finally, by executing the {\em Scatter Plot} node in \fig{rw-knime1}, all predecessor nodes are automatically executed for the user. The result of the sequential execution of these nodes in this workflow is shown in \fig{rw-knime3}.

%\clearpage

\begin{figure}[h]
\begin{center}
\includegraphics[width=5in]{figures/rw-knime3.png}
\end{center}
\caption[KNIME workflow]{The result of executing all nodes in the workflow. Note the scatter plot of clustered values. By selecting on each instance from the plot, its corresponding row is shown in the interactive table. Obtained from $http://www.knime.org/documentation/getting_started$}\label{fig:rw-knime3}
\end{figure}

  \subsection{Orange}

  Orange~\cite{orange-tool} is another popular open source alternative available to both novices and experts that allows data mining to be conducted through the use of either Python scripting or visual programming. By using only a few lines of Python (assuming Python is correctly installed on the system), a user can apply data mining modules to data sets, as in the following code:


\begin{verbatim}
import orange
data = orange.ExampleTable("voting")
classifier = orange.BayesLearner(data)
for i in range(5):
    c = classifier(data[i])
    print "original", data[i].getclass(), "classified as", c
\end{verbatim} 

Here, $data$ is assigned the data set labeled ``voting''. The classifier is set to\nbc, which is learned on the first five instances in the training data. The original class is then printed adjacent to the class assigned by the classifier. 

Visual programming is conducted in Orange by using the {\em Orange canvas} -- a space containing widgets, that when used together, create a $schema$. Orange makes use of widgets due to their modularity. Widgets are visual components on the canvas that supply instructions to other widgets through connected ``channels''. These channels provide the means for widgets to communicate by being the medium through which information is transmitted. The schemas built from these interconnected widgets can yield very informative visualizations. \fig{rw-orange1} shows a schema consisting of a $File$ widget, which ``feeds'' the data into a {\em Classification Tree} widget, and finally into both {\em Classification Tree Viewer 2D} and $Scatterplot$ widgets. The result is the visualization of the data itself in the scatter plot, as well as the branching nodes learned from the classification tree. Thus, Orange remains another powerful tool that can be used by both advanced and novice data miners. 

%\clearpage

\begin{figure}
\begin{center}
\includegraphics[width=4in]{figures/rw-orange1.png}
\end{center}
\caption[Simple Orange schema]{A simple schema constructed in Orange. Widgets represent modular parts of the schema that communicate to one another to form the resulting visualizations. Obtained from $http://www.ailab.si/orange/screenshots.psp$}\label{fig:rw-orange1}
\end{figure}


  \subsection{RapidMiner}

  RapidMiner~\cite{rapidminer-tool} is both an open source and commercially available data mining system, used worldwide. It provides not only a graphical user interface for processes, but also a custom scripting environment for certain desired operations. Workflows are constructed as per previously seen tools -- by combining nodes representing components, and connecting them to build a fully operational stream of execution. Data stores can be accessed from existing popular spreadsheet packages, database environments, and other forms such as text files. This data can then be inserted into the workflow, and following nodes can act on this data to accomplish specific data mining tasks. 
  
\begin{figure}
\begin{center}
\includegraphics[width=5in]{figures/rw-rapidminer1.png}
\end{center}
\caption[RapidMiner Plotting]{One of the many types of plotting facilities available in RapidMiner. Obtained from $http://www.ailab.si/orange/screenshots.psp$}\label{fig:rw-rapidminer1}
\end{figure}

Another potentially powerful aspect of RapidMiner is its many visualization tools and plotting facilities. Colorful plots and charts (\fig{rw-rapidminer1}), as well as accurate visualization of algorithms' results (\fig{rw-rapidminer2}) provide a good explanation medium for those new to data mining. 

%change wording? kinda sounds like an ad

\begin{figure}
\begin{center}
\includegraphics[width=5in]{figures/rw-rapidminer2.png}
\end{center}
\caption[RapidMiner Visualization]{One of the many types of visualizations available in RapidMiner. Here, a decision tree is represented as though it was drawn on a piece of paper -- from the root node, branching left and right. Obtained from $http://www.ailab.si/orange/screenshots.psp$}\label{fig:rw-rapidminer2}
\end{figure}


  \subsection{Rattle}

  Rattle (the R Analytical Tool To Learn Easily)~\cite{rattle-tool} is a freely available open source data mining toolkit. The authors of this toolkit attempt to provide an easy gateway into learning about the complex statistical language, R~\cite{r-tool}, while performing data mining through graphical user interfaces. By leading the user through the basics of data mining, it is hoped that he/she will become more familiar with R once it is illustrated how it is used to perform these basics operations.

  Rattle is used as an educational device in various places of the world, such as for the Data Mining Workshop in Canberra, Australia, and also in the Shenzhen Graduate School in China. Rattle is used professionally by others as well. For instance, Australia's largest data mining team in the Australian Taxation Office utilizes Rattle on a daily basis.

  \subsection{Weka}

  Weka (the Waikato Environment for Knowledge Analysis)~\cite{weka-tool}, developed in New Zealand, is an extremely popular, open source collection of machine learning algorithms used for various data mining tasks. The tools are written in Java, and thus have the ability to be executed on any system with the Java Runtime Environment installed. Standard operations such as data preprocessing, classification and clustering are available to the user through either a command line environment, as in \fig{rw-weka1}, or from an easily navigable graphical user interface like shown in \fig{rw-weka2}, \fig{rw-weka3}, \fig{rw-weka4}, and \fig{rw-weka5}. 

Weka allows access to practical portions of the toolkit through the GUI. These include the simple command line interface of \fig{rw-weka1}, the $Experimenter$ of \fig{rw-weka2}, the $Explorer$ in \fig{rw-weka3} and the $KnowledgeFlow$ from \fig{rw-weka4}. The command line interface is a valuable and powerful portion of Weka. With it, commands can be issued to the system to utilize each module of the environment. However, its tools are also available through a (e.g. UNIX) terminal. This allows for operations to be performed from an external setting, which can be called using any desired language. For instance, in order to call a simple \nbc classifier using a BASH~\cite{ramey94} function, the following code could be used


\begin{verbatim}

nb () { 
    local learner=weka.classifiers.bayes.NaiveBayes;
    $Weka $learner -p 0 -t $1 -T $2
}

nb train.arff test.arff

\end{verbatim}

where $\$Weka$ represents the path of the Weka compiled Java code. 

The $Experimenter$ option of the toolkit allows standard experiments to be run from a graphical user interface. This provides the means to experiment with various components of data mining. For instance, using this method, statistical comparisons can be made between different treatments on a data set. 

For possibly more immediately available results, the $Explorer$ option of the GUI gives the user the ability to execute single algorithms that provide quick insight into the properties of the data. For instance, suppose one wished to display the most important attributes ranked using Information Gain (whose results are shown in \fig{rw-weka5} for the popular $weather$ data set) as discussed in ~\cite{Mitchell97}, this interface yields rapid loading of data and execution. 

Popular workflows seen in many other existing tools can also be constructed using Weka's $Knowledge Flow$. In \fig{rw-weka4}, a fairly dense system of interconnected nodes perform complex operations. Thus, in short, Weka provides much variety in conducting data mining experimentation. 
  
\clearpage

\begin{figure}
\begin{center}
\includegraphics[width=6in]{figures/rw-weka1.png}
\end{center}
\caption[Weka's Command Line Interface]{Weka's command line interface (CLI) available to users of the toolkit.}\label{fig:rw-weka1}
\end{figure}

\clearpage

\begin{figure}
\begin{center}
\includegraphics[width=6in]{figures/rw-weka2.png}
\end{center}
\caption[Weka's Experimenter]{Weka's experimenter interface, allowing standard data mining experiments to be set up through the graphical user interface.}\label{fig:rw-weka2}
\end{figure}

\clearpage

\begin{figure}
\begin{center}
\includegraphics[width=6in]{figures/rw-weka3.png}
\end{center}
\caption[Weka's Explorer]{Weka's explorer interface, allowing quick execution of many machine learning algorithms to accompish a variety of data mining tasks.}\label{fig:rw-weka3}
\end{figure}

\clearpage

\begin{figure}
\begin{center}
\includegraphics[width=6in]{figures/rw-weka4.png}
\end{center}
\caption[Weka's Knowledge Flow Builder]{Weka's Knowledge Flow builder provides the opportunity to build a workflow seen in many existing toolkits, by attaching nodes sequentially to perform complex tasks. Obtained from $http://www.cs.waikato.ac.nz/~mhall/knowledgeFlow.png$.}\label{fig:rw-weka4}
\end{figure}

\clearpage

\begin{figure}
\begin{center}
\includegraphics[width=6in]{figures/rw-weka5.png}
\end{center}
\caption[Ranking Attributes using Weka's Information Gain]{Ranking attributes using Information Gain (available through the Explorer) on the popular $weather$ data set.}\label{fig:rw-weka5}
\end{figure}
 
\section{The Benefits of Textual Programming} 

%{\em "'A picture is worth a thousand words' -- isn't it? And hence graphical representation is by nature universally superior to text, isn't it -- isn't it? Why then isn't the anecdote itself expressed graphically?"} -- Marian Petre, {\em Why Looking isn't Alway Seeing: Readership Skills and Graphical Programming}. In this paper (~\cite{petre95}), Petre examines many conflicting arguments for and against graphical approaches to represent many problems 
 
	A variety of the more widely-used data mining toolkits have been assessed,
 and it is found that they are
are 
not suitable for  
publishing experiments.  
My complaint with these tools is same as that offered by
Ritthoff et al.~\cite{ritthoff01}:

\bi
\item 
Proprietary tools such as MATLAB are not freely available.
\item
 Real-world experiments
are more complex than 
running one algorithm. 
\item
Rather, such experiments or applications require {\em intricate combinations}
of a large 
number of tools that include data miners, data pre-processors and report regenerators.
\ei

%Many developers and data mining researchers have realized the need to "wire up" data mining tools in order create an interconnection between intricate combinations of tools required for publishable, large-scale experiments. These tools include data miners, data pre-processors and report generators. 

However, the need to ``wire up'' data miners within a multitude of other tools has been addressed in many ways. 
In WEKA's visual {\em knowledge flow} programming environment, for example,
nodes represent different pre-processors/ data miners/ etc while arcs represent how data flows between them.
A similar visual environment is offered by many other tools including ORANGE (see \fig{rw-orange1}).

%The need to extend data mining tools is well established.

%The need to ``wire up'' data miners within a multitude of other tools has been addressed in many ways. 
%In WEKA's visual {\em knowledge flow} programming environment, for example,
%nodes represent different pre-processors/ data miners/ etc while arcs represent how data flows between them.
%A similar visual environment is offered by many other tools including ORANGE (see \fig{orange}).
%The YALE tool (now RAPID-I) built by Ritthoff et al. formalizes these visual environments by generating
%them from  XML
%schemas describing
%{\em operator trees} like \fig{yale}.

Ritthoff et al. argues (and I agree) that
the standard interface of (say) WEKA does not support the rapid generation of
these intricate combinations. For many, these
visual environments are either
overly elaborate,
discouraging or distracting: 
\bi
\item
These standard data mining toolkits may be overly elaborate. As shown throughout this thesis, very simple UNIX scripting tools suffice for the specification,
execution, and communication of experiments.
\item Some data mining novices are discouraged by the tool complexity. These novices shy away
from extensive modification and experimentation.
\item Other developers may become  so enamored with the impressive software engineering
inside these tools that they waste much time  building environments to support data mining,
but never get around to the data mining itself.
\item According to Menzies~\cite{me00v}, while visual systems provide motivation for beginners, they fail in providing a good explanation device for many software engineering and 
knowledge engineering problems, as many of these problems are not inherently spatial. For example, suppose an entity-relationship diagram is drawn on a piece of paper. While inferences can be made from the diagram, they are not entirely dependent on the physical location of (e.g.) an entity. 
\ei
A similar experience is reported by our colleagues in WVU's Department of Statistics.
In order to support teaching, they 
hide these data mining tools inside high-level scripting environments. Their scripting environments
shield the user from the complexities of ``R''
by defining high-level LISP code that, internally, calls ``R''.
While a powerful language, many find that LISP can be 
arcane to many audiences. 

OURMINE is a data mining scripting environment. The current kit has tools written in
BASH/ GAWK/ JAVA/ PERL/ and there is no technical block to adding
other tools written in other languages. Other toolkits impose strict limitations of the usable languages:
\bi
\item
 MLC++ requires C++  
\item
Extensions to WEKA must be written in JAVA.
\ei
My preference for BASH~\cite{ramey94}/GAWK~\cite{awkbook} over, say, LISP is partially a matter of
taste but I defend that selection as follows.  Once a student
learns, for example, RAPID-I's XML configuration tricks, then those learned
skills are highly specific to that  toolkit.  On the other
hand, once a student learns BASH/GAWK methods for data pre-processing
and reporting, they can apply those scripting tricks to any number
of future UNIX-based applications.
