\begin{slide}[toc=Types]{Data/Knowledge Farming}
\begin{itemize}

\item  Andrew Kusiak : J. Ops. research, 2005: ``data farming''
\bi
\item DF generalizes DM (data mining)
\item DF = models + practices used to define most appropriate features for data collection/ transformation/
assessment
\item DF.effort greater than data mining effort
\ei
\item DOD : ``data farming''
\bi
\item Using a super-computer...
\item ... simulate the h\#ck out a model
\item e.g. the Marine Corps' {\em Project Albert}
\item understand the landscape of potential simulated outcomes, enhance intuition, find surprises and outliers, and identify potential options
\item Heavily augmented with interactive visualization tools
\item ??? no automatic summarization methods (e.g. data miners)
\ei
\item Me, ASE 2000: ``Knowledge farming''
\bi
\item
{\bf Plant} the seeds; i.e. build a simulator that can reproduce domain conditions;
\item 
{\bf Grow} the seedlings; i.e. Monte Carlo the simulator to generate data;
\item
{\bf Harvest} the crop;  i.e. use data miners to find patters in the data;
\item Core problem: how to build human-readable succinct rules from gigabytes of data
\ei

\end{itemize}
\end{slide}

\begin{slide}[toc=Why not KM?]{The Case Against Knowledge Farming (KM)}
\begin{itemize}
\item
Q: Won't you just re-learn the original model?
\bi
\item A: Nope
\item
Learned model = input data + summarization method;
\item Different summarization, different models;
\item KM + {\em relevancy tests} often prunes away variables that are noisy, low variance, under-sampled, not informative, etc etc
\item KM's harvest smaller than original.
\item Effects that are obscure in original are clear in summary
\ei
\item Q: Aren't you just learning quirks in the model?
\bi
\item A$_1$:If you have domain data, don't rely on a (possibly incorrect) model
\item A$_2$: What's the difference?
\bi
\item If the model is being used to make policy decisions, then we need to know the 
model, warts and all.
\ei
\item A$_3$: if your model has subtle quirks, how will you find them otherwise? (e.g. KARDIO)
\ei
\end{itemize}
\end{slide}

\begin{slide}[toc=Examples]{Examples of knowledge farming\newline (KARDIO: Bratko'89; 
ESA: Pearce'88)}
\begin{center}
\includegraphics[width=3.5in]{KARDIO.EPS}
\end{center}
\end{slide}

\begin{slide}[toc=More examples]{(More) Examples of knowledge farming\newline (Menzies\&Raffo: ASE'02;
Menzies\&Feather: RE'02)}
Optimizing a combined staffing/``worries'' model:
staffing: the COCOMO-II effort model;
``worries'': the Madachy software risk model.
\begin{center}
before:~\includegraphics[width=1.5in]{beforeless.eps}
after:~\includegraphics[width=1.5in]{afterless.eps}
\end{center}

Trade space studies at JPL:
benefits: requirements coverage;
costs: cost of risk mitigation strategies
(each dot is yes-no to 99 decisions).
\begin{center}
\includegraphics[width=1.5in]{jpl.eps}
\end{center}
\end{slide}

\begin{slide}[toc=Yet more examples]{(Yet more) Examples}
This talk: knowledge farming and SILAP
\end{slide}
