\section{Related Work}

reduction, by others

Many general tools and methodologies exist for {\em
modeling} such as distributed agent-based
simulations~\cite{clancey96a}, discrete-event
simulation~\cite{law00,harrell00,kelton02}\footnote{See also the
\url{http://www.imaginethatinc.com} web site.}, continuous
simulation (also called system dynamics)~\cite{hamid91,sterman00},
state-based simulation (which includes petri net and data flow
approaches)~\cite{akhavi93,harel90,martin00}, logic-based and
qualitative-based methods~\cite[chapter
20]{bratko01}~\cite{iwasaki89}, and rule-based
simulations~\cite{mi90}. Note that for software process
programming, elaborate new modeling paradigms may not be
required. For example, the Little-JIL process programming
language~\cite{cass00,wise00} just uses standard programming
constructs such as pre-conditions, post-conditions, exception
handlers, and a top-down decomposition tree showing sub-tasks
inside tasks.


{\em Simulations} can be based on {\em nominal} or {\em
off-nominal values}. Nominal simulations draw their inputs from
known operational profiles of system inputs~\cite{musa98}.
Off-nominal Monte-Carlo (also called stochastic) simulations,
where inputs are selected at random, can check for unanticipated
situations~\cite{gutjahr99}. Stochastic simulation has been
extensively applied to models of software process~\cite{raffo99}.
It is strange to report, but examples of execution of Little-JIL
are rare (an interpreter exists for Little-JIL models but examples
of its execution are not mentioned (i.e. in \cite{cass00,wise00}).


In a {\em sensitivity} analysis, the key factors that most
influence a model are isolated. Also, recommended settings for
those key factors are generated. We take care to distinguish
sensitivity analysis from traditional optimization methods. In our
experience, the real systems we deal with are so complex that they
do not always fit into (e.g.) a linear optimization framework.
Studying  data grown from simulators lets us investigate complex,
non-linear systems using a variety of data driven distributions.
These models can capture complex feedback and rework loops which
are not possible for traditional optimization methods.  Our
experience is that simulation models can look at processes in
detail as well as at a high level of abstraction which is where
the more analytic models must reside. Finally, simulation models
can capture multiple performance measures not able to be explored
using these optimization formulations.  This is not to say that
traditional optimization models are not useful. For certain
questions, traditional optimization formulations provide the best
fit for the question that is trying to be answered.   However, for
many questions, the standard optimization models are not the best
choice and something like {\em treatment learning} may be more
useful.


