%POM2 paper introduction

%\section{Introduction}
%For example, in the 1990s, the third author 
%published extensively on the supposed benefits of
%rule-based~\cite{mebfd92} and object-oriented programming~\cite{hame93}
%without hard evidence to back up those claims.  
%Many other prominent writers 
%were just as unconstrained.  
%In the mid-1990s Booch~\cite{booch94},
%That discussion is now closed-
% but not because any participant won the debate.
%Rather, a commercial company hired Booch and Rumbaugh and Jacobson to
% create one unified modeling language (UML~\cite{uml}).
%At a keynote address in 2007\footnote{\url{http://promisedata.org/?cat=130}} Fenton shocked
%his audience by saying:
%{\em ``....much of the current software metrics research is inherently irrelevant to the industrial mix ...
%any software metrics program that depends on some extensive metrics collection is doomed to failure''}. 
%The COCOMO~\cite{boehm00b} experience supports Fenton's pessimism.
%After 26 years of trying, Boehm et al. have collected less
%than 200 sample projects for the COCOMO database~\cite{me07f}.
There are too many examples in software engineering of 
positions being defended without empirical evidence.
For example, Rumbaugh~\cite{rum91} and Jacobson~\cite{jacobson92} engaged in a
high-profile and extended debate about the merits of different
styles of object-oriented modeling- without citing any empirical evidence.
%Nowhere in that debate was any
%empirical evidence that one kind of model was easier to
%write/ read/ debug/ maintain than any other kind. 

One explanation for this propensity to argue without evidence is the ``data drought'' reported by 
metrics-guru Norman Fenton. After years of advocating careful data collection~\cite{fenton96},
he now
despairs of that approach. 
%There are many reasons for this data drought including lack of 
%data being collected 
%as well as differences in how the metrics are defined, and collected
%Also, recently, we have seen 
%a decreasing willingness for organizations
%to share their operational data~\cite{me09e}.
When data-based evidence is missing, model-based evidence can fill in the gaps.
Sometimes,  software process models are difficult to build and understand;
%Also, some software process models are hard to understand-
especially when the uncertainties associated with the model lead to numerous wide
ranges. 
Analysts may find the output confusing, rather than clarifying.

%Not all process
%models are problematic. 
With the right automated support,
{\em software process modeling can be very easy}. Given a
well-defined theory and some automatic AI search tools, it is
possible to collect model-based evidence  about the value of variants on software processes.

Our case study concerns {\em requirements prioritization policies}.  
Such policies control the order in which a team implements the requirements.
The spectrum of these policies can be characterized by two extremes: traditional {\em plan-based} and {\em agile}.
%prioritization orders once, before any development. {\em Agile} development changes orderings often.

A standard argument is that agile should be used on highly dynamic projects, whereas plan based should be used on highly critical projects.
%In highly dynamic environments, the standard argument is that agile is a better method for reacting to
%the arrival of new requirements and/or existing requirements changing value.
Boehm  and Turner~\cite{btRisk} define five scales that can characterize the difference between plan-based and agile methods:
project size, project criticality, requirements dynamism, personnel, and organizational
culture. Previously, at ASE'08~\cite{pom1}, Port \& Olkov \&  Menzies (hereafter, POM) studied the effects of different prioritization
policies while adjusting the requirements dynamism.
An advantage of model-based evidential reasoning is the ability to repeat prior analysis.
For example, in the following study,
the POM model is extended along the lines proposed by Gr\"unbacher\cite{grunbacker08} then explored using an AI search engine.

Using the model and the AI search engine, we study plan-based and agile policies in different
contexts. We found (1)~no case where agile does worse than plan-based; (2)~in some cases, agile
performs much better. This leads to the conclusion: {\em The development methodology for an organization should be agile.}

%or the business sensitivity associated with the
%data
%and archived. 
%Model-based methods can be used to build an executable form of the intuitions
%and mental models of domain experts.
%If part of a model is  not known with certainty,
%model variables can become ranges, rather than point
%values. 
%For example Raffo~\cite{raffo05} spent two years carefully building and tuning
%a software process model for a North-Western software construction company. 
%These uncertainties can lead to voluminous output if the
%model is used within some Monte Carlo rig. 
%\bi
%\item  
%In traditional  {\em plan-based} prioritization,
%the ordering is performed once, prior to starting the work.
%\item
%In {\em agile} projects, orderings may change often.
%\ei
%In the literature, the agile versus plan-based debate is often cast as a ``one or the other'' proposition.
%This is a false dichotomy that ignores development practices that use interesting
%combinations of  of plan-based and agile development.
%XXX we use ai for deature subset selection. short theories.
%
%\begin{figure}[!b]\small
%\begin{tabular}{|c|}\hline
%{\bf Manifesto for Agile Software Development}\\
%~\\
%
%We are uncovering better ways of developing\\
%software by doing it and helping others do it.\\
%Through this work we have come to value:\\
%~\\
%
%{\em Individuals and interactions} over processes \& tools\\
%{\em Working software} over comprehensive documentation\\
%{\em Customer collaboration} over contract negotiation\\
%{\em Responding to change} over following a plan\\
%~\\
%
%That is, while there is value in the items on\\
%the right, we value the items on the left more.  \\\hline
%\end{tabular}
%\caption{The agile manifesto. From 
%\protect\url{http://agilemanifesto.org/}.}\label{fig:man}
%\end{figure}
%
%SAY SOMETHING MORE ABOUT EACH OF THE STRATEGIES?
%A more nuanced view is offered by 
%rate at which new requirements arrive and/or change value.
%Gr\"unbacher~\cite{grunbacker08} criticized the POM model (version 1), 
%noting that it explores only one of
%the five scales proposed by Boehm and Turner. 
%Once a model limitation is apparent, the model can be changed and re-simulated.

%Also, we show that (sometimes) model-based reasoning about software processes is easy.
%\bi
%\item
%The revised model
%was defined and implemented in eight weeks by a 
%graduate AI class working  ${\approx}20\%$ of their time on this task.
%This team had no experience in processing modeling.
% and little experience with
%the implementation language (LISP).
%\item
%The team ran the simulations on a CPU farm of 20 Linux machines. The results reported
%in this paper could be generated in one overnight run.
%\ei
%The simplicity of our analysis begs the question: why is so much
%time  wasted on evidence-less debates 
%when a simple combination of simulation plus 
%automatic search can mature the dialogue much faster? 
