%\subsection{POM2: Summary}
\noindent
POM2 shuffles $requirements$ through three stages: ``to do'', ``planned'', and ``finally completed''.
%\be
%\item They are {\em initially} in the ``to do'' heap;
%\item {\em Next} the requirements are revealed to a team;
%\item {\em Subsequently} a team moves  the requirement to their implementation plan;
%\item {\em Finally}, when implimented, the requirements are moved to the compleated list.
%\ee
%Step 4 is optional: if a project suffers from early termination, some requirements
%may never reach completion.
Within POM2 each $Requirement$ has a $cost$ and $value$. The requirements are
organized in $trees$ which are assigned to $teams$. $Requirements$ are completed in
$iterations$ according to the budget. The order $Requirements$ are completed is determined
by the $requirements$ $prioritization$ $policy$. $Projects$ experience $early$
$termination$ if funding is lost.

To simulate a $Project$, POM2 accepts 8 input values: $criticality$ (\tion{crit}),
$crit. modifier$ (\tion{crit}), $culture$ (\tion{cult}), $initial$ $known$ (\tion{dyna}), 
$inter-dependency$ (\tion{rthp}), $size$ (\tion{size}), $team$ $size$ (\tion{size}), and $dynamism$ (\tion{dyna}).
%\bi
%\item $Requirements$ have $costs$ and $values$. 
%\item $Projects$ have  trees of $requirements$;
%\item $Teams$ implement their inter-dependent sub-trees.
%\item $Teams\ sizes$ (\# of members) may differ;
%\item $Teams$ work in $iterations$ and each $iteration$ implements a small number of $requirements$.
%\item A $team$'s $plan$ is the set of  $requirements$ to be implemented in the next $iteration$ ordered by the $requirements$ $prioritization$ $policy$.
%\item The size of the $plan$ is defined by budget.
%\item When the $plan$ is done, $teams$ fill up the $plan$ again with $requirements$ from their sub-trees.
%\item The $project$ ends if the $requirements$ are done or management orders $early$ $termiantion$.
%\item $Teams$ order the $requirements$ in their $plans$ using a $requirements$ $prioritization$ $policy$.
%\ei
%POM2's inputs ranges are: 
%\bi
%\item .82 $<=$ $criticality$ $<=$ 1.26 (\S2.6.1)
%\item 2.0 $<=$ $crit. modifier$ $<=$ 10.0 (\S2.6.1)
%\item 0 $<=$ $culture$ $<=$ 100 (\S2.6.3)
%\item .4 $<=$ $initial$ $known$ $<=$ .7 (\S2.3)
%\item 0 $<=$ $inter-dependency$ $<=$ 100 (\S2.3)
%\item 3 $<=$ $size$ $<=$ 300 (\S2.6.4)
%\item 1.0 $<=$ $team$ $size$ $<=$ 44.0 (\S2.6.4)
%\item 1.0 $<=$ $dynamisim$ $<=$ 50.0 (\S2.6.2)
%\ei

These inputs effect the processing within POM2:
\bi
\item In high $criticality$ systems, $requirements$ $cost$ more to develop. 
\item In high $dynamism$, the $requirements$' $cost$ and $values$ change frequently.
\item Some organizational $cultures$ may
frequently re-order $requirements$ while other organizational $cultures$ 
are more prone to ignore value change.
\ei
%The following description of POM2 is somewhat lengthy.  Much of that discussion is a justification of
%different decisions. The actual code of POM2, shown in \fig{pom2} is quite short.
%\input{ariesbeck/words}
%
%The introduction to this paper mentioned that one of the advantages
%of model-based reasoning is that if can also prevent certain
%superfluous questions from begin explored.  Our results, shown below,
%illustrate this effect.  The rest of this section is a detailed
%description of the internals of POM2.  The reader may take issue
%with some of the details and prefer other modeling decisions.
%However, rather than peruse superfluous modeling details, we will
%use our AI search engines to guide our further work. As we shall
%see, the AI search engines show that very few of the following details are critical determiners of
%model output.  As to the remaining critical details, our future
%work section will discuss which parts of the following description
%might benefit from further elaboration.
%


%\subsection{Differences to POM1}
%\noindent
%POM2 differs from POM1 as follows:
%\bi
%\item POM1 assumed one team implemented
%the requirements while, in POM2,
%there are multiple teams.
%\item POM1's requirements had no dependents. POM2's requirements have dependents
%and a parent requirement cannot start till all its child requirements are finished.
%\item 
%The POM1 paper
%assumed small projects with about 10 developers
%per project and a maximum of 25 requirements.  Within our POM2 simulator, we
%work with projects involving up to 300 developers, we therefore expanded
%the maximum number of requirements from 25 to 750 
%($\frac{300}{10}*25=750$)
%\ei


\subsection{Requirements, Trees, Heaps, and Projects}\label{tion:rthp}
Within POM2, a project is divided into teams, each of which implements
a set of requirements with initial values and costs as defined by Port et al. \cite{pom1}.
% (assigned randomly
%based on the value and cost scales described by Port et al. \cite{pom1}).
%The values may change over the lifetime of a requirement.
These requirements are stored in acyclic trees representing
the work breakdown structure.
%The tree demands that all child-requirements must be completed
%before a parent can be started.  A requirement becomes
%``ready'' when all of its children are ``completed.''  
%
%Initially,
%all leaf requirements are ``ready,'' since by definition, they have no 
%child  that can block their start.
%
Each team is assigned a different tree from which new requirements are pulled.
%Initially, all teams are assigned different trees from which new requirements are pulled. 
%A standard business situation is that sometimes teams must wait for other teams to finish some requirements.
To model inter-team cooperation, at each level of the trees, one dependency is added between two trees at the same level.
%The inter-team dependencies are generated as follows:
%\begin{itemize}
%\item For all requirements at the same level in a tree, one dependency is
%  added to and from another tree, at the same level. 
%  Approaching 
%  generation in this manner removes the possibility of cyclic
%  dependencies (these cannot be completed).
%\end{itemize}
Depending on placement in the tree, inter-team dependency may significantly slow down the
requirement completion process.
%This method best mimics a common structure seen
%in industry: teams often share large sub-systems (requirements that use many
%sub-requirements) rather than very small requirement assemblies.

Trees are generated as follows. Nodes are assigned children
according to an exponential distribution: \(\frac{1}{X}\)\% of nodes have X children.
%(50, 24, 12, 6, 3, 1.5)\% of nodes have  (0,1,2,3,4,5) childeren (respectively).
%Our literature searches did not find papers describing the average structure
%of a software project and the number of dependents for each
%requirements. 
%We return to this issue below.

As a software project progresses it is not unusual for a development
team to discover that they have new requirements to complete. We model this
as follows: 
\begin{itemize}
\item $number$ $of$ $requirements = size * 2.5$\cite{pom1}
%\item If $size$ (number of personnel) is set to 200, the number of requirements 
%for the project will be set to 500 ($2.5 * size = requirements$\cite{pom1}).
\item We pre-generate the number %determine the number 
of new requirements to be added for each project iteration. 
Initially labeled as ``hidden'', the
new requirements are marked ``visible'' when the simulator encounters the iteration for which they were added.
%If we determine that we 
%will add 6 requirements after the first iteration we will randomly 
%generate these additional requirements before the simulation starts. 
\end{itemize}


%The above process inherently handles inter-team dependencies.  As
%we move further up the project tree, it becomes more likely that we
%will encounter a requirement that requires the completion of a requirement in another 
%teams tree.  That is, it will be less likely that we will encounter 
%inter-team dependencies near the leaves of the tree.  However, as more
%requirements are completed, larger sub-systems are built, which naturally
%must be combined with subsystems built by other teams.   
%Term Clarification
%The effect of this aspect of development is controlled by the ``inter-dependency'' 
%variable in our simulator. 




\subsection{Iterations}
Teams maintain a \textit{plan}, which is a set of requirements that have been pulled
from their heap. 
%Initially some percentage (``initial known'') of requirements are pulled from the heap 
%and placed in the plan. 
These requirements are  then marked as known. 
%Term Clarification
\textit{Plans} are processed in 2 to 6 iterations. Early terminations
are very common in iterative projects if (for example)
management decides that the resources of an organization
should be transferred to a new project.
After each iteration, the project has a probability of .9 to
continue to the next iteration. This means that a project only has a 53\% chance of finishing.


%Early termination is modeled by assigning a probability
%of .9 that a project may continue after iteration i.  This means that at 6
%iterations, a project has a 53\% probability of finishing.  This early
%termination rule models the industry reality that many projects are
%canceled before they're originally planned end date. 

The number of requirements that can be completed in each iteration is
controlled by the budget.  Following POM\cite{pom1},
the budget is defined as:
\begin{equation}
  budget=\frac{(total\; initial\; requirement\; cost\;)}{number\; of\; iterations}
\end{equation}
%Where ``total initial cost'' is computed from the cost of the team's
%plan, as it is known at the start of the project, after criticality is
%applied (discussed below). 

%\begin{comment}
%
%In removing this as we do not include personnel in our simulation anymore.
% Note that this base budget will be adjusted
%to reflect the effects of using less skilled programmers 
%(discussed below).

%\end{comment}
\subsection{Prioritization Policies}
\noindent
POM2 explores three prioritization policies.
\be
\item Plan Based (PB) : A non-agile development method
  where requirements are sorted once at the start of development
  using the original \(\frac{value}{cost}\) numbers assumed at the start 
  of the project.
%  Plan-based
%  policy adjusts requirement values at every iteration, ``but does not
%  resort them.''  \cite{pom1}
\item Agile2 (AG2) : A development method where requirements are sorted every iteration on  policies sort requirements (every iteration) on \(\frac{value}{cost}\). 
%  using value. According to Cao et al. \cite{lCao_bRamesh}
%  this is the standard method advocated by the agile community.
%\item Agile2 (AG2) : The same as AG, but sorts on value/cost. This is another standard method used throughout the agile
%  community.
\ee
%POM1 explored a fourth {\em hybrid} policy that 
%combined features of agile with plan-based methods.
%That policy assumed a linear structure of requirements
%and there are technical difficulties with implementing
%that structure over the trees of POM2. Hence, we leave
%exploration of hybrid methods for future work.

\subsection{Boehm and Turner Scales}
POM2 implements four of the five scales identified by Boehm and
Turner \cite{btBoehm}
that distinguished agile from traditional plan-based projects: project $criticality$, requirements $dynamism$, organization $culture$, and $size$.
% These four scales 
%are:
%\bi
%\item project $criticality$;
%\item requirements $dynamism$;
%\item organizational $culture$;
%\item $size$ (total number of developers in all teams);
%\ei
POM2 does not implement the fifth  scale ($personnel$) due to
theoretical reasons, see \tion{pers}.

\subsubsection{Criticality}\label{tion:crit}
%Boehm and Turner~\cite{btBoehm} comment that agile methods are untested on safety-critical 
%products as they present potential difficulties with simple design and lack of 
%documentation. Conversely, plan-based methods evolved 
%to handle highly critical products which are hard to tailor down efficiently 
%to low-criticality products. 

Boehm and Turner\cite{btBoehm} measure criticality in terms 
of losses due to impact or defects and ranges from ``none'' (best for agile 
development) to ``loss of many lives''. Thus, plan-based methods are best 
suited to projects that must be carefully planned, lest defects cause loss of 
many lives. 

%According to the COCOMO research \cite{coco}
%the effect criticality has on cost is displayed in 
%Figure \ref{figure:crit}.
%\begin{figure}\scriptsize
%\centering
%\begin{tabular}{|c|c|p{1cm}|p{1cm}|c|c|} \hline
% & 1 & 2 & 3 & 4 & 5  
%\\ [0.5ex]
%\hline
%Level  & very low & low   & nominal  & high & very high \\
%\hline
%%Impact & none     & impact on discretionary funds   
%%       & impact on essential funds  & single life & many lives \\
%%\hline
%Cost   & 0.82     & 0.92  & 1.00  & 1.10 & 1.26 \\ [1ex]
%\hline
%\end{tabular}
%\caption{Effort multipliers on cost due to criticality. Learned via regression
%from 161 projects. From COCOMO~\cite{boehm00b}. }
%\label{figure:crit}
%\end{figure}
%%POM2 assumes that all requirements performed by one team are of equal criticality
%%(this reflects the industrial standard that specialist teams work on
%%particular portions of the code base).
We assume that X\% of the teams are affected by the criticality, where
\(X=2 \ldots 10\).
%Term Clarification
  Within our simulator, we refer to X as the criticality modifier.
 We adjust the cost of each requirement in a selected teams tree as follows:
\begin{equation}
  \label{eq:criticality}
    cost'= cost * X^{criticality}
\end{equation}

\subsubsection{Dynamism}\label{tion:dyna}
Project dynamism measures how frequently new requirements are created
and how often existing requirements change value.  
%As
%dynamism {\em increases}, we discover {\em more} new requirements and the value
%of the requirements becomes $more$ variable.

Boehm and Turner measure dynamism in terms of the percent of
requirements changed each month and has the range 50\% (best for
agile) to 1 (best for planned-based).

To implement this factor, the POM1 assumptions for dynamism \cite{pom1} 
are adopted:
\begin{itemize}
\item Initially, we only mark $30\% \leq 70\%$
%\[30\% \leq 40*N(0,1) \leq 70\]
of the requirements
  in the project tree as ``visible.''  At each iteration,
  we make visible $new=Poisson(\lambda)$ more requirements.
\item Another parameter controlling dynamism is $\sigma$.  At each iteration, every 
requirement is visited, and and its value is altered by:
    $value' \mbox{~+=~}  maxValue * N(0,\sigma)$
  
\item 
%Note that POM2 links $\sigma$ and $\lambda$ are linked such that high $\sigma$
% values
%  implies high $\lambda$ values (and vice versa). Specifically, 
After setting $\sigma$, we set $\lambda$ to 10 \% of $\sigma$.
% \[\lambda = \frac{\sigma}{10}\]
\end{itemize}

%There are some special cases of the above process that requires further discussion. If the dynamism parameter
%tells a team to find a ``new'' number of requirements, but there are not that 
%many ``visible'' or ``ready'' requirements, then a team's plan may cost less to
%complete than the budget for that iteration. In the case where there is some remaining 
%budget, and there are no requirements that can be completed, the budget is burned. We call 
%this "twiddling thumbs" where a team of employees ceases to work while they wait on
%the completion of work from another team. 

%In another case,  a team may encounter a 
%situation where their remaining budget is not large enough to complete the next 
%available requirement. If this happens, the budget is kept and carried over to the next
%iteration.  This is accomplished by maintaining two numbers: 
%\textit{totalAccumulatedBudget} and \textit{totalSpentBudget}.  Budget is
%burned by setting the \textit{totalSpentBudget} to the
%\textit{totalAccumulatedBudget}.  At the start of each iteration,
%\textit{totalAccumulatedBudget} is incremented by the budget for the
%iteration.

%Before continuing, we digress to discuss another method (which we
%do not use) for handling leftover budgets.  We considered allowing
%a team to work on requirements from other teams.  However, Brook's Law 
%\cite{brooks}
%(adding programmers to a late requirement makes it later) convinced us of the
%folly of that approach.  Teams are specialists in the quirks and
%capabilities of their own code base.  An outsider coming in to work
%temporarily on a small part of another team's code base can be quite
%unproductive (since they do not know the quirks of that code).  
%Further, they can slow up the remaining team (while they teach the
%newcomer the tricks of that code).

\subsubsection{Culture}\label{tion:cult}
For a fully agile project, changing requirement values means resorting the
requirements in the plan to ensure that most cost-effective requirement is
completed first. According to Boehm and Turner,
culture is measured in terms of the percent of the staff thriving on
chaos and has the range 90\% (best for agile, flexible) to 10\% (best for plan-based, rigid).  
%At \(culture=90\%\), the changes to a requirements
%value described above (see $Dynamism$) are used when resorting for the
%next iteration.  However, at \(culture=10\%\), developers are loathed
%to change the initial project plan since this introduces a degree of 
%disorder into their work life.

The calculation used for the accepted value is:
%The value used for resorting is calculated as follows:
%We therefore distinguish between the true value and the accepted value (the value used while resorting) of 
%requirement, calculated as follows:
\begin{equation}
\label{eq:culture}
%  accepted = original + ((actual - origional) * culture)
%(actual * culture) + (original * (1 - culture))
  accepted = value + (value * N(0,\sigma) * culture)
\end{equation}

%As the percent thriving on chaos decreases, ``culture'' drops to 0
%and the accepted value remains as the old value.  Note that the accepted
%value is used to resort (exception: not for the plan-based policy,
%which never resorts) the requirements, but when performance statistics are
%gathered, we use the true value.

\subsubsection{Size}\label{tion:size}
$Size$ is measured in terms of the number of personnel and has the range
3 (typically, best for agile), 10, 30, 100, and 300 (typically, best for plan-based).  The
size of a POM2 project is picked at random from this range and the
number of requirements is then set to $size*2.5$.

We planned to build teams using the results of \cite{pend07}
who report that the size of software development teams has
\((min,mean,sd) = (1,8,20)\).  However, this leads to a 
large number of single-person teams.  
We modified that result
slightly in consultation with some of our NASA colleagues. 
%POM2
%repeatedly selects team size randomly from the distribution of \fig{ssizes}
%until the total team size exceeds $size$.


%\begin{figure}[!h]\renewcommand{\baselinestretch}{0.7}.
%{\scriptsize 
%\begin{center}
%\begin{tabular}{r|rl}     
%X = \# of personnel & \multicolumn{2}{l}{Y = frequency of X}\\\hline
%1&    3&   \rule{3mm}{2mm}                \\
%3&    9&   \rule{9mm}{2mm}                         \\ 
%6&    18&   \rule{18mm}{2mm}                \\
%9&   28&  \rule{28mm}{2mm}        \\
%12&  22&  \rule{22mm}{2mm}    \\
%15&  18&  \rule{18mm}{2mm}   \\
%18&  15& \rule{15mm}{2mm}    \\
%21&  12& \rule{12mm}{2mm}    \\
%24&  8&  \rule{8mm}{2mm}    \\
%27&  3&\rule{3mm}{2mm}      \\
%30&  2&  \rule{2mm}{2mm}    \\
%33&  1& \rule{1mm}{2mm}    \\
%36&  1& \rule{1mm}{2mm}     \\
%39&  1&  \rule{1mm}{2mm}    \\
%42&  1& \rule{1mm}{2mm}    
%\end{tabular}
%\end{center}}
%\caption{Distribution of team sizes.}\label{fig:ssizes}
%\end{figure}

\subsubsection{Personnel}\label{tion:pers}
Having described the scales implemented in POM2, we now discuss the theoretical
problems that lead to an unimplemented
$personnel$ scale. Boehm and Turner describe project $personnel$ using the Cockburn
mixtures model. In summary, the mixtures model divides programmers into three groups:
\begin{description}
\item[Alpha:] 
~~The most productive/flexible programmers.
\item[Beta:]
~~Able to perform discretionary method steps. 
%such as 
%sizing stories to fit increments, composing patterns, compound 
%refactoring, or complex COTS integration.
\item[Gamma:]
~~Unable/unwilling to follow shared methods
\end{description}
Applying the conventions of the Boehm and Turner {\em personnel scale}, 
lower personnel values indicate {\em more}
alpha programmers on the team. Combining the COCOMO effort multipliers~\cite{boehm00b} and the Cockburn mixtures model, 
%
%We planned to use the COCOMO effort multipliers~\cite{boehm00b}
% to derive a cost
%delta for requirements implementation (the {\em more} alphas on the team, the {\em less}
%the development effort). Strangely, however, when we applied those effort multipliers,
the net effect was nearly zero.\footnote{For a detailed explanation of the ``zero effect'' described, reference http://menzies.us/pdf/09pom2.pdf}
%To see ``zero effect'', consider the COCOMO multipliers relating to programmer
%and analyst capability:
%
%{\scriptsize\begin{center} \begin{tabular}{r|rrrrr} 
%capability &   very low &   low  &      nominal  &     high  &    very high\\\hline
%acap= analyst & 1.42& 1.19& 1.00& 0.85& 0.71 \\    
%pcap= programmer & 1.34& 1.15& 1.00& 0.88& 0.76   
%\end{tabular}\end{center}}
%
%After averaging the rows, 
%combining the lows and the highs, and expressing
%the results as ratios of the very high values, we arrive at
%the following ``slowdown
%factors'' representing the effects of using different kinds of programmers:
%$\{1.6, 1.22, 1\}$ for using \{gamma, beta, alpha\} programmers, respectively.
%With these factors, we had planned to shrink the budget for an iteration according to
%how many less-than-good programmers were assigned to the iteration.
%\begin{equation}\label{eq:b}\small
%budget = budget/(1.6*\gamma + 1.22*\beta + \alpha).
%\end{equation}
%where $\gamma$ and $\beta$ and $\alpha$ are the ratios of different kinds of assigned
%programmers. 
%
%Boehm and  Turner define personnel to range 1 to 5 where each step
%defines ratios of alpha, beta, and gamma programmers. For example, as shown in 
%\fig{person}, if $personnel=5$ then 65\% of the team comprises alpha programmers.
%The last row of that figure demonstrates ``zero effect''. This row shows the
%weighed sum of the ratio of programmer types as calculated by \eq{b}. Note that
%the weighted sums are nearly all the same. That is, the net effect of these personnel
%types on productivity is almost constant.
%
%
For that reason, POM2 ignores Personnel and will explore this issue
in future work.

%\begin{figure}[!h]
%{\scriptsize
%\begin{center}
%\begin{tabular}{rr|rrrrr}
%                            &&\multicolumn{5}{c}{Boehm \& Turner's $size$ scale}\\\cline{3-7}
%    &      programmer types                  & 1 & 2 & 3 & 4 & 5\\\hline
%$\alpha$&\% alpha &45\% &50\% &55\%&60\%&65\%\\
%$\beta$&\% beta &40\% &30\%&20\%&10\%&0\%\\%\
%$\gamma$ &\% gamma  &15\% &20\%&25\%&30\%&35\\\hline
%SUM:& using \eq{b} &118 &119&120 &121&121
%\end{tabular}
%\end{center}}
%\caption{Boehm \& Turner's $personnel$ scale selects for different ratios
%of alpha, beta, and gamma programmers of a project.}\label{fig:person}
%\end{figure}

