In this chapter, we will present the concept of ``better, faster, cheaper'' (BFC) and its history, followed by a study conducted on BFC. 

\section{History of BFC}\label{sec:bfchist}
``Faster, Better, Cheaper'' or ``Better, Faster, Cheaper'' (BFC)  was a philosophy in designing and implementing projects that was pushed in NASA by Daniel Goldin, who served as the NASA Administrator from April $1^{st}$, 1992 to November $17^{th}$, 2001. It was mainly pushed to reduce the expenditure of NASA, and was in-line with the direction that the Clinton administration's approach of doing more for less. While starting out on the right foot with such successes as Pathfinder, later failures caused a wealth of criticism of these policies. While BFC hasn't been abandoned, its application has since then evolved to rectify its implementation shortcomings.

\input{figures/misc/bfc}

\subsection{The Start and Successes of BFC}\label{sub:bfcsuc}
The view of pre-Goldin BFC was that at one time in a project, only two of the aspects of BFC could be achieved. This was expressed by the common phrase ``Better, Faster, Cheaper: pick any two'', illustrated in \fig{bfc}. Goldin believed that it was in fact possible to have all three aspects, and so initiated new policies within NASA based on his views.

The main approach to implementing BFC within NASA was to downsize projects and reduce their cost and complexity, concentrating on producing missions in volume. Reducing funding naturally meant that less verification and testing was possible within budget and schedule constraints. The reasoning behind this however was to be able to produce a larger volume of unmanned missions, which would counteract the expected higher rate of mission failure. This would, optimally, yield more successful missions as well as more scientific data produced by these projects. Another focus in this policy was allowing teams to take acceptable risks in projects to allow for cost reduction, and possibly using new technology that could reduce cost while possibly providing more capabilities. This was accompanied by the the new view that was being pushed at NASA by Goldin that ``it's ok to fail'' \cite{spear00(2)}, which was rather misunderstood. This new policy was meant to eliminate huge budget missions of the past, that upon possible failure would yield large losses. Project cost used to routinely exceed the \$1 billion mark, while the first BFC project, the Mars Pathfinder, was completed for a fraction of the cost, netting at about \$270 million \cite{david00}.

Some within NASA, such as 30 year veteran Frank Hoban, supported these policies~\cite{david00}. Some viewed these new policies as a necessary break from traditional policies that were very risk averse. The additional cost reduction, accompanied by the additional risk, was to allow for a path to cheap and commercial space flight. Even given the reduced funding, the Mars Pathfinder mission, along with other first generation BFC missions, were successes. This fueled enthusiasm to apply BFC across all of NASA and further reduce spending per mission, as well as reduce NASA expenditure by reducing the workforce by a third. BFC was extended to be applied on manned space missions as well, where funding was also reduced. Coming into a space shuttle program that was starting to age and in need of updates, the new policies imposed cuts in funding from 48\% of the NASA budget to 38\% \cite{key03}, further straining that program. Further more, a single prime contractor (Lockhead Martin) was used for missions in another bid to reduce cost and managerial complexity~\cite{young00, NASAwatch}.

This produced opposition within NASA, where traditionally issues pertaining to the shuttle were designated LOVC (Loss of Vehicle and Crew) and given priority over all other issues, including cost. However the cost cuts and layoffs that ensued were too much for teams, and caused a blow to morale. In addition there was a progressive loss of veteran scientists, engineers and managers who had accepted offers for early retirement that were extended to them \cite{key03}. 

Despite this, additional projects were on the way in the form of the Mars Climate Orbiter and the Mars Polar Lander. These two projects were more aggressive implementations of BFC, especially when it came to the Faster-Cheaper part of those policies. Costs of the Orbiter and the Lander were brought down to \$125 million and \$165 million respectively~\cite{tuite07}. This was much lower that the previous Pathfinder mission, which itself cost slightly less than \$300 million. The success of these missions would've furthered the BFC mantra within NASA and JPL, and would've been seen as breaking new ground in terms of mission completion with the kind of staff and cost reductions they had compared to previous missions, even the Pathfinder~\cite{hardin99}.

\subsection{The Legacy of BFC}\label{sub:bfcleg}
Given its early success in terms of mission delivery, BFC started being more aggressively applied to missions in NASA. One product of this were the above mentioned Mars Climate Orbiter and Polar Lander. Each cost about 40\% less than the previous Pathfinder mission, which is extraordinary given that Pathfinder had been touted as a money saver by NASA when compared to previous missions like Viking, which cost about \$935 million in 1974 Dollars (equivalent to \$3.5 billion in 1997 dollars). Both of these missions however failed. Using a single contractor had weakened quality assurance and resulted in flaws that caused the loss of these two Mars missions. These flaws had been software flaws that could have easily been rectified if they had been discovered on the ground. One of these flaws was a failure to convert from imperial to metric units, causing the loss of the Climate Orbiter~\cite{nasa99}. The Mars Program Independent Assessment Team Report~\cite{young00} found that these missions were understaffed, underfunded by at least 30\%, and too tightly scheduled.

Elsewhere, across the Atlantic in the UK, another mars mission to deliver a lander, designated the Beagle 2, was under way. This mission way also developed cheaply, applying the same concepts in design and implementation that NASA was at the time using. The lander however was declared lost after not establishing contact after separation from the mars express vehicle~\cite{beagle2}.

One other failure that BFC was blamed for was the Columbia Shuttle disaster in 2003. This was post-Goldin, at a point where NASA had realized the excessive cost cutting and staff reducing policies needed to be changed. After that disaster, critics quickly pointed the finger to these missions being under funded due to BFC. There were many calls, especially politically, for throwing BFC ``in the waste basket"~\cite{cowig03}. One report that particularly criticized BFC, and Goldin's policies in implementing it, is the International Federation of Professional and Technical Engineers' (IFPTE) report on the Effectiveness of NASA's Workforce \& Contractor Policies~\cite{ifpte03}, which stated the following:

\begin{quote}``Using FBC as a way to contract out services and move more of NASA's resources into the private sector, Goldin eliminated much of the civil service infrastructure that monitored and held technical knowledge of the service and products contractors provided and oversaw NASA's safe and successful operation. Critics of FBC always doubted NASA's ability to fulfill FBC without sacrificing either the `faster', the `better', or the `cheaper'. Concerns became widespread after the highly publicized Mars missions failed in 1999. Further concerns arose as NASA's workforce reductions and increased contractor workforce, jeopardized the safety of space shuttle operations. Enough evidence existed in failed missions, close calls, and government reports that suggested the trade offs of FBC were inexperienced and reduced workforce capability; increased safety risks; and minor oversights that resulted in lost spacecraft.''\end{quote}

Despite all the criticism of these policies, it should be noted that BFC was successful/partially successful in 136 of the total of 146 missions launched during the period that Goldin was administrator. This would be called an overall success if it hadn't been for the largely publicized failure, as it had been suggested that NASA could've used better PR~\cite{turner03}. Also, the major failures in missions attributed to the BFC policies seemed to be due to communication and managerial failures, rather than the BFC policies directly. This can be seen as a result of how BFC was implemented, where staff reductions, as well as the aforementioned loss in veteran engineers and managers to retirement, causing experienced managerial staff to be stretched too thin given tight scheduling~\cite{young00}. This forced projects to use inexperienced managers which caused management mix ups and human error.

It seamed that BFC was more of a front for faster and cheaper, as the extreme cost cutting and tight scheduling seemed to eat into the quality, i.e. the better, of the missions. This implies that BFC was not the issue in itself, but rather its implementation. In support of this, Tony Spear, a JPL veteran engineer from 1962 to 1998, testified~\cite{spear00} to the possible effectiveness of BFC, and emphasized the importance of a cohesive and communicative team for it to work. He also mentions a fixation that happened in NASA on cost, causing cost cuts that were too much for $2^{nd}$ generation BFC projects. However, far from recommending throwing BFC out the window, he recommended a more focused way of implementing it, concentrating on three aspects: people, technology, and methods~\cite{spear00,spear00(2)}. This would entail more staffing and building and retaining talent, taking advantage of advancements in technology such as the Internet, and advancing methods used in project development and verification. One of the recommended methods include peer review, as well as also developing multi-mission technology. This would make the projects and the process by which these projects are developed more mature. Spear elaborated on this in the NASA FBC Task Final Report~\cite{spear00(2)}.

A more detailed time line of BFC and its woes within NASA can be seen online~\cite{NASAwatch}.

\section{Analysis of BFC: Pick Any Two?}\label{sec:bfcpa2}
Having presented the history of BFC at NASA, our purpose behind this section is to attempt to investigate the viability of achieving a project that has high reliability and quality, was cheap to develop and was quick to get to market. The general consensus before the Goldin era at NASA was that you had to have a compromise somewhere. Hence the common phrase "Better, Faster, Cheaper: pick any two". 

To apply study this, we needed to apply the different scenarios of BFC within STAR. These include BFC, FC, BF and BC, where the latter three options are the ``pick any two'' variants. For this study, BFC was interpreted as an indication of what models the weighting is on in~\eq{energy}. More specifically, it was interpreted as the following:
\bi
\item
BFC: maintain default weights in the fitness function: $\alpha$ = 1, $\beta$ = 1, $\gamma$ = 1, $\delta$ = 1 and $relydefect$ = 1.8.
\item
FC: Apply higher weight on the effort and development time aspects in the fitness function, and eliminate all the others: $\alpha$ = 2, $\beta$ = 0, $\gamma$ = 0, $\delta$ = 2 and $relydefect$ = 0.
\item
BF: Apply higher weight on the defects and development time aspects in the fitness function, and eliminate all the others: $\alpha$ = 0, $\beta$ = 2, $\gamma$ = 0, $\delta$ = 2 and $relydefect$ = 1.8.
\item
BC: Apply higher weight on the defects and effort aspects in the fitness function, and eliminate all the others: $\alpha$ = 2, $\beta$ = 2, $\gamma$ = 0, $\delta$ = 0 and $relydefect$ = 1.8.
\ei

\input{figures/bfc/winloss-bfc}

For each of the NASA projects presented in \tion{nasaprojpol}, STAR was run ten times for each of the variant weighting schemes, all while employing default analysis. The results from these runs were post processed to produced the quartile charts presented in \fig{bfceffort}, \fig{bfctime}, and \fig{bfcdefects}. Note that these charts only show the middle two quartiles in an effort to keep the charts legible, and in order to avoid reporting outliers. These quartiles have also been normalized to further emphasize the differences between the different schemes. The rank field in those charts indicates the rank given to the particular weighting scheme when statistically compared to the other schemes using the Mann-Whitney unit test~\cite{mann47}. Threats weren't included in this study due to results that had it zeroed out accross the board, with occational outliers which produced messy, inconclusive results, a pattern seen several times throughout this thesis.

Looking at \fig{bfceffort}, the figure describing effort results, we observe that BFC seems to perform mid-pack compared to the rest of the weighting schemes, while FC ranks first in all but one project (OSP2). For OSP2, BC ranks first among the weightings, with FC ranking last. Note that in all the other projects, BC actually ranked last. However, those rankings can't be considered a clear loss for BC since the range of effort estimates produced by BC overlaps fairly well with the estimates from the other weighting schemes. FC's loss in OSP2 however is a more clear loss.

Moving on to \fig{bfctime}, the figure describing development time results, we can again see how BFC performs mid-pack compared to the other schemes. This time around, FC wins across the board, with BF trailing close behind, and BC rounding out the pack. The more decisive wins for FC take place with the more constrained projects, OSP and OSP2, as with the more decisive losses of BC.

Finally, in \fig{bfcdefects}, the figure describing defects results, we observe that BFC does very well with OSP and OSP2, performing on par with BC and BF for these projects. On the other hand, FC does very poorly for these projects, clearly driving defects up due to a lack of emphasis on defects. The performance in the other two projects is closer, however note that BFC does slightly worse than everything else with \textit{ground}.

From the above observations, we present the following analysis relating to BFC:
\be
\item
BFC is a ``jack of all trades, master of none''. It works well in most instances, and provides a compromise solution in balancing out model estimates. This supports the view that you can't fully achieve BFC with out some compromises, and that you're better off ``picking any two'' aspects to concentrated if there is a focused end result is needed.
\item
FC speeds  projects, while having the side effect of drastically increasing defects for more defined and constrained projects. This seems to suggest against rushing well defined and constrained projects out the door, and concentrating on on keeping some ``better'' aspect in consideration to avoid the defect spike, especially for mission critical projects.
\item
Finally that BC, while driving down defects, also drives up development time, especially for more constrained projects. This also applies for effort for all projects other than the most constrained (OSP2)
\ee

So as a final conclusion to this section, this study seems to support ``better, faster, cheaper: pick any two'', providing an empirical study that opposes the BFC policies applied throughout NASA in the 1990's to the early 2000's.
