The case studies in this chapter use models developed with the
Defect Detection and Prevention (DDP) \cite{corn01,feather03}
application.
To familiarize the reader with this application,
this section will describe DDP.

%%%% Ripped from the bore paper
DDP  is used at NASA's Jet Propulsion Laboratory
to record a group's qualitative
knowledge about design options of future deep-space satellite
missions, their associated risks, and the costs of mitigations that
can reduce those risks.  

In  DDP, a ``design'' is a decision about which set
of mitigations to apply.
One such ``design'' is better than another
when it  costs less, reduces the risks
more, or achieves more requirements than some alternative
design.


The design of DDP reflects
the reality of group decision-making at JPL.  Six to twenty experts are
gathered together for short, intensive knowledge acquisition sessions,
typically three or four half-day sessions.  These sessions must be short
since it is hard to gather together these busy experts for more than a
very short period of time.

In those sessions, the DDP tool supports a graphical interface for the
rapid entry of the assertions. Such rapid entry is essential, lest
using the tool slow up the debate.  Assertions from the experts are
expressed by using an ultra-lightweight decision ontology. The
ontology must be ultra-lightweight since: \\
\bi
{Only brief assertions can be collected in short knowledge acquisition sessions.}
{If the assertions get more elaborate, then experts may be unable to understand 
technical arguments from outside their own field of expertise.}
\item
 Design rationale
research cautions against elaborate notation languages since they can
confuse the users ~\cite{moran96}. 
Successful rational languages (e.g. QOC~\cite{shum94}, DDP,
\etc) all use very simple notations.
\ei
\noindent
Hence, DDP assertions are either: \\
\bi
{{\em Requirements}  (free text) describing 
the objectives \& constraints of the 
mission and its development 
process;}
{{\em Weights} (numbers) of requirements, reflecting their  importance;}
\item	{\em Risks} (free text), i.e. 
 events 
that  damage requirements;
\item	{\em Mitigations} (free text) describing 
actions that can reduce risks;
\item	{\em Costs} (numbers) of mitigations, \ie
the repair costs for correcting risks 
with these mitigations;
\item	{\em Mappings} that connect
requirements, mitigations, and risks; or
\item	{\em Part-of relations,} which structure the 
collections of requirements, risks 
and mitigations.
\ei

This ontology is deliberately quite restrictive
but even in this limited form, it has been useful for structuring and simplifying debates 
between NASA experts.  For example, DDP has been applied to over a dozen 
applications to study advanced technologies such as (1) a computer memory device; (2) 
gyroscope design; (3) software code generation; (4) a low temperature experiment's 
apparatus; (5) an imaging device; (6) circuit board like fabrication; (7) micro 
electromechanical devices; (8) a sun sensor; (9) a motor controller; (10) photonics; and 
(11) interferometry. The DDP sessions have found  cost savings exceeding \$1 million in 
at least two of these studies, and lesser amounts (exceeding \$100,000) in the other 
studies. These meetings have also generated numerous design improvements such as a
savings of power or mass and a shifting of risks from uncertain architecture to better 
understood design. Also, at these meetings, some non-obvious significant risks have been 
identified and mitigated.

Specifically in this thesis, 
we will study three DDP models: \aero, \holo, and \cobns.
Full details of these models
cannot be disclosed
(since they are  proprietary) but some brief notes follow.

\Aero is a ``portfolio'' level-model, where each of the mitigations
is a research program rather than an activity.  The overall 
\aero program was a complex system involving hardware, software and
operators. Further, the risks (challenges) described in \aero span
technical and organizational concerns.

\Cob and \holo are two non-portfolio models.
Both are examples of 
JPL Technology Infusion Maturity
Assessment (TIMA) studies.
Such TIMA sessions involve over a dozen stakeholders whose
total experience spans the domains of systems engineering,
space experiments, avionics, materials, packaging, manufacturing,
testing, experimental design, failure analysis,
quality assurance, mission technologies, MEMS research, and
program management.
A typical TIMA  session might 
call together a set of stakeholders to discuss
 whether or not the
MEMS technology was suitably mature and appropriate for
the intended applications, and to construct a cost-effective
development and testing plan.
Specifically, \holo was a study to identify
risks that would arise in maturing a particular piece of technology
to flight readiness.  

After a model is developed, the major design decision
is what set of mitigations to apply.
Recall that mitigations cost money and affect the total
benefit of a project\footnote{Some mitigations decrease the expected benefit
in some way, \eg increased vibration testing
might damage circuit boards}.
The goal then is to balance cost and benefit.
This will be our main performance metric.
The \holons, \aerons, and \cob models have
99, 83, and $58$ mitigations respectively.
This leads to design spaces with a cardinalities of
$10^{29}, 10^{24},$ and 10$^{17}$ respectively.
Clearly then, the set of possible designs can
not be exhaustively searched, or even enumerated.
%%%%%% ripped from the bore paper

Note that DDP can model interdependency between the objectives
and mitigations.
Recall from \tion{reqAnalysis} that some earlier work on
 prioritizing requirements
(which is just a different terminology for objectives) \cite{karlsson97}
used a simpler matrix based approach that could not capture interdependencies.


%%% Local Variables: 
%%% mode: latex
%%% TeX-master: "../thesis"
%%% End: 
