\section{Problems with PITS}

Based on our
experience with data mining
at other corporations, we assert that PITS is a fairly
standard corporate database:
\bi
\item
It is
useful for storing day-to-day information and generating small-scale
tactical reports (e.g. ``list the bugs we found last Tuesday'').
\item
However, it is 
difficult to  use PITS for high-end business strategic analysis
(e.g.. ``in the past, what methods have proved most cost effective
in finding bugs?'').
\ei

One problem with PITS is that 
there is a lack of consistency in how each
of the projects collected issue data.  In virtually all instances,
the specific configuration of the information captured about an
issue is tailored by the IV\&V project to meet its particular
needs.  This has
created consistency problems when metrics data is pulled across
projects.  While there was a set of required data fields, the
majorities of those fields do not provide information in regards
to the quality of the issue and are not very suitable for comparing
projects.

Hence, It is difficult, to say the least, to generate conclusions from a moving
target like PITS. 
In rare cases, some general conclusions were made~\cite{dabney06,dabney07}.
In all those cases,
the conclusions were reached
after tedious and complex manipulations of the database.
Worse, the extracted data was only accessible
with the help of NASA civil servants- a scarce and expensive
resource. 

NASA is very aware of the problems with PITS and is taking
active steps to improve it.
\bi
\item
There is an on-going effort to implement a mandatory data
set in each IV\&V project database to support IV\&V effectiveness
metrics.  
\item
The original PITS database only stored {\em outputs}
from the IV\&V process; i.e. the issues found. Initially, PITS
did not store IV\&V {\em inputs}; i.e. project features seen {\em before}
IV\&V started. 
To correct this, the IV\&V team devised the
{\em SILAP} model (described below) that captures information
about a project as it {\em arrive} at IV\&V.
\ei
This effort has been in development for about a year and
is currently being executed by several projects.  Progress has been
made, but mostly for the more recent IV\&V tasks.
For example, the SILAP data (used below) is only available for projects
seen since\footnote{when??}.
As to the effectiveness metrics, that is an on-going task and there is
currently insufficient information to draw definitive concluions.




