\section{Conclusion}

In this paper we evaluate and analyze the performance of the KEYS and
KEYS2 model optimization algorithms using artificially-grown models.
We then identify and address their deficiencies and inefficiencies and
attempt to speed up their runtimes so optimal solutions for
incrementally larger models can be obtained near real-time.

Our analysis identifies the execution of the model as the main
bottleneck in the efficiency of the algorithms. KEYS spends 86\% of
the runtime, running model simulations.  Our modified versions attempt
to reduce the number of times the model is called without
significantly affecting the ability to optimize the search for a
solution. The technique employed to achieve that reduction consists of
maintaining valid model instances between eras and generating and
executing new instances only to replace the invalid ones.

The results show that this fairly simple technique significantly
reduces the runtimes at a slight decrease in cost minimization. The
time reduction is particulaly large (more than 50\%) for the KEYS
algorithm.

\section{Future Work}

We believe that time savings benefit of our improvemnt technique
increases as the models become larger. As part of the future work we
would like to perform of the relevant experimentation with even bigger
models.  

Another aspect worth exploring is multi-threading. In theory this
technique is promising for large to huge models due to the fact that KEYS
bottleneck lies in calling the model scoring function millions of
times but these calls can be done in paralel at every era. Early
experimentation shows a slightly better runtimes when using available
cores in a multi-processor, multi-core server environment. Cloud
computing is the logical next step for our future experimentation.

