Add more statistics support to ExecutionTimeComparator validator.
This patch adds optional enhancements to the validator, which
include:
- Use of fixed or variable sampling size (query repetitions).
- Evaluation of result stability by use of statistical
properties such as standard deviations.
- Detailed statistics for each query if --debug is enabled.
- All of the above is possible only if Statistics::Descriptive
perl module is available and a sampling size (MIN_SAMPLES
and MAX_SAMPLES) is set.
Default behavior is still to simply compare original execution
times from each server, without further sampling or statistical
considerations.
Note:
- Constant QUERY_REPEATS is replaced by the combination of
MIN_SAMPLES and MAX_SAMPLES.
If Statistics::Descriptive is not available, a query will
be repeated MAX_SAMPLES times if both constants are larger
than 0.
If Statistics::Descriptive is available, a query will be
repeated at least MIN_SAMPLES times and at most MAX_SAMPLES
times if both constants are larger than 0.
If either MIN_SAMPLES or MAX_SAMPLES is 0, only the
original result will be used for comparison.
- File output format is changed in certain cases, i.e. if
Statistics::Descriptive is available and a sample size is
set.