### ITimingTest

  An Abstract base class that represents the tests that measure the median and time required to execute a well-defined operation. It is the main class used to measure CPU performance.

  Instead of the usual setup, test, and cleanup framework methods, ITimingTest has timingSetup, timingTest, and timingCleanup framework methods. These methods behave somewhat differently from their namesakes. Subclasses place the code to be timed in timingTest. The methods timingSetup and timingCleanup allow subclasses to do any setup and cleanup for the timingTest method; code in these methods is NOT timed.

  The ITimingTest base class enables you to write tests that measure the time taken by a specific operation to complete. The derived class IStartStopTimingTest allows you more control in timing events.

  ITimingTest expects the operation to show some variation in execution time due to multitasking, varying CPU load, and other variable conditions. Consequently, the class takes multiple measurements and reports the median.

  ITimingTest measures the time needed to execute the TimingTest member function. It does this by taking some number of samples, the "sample count". Each sample consists of a call to TimingSetup, some number of calls to TimingTest, and a call to TimingCleanup.

  The calls to TimingTest happen in a tight loop, the "timing loop". Derived classes, or the caller through a RunTest option, can control how many iterations comprise the timing loop by changing the "timing count". The total time taken by the timing loop is divided by the number of iterations to get a single "sample time". The median of all sample times is taken as the estimate of the time to execute the TimingTest member function.

  To create a timed test, do the following:

  1. Derive your test class from ITimingTest.
  2. Override TimingSetup, if necessary.
  3. Override TimingTest.
  4. Override TimingCleanup, if necessary.

  The default settings are as follows:

           kDefaultTimingCount = 500,
           kDefaultSampleCount = 10,
           kDefaultMaximumSampleWindows = 10

           kDefaultAutomaticSamplingTolerance = 0.1
           kDefaultMinimumCorrelation = .4

### ITimingTest - Member Functions and Data by Group

### Constructors & Destructor

Class constructors and destructors.

~ITimingTest

      public:
      virtual ~ITimingTest()

  Destructor.

Supported Platforms

  Windows | OS/2 | AIX
  Yes | Yes | Yes

ITimingTest

  Overload 1
  Standard copy constructor.

      public:
      ITimingTest(const ITimingTest& source)

  source
    ITimingTest to be copied.

Supported Platforms

  Windows | OS/2 | AIX
  Yes | Yes | Yes

  Overload 2

      protected:
      ITimingTest( unsigned long timingCount,
                   unsigned long sampleCount )

  Constructor that explicitly sets timing count and sample count.

  timingCount
    explicit timing value.

    sampleCount - explicit sample count value.

Supported Platforms

  Windows | OS/2 | AIX
  Yes | Yes | Yes

  Overload 3
  Default constructor.

      protected:
      ITimingTest()

  This default constructor sets the timing count and sample count to default values, with no maximum allowed median time.

  Constructs an ITimingTest specifying a timing count, sample count, maximum sample count, minimum correlation, and an automatic sample tolerance. The default values are:

           kDefaultTimingCount = 500,
           kDefaultSampleCount = 10,
           kDefaultMaximumSampleWindows = 10
           kDefaultAutomaticSamplingTolerance = 0.1
           kDefaultMinimumCorrelation = .4
           automatic sampling ON
           no maximum allowed median time

Supported Platforms

  Windows | OS/2 | AIX
  Yes | Yes | Yes

### Advanced Subclass Specialization Methods

Advanced timing member functions. These should only be used by those users with a good understanding of the timing tests.

doUntimedTest

      protected:
      virtual void doUntimedTest()

  Run the timingTest without performing the timing. (e.g., execute only timingSetup, timingTest, and timingCleanup). Useful to debug the timingTest without concern for the timing aspects.

Supported Platforms

  Windows | OS/2 | AIX
  Yes | Yes | Yes

oneEmptySample

      protected:
      virtual double oneEmptySample()

  Runs an empty sample. Useful to determine the amount of time necessary for the overhead of running timingTestt, the function call to an empty test function (e.g., timingTest return 0.0), and timingCleanup.

Return
  theTime The time to run the empty sample

Supported Platforms

  Windows | OS/2 | AIX
  Yes | Yes | Yes

oneSample

      protected:
      virtual double oneSample()

  Determine the time it takes to run one sample of the timingTest.

Return
  theTime The time to run the one sample

Supported Platforms

  Windows | OS/2 | AIX
  Yes | Yes | Yes

samplingErrorMicroseconds

      protected:
      virtual double samplingErrorMicroseconds() const

  Returns the sampling error.

Return
  double The sampling error

Supported Platforms

  Windows | OS/2 | AIX
  Yes | Yes | Yes

### Assignment Operator

Use this operator to assign an ITimingTest class to another.

operator =

      public:
      ITimingTest& operator =(const ITimingTest& rhs)

  Standard assignment operator.

  rhs
    ITimingTest object used for assignment values.

Supported Platforms

  Windows | OS/2 | AIX
  Yes | Yes | Yes

### Automatic Sampling Methods

Member functions used to configure the automatic timing sampling.

automaticSampleTolerance

      public:
      double automaticSampleTolerance() const

  Returns the threshold value for automatic sampling.

Return
  double The current threshold value for automatic sampling

Supported Platforms

  Windows | OS/2 | AIX
  Yes | Yes | Yes

automaticSampling

      public:
      bool automaticSampling() const

  Returns true if automatic sampling is done, false otherwise.

Return
  bool State of automatic sampling setting

Supported Platforms

  Windows | OS/2 | AIX
  Yes | Yes | Yes

maximumSampleWindows

      public:
      unsigned long maximumSampleWindows() const

  Returns the maximum number of sample windows.

Return
  The maximum number of sample windows

Supported Platforms

  Windows | OS/2 | AIX
  Yes | Yes | Yes

minimumSampleCorrelation

      public:
      double minimumSampleCorrelation() const

  Returns the minimum acceptable correlation between samples.

Return
  The minimum acceptable correlation between samples

Supported Platforms

  Windows | OS/2 | AIX
  Yes | Yes | Yes

setAutomaticSampleTolerance

      public:
      void setAutomaticSampleTolerance(double tolerance)

  Sets the threshold value for automatic sampling. Samples will continue to be taken until the error on the measured mean is below this threshold value. For example, if the Tolerance is 0.05, then samples will continue to be taken until the error on the mean is less than or equal to 5 percent.

  tolerance
    The threshold value for automatic sampling

Supported Platforms

  Windows | OS/2 | AIX
  Yes | Yes | Yes

setAutomaticSampling

      public:
      void setAutomaticSampling(bool doAutomaticSampling)

  Controls whether automatic sampling is done.

  doAutomaticSampling
    If true, samples are teken until the error on the measured mean reaches a desired percentage, the tolerance. This allows the operation being timed to "settle" if the first measurements being taken are too high or low for some reason. Samples taken prior to settling are discarded. If false, then the first sampleCount samples are always taken and accepted.

Supported Platforms

  Windows | OS/2 | AIX
  Yes | Yes | Yes

setMaximumSampleWindows

      public:
      void setMaximumSampleWindows(unsigned long maxWindows)

  Sets the maximum number of sample windows that are taken.

  maxWindows
    Number of sample windows to take

Supported Platforms

  Windows | OS/2 | AIX
  Yes | Yes | Yes

setMinimumSampleCorrelation

      public:
      void setMinimumSampleCorrelation(double minSampleCorr)

  Sets the minimum acceptable correlation between samples. If the correlation between samples exceeds this number, a message will be printed and the actual correlation will be posted.

  minSampleCorr
    The minimum correlation: 0 < minSampleCorr < 1

Supported Platforms

  Windows | OS/2 | AIX
  Yes | Yes | Yes

kDefaultAutomaticSamplingTolerance

      public:
      static const double kDefaultAutomaticSamplingTolerance

  The default automatic sampling tolerance. Initial value set to 0.1.

Supported Platforms

  Windows | OS/2 | AIX
  Yes | Yes | Yes

kDefaultMinimumCorrelation

      public:
      static const double kDefaultMinimumCorrelation

  The default minimum correlation. Initial value set to 0.4.

Supported Platforms

  Windows | OS/2 | AIX
  Yes | Yes | Yes

### Timing Methods

Class member functions used to control and perform the timing tests.

calibrate

      public:
      void calibrate()

  Measures the empty method time.

Supported Platforms

  Windows | OS/2 | AIX
  Yes | Yes | Yes

calibratedMeanMicroseconds

      public:
      double calibratedMeanMicroseconds()

  Returns the mean of all sampled times.

Return
  double The mean of all sampled times

Supported Platforms

  Windows | OS/2 | AIX
  Yes | Yes | Yes

calibratedMedianMicroseconds

      public:
      double calibratedMedianMicroseconds()

  Returns the estimated median time for the timingTest method to execute minus the empty method time, in microseconds.

Return
  The estimated median time for the timingTest method

Supported Platforms

  Windows | OS/2 | AIX
  Yes | Yes | Yes

clockNowOverhead

      public:
      double clockNowOverhead() const

  Returns the overhead required to read the clock. This is a measured value used to compute other error tolerances.

Return
  double Overhead required to read the clock

Supported Platforms

  Windows | OS/2 | AIX
  Yes | Yes | Yes

continualCalibration

      public:
      bool continualCalibration() const

  Returns true if calibrate is called for each sample; false otherwise.

Return
  bool True if calibrate is called for each sample; false otherwise

Supported Platforms

  Windows | OS/2 | AIX
  Yes | Yes | Yes

copySamples

      public:
      void copySamples(long& count, double *&) const

  This method sets a pointer to point at an ordered array of each sample value.

  Note: The caller owns its storage for the elements in the array.

  count
    The number of elements of the array pointed to

    double *& - the array pointer to set

Supported Platforms

  Windows | OS/2 | AIX
  Yes | Yes | Yes

correlationMicroseconds

      public:
      double correlationMicroseconds() const

  Returns the correlation between samples.

Return
  double The correlation between samples

Supported Platforms

  Windows | OS/2 | AIX
  Yes | Yes | Yes

correlationTooLarge

      public:
      bool correlationTooLarge() const

  Returns true if the correlation between samples exceeds the minimum correlation specified by the user (using setMinimumSampleCorrelation).

Return
  bool

Supported Platforms

  Windows | OS/2 | AIX
  Yes | Yes | Yes

errorMicroseconds

      public:
      double errorMicroseconds() const

  Returns the 95% confidence interval for the test.

Return
  double The 95% confidence interval for the test

Supported Platforms

  Windows | OS/2 | AIX
  Yes | Yes | Yes

errorTooLarge

      public:
      bool errorTooLarge() const

  Returns true if the relative error from relativeErrorMicroseconds is greater than the tolerance specified by the tester on the command line or by calling setAutomaticSampleTolerance.

Return
  bool

Supported Platforms

  Windows | OS/2 | AIX
  Yes | Yes | Yes

maximumAllowedMedianMicroseconds

      public:
      double maximumAllowedMedianMicroseconds() const

  Returns the time limit for this timing test.

Return
  double The time limit for this timing test.

Supported Platforms

  Windows | OS/2 | AIX
  Yes | Yes | Yes

relativeErrorMicroseconds

      public:
      double relativeErrorMicroseconds() const

  Returns the ratio of the 95% confidence interval for the mean (as returned by errorMicroseconds() ) to the mean itself.

Return
  double The ratio of the 95% confidence interval

Supported Platforms

  Windows | OS/2 | AIX
  Yes | Yes | Yes

sampleCount

      public:
      unsigned long sampleCount() const

  Returns the number of samples that will be taken for timing and calibration.

Return
  long Number of samples that will be taken

Supported Platforms

  Windows | OS/2 | AIX
  Yes | Yes | Yes

samplesDiscarded

      public:
      unsigned long samplesDiscarded() const

  Returns the number of samples discarded in the case that automatic timing is on.

Return
  long The number of samples discarded

Supported Platforms

  Windows | OS/2 | AIX
  Yes | Yes | Yes

samplesTaken

      public:
      unsigned long samplesTaken() const

  Returns the number of samples actually taken for a full test.

Supported Platforms

  Windows | OS/2 | AIX
  Yes | Yes | Yes

setContinualCalibration

      public:
      void setContinualCalibration(bool continuallyCalibrate)

  Controls whether calibrate is called for each sample.

  Continual calibration is off by default under the assumption that the empty method time is relatively constant. If the client or subclass is varying the load, then continual calibration may provide more accurate timing results.

  continuallyCalibrate
    bool; If true, timing is set to be continually calibrated

Supported Platforms

  Windows | OS/2 | AIX
  Yes | Yes | Yes

setMaximumAllowedMedianMicroseconds

      public:
      void setMaximumAllowedMedianMicroseconds(double timeLimit)

  Sets the time limit for this timing test. If the final calibrated median time exceeds the specified time limit, then the test is considered a failure and setSuccess(false) is called. If setMaximumAllowedMedianMicroseconds is never called (the default case) then no time limit will be set, and all runs will be reported to be passing tests.

  timeLimit
    Maximum allowed time

Supported Platforms

  Windows | OS/2 | AIX
  Yes | Yes | Yes

setSampleCount

      public:
      void setSampleCount(unsigned long sampleCount)

  Sets the number of samples that will be taken for timing and calibration. This is also the number of samples that will be taken (in a moving window) for automatic timing.

  sampleCount
    The number of samples to take

Supported Platforms

  Windows | OS/2 | AIX
  Yes | Yes | Yes

setTimingCount

      public:
      void setTimingCount(unsigned long timingCount)

  Sets the number of times the timingTest method will be called in a loop during a single sample.

  Note: This timing count also applies to the calibration method.

  timingCount
    The number of calls to make within a single sample

Supported Platforms

  Windows | OS/2 | AIX
  Yes | Yes | Yes

setTimingOn

      public:
      void setTimingOn(bool timingOn)

  Controls whether timing operations are done for the timingTest method.

  timingOn
    A bool. If true, multiple samples are taken, each sample consisting of a call to timingSetup, many calls to timingTest, and a call to timingCleanup. If false, each run makes a single call to timingSetup, timingTest, and timingCleanup. In addition, echoing to the console is suppressed.

Supported Platforms

  Windows | OS/2 | AIX
  Yes | Yes | Yes

timingCount

      public:
      unsigned long timingCount() const

  Returns the number of times timingTest will be called in a loop during a single sample.

Return
  long Number of times timingTest will be called

Supported Platforms

  Windows | OS/2 | AIX
  Yes | Yes | Yes

timingOn

      public:
      bool timingOn() const

  Returns true if timing operations are done for the timingTest. Otherwise returns false.

Return
  bool

Supported Platforms

  Windows | OS/2 | AIX
  Yes | Yes | Yes

cleanup

      protected:
      virtual void cleanup()

  This is a framework cleanup method. It is called once after the test is run and should contain whatever code is necessary to cleanup after setup. It also must call the parent ITimingTest::cleanup method.

Supported Platforms

  Windows | OS/2 | AIX
  Yes | Yes | Yes

print

      protected:
      virtual void print(ITieredTextBuffer& out)

  Prints information about the object to the given tiered text buffer.

  out
    The buffer where information about the test is printed.

Supported Platforms

  Windows | OS/2 | AIX
  Yes | Yes | Yes

setup

      protected:
      virtual void setup()

  A framework setup method. It is called once before the test is run.

  The test should override this function to execute any setup code that only needs to be called once.

  It is also a good place to parse command line arguments. If this method is overridden it must call the parent ITimingTest::setup method.

Supported Platforms

  Windows | OS/2 | AIX
  Yes | Yes | Yes

test

      protected:
      virtual void test()

  A framework test method. ITimingTest overrides the ITest::test method; subtests of ITimingTest should not override it. ITimingTest::test first calls calibrate if this has not been done before. It then takes sampleCount timing samples if automatic sampling is off, or automatically takes enough samples to achieve the desired tolerance, as returned by automaticSamplingTolerance, up to a maximym of maximymSampleCount samples.

  Note: Do NOT override this method!

Supported Platforms

  Windows | OS/2 | AIX
  Yes | Yes | Yes

timingCleanup

      protected:
      virtual void timingCleanup()

  Performs any required cleanup for n consecutive calls to timingTest, where n is the timing count. Called by the framework once per sample. Subclasses must override timingCleanup if any cleanup is necessary after calls to timingTest.

Supported Platforms

  Windows | OS/2 | AIX
  Yes | Yes | Yes

timingSetup

      protected:
      virtual void timingSetup()

  Performs any required setup for n consecutive calls to timingTest, where n is the timing count. Called by the framework once per sample. Subclasses must override timingSetup if any setup is necessary before calls to timingTest.

Supported Platforms

  Windows | OS/2 | AIX
  Yes | Yes | Yes

timingTest

      protected:
      virtual void timingTest() = 0

  A framework method for timing tests. Subclasses must override timingTest and place the code to be timed in the overridden method. If this code requires any setup and/or cleanup, subclasses should override timingSetup and timingCleanup and place the appropriate code in each.

  For each timing sample, ITimingTest calls timingSetup once, followed by timingCount calls to timingTest, followed by a call to timingCleanup. The loop of calls to timingTest is timed, and this time is divided by timingCount to get the uncalibrated sample time. The median empty time is subtracted from this to get the calibrated sample time.

Supported Platforms

  Windows | OS/2 | AIX
  Yes | Yes | Yes

### Miscellaneous Members

copyInfo

      public:
      virtual void copyInfo()

  Adds information about input syntax and a timing test description. Can be overridden to provide metainformation.

Supported Platforms

  Windows | OS/2 | AIX
  Yes | Yes | Yes

### ITimingTest - Enumerations

      enum  { kDefaultTimingCount=500,
              kDefaultSampleCount=10,
              kDefaultMaximumSampleWindows=10 }

  Default constants used by the timing tests.

Supported Platforms

  Windows | OS/2 | AIX
  Yes | Yes | Yes

### ITimingTest - Inherited Member Functions and Data

#### Inherited Public Functions

  ITest

  -     virtual ~ITest()

  -     void copyInputs(int& argc, char * *& argv) const

  -     virtual EStage exceptionStage() const

  -     virtual bool lookupInfo(const IString& key, IString& info)

  -     ITest& operator =(const ITest& test)

  -     ITieredTextBuffer& outputTextStream()

  -     virtual void run()

  -     virtual unsigned long runCount() const

  -     void setInputs(const int argc, const char * const* argv)

  -     bool success() const

#### Inherited Public Data

  ITest

  -     static const IString& kComponentKey

  -     static const IString & _Import kComponentKey

  -     static const IString & _Import kDescriptionKey

  -     static const IString& kDescriptionKey

  -     static const IString& kInputSyntaxKey

  -     static const IString & _Import kInputSyntaxKey

  -     static const IString& kTargetClassKey

  -     static const IString & _Import kTargetClassKey

  -     static const IString& kTestNameKey

  -     static const IString & _Import kTestNameKey

  -     static const IString & _Import kTestTypeKey

  -     static const IString& kTestTypeKey

  -     static const IString& kTestVersionKey

  -     static const IString & _Import kTestVersionKey

#### Inherited Protected Functions

  ITest

  -     virtual void addInfo(const IString&, const IString&)

  -     virtual bool isReset() const

  -     virtual bool isSuccessUndecided() const

  -     ITest()

  -     ITest(const ITest& test)

  -     virtual void print(ITieredTextBuffer&)

  -     virtual void reset()

  -     virtual void setStickySuccess(bool success)

  -     virtual void setSuccess(bool success)

  -     virtual void test() = 0

#### Inherited Protected Data

  - None