|
Data analysis
software verification Function of testCheck the computation of probability density functions for three types of signal. Signal DetailsThe files required for this benchmark are: EIS_DATA_C1_long_V.txt (Sine
Wave as used in benchmark 3) Analysis parameters to be used The data are supplied in ASCII form and each file represents one signal. When loaded into an analysis package the data should be set as follows:
Note that normalising the signal to zero mean and unity standard deviation requires subtracting the estimated mean value and then dividing by the estimated standard deviation. The analysis ranges are then effectively in terms of a number of standard deviations. This is carried out by finding the mean value of the signal, xbar say, and its standard deviation, SD say. The required signal y(t) is given by Reference resultsThe reference results quoted below have been computed using DATS for Windows Version 6.0.23 Basic StatisticsThe basic statistics for the three sections of data to be analysed are:
Probability Density results.Results of the probability density analyses are detailed in the table below. It is recommended that spot checks on the probability results be carried out at 4 spot point in each of the results as shown in the table. Three reference data sets of the probability results are included named as shown in the table below. These datasets can be used as overlays to check the results computed.
NOTES: The points chosen for comparison have no particular significance having been selected at random. The curves shown in dotted lines are the theoretical curve for a sine wave and a Gaussian distribution for the two random data sets. A useful check on the probability densities is to integrate the distributions and check that the final value is 1.0. The reference results datasets names are of the form EIS_BM_6 _RESULT_{qualifier}.TXT where qualifier is as given above. All of these data sets have the same base and increment values which are: base = –3.960396 and increment = 0.079208 The scatter in results will obviously increase as the probability decreases. If we have a sine wave of amplitude A then the width of the probability density curve is ±A. As we have normalised to unit standard deviation the amplitude of the normalised sine wave is ±Ö2. Also the probability at the zero amplitude is 1/pA. With the normalised data this is 0.22508. The equation for the theoretical probability density functions are shown below: Sine wave
Where A is the amplitude of the sine wave. Gaussian
where s is the standard deviation and
Figure 1 Probability Density of Sine Wave (Data C1_long)
Figure 2 Probability of Pseudo Random Signal (Data C14_long)
Figure 3 Probability of Real World Data (Data D2_4) Acceptable tolerance in computed resultsThe tolerance on the results of the specified analyses should as follows: Signal StatisticsNumber of samples processed should be exact. Other results should be accurate to 4 significant figures. (This is considered to be acceptable for normal analysis purposes) Probability densitiesTypical benchmark 6 erroneous results and their causesIf good agreement is not achieved for the basic statistics then the tests specified in groups A-C should be run to check that good agreement for basic statistics for deterministic signals can be obtained to ensure that the computation of the statistics is correct. If results from groups A-C are correct then the length of data in the selected section and its position within the original data should be checked. Overlaying the supplied reference probability densities on the locally derived spectra best checks computed probability densities. Possible causes of differences are: Differences in methods employed for normalizing to zero mean and unity standard deviation. Difference in setting the amplitude levels for the analysis. Selecting end of bin rather than centre Results return documents.In order to aid the working party in the development of benchmark data users are invited to submit the results obtained when employing the tests. These submissions will be assessed by the working party and summaries may be used in presentations and publications of the working party’s activities. If you wish to submit results please use the document contained in Appendix A. Appendix ABenchmark 6. Analysis results return. Name: Organisation: Date:Address: Email: Tel: Fax: Analysis Package used: Version: Computer Hardware and operating system details Processor: Speed: Memory size: Mb: Operating System: Version: Results for basic statistics
Probability Density ResultsComparison of reference points
Please detail any differences found and supply the data in ASCII form in datasets named {Identifier}_BM_6_RESULT_{signal}.TXT, where Identifier your name and signal is the appropriate signal name. Comments:
Suggestions for further benchmark tests:
Return results to: Dr. B. J .May 1, Westhall Road, Mickleover, Derby. DE3 0PA. UK
Email: verification@e-i-s.org.uk Tel/Fax: +44 (01332) 737034 NOTES: 1.
Numeric
precision All results should be quoted to 4 significant figures. 2. Trailing zeros Trailing zeros may be suppressed. 3.
Use of
results The
working party may make use of the information received in Analysis results
returns for the purpose of promoting the work of the party and for informing
the Data Analysis community of the variation of results obtained. When
results are used for presentations no reference will be made to either the
source or analysis package employed. |
Copyright(c) 2003 Engineering Integrity Society. All rights reserved.
verification@e-i-s.org.uk