INTRODUCTION_TO_MEASUREMENT_UNCERTAINTY

INTRODUCTION TO MEASUREMENT UNCERTAINTY Steve Clyens, CET Gas Specialist, Ontario Region CGA Gas Measurement School Edmonton, June 2006 REFERENCES 1. Guide to the Expression of Uncertainty in Measurement. BIPM, IEC,IFCC, ISO, IUPAC, IUPAP, OIML, 1995. 2. International Vocabulary of Basic and General Terms in Metrology (VIM). International Organization for Standardization, 1993. 3. Measurement Good Practice Guide No. 11 (Issue 2), A Beginner's Guide to Uncertainty of Measurement, Stephanie Bell 4. The Uncertainty of Measurements: Physical and Chemical Metrology: Impact and Analysis, Kimothi, S. K., 2002 DISCLAIMER This presentation does not represent or communicate Measurement Canada policy relating to the determination, evaluation, expression and/or use of measurement uncertainty in trade measurement and related applications. This presentation is intended for those people who know little or nothing about Measurement Uncertainty but are involved in measurement in some form or other as part of their daily activities. INTRODUCTION Many technical papers and guides on this subject are loaded with advanced statistical terminology and concepts, complex-looking mathematical equations, not to mention the language of differential calculus used by scientists and engineers. The subject of Measurement Uncertainty can be somewhat intimidating to many people. This presentation WILL NOT teach you how to perform measurement uncertainty evaluations and calculations. The main objective is to get you thinking more about the concept of measurement uncertainty and why it is important and necessary to include it in your own particular measurement activities. HOW LONG IS THE PAPER CLIP? ruler may have inherent error, was it calibrated? ruler has limited resolution, limited precision (# significant digits) length of ruler and/or the paper clip may change with a change in temperature paper clip may not be lined up parallel with the ruler bottom of paper clip may not be lined up with zero of the ruler different people may report different results (parallax error, etc.) and other things we haven't considered or don't know about These are all potential sources of error that, if not fully accounted for and/or corrected, can cause doubt or uncertainty about the quality of the measurement result. If we want a more accurate and precise result, we need to consider certain factors and quantities that could influence the outcome of the measurement. If all we want is a ball park figure we can look at the scale and take our best guess. THE PERFECT MEASUREMENT It is these imperfections that give rise to error in the result of a measurement. Making a perfect measurement would require: perfect measurement equipment, perfect measurement processes, perfect conditions for measurement, perfect people making the measurement Of course, none of these exist. All have imperfections to some degree or other. We can report that a measurement result is 100% accurate but we can never be 100% certain about such a result. In fact, we can never be 100% certain about any measurement result. This would require a perfect measurement. Unfortunately, there is no such thing. MEASUREMENT ERROR Error (of measurement) = Systematic Error + Random Error Error (of measurement) = Measurement Result* - True Value Measurand Particular quantity subject to measurement. The length of the paperclip at 70 oF in centimeters and expressed to 2 decimal places. * May be based on a single measurement or the mean of a series of repeat measurements. Traditionally, error of measurement is viewed as having a systematic component and a random component. Error (of measurement) Result of a measurement minus the true value of the measurand. SYSTEMATIC ERROR remains constant over a series of repeat measurements, or varies in a predictable way over a series of repeat measurements, or may occur randomly over repeat measurements, samples, time etc. The recognized systematic effect could be a positive or negative offset of the measurement result from the true value that: readily identifiable (and so correctable), or very difficult if not impossible to identify Systematic errors may be: Systematic errors arise from recognized (systematic) effects of influence factors or quantities on a measurement result. SYSTEMATIC ERROR bias in measuring equipment, ie. zero offset error, span error, errors in calibration, hysteresis incorrect measurement methods, ie. selection and use, temperature effects on dimensional measurements human factors, ie. parallax error, incorrect assumptions of linear response use of measurement equipment under conditions differing from calibration conditions time, ie. drift of metrological characteristics such as equipment bias Some of the influence factors or quantities that can lead to systematic error are: The greater the magnitude of the overall systematic error, the poorer the accuracy of the measurement result unless the systematic error is corrected. RANDOM ERROR Random errors arise from the (random) effects of unpredictable variations of influence factors or quantities on a measurement result. When a series of repeat measurements of the same measurand are made, random errors: If all systematic error could be accounted for and eliminated, the expected value of the mean of a very large (approaching infinite) number of normally distributed random errors would be zero. are characterized by random variation in the individual measurement results will have varying magnitude and sign (+ or -) will tend to be normally distributed about the mean (average) of the individual measurements RANDOM ERROR unpredictable variations in environmental and/or measurement conditions, ie. temperature, pressure, humidity, vibration, etc. inherent instability of measuring equipment, ie. repeatability & reproducibility differences between persons making measurements, ie. degree of expertise and performance instability of the "thing" being measured Some of the influence factors or quantities that can lead to random error are: The greater the magnitude of the random errors: the greater the degree of variation amongst repeat measurements the wider the dispersion of individual measurement results about their mean QUANTIFYING VARIATION - Standard Deviation where, x i is the result of the i'th measurement Sample (or experimental) standard deviation is used to quantify the extent of variation in a sample of repeat measurements of the same measurand. is the sample mean (average) n is the sample size Standard deviation has the same units of measure as the measurement result. In metrology, standard deviation is used as an indication of: repeatability - closeness of agreement between results of measurements of the same measurand carried out under same conditions of measurement reproducibility - closeness of agreement between results of measurements of the same measurand carried out under changed conditions of measurement standard uncertainty - established using statistical or non-statistical methods REDUCING MEASUREMENT ERROR removing known systematic error from the measurement process, making calibration adjustments to the measuring device, applying corrections from calibration certificates to measurement results increasing the number of repeat measurements (sample size) and taking the mean as the measurement result (not always practical or cost effective) identifying the measurement's random influence factors or quantities and attempting to minimize their variation during the measurement process Random error cannot be completely eliminated but can be reduced by: Systematic error cannot be completely eliminated but can be reduced by: BEST ESTIMATE OF THE MEASURAND imperfect determination of corrections for systematic effects, and incomplete knowledge of all existing systematic effects, and random variations of repeat measurements There will always be some doubt or uncertainty about the goodness or quality of an estimate of the value of the measurand; that is, a measurement result. Even after reducing recognized systematic and random errors in a measurement process, any subsequent measurement result is still only an estimate of the value of the measurand because of: The best estimate of the measurand is considered to be the mean of a sample of repeat measurements. The larger the sample size the better the estimate. UNCERTAINTY OF MEASUREMENT Parameter, associated with the result of a measurement, that characterizes the dispersion of the values that could reasonably be attributed to the measurand. Measurement Result (MR) ± Uncertainty (U) This provides for an interval estimate that allows us to state, with a certain level of confidence, that the measurand lies somewhere between: The ISO definition of Measurement Uncertainty is: When a measurement result is reported with no indication of its uncertainty, ie. as a point estimate of the measurand, we have zero confidence in its goodness or quality. Uncertainty of measurement is due to the unrecognized and/or uncorrected errors in the measurement result. (MR - U) and (MR + U) WHY IS IT IMPORTANT? Any decision made on the basis of measurement results requires some indication of the quality of those results. Decision Making Without such an indication, it is impossible to judge the fitness of the measurement results as a basis for making decisions relating to health, safety, commerce or scientific research. Most, if not all, of our companies or agencies have quality programs, quality controls, quality manuals and quality procedures all with the aim of producing quality products and/or quality test results. Uncertainty is used as a quantitative indication of the goodness or quality of a measurement result. The measurements we make should be quality measurements. Quality WHY IS IT IMPORTANT? Fit For Purpose An evaluation of the overall uncertainty of a measurement process (methods, equipment, procedures and operators) provides a means of establishing that the process will allow valid measurements and results to be obtained. Tolerances and Specifications The uncertainty of a measurement result should be taken into account when it is compared against performance specifications or tolerance limits. The smaller the uncertainty, the closer the measurement result can approach the tolerance limit without being rejected. An estimation of measurement uncertainty is required for testing and calibration laboratories with ISO/IEC 17025 accreditation. Laboratory Accreditation In calibration, uncertainties have to be stated in the calibration certificate as they are required by the user of the calibrated equipment. WHY IS IT IMPORTANT? Uncertainty allows meaningful comparison of results from different facilities or within a facility or with reference values given in specifications or standards. Comparisons 66.80 oF 67.00 oF ± 0.15 oF ± 0.19 oF An uncertainty budget (a list of the uncertainty components and their respective uncertainty contributions) can be used to determine which sources of uncertainty contribute most so that financial and other resources can be applied wisely. Process Planning and Improvement A consideration of individual uncertainty components also indicates aspects of the measurement process to which attention should be directed to improve procedures. How do these values compare? How about now? UNIFORM METHOD OF DETERMINATION Ideally, the method for evaluating and expressing uncertainty should be readily implemented, easily understood and generally accepted internationally. In 1993 a guide, prepared by a joint working group of experts nominated by the BIPM, IEC, ISO and OIML organizations, was published. The GUM recommends that all recognized systematic error be corrected first. Guide to the Expression of Uncertainty in Measurement (GUM), 1993 The GUM establishes general rules for evaluating and expressing uncertainty in measurement and is analytical in nature. The need for such a method was recognized by the Comite International des Poids et Measures (CIPM), the world's highest authority in metrology, in 1977. 1. Specify or define the measurand THE GUM METHOD The GUM method of uncertainty evaluation consists of eight generalized steps. What follows is a very basic description of each of these steps. 2. Express mathematically the relation between the measurand and any input quantities State the quantity subject to measurement and, where applicable, the conditions of measurement. Quantity of interest is either: a. measured directly, ie. length of paperclip, or b. the output, y, of an equation composed of a number of input quantities, x n. y = f (x 1, x 2, x 3, ..... x n) Each input value, x i, is an estimate of a measurand with its own uncertainty. THE GUM METHOD - Sources of Uncertainty a. incomplete definition of the measurand; (significant unaccounted for influences) b. imperfect realization of definition of measurand; (limitations of test conditions) c. nonrepresentative sampling; (conditions of use differ from those of calibration) d. inadequate knowledge of the effects of environmental conditions on the measurement or imperfect measurement of environmental conditions; e. personal bias; (influence introduced by person(s) making measurements) f. finite resolution; (reading a scale, quantity per A/D count) g. inexact values of reference standards and materials (inherited uncertainty); h. inexact values of constants and other parameters obtained from external sources (value or constants determined empirically); i. estimates and assumptions used in the measurement process (linear response, no hysteresis); j. variations in repeat measurements under apparently identical conditions (random error). 3. Identify all sources of uncertainty Some potential sources of uncertainty in a measurement result, include: 4. Evaluate the input quantities and quantify the standard uncertainty of each THE GUM METHOD - Type A and B Evaluations The GUM does not use a systematic and random uncertainty approach to evaluation. Instead, individual uncertainty components are evaluated using one of two methods: Once evaluated, an uncertainty component's quantified uncertainy, when expressed as one standard deviation, is called its standard uncertainty. Type A - Uncertainty is evaluated based on the statistical distribution of the results of a series of measurements and can be characterized by experimental standard deviations. Type B - Uncertainty is evaluated from assumed probability distributions based on: past experience of the measurements and/or behaviour of equipment data provided in calibration and other certificates, manufacturer’s specifications, uncertainties assigned to reference data taken from handbooks 5. Identify the covariances of correlated input quantities THE GUM METHOD - Combined Uncertainty 6. Calculate Combined Uncertainty (for each input quantity and the final output) Does any input quantity influence the behaviour of any other input quantity? Individual uncertainty components, expressed as standard uncertainties, are combined (by Root Sum Square) to give the combined standard uncertainty. The sensitivity of the final output to each input quantity (change in output caused by a given change in an input) is accounted for in this step thru the application of sensitivity coefficients. These coefficients can be established using partial derivatives (calculus ) or numerical methods (algebra). Any correlation between input quantities and the resulting covariances (+ or -) are also accounted for in this step. If yes, they are said to be correlated and their covariance must be quantified. 7. Calculate the Expanded Uncertainty THE GUM METHOD - Expanded Uncertainty For degrees of freedom >29, Level of Confidence Coverage Factor, k 68.26% 1 95.46% 2 99.73% 3 Expanded Uncertainty = Coverage Factor (k) x Combined Standard Uncertainty A coverage factor, usually k=2, is applied to the final combined standard uncertainty to give the expanded uncertainty. The expanded uncertainty, when applied to the measurement result, provides for an interval estimate of the measurand at the desired level of confidence (usually approx. 95%). THE GUM METHOD - Reporting Result and Its Uncertainty This device's corrected indication has an expanded uncertainty of ±0.15 oC based on a standard uncertainty multiplied by a coverage factor k = 2, providing a level of confidence of approximately 95%. This uncertainty was estimated according to the ISO Guide to the Expression of Uncertainty in Measurement (GUM). Corrected Indication 15.00 oC with associated Uncertainty ± 0.15 oC or ± 0.3% Full Scale This means approximately 95 times out of 100, the measurand will be within: A thermometer's calibration certificate might include the following statement of uncertainty: 8. Report the measurement result with the estimated expanded uncertainty 14.85 and 15.15 oC Let's see how this might be applied. CONFIDENCE IN OUR MEASUREMENT RESULTS How do we establish some degree of confidence in our estimate of the value of the measurand (measurement result)? 1. Take all reasonable steps to reduce error (systematic and random): (a) in the measurement equipment (b) in the measurement process 2. Quantify the total uncertainty associated with the estimate (measurement result) produced by the measurement equipment / process. 3. Establish an interval estimate of the value of the measurand at a pre-determined level of confidence using the measurement result and the quantified expanded uncertainty. QUESTIONS ?? Steve Clyens, CET Gas Specialist, Ontario Region Measurement Canada 232 Yorktech Drive Markham, ON L6G 1A6 905-943-8737 (office) 416-938-5712 (cell) clyens.steve@ic.gc.ca