Differences Between Accuracy, Error, Tolerance, and Uncertainty in a Calibration Results

with 10 Comments

Are you familiar with the differences and relationships between Accuracy, Error, Tolerance, and Uncertainty in calibration Results?

There are so many terms that we always use or read during our measurement process. Knowledge in these terms is the key to proper understanding and execution of your measurement results. 

I have recently received the below question to most of the comments in my posts, And therefore, it is worth a topic to discuss.

Q1. I am unable to understand the relation between Accuracy, Error, and uncertainty. Can you tell me with example?

Q2. What is the difference between Tolerance and Uncertainty?

One way to easily learn, implement the results in a calibration certificate, and to properly understand most of the calibration procedure is to understand the measurement terms in it.

In my last article, I have presented the difference between Calibration, Verification, and Validation in the measurement process. See this link if you did not read it yet >> Calibration-Verification-Validation

Now in this article, I will present the difference, relationships and Interpretations of the following terms: Accuracy, Tolerance, Error, and Uncertainty.


Moreover, I will share with you below topics to answer the questions above:

  1. The Difference Between  Accuracy and Error (Accuracy vs Error)
  2. The difference between Error and Uncertainty (Error vs Uncertainty)
  3. The Difference between Tolerance and Uncertainty (Tolerance vs Uncertainty)
  4. The relationships between Accuracy, Error, Tolerance, and Uncertainty in Calibration Results

As per JCGM 200 and 106: 2012, below are the actual definitions:

  1. Accuracy = closeness of agreement between a measured quantity value and a true quantity value of a measurand
  2. Error or measurement error = measured quantity value minus a reference quantity value
  3. Tolerance =difference between upper and lower tolerance limits
  4. Uncertainty or measurement uncertainty = non-negative parameter characterizing the dispersion of the quantity values being attributed to a measurand, based on the information used- make more meaningful

First Let me present each term in a simple way that I understand (I hope for you too)


Accuracy

Accuracy is the closeness of UUC results to the STD (true) value. This ‘closeness’ is usually represented in percentage value (%) and can be shown in the same unit by converting it into an error value ( %error). The more close the percentage value to ZERO (0%), the more accurate.

Accuracy is more on a qualitative description which means that it does not present an exact value.

Accuracy is equivalent to or a percent error (%error). This is where the value of error will be used.

Accuracy can be calculated using the formula: 

the formula on how to calculate accuracy

If you know the error from a specific range, you can calculate the accuracy.


Error

Error is simply the difference between the UUC and STD results after calibration. It has the same unit as the measured parameter.

See the below example from a calibration result based on the photo above.

Calculating error in calibration result


Tolerance

It is the maximum error or deviation that is allowed or accepted in the design of the user for its manufactured product or components.

Tolerance Formula, Tolerance Interval and Tolerance limits
Tolerance Formula, Tolerance Interval and Tolerance Limits

Tolerance is a range of values that is acceptable or permitted by the user from the result of the process or product measurement.

It is the permitted error that is:

  1. Calculated from the process design by the user
  2. Prescribed by regulatory bodies (based on Accuracy Class)
  3. Manufacturer specifications (based on Accuracy)

The formula is Upper limit – lower limit (UTL-LTL)

If we perform a measurement, tolerance value will tell us if the measurement we have is acceptable or not. 

If you know the calibration tolerance limits, it will help you answer the questions like:

1.        How do you know that your measurement result is within the acceptable range?
2.        Is the final product specification pass or fail?
3.        Do we need to perform adjustments?

 “The bigger the tolerance, the more product or measurement results will pass or accepted.”


Uncertainty

I will only explain here what measurement uncertainty is, not how to calculate measurement uncertainty. 

This will be in a different post because there are many processes involved before we can come up with a single expanded uncertainty result. My point here is to show you the difference and relationships of uncertainty results with the other measurement terms.

Ok let’s go…

Uncertainty or Measurement Uncertainty is defined as the quantification of the doubt. There is always a doubt that exists, an error included in the final result that we do not know, therefore, there are no perfectly exact measurement results,

Why do we have uncertainty or doubt in our measurement?

Some of the main reasons why we have doubt or uncertainty in measurements are:

  1. Inadequate knowledge of the Effects of the environmental conditions on the measurement;
  2. Personal bias in reading analog instruments, an example is the resolution or smallest value that you can read.;
  3. inexact values of measurement standards and reference materials;
  4. approximations and assumptions incorporated in the measurement method and procedure;
  5. variations in repeated observations of the measurand under apparently identical conditions – Repeatability 

You can read more under JCGM 100:2008 also known as the GUM 

We do not know this error that is added to our measurement results, and therefore, we cannot remove or correct it.

Because we cannot correct it,  what we can do is to determine or estimate the range where the true value is located, this range of true value is the measurement uncertainty result.

See below image  with calibration result as an example:

A measurement result with the Measurement uncertainty
A measurement result with the Measurement uncertainty

  “The smaller the measurement uncertainty, the more accurate or exact our measurement results.”



Differences Between Accuracy, Error, and Tolerance, their Relationships and  Interpretations in Measurements

Accuracy Versus Error

Accuracy is for gauging how small/large the error is (a qualitative description),  while the Error is the actual representation of accuracy in the same units as the measured parameter (measurand). In other words, the error shows the quantity of accuracy in the unit of measurement used.

Accuracy and error have opposite relationships (indirectly proportional) but they are directly related to each other.

Accuracy is a qualitative form, meaning no exact value or measurement result is presented, only a presentation (usually in percentage form) of how good or bad or how far and near but no exact value, while error shows the absolute value or actual value.

In order to show the exact or absolute value, we need to use the error. The error shows the exact distance of the measurement result from the true value.

Sometimes, accuracy is presented in a quantitative form which is actually the error at a certain range. See below example based on the photo above:

Example: accuracy of 0.1% of Full scale is more accurate than a 0.5% of full scale reading.

@Full scale = 5000 psi

For 0.1% accuracy = 0. 1%*5000 = 0.001*5000 = 5 psi

For 0.5% accuracy = 0.5% *5000 = 0.005*5000 = 25 psi

From the results above, it is clear that 5 psi is a much smaller error compared to 25 psi,  which means that 0.1% is more accurate than 0.5%


Accuracy Class and MPE: A Manufacturer Tolerance Value

At this point, accuracy can be used as a Tolerance based on manufacturer specifications. We call this type of accuracy as the Accuracy Class or Grade. 

Then from accuracy class, it is calculated to its equivalent error which is called the MPE (maximum permissible error) as required by standards like ASTM and  ISO or manufacturer’s specifications. This is then used as the tolerance limits, and afterward, the tolerance value. See the below image presentation.

An example of how an accuracy class is used as a Tolerance
An example of how an accuracy class is used as a Tolerance

.



The Difference  Between Error and Uncertainty and its Relationship in Measurement Results

Error vs Uncertainty

As we know now, Error is the difference between UUC – STD reading. The smaller the error, the more accurate the measurement results.

UUC -TSD = error

Any error that we know can be corrected. From calibration certificate results, where a standard value is given, we can now determine the error. And since the error is determined, we can correct it by either adding or subtracting the correction factor which is the opposite of the error.

Correction = STD-UUC.

From the table above, we now know that the error is a +3, or more than 3, therefore, in order to achieve the most accurate result during use in measurement, we need to remove the excess 3, hence minus 3. Now, the final value of our measurement result is 497.

In reality, the exact error is not known, therefore, what we can do is to estimate it. This estimated error is the measurement uncertainty. A range of estimated errors. 

Uncertainty is the ‘range of values’ where the true value or actual location of the measurement results (UUC) lie, while the Error is the ‘exact result’ of the difference between the UUC and STD which shows how accurate the measurement result is by showing the actual distance to the true (STD) value.

Uncertainty is a boundary within the measurement results to show the range of its actual location based on a given confidence level (95%, k=2)). See the below photo.

Error vs. Uncertainty
Error vs. Uncertainty: The difference Between Error and Uncertainty

When the uncertainty results are included in the measurement results,  we are 95% sure that the true value lies somewhere in the range 496.1 to 503.9 psi. 

The smaller the measurement uncertainty, the more accurate the result, because it shows that the range of estimated errors are very small.



The Difference  Between Tolerance and Uncertainty  


The Difference  Between Tolerance , Error and Uncertainty

The Difference  Between Tolerance, Error, and Uncertainty  

Based on the image above, it shows the difference between Tolerance and Uncertainty, such as:

  1. The Tolerance is the boundary of permissible value or limits of errors; 
  2. The Uncertainty shows the boundary or limits of an estimated error where the exact measurement result’s location.

Calibration tolerance limits are provided by the user, by regulatory bodies or as per specifications while Uncertainty is calculated based on the combined errors related to the STD and UUC.

With a given Tolerance and Uncertainty, TUR (Test Uncertainty Ratio) can be calculated. A ratio of 4:1  is recommended. 

The formula for TUR is equal to tolerance/uncertainty. See the below formula and example. The TUR for this is equal to 12.8:1

The formula for TUR is equal to tolerance/uncertainty


Tolerance and Uncertainty As a Basis for Decision Rule as per ISO 17025:2017

Uncertainty is used or included when determining compliance with specifications as per the requirement of ISO 17025:2017. One way to tell if a product has passed or failed based on a given tolerance, a decision rule.

What is a Decision Rule

As per ISO 17025:2017, clause 3.7, it is a rule that describes how measurement uncertainty is accounted for when stating conformity with a specified requirement.

Below are the decision rules (as an example based on the image below):

  1. If the results of our measurements are within the tolerance indicated including the uncertainty results, then it is a ‘pass’.
  2. If the results of our measurements are outside the tolerance including the uncertainty results, then it is considered  ‘fail’ or ‘out of tolerance’.
  3. If one of the uncertainty limits is outside the tolerance while the other limits are inside the tolerance limit, then it is not a pass or a fail, we call it ‘Indeterminate’. The decision now is based on the user. 

The requirement is that, during the assessment of the statement of conformity,  we should consider the uncertainty results and apply the above decision rule. See below presentation to explain more:

 The use of tolerance and uncertainty results to come up with a decision rule
The use of tolerance and uncertainty results to come up with a decision rule. The UUC results including the uncertainty are within tolerance and therefore, it is ‘Passed’

Based on the above results/presentation, it is “passed” because the result (UUC reading) including the uncertainty results is inside the tolerance limits.

For a detailed explanation regarding the ‘decision rule’ and ‘reporting statement of compliance to specifications’, I have read a good guide, which presents almost all that you need to know regarding this topic. Check out this link>> Guide for statements of conformity and decision rules



The Relationships Between Accuracy, Error, Tolerance, and Uncertainty – The Interpretation from a Calibration Result

Accuracy shows the degree of closeness of a measurement result to the true or reference value.

The degree of closeness from the reference value is presented in the actual value (not a percentage (%) of) through the calculated Error (UUC-STD).

The error shows how the measurement results have deviated from the true value.

While accuracy is calculated based on error and true value,  Uncertainty is calculated based on the combined errors or inaccuracy of reference standards (STD) and the measurand (UUC).

Uncertainty shows the range where the measurement results (UUC)  actually located. An estimated location of true UUC value which is limited by the confidence interval (usually @ 95%, k=2).

In order for the result to be acceptable, uncertainty results should stay within the tolerance limit.

Not in all cases uncertainty is larger than the error as presented here. But in all cases to have a sure ‘Pass’ remarks, when included in measurement results, it should stay within the tolerance limit.

Tolerance shows the permissible error of measurement results and it is the difference between the UTL and LTL (UTL-LTL)

To sum it all, see below image. 

 The Relationships Between Accuracy, Error, Tolerance, and Uncertainty from a calibration results
The Relationships Between Accuracy, Error, Tolerance, and Uncertainty from a calibration results



Conclusion

I hope I have presented with you simple definitions and explanations regarding the differences, relationships, and interpretations of accuracy, error, tolerance, and uncertainty.

These are the most used terms when it comes to reporting calibration results, understanding and creating a calibration procedure or just simply understanding a calibration process.

These will help us understand more about the calibration process and therefore we can have a proper and correct way of interpretations thus leading to correct measurement implementation.

In this post, I have presented the following:

  1. The Difference Between  Accuracy and Error ( Accuracy vs Error)
  2. The difference between Error and Uncertainty (Error vs Uncertainty)
  3. The Difference between Tolerance and Uncertainty (Tolerance vs Uncertainty)
  4. The relationships between Accuracy, Error, Tolerance, and Uncertainty from a Calibration Results

When it comes to decision making regarding the results of our measurements, these are the terms that we need to understand.

For the relationships between Accuracy, Precision, and Tolerance, visit my other post HERE

Please comment and share to LinkedIn

Connect with me on my Facebook page.

Best Regards,

Edwin

10 Responses

  1. ARTURO
    | Reply

    Good night
    Thank you very much Edwin very well explained
    A query when a pattern comes out not compliant can I continue to use to calibrate other instruments

    • edsponce
      | Reply

      Hi Arturo,

      You are welcome. I appreciate your comment.

      Non-compliant means it is a fail or out of tolerance, therefore it needs adjustment and recalibration before use.

      I hope I understand your question.

      Thanks for reading my post.
      Edwin

  2. M. Saleh AL-Mufti
    | Reply

    Sir,
    Really,it’s very great job & helpful too with high explanation, many thanks to your efforts.

    • edsponce
      | Reply

      Hi Saleh,
      You are welcome. Your comment is highly appreciated.

      Best regards,
      Edwin

  3. Naveed Ahmed
    | Reply

    Thanks sir very good brief all queries in my mind now clear thanks again

    • edsponce
      | Reply

      Hi Naveed,
      You are welcome. I am glad that you liked it.

      Best Regards,
      Edwin

  4. Vinod
    | Reply

    Ultimate guide sir, you clear all my doubts.
    Finally i want to know that as per example you have taken the tolerance is very high. if it is applied for temperature calibration and accuracy is 0.5 % of reading and range of equipment is 0 to 200 deg. cel. then we can say that tolerance is 199 to 201 deg. cel.@ 200 deg. cel.
    After calibration we have found the error of 0.6 deg. cel. and expanded uncertainty is 1.3 deg. cel. then what is means for that equipment’s acceptability? Have we correlate the uncertainty with tolerance or error with tolerance for adding both (uncertainty and error). Your valuable response is awaited.

    • edsponce
      | Reply

      Hi Vinod,

      Thanks for the comment.

      Please see the below image if it answers your concern.

      Indeterminate Decision Rule

      The decision rule that applies for the above result is Indeterminate, meaning, it is up to the user on how he will decide into it.

      Some ways to fix this are:
      1. Perform adjustment if possible or use the correction factor.
      2. Look for a calibration service provider with a good CMC
      3. Use a reference standard with much higher accuracy
      4. Implement guardbanding

      I hope this answers your questions.

      Edwin

      • Vinod
        | Reply

        A lot of thanks, i understood it that from a calibration certificate, an equipment is acceptable if UUC Reading +/- expanded uncertainty < tolerance for that equipment. Is it right?

        • edsponce
          | Reply

          Hi Vinod,
          Yes, You are correct. There is no doubt that it is acceptable if the results including the uncertainty are within or < than tolerance limits. Best regards, Edwin

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.