You received a pressure gauge calibration certificate with the following results:

You are asked to interpret these results to determine if the results of the calibration have passed or failed by accounting the measurement uncertainty in the measurement results before deciding.

How do you decide? What decision rule can you follow to determine conformity if it is passed or failed?

In other terms, is it complying with the specifications? This is a question most users need to answer clearly

During the reporting of results, if you put a remark in a calibration report like pass, fail, ok, not-ok, in-tolerance, out-of-tolerance, you are already providing a decision for conformity.

When providing a decision, you need a rule as a guide that will help you to come up with that decision. And this rule is what we call the “Decision Rule”.

Most of us are defaulted to the rule that when the measurement result is within the tolerance limit  provided, then it is passed, else, it is failed. While this is not wrong, this is not enough. There are other details that we need to consider that I will present it here.

There are so many guides out there but I will present here what I understand and believe to be simple to those starting out to understand this topic. Once you have understood this part, you can now begin to expand more by using other references with more advanced technique.

I only choose the most basic parts in the reference guides that I listed in the last part of this article.

In this post, I will present the following topics:

1. Understanding the decision Rule
2. 5 Steps to Implement an ISO 17025 Decision Rule for Conformity Assessment in Calibration (with Flow chart)
3. 3 Types of Decision Rules: The Basis for the Statement of Conformity
4. The Requirements of Decision Rule in The User Side- Why do you need to Implement Decision Rule?
5. ISO 17025 Decision Rule Requirements
   1. When to apply the Decision Rule
   2. When not to apply the Decision Rule
6. References
7. Conclusion

This topic of decision rule will be focused more on the user side. There is no use of the decision rule if the main owner or user does not understand its importance. 

I have presented this topic where mathematics and statistics are limited to make it simple (and honestly I am not a mathematician).  My objective is to present a decision rule that is simple enough to understand that will become your starting point for implementation.

.

Understanding The Decision Rule 

What is a Decision Rule?

Before we can decide if the measurement result of our instrument has passed or failed, we need to choose what decision rule we can use.

The meaning of decision rule is  defined by ISO 17025:2017, under clause 3.7,  it states that: >>>  rule that describes how measurement uncertainty is accounted for when stating conformity with a specified requirement

Decision Rule is a statement of rules describing how you will use the Measurement Uncertainty (MU) in relation to the measurement results and tolerance to come up with a pass or failed decision.

It is the same with an instruction, telling you what to do in order to decide if an instrument has passed or failed  based on a criteria.

Decision rule will answer the question “How did you use the Measurement Uncertainty to determine if the  instrument has passed or failed based on your tolerance?

There are 3 ways in which the MU is used (accounted for)  in order to come up with a decision rule.  Actually there are more ways to account MU but I will share with you the simplest for us to understand the basics and its principles.

The objective of having a decision rule is to detail the rules to follow on how we determine a pass or failed results of our Instruments based on a specified requirement, like the manufacturer specifications.

A pass or failed decision is the end result of conformity assessment. Conformity assessment as per ISO guide 98-4 means,  “activity to determine whether specified requirements relating to a product, process, system, person or body are fulfilled”.

In our case, it is a verification if it complies with standards or specifications – examples are the manufacturer specifications, regulations, standards, or customer process requirements.

Therefore, by having a Decision Rule, conformity assessment in calibration or testing results  will be easily fulfilled.

 There are more ways to come up with a Decision Rule. Same as Calibration Interval Implementation, there are no single rule that can cover all the needs of every user, therefore, these methods that we read and other reference documents are all guidelines and recommendations. In the end, still, you, as the user has the right to choose what is applicable for you.

5 Steps to Implement an ISO 17025 Decision Rule for Conformity Assessment in Calibration

1. Define Specifications from the Calibration Results
2. Account for the measurement uncertainty
3. Identify the Zone for acceptance and Rejection
4. State the Decision Rule
5. Use the decision Rule – Decide for conformity

Step 1: Define the Specifications – Interpret the Calibration Results

In this initial step, there are 3 basic requirements which are needed in order to come up with a decision for compliance. These data are based on our calibration results..

Calibration results consists of the following 3 basic requirements:

a.       Measured Value (MV)

b.      Tolerance Limit (TL)

c.       Measurement Uncertainty (MU)

Measured value is simply the output reading of the instrument involved (the UUC).

The tolerance limit is the permissible error from our process tolerance or from the manufacturer specifications. Another term is MPE (Maximum Permissible Error).

The question now is, where can I find the decision rule MU? Do I need to calculate this?

The good news is the Measurement Uncertainty for decision rule calculation is already given in the certificate, just copy it directly. I am referring to this Implementation where a calibration certificate is already available.Calculation of MU is not necessary because it is already given. But if not, then you need to calculate the measurement uncertainty.

Below is a sample from a calibration certificate that shows the measurement  results with the Measurement Uncertainty.

Step 2: Account for the Measurement Uncertainty.

Calculating The Results for a Decision Rule: How is Measurement Uncertainty (MU) Accounted for? 

Decision Rule content is the combination of Measurement Results and Measurement Uncertainty (MU). Therefore, a calculation is needed.

But why do we need to account for the measurement uncertainty when determining compliance with specifications?

Since no measurement is exact, we need to know not just the measured value but the true value or range of true values. This is because when MU is combined with the measured value, what we have now will be the range of true value (or just true value).  See photo under method A below

In these steps, we will Use the measurement results (as given in 1) to calculate the Results for a Decision Rule. We will  integrate the MU in our measurement results (measured value and tolerance value).

I will show you how to account  for the MU and to provide a decision rule as per ISO 17025:2017 by a simple calculation. The Decision Rule calculation procedure

Below are  3 methods on how you can directly account for MU.

1. Method A: The True Value (or a range of true value)
2. Method B: Guard banding  .
3. Method C: Combination of Method A and B

Method A: The True Value – Complete Statement of Result

The result of adding and subtracting the Measurement Uncertainty in the Measured Value is a term called the ‘True Value’ also defined as “Complete Statement of Result”’ as per NPL GPG 79.

It is a complete statement of results because what you have is the range of location of the true value, compared to MV where only a single location of the UUC output reading is known. Once you have determined the True Value (or range of true value), your confidence will become higher, more so if you observed that it is within the tolerance limit.

How to calculate for Method A:

 Method A = MV+/- expanded measurement uncertainty (MU)

Method B: Guard banding  

The main objective of this method is to reduce the ‘Tolerance limit’ to a new limit called the ‘Acceptance Limit’ using a guard band in order to determine the acceptance interval or acceptance zone.

 Let us first define a guard band: as per ILAC G8. “it is a safety factor built into the measurement decision process by reducing the acceptance limit below the tolerance limit”

The purpose is to reduce the risk of making an incorrect conformance decision or passing (accepting) a non-conforming item.

How To calculate Method B

 For the simplicity of this article, the Guard band is equal to Measurement Uncertainty (MU) (Guard band = MU). (Note that the formula of guard band that I presented here is not fixed, depending on the required risk, the guard band will change. But this is in more advanced analysis and use so we will just focus on the most basic).

For actual calculation, below are the steps:

1. Record the MU;

2. Determine the Guard band(w)

where: Guard band (w) = MU
Guard band (w) =TL-AL

2. Determine the Acceptance Limit basing it on the chart below (Upper and Lower Acceptance Limit)
UAL = UTL – w
LAL = LTL + w

3. Determine the Acceptance Zone or Acceptance Interval

Acceptance Zone = UAL – LAL

Once the guard band and the Upper and Lower Acceptance Limit is in place, we have now the acceptance interval or acceptance zone.

Method C: Combination of Method A and B

This method is simple once you have completed Method A and B. Just combine the results of Method A and B. Both methods are used in one chart creating another way to account for a measurement uncertainty.

By choosing how to account for measurement uncertainty will lead us with the types of a Decision Rule to be used. 

Step 3: Identify the Rejection and Acceptance Zone

After a successful determination of acceptance limits, label the Rejection and Acceptance Zone to easily identify conformity.

Step 4: State the Decision Rule

Now that the criteria for determining a pass or fail decision is clear, it is time to Implement and declare our decision rule statement. 

A statement of the rule you just implement from number 2 and 3 above to determine the Pass and Fail results. It is describing how you come up with a pass or fail decision.

The decision rule statement is as follows:

Pass - if MV +/-MU is within the Acceptance Zone and MV does not exceed the  Acceptance Limit

Fail -  if measured value is outside the Acceptance Limit or within the Rejection Zone

The statement above is what we will include in our calibration report

(See this type of Decision Rule under “3 types of Decision Rules” below).

Step 5: Decide for Conformance-  Use the Decision Rule

Choose Decision Below:

1. Passed  or Failed
2. In-Tolerance or Out-of-Tolerance
3. OK or Not OK

3 Types of Decision Rules: The Basis for the Statement of Conformity

There are many types of decision rules that can be applied depending on the need and understanding of the user. But for simplicity, I will only present 3 types which are the most used and recommended Decision Rule. 

1. Binary Decision Rule Simple Acceptance 

2. Non-Binary Decision Rule

3. Guarded Decision Rule

1.    Binary Decision Rule –  Simple Acceptance

Simple Acceptance Rule – the basic decision rule where acceptance is made when measurement results are within the Tolerance Interval.

It is a binary means there are only 2 choices:

1. Pass – Measured value is within the tolerance limit
2. Fail – measured value is outside the tolerance limit
   

This method of accounting measurement uncertainty is the simplest. As long as we calculated the TUR to be equal or greater than to 4:1 (as per agreement). then we are all set. (reference from ASME B89.73.1)

To calculate TUR:  

2.    Non-Binary Decision Rule 

 What if one part of the range of true value is outside the tolerance limit?

There are cases where the measured value +/- uncertainties were found out of tolerance on either the high side or low side. See the below photo.

This will result to a decision rule known as Non-Binary because you have more than 2 choices as follows: (note: the meaning of ‘True Value’ is explained under method A above)

1. Pass – if the True value is within the Tolerance Limit (TL)
2. Fail- if the True value has exceeded the Tolerance Limit (TL)
3. Indeterminate  – final result is not sure therefore no decision is made for the user. This will result in additional choices namely:
   1. Conditional Pass – part of the true value (or MU) has exceeded the TL while the measured value is inside the TL.
   2.  Conditional Fail – the true value (or MU)  including the MV has exceeded the TL  but part of the True value remains inside the TL.

Indeterminate results mean we are not sure. It is conditional. The decision is now based on the user’s knowledge and experience regarding the instrument. This scenario will lead us to a probability in rejecting good instruments or accepting bad ones. How to address this ‘no-decision’ scenario in a decision rule?

The problem regarding the Indeterminate decision will now be solved by the guard band. See the 3^rd  Type of Decision Rule below

3.    Guarded Decision Rule

This decision Rule is also known as Binary Decision Rule with Guard band

1. Pass – if MV +/-MU is within the Acceptance Zone and MV does not exceed the  Acceptance Limit
2. Fail – if the measured value is outside the Acceptance Limit

We will use this method if one of the true range values is outside or crossed the outside the tolerance limit. In Method A above, an indeterminate decision will result if this happened, and we do not want indeterminate results, the PASS and FAIL decision should be clear for us.

Therefore, as a solution, we will put a guard band. In which a new limit will be created know as Acceptance Limit (AL).

Any results  that are within the Acceptance Limit (AL) will be accepted.

The Requirements of Decision Rule in the User Side- Importance of Implementing the Decision Rule

One of the basic things to do once we receive a calibration report is to review it. Reviewing calibration results is a necessary activity. Interpretation  of results for verification is the next move once we receive a calibration report.

In this part, my main purpose is to show the users how important it is to understand the use of a decision rule that will result to a proper decision to determine compliance with specifications.

The following are some of the benefits of having a documented Decision Rule:

1. The decision rule will guide us regarding the rule to follow or use in order to execute the decision process for conformity assessment, to come up with  a pass or failed decision easily and objectively.
2. It is one way to appreciate the importance of a measurement uncertainty reflected in a calibration certificate. You will observe the impact of having high uncertainty results.
3. In connection to number 2, you will also appreciate the importance of observing and maintaining the use of TUR (Test Uncertainty Ratio) during calibration.
4. You will be aware of what tolerance to consider and adjust it appropriately in the process.
5. You will be more aware about the performance of your measuring instrument. Most labs do not interpret the calibration certificate unless it is requested by the user. So the responsibility for review and interpretation lies with the UUC owner.
6. By providing a specific decision rule will be good evidence or supplement  of compliance with  clause 7.1.5.1 of ISO 9001, Monitoring and Measuring Resources
   1. You can determine suitability of Measuring and monitoring Instruments
   2. Ensuring the instrument’s ‘fitness for their purpose’

But how do we use the  decision rule to  interpret the calibration results? You can implement the decision rule by following the 5 steps above.

ISO 17025 Decision Rule Requirements

The first part is the user side, now we will focus on the calibration laboratory side.

With this new version of the ISO 17025 standards, Decision Rule Implementation is mentioned or implied to be implemented in various activities such as in contract review, reporting of results, interpretation of results, verification process, and conformity assessment.

When to Implement or Apply  the Decision Rule?

Whenever we hear or read the term conformity assessment or statement of conformity, it is referring to the decision rule to determine conformance with specifications – the pass or fail decision.

The following are some of the related activities (with clauses) that prompt us when to implement the decision rule as per ISO 17025.

1. When it is required by a customer as part of the contract review as per clause 7.1.3.
   – The initiation of a decision rule implementation starts when a customer requests it. Therefore, it is our job to ensure that we get the request of the customer regarding their tolerance and the decision they want to use, on the other hand, it should be clear for us (and the customer)  that the customer has NOT required it.
   
2. When the uncertainty of measurements has an effect on the validity of the measurement results (clause 7.8.3.1c) – This is the main requirement of a decision rule, accounting for the measurement uncertianty to determine the validity of measurement results. An example of this is the 4:1 TUR Decision Rule and Guarded Acceptance decision rule.
   
3. During the interpretation of results to determine compliance with specifications (Clause 7.8.3.1b) – a decision rule that will be displayed in the calibration report based on interpretations of the lab.
   
4. The calibration certificate shall include where relevant, a statement of conformity with requirements, or specifications (Clause 7.8.4.1e).
   
5. During the reporting of Statements of conformity (Clause 7.8.6.2) – it should be clear what decision rule is used during the interpretation of results for conformity assessment.

And in connection with this, any person that is assigned to assess a statement of conformity should be authorized as per clause 6.2.6.

When Not  to Apply ISO 17025 Decision Rule ?

1. If there is a standard guide or method that we follow, usually prescribed by regulatory bodies like ASTM standards and government-provided acceptance criteria, there is no need to implement the decision rule prescribed by ISO 17025. 
2. When a customer requires a conformity assessment without accounting the Measurement Uncertainty (MU).
3. When MU is negligible that it is not needed to be included in the Decision Rule – this happens when you have very accurate instruments. But in reality, this is difficult or very expensive to attain.
4. When the TUR is greater than 4:1 (the no 1. Type above- simple acceptance). Actually this is an indirect type of decision rule. But if you agree with this type of decision rule, you do not need to think of accounting the measurement uncertainty as long as you maintain the 4:1 requirement, just directly use the simple acceptance type of decision rule.

Examples of Decision Rule from a Calibration Certificate

You may check more examples and presentations in this link >> https://www.isobudgets.com/statements-of-conformity-and-decision-rules/

References for Decision Rule Implementation

1. ILAC_G8_09_2019_Guidelines on decision rules and statements of conformity
2. JCGM_106_2012_E role of Conformity Assessment
3. ASME B89.7.3.1-2001 Guidelines For Decision Rules

Conclusion

Decision Rule is a statement of rules explaining how measurement uncertainty is used when performing conformity assessment.

Conformity Assessment is an activity to determine conformance to specifications using the decision rule. This is where we determine a pass of fail decision.

Decision rule requirements are mostly applicable to the customer since they are the ones who need to determine conformance to their instruments. 

But in order for them to understand, the lab should make the effort to educate and help them decide what decision rule is best for them. This as a result will help the lab how to properly implement it.

Since it is a requirement during conformity assessment, we cannot avoid it but more so, we need to understand it. 

In this post, I have presented the following:

1. Understanding the decision Rule
2. 5 Steps to Implement an ISO 17025 Decision Rule for Conformity Assessment in Calibration (with Flow chart)
3. 3 Types of Decision Rules: The Basis for the Statement of Conformity
4. The Requirements of Decision Rule in The User Side- Why do you need to Implement Decision Rule?
5. ISO 17025 Decision Rule Requirements
   1. When to apply the Decision Rule
   2. When not to apply the Decision Rule
6. References

I hope this article is a big help to easily understand this topic.

Thank you for reading my article.

Please comment and subscribe!

Connect with me on my Facebook page.

Best Regards,

Edwin