I have received the below comments in relation to the last post about the differences between calibration, verification, and validation.
These are the comments:
- Can you give an example instrument with a specification that we can determine or differentiate calibration, verification, and validation?
- When is verification, validation or calibration is the most appropriate for the intended M&TE?
I presented here a concrete example to differentiate these three terms in a direct and hopefully clear way.
Also, through these examples, it will show if your intention to use these terms regarding your M&TE (Measurement and Test Equipment) is appropriate.
Let us start!
As an example to, we will perform a calibration, verification, and validation in a Liquide Oxygen (LOX) Tank. The tank has Differential Pressure Level Gauge (DPLG) and a pressure gauge to determine its level and actual pressure together with Pressure Safety Valve and Pressure Transmitter.
But in this example, I will focus only on the DPLG and pressure gauge for simplicity.
CALIBRATION
By definition, calibration is the “comparison” of the unknown reading of a UUC to a known reading of the Reference Standard.
During calibration, our objective is to determine how accurate the Differential Pressure Level Gauge by comparing its output reading and computing the error.
Buy determining the error, we can see how far (or close) the value of the UUC to the value of the reference standard, thus, we can also determine the correction value.
Calibration Result:
For DPLG, based on the photo above, the standard reads at 1519 mmWC while the UUC reads 1500 mmWC.
Error = Measured value – true value
Error = 1500-1519 = -19 mmWC
Therefore, the Correction Factor is +19 mmWC
Moreover, calibration is incomplete without the measurement uncertainty results because this will inform us the exact location (boundary) of the true value. When we know the location of the true value, then a more accurate estimate is determined with higher confidence. Based on the measurement uncertainty results in the above example, we are 95% confident that the exact value of 1519 is within the interval or boundary of +/- 38.5 (between 1480.5 – 1557.5).
Take note that during calibration, we follow a standard procedure to determine how accurate or close (or far) the result is to the true value.
Now, because of calibration, we have determined the error and correction factor. We can use this to improve the accuracy of our measurement results by performing adjustments to return the UUC to its most accurate reading, or if an adjustment is not possible, add/subtract the correction factor to the final results.
The same procedure applies to the pressure gauge.
VERIFICATION
From our simple definition, Verification is a process of “confirming” that a given specification is fulfilled.
How to verify after calibration? After the process of calibration, immediately comes verification to determine if the results are within the set tolerance.
During verification, our objective is to determine if the DPLG has an acceptable output reading based on a specification or user requirement. For simplicity, we will use the manufacturer specs regarding tolerance to determine a pass or a fail status.
We will verify if the UUC is within the limits defined by the manufacturer.
The accuracy of the DPLG based on manufacturer specs is 2.5% of Full Scale (FS) (see photo)
The full scale =2000 mmWC
Therefore,
the tolerance limit is 2000*0.025 = +/-50 mmWC,
Verification Result:
The error of the DPLG based on calibration result at 1500 is 19.
Based on the manufacturer specs, it is within the +/- 50 tolerance limit, therefore it is PASSED.
The same process of verification with the pressure gauge is applied.
Take note that during verification, we follow a specification to determine the acceptance criteria.
Because of verification, we confirmed that the DPLG is within manufacturer specification and therefore, re-calibration and/or adjustment is not needed.
VALIDATION
As we continue in this stage, we define again validation as “ensuring” the acceptability of the implemented measurement process.
Which is in this example, the suitability of the tank to be used as a storage and monitoring for Liquid Oxygen (L0X)
During validation, we will consider the entire system, where all the instruments installed are functioning to its specification, resulting in an output based on the intended purpose.
Let us assume that the intended purpose is to display the exact level of liquid oxygen with an acceptable pressure range.
.
The objective of the validation is to determine that:
- The DPLG will display the exact differential pressure at a determined level based on the tank specifications. See in the photo below.
Tank Specifications: Conversion of Differential Pressure to Level - The pressure should not exceed 35 bar.
- The transmitter is transmitting the right information (level) for remote monitoring.
Validation Result:
The tank displays the exact level of liquid oxygen with an acceptable range of pressure as per calibration and verification results with the Tank Specifications.
Therefore, the whole system (tank) is suitable for storing and monitoring liquid oxygen (LOX).
Take note, that prior to validation, we performed calibration and verification in order to support our objective above.
Through calibration, we have determined the instrument accuracy, while through verification, we have confirmed that it is within manufacturer tolerance/specifications.
Validation results will conclude if the use of the LOX tank is fit for its purpose (storing and monitoring) and therefore leads to the approval or rejection decision by the concerned individuals.
This is just an example, in actual, more test and verification is performed during the actual validation process which includes a leak test, the safety valve, the transmitter, and others.
Final results of validation will be documented through a validation report with signatures of approval for ‘fit to be used‘ on its purpose
Check out in this link the whole post about the differences between calibration, verification, and validation.
I hope this makes all clear.
Please comment and subscribe.
Edwin
18 Responses
Alvin
Hi S’Edwin,
Good Day!
Now I understand well their differences. I appreciate you sir. I hope my appreciation will boost your enthusiasm to post more about calibration or other topics and be as expert as you someday. Thanks a lot!
Regards,
Alvin
edwin
Hi Mr. Alvin,
You are welcome. I am happy to know that you understand it.Your appreciation will truly boost my motivation and enthusiasm to create more post that I believe can benefit us all.
Thank you for your valuable comments. I also appreciate it.
Any more comments or concern with regards to calibration, do not hesitate to message me.
Best Regards,
Edwin
Siva
Dear Edwin,
I want to know ,,
how to calculate uncertainity value (+or -) from Standard Reading and UUC Reading and Error..
edsponce
Dear Siva,
Thank you for reading my post.
Calculating measurement uncertainty requires more steps and considerations of other errors. But for simplicity, I will just focus directly on your inquiry.
For standard reading, you need to have the calibration certificate and look there the measurement uncertainty value.
That value, you divide it by 2 (normal distribution), example, uncertainty from cal cert = 0.5 therefore, 0.5/2 = 0.25 psi
For UUC reading, compute for the standard deviation of the repetitions you made then divide by Sqrt of 2 = 1.41 (for 2 repetitions)
For example, trial 1 = 2.2 and trial 2= 2.3, using excel to compute for the standard deviation will lead to an answer of 0.07,type in excel, =STDEV(2.2,2.3)
then 0.07/1.41= 0.05 psi
And for the error, we will substitute the error for hysteresis error, for example, equal to 0.2, then 0.2/ sqrt of 3 for a rectangular distribution. Sqrt of 3 = 1.73,
therefore, 0.2/1.73 = 0.115. psi
Now we will combine all the uncertainties to one result using the Root Sum Square (RSS) method.
It is easy to calculate with the help of an excel sheet. Just type below:
=SQRT(SUMSQ(0.25,0.05,0.115)) = 0.28 psi
Multiply 0.28 by 2 for a 95% confidence level which is equal to 0.56 psi
+/- 0.56 >>> the final uncertainty value that you will use.
This is just a simple example. Other sources of error are still needed to be determined and include in the calculation.
I believe you have a background about uncertainty calculation in order to understand this.
I hope this helps,
Edwin
Jagadeesh
Hi Sir,
Good day,
During external audit got one minor NC regarding method verification record not available,
We are not using any Non standard method even our quality manual also address the same, we are using Is standard only…
But auditor told me to verification records need for IS standard also,
So kindly I request you to please clarify how to done method verification record….
edsponce
Hi Jagadeesh,
Just to let you know that the auditor is correct, all methods used even if it is a standard method should be verified or validated for appropriateness before use.
If you notice, the standard method we use does not recommend or specified a specific reference standard instrument to use, including the exact knowledge and skills necessary to do the job. Therefore, once we adapt this standard method, we need to perform verification using our own reference standard and skilled personnel.
With this in mind, we need a record showing that before we use the standard method, there is an evidence that it has an acceptable results. There is a study performed to verify and show acceptable results in using the method with the instrument we have.
There are no specific rules or guidelines for verification that are required but below are my suggestions.
Perform one calibration of instrument and verify results showing that:
1. It has acceptable results – within specifications
2. It has a good repeatability and reproducibility (precision and bias)
3. Measurement uncertainty is within your CMC
4. Acceptable measurement uncertainty by implementing intra-lab or Inter-lab comparison using the same statistical technique (EN value or Z-score) for acceptance criteria.
Take note that all these should be recorded and approved by an authorized person.
For reference, you may use Eurachem Guide, The Fitness for Purpose of Analytical Methods. This is free to download. Just choose the process that is applicable for you.
I hope this helps,
Edwin
Uma
Dear Sir,
Good Day!
Now I understand well their differences. Can you please share sample verification record of any one calibration method?
edsponce
Hi Uma,
Any record where calibration is performed with a remarks of passed or failed based on tolerance is a verification record. You can see more of this when you are performing a quality check or monitoring. One example is the intermediate check performed in the lab or a preventive maintenace report.
See example of the PM report on my other post in this link >> preventive-maintenance-in-calibration
Best regards,
Edwin
Anna
Hi,
I come from a water/wastewater testing lab. One parameter we test is pH. We were recently audited and got an NC for not having records of verifying our new pH meter & probe before being placed in our lab for use. We also don’t have a procedure for verification of equipment. I’m in the process of developing a procedure.
Can you confirm if the following is sufficient for verifying the equipment before placing it in service (in alignment to ISO/IEC 17025:2017 std);
1. Checks to ensure it meets the Manufacturer’s specifications
-in this case, verifying the proper installation of the probe and meter settings
– successful calibration and acceptable verification of calibration with a reference material
2. Calculation of the equipment MUs
3. Statement is “fit for use” after successful verification
edsponce
Hi Anna,
The procedure is ok but my suggestion is to separate the calibration part from verification.
These are my suggestions:
1. If not calibrated, perform calibration. The calibration part includes:
a. Determination of the error and MU. Provide a calibration certificate for this. This could be every 6 months or yearly depending on your frequency as per criticality of usage.
b. then perform verification by assessing if the error is within the set tolerance limit and determine if it is fit for use.
2. Now, here comes the requirement of verification before putting it to service. This is a pH meter that needs regular verification.
a. Make a regular schedule, it can be verified before using the pH solution or once every week. Since this is a regular verification, no need to calculate the MU, this is already done in the calibration stage. Your goal for verification is to determine only if it is fit for use or not based on the set tolerance. You only need a verification record.
I hope this helps,
Edwin
Anna
Thank you for the feedback.
Could you elaborate what is meant by “Determination of the error”. and “perform verification by assessing if the error is within the set tolerance limit and determine if it is fit for use”.
edsponce
Hi Anna,
‘Determination of error’ means to calculate the error based on the calibration results. Error can be determined or calculated using the formula, Error= UUT value – Standard value.
Once you have determined the error, use this value to assess if it is within the tolerance limit. If it is within the set tolerance limit then you can assess or decide that it is fit for use. The decision made (fit or not fit/pass or fail) is now the verification result.
For example, the error is equal to 0.3, the tolerance limit = +/-0.5. It is clear the 0.3 is within the tolerance limit, therefore it is ‘passed.’
I hope it is clear now.
Edwin
Rasli
Hi Edwin,
Thanks a lot for your explanations about Calibration, Verification and Validation.
Looking at the ISO 9001:2015, clause 7.1.5.2 Measurement traceability, it reads:
When measurement traceability is a requirement, or is considered by the organization to be an essential part of providing confidence in the validity of measurement results, measuring equipment shall be:
a) calibrated or verified, or both, at specified intervals, or prior to use, against measurement standards traceable to international or national measurement standards; when no such standards exist, the basis used for calibration or verification shall be retained as documented information;
b) identified in order to determine their status;
c) safeguarded from adjustments, damage or deterioration that would invalidate the calibration status and subsequent measurement results.
The organization shall determine if the validity of previous measurement results has been adversely affected when measuring equipment is found to be unfit for its intended purpose, and shall take appropriate action as necessary.
My concerns and questions:
1) When looking at 7.1.5.2 a) above, it says “calibrated OR verified, OR both, at specified intervals, or prior to use, AGAINST measurement standard traceable to international or national measurement standards……..as documented information;
In above article (please correct me I am wrong), it says that for “Calibration”, it is comparing the readings of UUC against the readings of STD and the STD shall be traceable to international or national standard. However, for “Verification”, one will not compare the readings of UUC versus the readings of STD.
However, when I read the ISO 9001 clause 7.1.5.2 a) above, seems like both, “Calibration” and “Verification” shall be “against measurement standard…”. Means, from what I understand, both “Calibration” and “Verification” are actually comparing the readings of UUC against the reading of STD.
2) If one is doing the so-called “Calibration” (comparing the readings of UUC against the readings of STD), BUT, he/she does not calculating the measurement uncertainty, can we call it a “Calibration”? If no, what do we call such task?
Sorry for the long comments and inquiry, above. However, I hope you can help me on this. Thank you.
Concerning Calibration –> in artcile
edsponce
Hi Rasli,
Below is my response to your inquiry:
1. For number 1 question:
For us not to confuse about the meaning of verification and calibration, let us go back to the simple understanding.
Calibration is the comparison of the UUC reading to the STD reading, in this way, an error or a correction and measurement uncertainty can be obtained.
While for verification, it is the comparison of the UUC reading to the specifications of the UUC or this is what we call the tolerance limit, this is to determine if the UUC has passed or failed the performed measurement or calibration. During verification, we need the measurement results from the calibration we performed, specifically the “error” and “uncertainty” values. This is where the “decision rule” will also come in.
Now, during verification, in order to for us to be sure that the results are correct or with confidence, the standard we used to verify the UUC should be calibrated.
Because of this, it is understandable that before we perform verification, we should calibrate first the UUC using the available reference standard. This is why calibration comes first before verification. The calibration result is now used for verification using the specification of the UUC.
2. for number 2 question
If we take deeper the meaning of calibration, it is referring to the establishment of a relationship between the quantity values with measurement uncertainties, it is, therefore, an incomplete calibration, or it is not calibration at all if the measurement uncertainty is not available. You are just performing a simple measurement procedure.
In the user part, there are instances that they do not need the measurement uncertainty results, but during calibration, it is a must that we calculate the measurement uncertainty, more so if you are a lab that seeks accreditation where it is one of the main requirement of ISO 17025.
Remember that our main goal in calibration is to determine how accurate our measurement result is, measurement uncertainty will inform us of the exact location of the measurement results, thus leading to a more accurate representation of the exact location of the true value.
I hope this helps, thanks for reading my posts.
Best regards,
Edwin
Ulf P.
Dear Edwin,
trying to get my head around the example used for your calibration and verification to understand if the instrument actually is within calibration.
The tolerance limit is set to ±50mmWC
The error is 19mmWC
The uncertainty is ±38,5mmWC
So if I take the error plus the uncertainty I will get 57,5mmWC, which then is greater than 50mmWC. So the potential reading with the uncertainty and the error takes me outside of the permissible tolerance given on the gauge.
Question: Is the instrument within or out of calibration?
edsponce
Hi Ulf,
If you are referring to the Decision Rule wherein, we account the measurement uncertainty to make a compliance, you are correct, the instrument is out of calibration or out of tolerance limit.
The solution in this case is either we perform the necessary adjustment or correction factor and/ or look for a calibration lab with a better CMC.
Thanks for reading my post.
Best regards,
Edwin
Phuc
Hi Sir,
Could you please explain me the different between Calibration failed vs Failed calibration?
Thanks you,
Phuc
edsponce
Hi Phuc,
There are no specific definition that I am aware of, this depends on how the terms are used. if you can share where you have read this it would be more clear and helpful to understand.
Best regards,
Edwin