Gyan Repository







National Institute of Cholera and Enteric Diseases || Uncertainty of Measurement for ELISA
Case Studies   »  
(April 11, 2019)

Key Words:        ELISA; GUM; ISO 15189; Relative Standard Deviation; Uncertainty, Equivocal

Abstract: Analytical results estimating Uncertainty of Measurement (MU) signifies the confidence level of the concerned assay. Enzyme-linked Immunosorbent Assay (ELISA) is an extensively used important tool for infectious disease sero-diagnosis, nevertheless, MU for ELISA result is reported since procedure of MU estimation is hardly available. International Organization for Standardization (ISO) provides the Guide to the Expression of Uncertainty in Measurement (GUM) describing set of guidelines; however, no detailed procedures or instructions for evaluating specific measurement processes are described. This article aims step-by-step procedure for MU estimation in a serology laboratory by describing the potential sources of uncertainty during each step of ELISA.

Methods

HIV sample was tested by commercial ELISA following routine procedure. Uncertainty was estimated by specification of measurand, identification of uncertainty sources, quantification of values attributed to the sources of uncertainty and calculation of the combined standard uncertainty following GUM by converting all the standard uncertainties of ELISA into dimensionless relative standard uncertainties. The combined standard uncertainty uC  was calculated using the propagation principle

 

Results

Uncertainty arising from systematic error (RSD 0.10966)   and routine analytical imprecision (RSD 0.04938)  were considered as the sources of uncertainty contributing to the total MU of this routine qualitative diagnostic method. Analysis of the data revealed that results obtained for the relative expanded uncertainty U rel  was 24% with 95% confidence level (k=2).

Conclusion

Quantification of uncertainty by combining systematic error and analytical imprecision seems a feasible method of MU for ELISA. The estimated MU reflects the extent of variation for the cut-off absorbance for equivocal outcome of the qualitative interpretation.

Introduction

Uncertainty of Measurement (MU) is a parameter associated with the result of a measurement that characterizes the dispersion of the values that could reasonably be attributed to the analyte measurand (1). Some of the components in MU may be evaluated from the statistical distribution of the results obtained from series of measurements and can be characterized by standard deviations. Some other components can be evaluated from assumed probability distributions based on experience or other information (2). The objective of the validation of the analytical method is to ensure that every future measurement in routine analysis will be close enough to generate the desired robustness that the unknown true value can be approximated with minimum error for the content in the sample. MU of test result can be demonstrated as the probability of not observing the same quantitative test result while retesting the same sample in identical methods and settings. Uncertainty for serological assay can be estimated following the “Guide to the Expression of Uncertainty in Measurement” (GUM). While GUM provides general rules for conceptualization and analyses of the uncertainty in a certain parameter, it does not provide detailed procedures or instructions for evaluating specific measurement processes (3-5). In many countries regulatory authorities follow the standards prescribed by International Organization for Standardization (ISO) to assess laboratory competence for estimating the MU of laboratory assays. One of the requirements of ISO 15189 is the determination of uncertainty of results where relevant and necessary (6, 7).

In serological testing there are many potential uncertainties that can substantially affect test results and it is important to identify such factors (8). Although pre- and post-analytical steps may well become sources of uncertainty, they hardly affect the inherent uncertainty of the testing procedure and therefore, such factors can be excluded from the estimation process of MU (9). Uncertainties arising from the measurand of reference material and routine analytical imprecision may be considered as two major sources of uncertainty contributing to the total MU of a routine diagnostic method (10). At first efforts can be made to measure the uncertainty associated with the numerical values assigned to the measurand for the reference specimen used in the routine method using Type A (evaluated using statistical analysis of a series of observations) and ‘Type B’ (evaluated using means other than statistical analysis of a series of observations) bottom-up approaches as described in GUM (5). In the next step, measurement of the random error that normally occurs while conducting the test may be incorporated as the uncertainty associated with the value of a test result (11). The later component of uncertainty is conventionally demonstrated by the dispersion of values or imprecision observed when a measurand in the same specimen is repeatedly measured. Quantification of imprecision obtained from the routine application of internal quality control methods is recommended as a part of the quantitative estimation of the MU (8). When the estimate for uncertainty is known for both the reference material and the routine analytical imprecision of a test procedure, the total estimate for MU of the test results can be calculated by addition. As in a laboratory  these measurement procedures are usually employed over a longer time span, the most relevant findings usually interpreted as the estimate for MU against fixed reference values are the imprecision of the test results across as many routine operating conditions as possible (12). For well-established methods, it is recommended that a minimum of six months internal quality control (IQC) data should be used to calculate routine imprecision, updated at least annually where possible (13).

Enzyme-linked Immunosorbent Assay (ELISA) has been adopted for use in many diagnostic tests, including serological diagnoses of microbial infections (14). In addition to a high level of sensitivity which may allow the use of serology to diagnose infection within days, ELISA is a reproducible and objective test that can be performed on small amounts of serum (15). In recent past MU have been assessed in different estimation processes by using different techniques (16-18), but hardly any published article is available on MU related to ELISA. Therefore, an attempt has been made for estimation of MU of ELISA in a serological laboratory following standard guidelines. The objective of the study was to estimate uncertainty during ELISA procedure based on numerical values generated using reference material for systematic error and imprecision data obtained from the routine analytical procedure following guidelines of ISO 15189, EURACHEM and GUM document. This article aims to provide step-by-step procedure for MU estimation in a serology laboratory even with limited resources.

Materials and Methods

General setting

The study was carried out at a National HIV Reference Laboratory, accredited as per International Standard ISO 15189, designated for quality assurance of HIV-testing and evaluation of diagnostic kits including ELISA.

Ethics statement

The blood sample used as reference material was received from blood bank following permission from Drug Controller General of India, the competent authority. HIV positive samples were obtained from an unknown and anonymous voluntary donor source. As per blood bank protocol the HIV positive blood has to be discarded as it is not fit for transfusion and supposed to be destroyed. We have received permission from Drug Controller General of India to use it for the purpose of this research. Therefore, taking consent from the donor seemed dispensable. Institutional Ethics Committee of National Institute of Cholera and Enteric Diseases, Kolkata approved the study.

Equipment

Micropipettes (Finnpipette, Finland) of 50 µl, 300 µl and 1000 µl, timer (Fisher Scientific, India), thermometer (Fisher Scientific, India) and ELISA reader (Thermo labsystems, USA) were used for the study. The pipettes and timer were calibrated by a calibration laboratory following international standard (ISO 15189) procedure and protocol. ELISA reader was calibrated by the service engineer of the manufacturer. The measurement of absorbance was carried out at 450 nm wavelength at kit manufacturer’s specified conditions regarding temperature and humidity. All the dilution and reaction was carried out in a temperature controlled environment.

Reagents and kits

Commercial ELISA kit Vironstika, HIV Ag/Ab (Biomerieux, Holland) was used to test the reference sample for this study. This kit consisted of 96-well ELISA plates coated with a mixture of HIV-1 gp 160, HIV-1 ANT70, HIV-2 env and anti-HIV-1 p24. Each well contained a pearl-shaped sphere, containing lyophilized HRP-labeled HIV-1 gp 160, HIV-1 ANT70, HIV-2 env conjugate and anti- HIV-1 p24, and utilized urea peroxide and tetramethylbenzidine for detection of antibodies to HIV antigen in human serum.

Reference sample

HIV-positive plasma sample received from blood bank and confirmed by three ELISA, three rapid assays and a Western Blot assay was defined as the reference sample or reference material. This sample was used for ELISA by multiple laboratories for calculating the systematic error. An aliquot of 5ml sample was transported to each laboratory maintaining cold chain (2-8oC) and stored at -20 oC freezer till the ELISA was performed.

Assay procedure

The assay was performed following manufacturer’s instructions. 50 µl sample was diluted with 100 µl Tris-buffer saline containing 1% Tween 20 (TBS-T), dispensed into each well and incubated for 1 hour at 37 0C. The plates were washed six times by phosphate buffer, then 100 µl TMB substrate was added to each well and incubated for 30 minutes at 15 to 30 0C. The reaction was then stopped by the addition of 100 µl of 1M sulphuric acid, and absorbance/optical density (OD) of each well was read at 450 nm. Cut-off value was calculated asNCx+0.100 , where NCx  was OD of the negative control supplied in the kit.

Calculating MU: Step-by-step procedure

Reference sample was tested in the same manner as done routinely for patient samples. In this study, we have taken into consideration the important parameters which contributed towards the uncertainty as shown in Fish-bone or cause-and-effect diagrams. The uncertainty arising from measurement of ELISA has been presented in Figure 1.

Calculating systematic error

The reference sample (HIV positive plasma) was tested by 5 different laboratories using same kit (Vironstika, HIV Ag/Ab) and repeated independent results were produced by 20 analytical run from each laboratory. The technical staff members were blinded to the category of the samples and the reference sample was included in the laboratories routine sample flow. Absorbance values of all test runs were used to calculate the systematic error.

Calculating imprecision error

As shown in Figure 1, the major sources of uncertainty in the measurement of ELISA were the steps involved in antigen-antibody reaction, substrate reaction and measurement of absorbance. Uncertainties associated with both the reaction steps did arise from the micropipette used for dispensing, the timer use for the measurement of time and the thermometer used for the measurement of the ambient temperature during reaction. To determine the values of relative standard uncertainty of pipetting volume in each step, the dial of the working micropipette was set at the desired level. A solution was aspirated and transferred into a beaker on a microbalance. The weight of the pipette solution was measured 20 times and the value for the relative standard uncertainty was calculated. A calibrated timer with 1second resolution and a calibrated thermometer with 0.1oC resolution were used to calculate relative standard uncertainty due to time and temperature measurement respectively. Uncertainty due to ELISA reader during measurement of absorbance was calculated from the linearity and repeatability information of the instrument specifications.

Calculating combined uncertainty

To calculate the total uncertainty, the uncertainty propagation law was used for which there were only divisions and products. All individual components were converted into relative uncertainties for avoiding mixed units arising out of different sources of uncertainties. Relative standard uncertainty was expressed as Relative Standard Deviation (RSD) which was calculated as standard deviation divided by mean of a set of results. In accordance with GUM (20), the combined uncertainty of the mathematical model, which was a product or quotient, could also be considered as the model equation, as following:

                                                                                                             (Eqn. 1)

where y(x1, x2.....) was the function of several parameters x1,x2,....   and Ci was sensitivity coefficient, for all the cases assumed as 1. Each variable’s contribution was measured as the square of the associated uncertainty expressed by the standard deviation multiplied with the square of the relevant sensitivity coefficients.

As both systematic error and routine analytical imprecision were considered for the calculation of MU, the combined uncertainty could be calculated using the following model:

                                                                                      (Eqn. 2)

 

Uncertainty expression / MU calculation.

Expanded uncertainty U  was a parameter allowing the determination of the limits of confidence interval comprising an unknown true value µ with a defined probability α:

 

 

                                                                                                                                            (Eqn. 3)

Where the expanded uncertainty U  was expressed as: 

                                                                                                                                                                      (Eqn. 4)

Uncertainty was expressed in the form of a range with a specific level of confidence. A common form of expression was  units (at a confidence level of N%) where X   was the “best estimate” measurement result; was the uncertainty associated with that result; and N was the confidence interval (19).

Results and Discussion

A sum of quantifiable numerical values arising from sources contributing uncertainty for ELISA was essential for estimating the extent of uncertainty.

Uncertainty due to systematic error

When the result of five laboratories were combined, the mean of absorbance for all (n=100) the results was 3.3581 and standard deviation was 0.36825. Relative standard uncertainty due to reference sample was calculated as 0.10966. A laboratory result can be precise but may not be accurate. Inaccuracy in such cases is attributable to lack of validity, a measure of systemic error quantified as bias. Certified reference materials (CRM) are ideal control materials when available and can be regarded as ultimate standards of validity for measuring systematic error of any analytical method. The use of commercially available CRM may sometimes not be appropriate for laboratories due to the high cost of CRMs and non-availability in sufficient amounts for use as reference sample over extended periods. Alternately, a defined reference sample/proficiency sample tested by multiple laboratories on the same assay can be considered as an optimized option manageable in a resource limited setting.

Uncertainty due to routine analytical imprecision

As presented in Figure 1, the sources of uncertainty due to antigen-antibody reaction step were: dispensing of diluents, samples and kit controls into the well and reaction time. The sources of uncertainty due to TMB substrate reaction step were: mixing of TMB and substrate solution, dispensing of TMB substrate into the well and reaction time. Relative standard uncertainty due to reaction steps was calculated as following.

      (Eqn. 5)

                                                             (Eqn. 6)

Reading for the absorbance of the sample measured by the ELISA reader was expected to vary due to the non-identical repetitions, so there existed a random error for the measured absorbance forming a rectangular distribution. The main uncertainties were due to the non-homogeneity of the 96 well plates or edge effects. Uncertainty associated with the absorbance measure was quoted by the manufacturer as linearity (0.02) and repeatability (0.05).

Uncertainty due to absorbance measure =                                                                 (Eqn. 7)

Relative standard uncertainty was then quantified as                                                            (Eqn. 8)

Relative standard deviation due to routine analytical imprecision was calculated by using the following equation:

                                                                                                  (Eqn. 9)

 

Combined uncertainty and uncertainty budget

The evaluation of uncertainty mainly depends on detailed knowledge of the measurand and the measurement procedure used. The quality and utility of the uncertainty quoted for the result of a measurement did depend on the understanding of the critical analysis and integrity of those who contributed to the ascertainment of its value. GUM proposed a “bottom-up approach” to quantitatively evaluate the ‘combined uncertainty’ of a measurement. This method quantified the MU associated with the individual effects and analytical steps that resulted in the random and systematic errors of the measurements (6). Alternatively, the “top-down” approach was intended to obtain a reliable estimate for the uncertainty, without necessarily knowing any of the sources individually. It relied on the overall reproducibility estimates either from in-house or inter-laboratory measurement trials. Also, a combination of the different approaches were needed often to assess the uncertainty (21). In the present study, the measurement of uncertainty of the method was calculated following a GUM “bottom up approach” that was based on long experience and reflected common practices (4, 22).

As an illustration, the contribution of each component to the total measurement uncertainty of the absorbance measurement in this study is given in Table 3 and Figure 2. We converted all the standard uncertainties into dimensionless relative standard uncertainties and then combined standard uncertainty uC was calculated using propagation principle

                                                                                  (Eqn. 10)

Where OD denoted the absorbance of the sample and uC(OD)  was combined uncertainty of an ELISA run. RSDsystematic error  was obtained from data from 5 different laboratories using reference specimen and RSDRoutine analytical imprecision was obtained from Eqn .9

Combined standard uncertainty   and the combined relative standard uncertainty was calculated as 12%.

Expanded uncertainty was calculated by multiplying the combined uncertainty by the coverage factor k, which as usual was chosen to be k=2 to achieve an approximately 95% confidence interval. The relative expanded uncertainty U rel (K=2) was found to be 24%.

In a test, the variations in the reference sample and patient samples were assumed to be equivalent. Thus, this result might be interpreted as the potential variation or MU in the OD following ELISA procedure that was measured as ±24% with 95% confidence interval for a patient sample also. This might have indicated that the result was probably equivocal within the range of ±24% OD value with reference to the cutoff value for qualitative interpretation also.

Estimation of MU is a time consuming and demanding task for laboratories (8). A large number of experiments as well as data are the pre-requisite before all required uncertainties are obtained. The requirement for uncertainty estimation should be considered for the stages of work procedure and measurement method design so that all operations are performed in such a way as to reduce potential uncertainties and simplify their calculation.

 

Conclusions

Quantification of combined uncertainty by cumulating the contributions arising from routine analytical imprecisions and systematic error using a reference sample may be considered as a practical approach for estimating MU. Where commercial reference material with defined tractability is difficult to obtain, a well-defined in-house reference material tested by different laboratories to obtain the estimation of bias (systematic error) may be an acceptable alternative for resource poor laboratories. The potential for variation or MU in the OD following ELISA procedure, as found in this study, could be ±24% with 95% confidence interval for a patient sample also. This procedure of estimating bias and analytical imprecision may be considered as a generalized scheme for planning a simple and practical method to estimate MU as demonstrated. This method might be able to establish itself, over the years, as a credible time and cost-efficient approach for the determination of MU in ELISA in a serological laboratory.

 

 

References

  1. Kessel W (2002) Measurement uncertainty according to ISO/BIPM-GUM. Thermochim Acta 382: 1-16.
  2. Gómez-Hens A, Aguilar-Caballos MP (2007) Modern analytical approaches to high-throughput drug discovery. TrAC Trend Anal Chem 26: 171-182.
  3. Ellison SLR, Williams A, editors. (2007) EURACHEM/CITAC Guide: Use of uncertainty information in compliance assessment.  Geneva, Switzerland.
  4. EURACHEM/CITAC. (2000) EURACHEM/CITAC Guide: Quantifying Uncertainty in Analytical Measurement, Teddington, England (UK).
  5. ISO. 1995. Guide to the expression of uncertainty in measurement. International Organization for Standardization (ISO), Geneva, Switzerland.
  6. ISO. 2007. Medical laboratories-particular requirements for quality and competence (ISO 15189). International Organization for Standardization (ISO), Geneva, Switzerland.
  7. ISO/IEC. 2005. General requirements for the competence of testing and calibration laboratories (ISO/IEC 17025:2005). International Organization for Standardization (ISO), Geneva, Switzerland.
  8. Dimech W, Francis B, Kox J, Roberts G (2006) Calculating uncertainty of measurement for serology assays by use of precision and bias. Clin Chem 52: 526-529.
  9. White GH, Farrance I (2004) Uncertainty of measurement in quantitative medical testing: a laboratory implementation guide. Clin Biochem Rev 25: S1-24.
  10. Thompson M, Ellison SL, Wood R (2002) Harmonized guidelines for single-laboratory validation of methods of analysis (IUPAC Technical Report). Pure Appl Chem 74: 835-855.
  11. Guevara-Riba A, Sahuquillo A, López-Sánchez JF, Rubio R (2006) Comparison of three strategies to evaluate uncertainty from in-house validation data. A case study: mercury determination in sediments. Anal Bioanal Chem 385: 1298-1303.
  12. Fuentes-Arderiu X (2000) Uncertainty of measurement in clinical laboratory sciences. Clin Chem 46: 1437-1438.
  13. Farr AJ, Freeman KP (2008) Quality control validation, application of sigma metrics, and performance comparison between two biochemistry analyzers in a commercial veterinary laboratory. J Vet Diagn Invest 20: 536-544.
  14. Engvall E, Perlmann P (1971) Enzyme-linked immunosorbent assay (ELISA). Quantitative assay of immunoglobulin G. Immunochem 8: 871-874.
  15. Avrameas S (1992) Amplification systems in immunoenzymatic techniques. J Immunol Methods 150: 23-32.
  16. Kumar AR, Riyazuddin P (2007) Underestimation of total arsenic concentration in groundwater samples determined by hydride generation quartz furnace atomic absorption spectrometry due to sample characteristics. Accred Qual Assur 12: 455-458.
  17. Singh N, Ojha V, Kayal N, Ahuja T, Gupta PK (2011) Quantifying uncertainty in the measurement of arsenic in suspended particulate matter by Atomic Absorption Spectrometry with hydride generator. Chem Cent J 5: 17.
  18. Synek V, Subrt P, Marecek J (2000) Uncertainties of mercury determinations in biological materials using an atomic absorption spectrometer – AMA 254. Accred Qual Assur 5: 58-66.
  19. Blues J, Bayliss D, Buckley M (2004) The calibration and use of piston pipettes, National Physical Laboratory Teddington, Middlesex, United Kingdom.
  20. ISO. 2010. Guidance for the use of repeatability, reproducibility and trueness estimates in measurement uncertainty estimation. In ISO 21748. ISO, Geneva, Switzerland.
  21. EUROLAB. 2007. Measurement uncertainty revisited:  Alternative approaches to uncertainty evaluation France.
  22. Chudzinska M, Debska A, Baralkiewicz D (2012) Method validation for determination of 13 elements in honey samples by ICP-MS. Accred Qual Assur 17: 65-73.