National Institute of Cholera and Enteric Diseases || Uncertainty of Measurement for ELISA
Case Studies »
(April 11, 2019)
Key Words: ELISA; GUM; ISO 15189; Relative Standard Deviation; Uncertainty, Equivocal
Abstract: Analytical results estimating Uncertainty of Measurement (MU) signifies the confidence level of the concerned assay. Enzyme-linked Immunosorbent Assay (ELISA) is an extensively used important tool for infectious disease sero-diagnosis, nevertheless, MU for ELISA result is reported since procedure of MU estimation is hardly available. International Organization for Standardization (ISO) provides the Guide to the Expression of Uncertainty in Measurement (GUM) describing set of guidelines; however, no detailed procedures or instructions for evaluating specific measurement processes are described. This article aims step-by-step procedure for MU estimation in a serology laboratory by describing the potential sources of uncertainty during each step of ELISA.
HIV sample was tested by commercial ELISA following routine procedure. Uncertainty was estimated by specification of measurand, identification of uncertainty sources, quantification of values attributed to the sources of uncertainty and calculation of the combined standard uncertainty following GUM by converting all the standard uncertainties of ELISA into dimensionless relative standard uncertainties. The combined standard uncertainty uC was calculated using the propagation principle
Uncertainty arising from systematic error (RSD 0.10966) and routine analytical imprecision (RSD 0.04938) were considered as the sources of uncertainty contributing to the total MU of this routine qualitative diagnostic method. Analysis of the data revealed that results obtained for the relative expanded uncertainty U rel was 24% with 95% confidence level (k=2).
Quantification of uncertainty by combining systematic error and analytical imprecision seems a feasible method of MU for ELISA. The estimated MU reflects the extent of variation for the cut-off absorbance for equivocal outcome of the qualitative interpretation.
Uncertainty of Measurement (MU) is a parameter associated with the result of a measurement that characterizes the dispersion of the values that could reasonably be attributed to the analyte measurand (1). Some of the components in MU may be evaluated from the statistical distribution of the results obtained from series of measurements and can be characterized by standard deviations. Some other components can be evaluated from assumed probability distributions based on experience or other information (2). The objective of the validation of the analytical method is to ensure that every future measurement in routine analysis will be close enough to generate the desired robustness that the unknown true value can be approximated with minimum error for the content in the sample. MU of test result can be demonstrated as the probability of not observing the same quantitative test result while retesting the same sample in identical methods and settings. Uncertainty for serological assay can be estimated following the “Guide to the Expression of Uncertainty in Measurement” (GUM). While GUM provides general rules for conceptualization and analyses of the uncertainty in a certain parameter, it does not provide detailed procedures or instructions for evaluating specific measurement processes (3-5). In many countries regulatory authorities follow the standards prescribed by International Organization for Standardization (ISO) to assess laboratory competence for estimating the MU of laboratory assays. One of the requirements of ISO 15189 is the determination of uncertainty of results where relevant and necessary (6, 7).
In serological testing there are many potential uncertainties that can substantially affect test results and it is important to identify such factors (8). Although pre- and post-analytical steps may well become sources of uncertainty, they hardly affect the inherent uncertainty of the testing procedure and therefore, such factors can be excluded from the estimation process of MU (9). Uncertainties arising from the measurand of reference material and routine analytical imprecision may be considered as two major sources of uncertainty contributing to the total MU of a routine diagnostic method (10). At first efforts can be made to measure the uncertainty associated with the numerical values assigned to the measurand for the reference specimen used in the routine method using Type A (evaluated using statistical analysis of a series of observations) and ‘Type B’ (evaluated using means other than statistical analysis of a series of observations) bottom-up approaches as described in GUM (5). In the next step, measurement of the random error that normally occurs while conducting the test may be incorporated as the uncertainty associated with the value of a test result (11). The later component of uncertainty is conventionally demonstrated by the dispersion of values or imprecision observed when a measurand in the same specimen is repeatedly measured. Quantification of imprecision obtained from the routine application of internal quality control methods is recommended as a part of the quantitative estimation of the MU (8). When the estimate for uncertainty is known for both the reference material and the routine analytical imprecision of a test procedure, the total estimate for MU of the test results can be calculated by addition. As in a laboratory these measurement procedures are usually employed over a longer time span, the most relevant findings usually interpreted as the estimate for MU against fixed reference values are the imprecision of the test results across as many routine operating conditions as possible (12). For well-established methods, it is recommended that a minimum of six months internal quality control (IQC) data should be used to calculate routine imprecision, updated at least annually where possible (13).
Enzyme-linked Immunosorbent Assay (ELISA) has been adopted for use in many diagnostic tests, including serological diagnoses of microbial infections (14). In addition to a high level of sensitivity which may allow the use of serology to diagnose infection within days, ELISA is a reproducible and objective test that can be performed on small amounts of serum (15). In recent past MU have been assessed in different estimation processes by using different techniques (16-18), but hardly any published article is available on MU related to ELISA. Therefore, an attempt has been made for estimation of MU of ELISA in a serological laboratory following standard guidelines. The objective of the study was to estimate uncertainty during ELISA procedure based on numerical values generated using reference material for systematic error and imprecision data obtained from the routine analytical procedure following guidelines of ISO 15189, EURACHEM and GUM document. This article aims to provide step-by-step procedure for MU estimation in a serology laboratory even with limited resources.
Materials and Methods
The study was carried out at a National HIV Reference Laboratory, accredited as per International Standard ISO 15189, designated for quality assurance of HIV-testing and evaluation of diagnostic kits including ELISA.
The blood sample used as reference material was received from blood bank following permission from Drug Controller General of India, the competent authority. HIV positive samples were obtained from an unknown and anonymous voluntary donor source. As per blood bank protocol the HIV positive blood has to be discarded as it is not fit for transfusion and supposed to be destroyed. We have received permission from Drug Controller General of India to use it for the purpose of this research. Therefore, taking consent from the donor seemed dispensable. Institutional Ethics Committee of National Institute of Cholera and Enteric Diseases, Kolkata approved the study.
Micropipettes (Finnpipette, Finland) of 50 µl, 300 µl and 1000 µl, timer (Fisher Scientific, India), thermometer (Fisher Scientific, India) and ELISA reader (Thermo labsystems, USA) were used for the study. The pipettes and timer were calibrated by a calibration laboratory following international standard (ISO 15189) procedure and protocol. ELISA reader was calibrated by the service engineer of the manufacturer. The measurement of absorbance was carried out at 450 nm wavelength at kit manufacturer’s specified conditions regarding temperature and humidity. All the dilution and reaction was carried out in a temperature controlled environment.
Reagents and kits
Commercial ELISA kit Vironstika, HIV Ag/Ab (Biomerieux, Holland) was used to test the reference sample for this study. This kit consisted of 96-well ELISA plates coated with a mixture of HIV-1 gp 160, HIV-1 ANT70, HIV-2 env and anti-HIV-1 p24. Each well contained a pearl-shaped sphere, containing lyophilized HRP-labeled HIV-1 gp 160, HIV-1 ANT70, HIV-2 env conjugate and anti- HIV-1 p24, and utilized urea peroxide and tetramethylbenzidine for detection of antibodies to HIV antigen in human serum.
HIV-positive plasma sample received from blood bank and confirmed by three ELISA, three rapid assays and a Western Blot assay was defined as the reference sample or reference material. This sample was used for ELISA by multiple laboratories for calculating the systematic error. An aliquot of 5ml sample was transported to each laboratory maintaining cold chain (2-8oC) and stored at -20 oC freezer till the ELISA was performed.
The assay was performed following manufacturer’s instructions. 50 µl sample was diluted with 100 µl Tris-buffer saline containing 1% Tween 20 (TBS-T), dispensed into each well and incubated for 1 hour at 37 0C. The plates were washed six times by phosphate buffer, then 100 µl TMB substrate was added to each well and incubated for 30 minutes at 15 to 30 0C. The reaction was then stopped by the addition of 100 µl of 1M sulphuric acid, and absorbance/optical density (OD) of each well was read at 450 nm. Cut-off value was calculated asNCx+0.100 , where NCx was OD of the negative control supplied in the kit.
Calculating MU: Step-by-step procedure
Reference sample was tested in the same manner as done routinely for patient samples. In this study, we have taken into consideration the important parameters which contributed towards the uncertainty as shown in Fish-bone or cause-and-effect diagrams. The uncertainty arising from measurement of ELISA has been presented in Figure 1.
Calculating systematic error
The reference sample (HIV positive plasma) was tested by 5 different laboratories using same kit (Vironstika, HIV Ag/Ab) and repeated independent results were produced by 20 analytical run from each laboratory. The technical staff members were blinded to the category of the samples and the reference sample was included in the laboratories routine sample flow. Absorbance values of all test runs were used to calculate the systematic error.
Calculating imprecision error
As shown in Figure 1, the major sources of uncertainty in the measurement of ELISA were the steps involved in antigen-antibody reaction, substrate reaction and measurement of absorbance. Uncertainties associated with both the reaction steps did arise from the micropipette used for dispensing, the timer use for the measurement of time and the thermometer used for the measurement of the ambient temperature during reaction. To determine the values of relative standard uncertainty of pipetting volume in each step, the dial of the working micropipette was set at the desired level. A solution was aspirated and transferred into a beaker on a microbalance. The weight of the pipette solution was measured 20 times and the value for the relative standard uncertainty was calculated. A calibrated timer with 1second resolution and a calibrated thermometer with 0.1oC resolution were used to calculate relative standard uncertainty due to time and temperature measurement respectively. Uncertainty due to ELISA reader during measurement of absorbance was calculated from the linearity and repeatability information of the instrument specifications.
Calculating combined uncertainty
To calculate the total uncertainty, the uncertainty propagation law was used for which there were only divisions and products. All individual components were converted into relative uncertainties for avoiding mixed units arising out of different sources of uncertainties. Relative standard uncertainty was expressed as Relative Standard Deviation (RSD) which was calculated as standard deviation divided by mean of a set of results. In accordance with GUM (20), the combined uncertainty of the mathematical model, which was a product or quotient, could also be considered as the model equation, as following:
where y(x1, x2.....) was the function of several parameters x1,x2,.... and Ci was sensitivity coefficient, for all the cases assumed as 1. Each variable’s contribution was measured as the square of the associated uncertainty expressed by the standard deviation multiplied with the square of the relevant sensitivity coefficients.
As both systematic error and routine analytical imprecision were considered for the calculation of MU, the combined uncertainty could be calculated using the following model:
Uncertainty expression / MU calculation.
Expanded uncertainty U was a parameter allowing the determination of the limits of confidence interval comprising an unknown true value µ with a defined probability α:
Where the expanded uncertainty U was expressed as:
Uncertainty was expressed in the form of a range with a specific level of confidence. A common form of expression was units (at a confidence level of N%) where X was the “best estimate” measurement result; U was the uncertainty associated with that result; and N was the confidence interval (19).
Results and Discussion
A sum of quantifiable numerical values arising from sources contributing uncertainty for ELISA was essential for estimating the extent of uncertainty.
Uncertainty due to systematic error
When the result of five laboratories were combined, the mean of absorbance for all (n=100) the results was 3.3581 and standard deviation was 0.36825. Relative standard uncertainty due to reference sample was calculated as 0.10966. A laboratory result can be precise but may not be accurate. Inaccuracy in such cases is attributable to lack of validity, a measure of systemic error quantified as bias. Certified reference materials (CRM) are ideal control materials when available and can be regarded as ultimate standards of validity for measuring systematic error of any analytical method. The use of commercially available CRM may sometimes not be appropriate for laboratories due to the high cost of CRMs and non-availability in sufficient amounts for use as reference sample over extended periods. Alternately, a defined reference sample/proficiency sample tested by multiple laboratories on the same assay can be considered as an optimized option manageable in a resource limited setting.
Uncertainty due to routine analytical imprecision
As presented in Figure 1, the sources of uncertainty due to antigen-antibody reaction step were: dispensing of diluents, samples and kit controls into the well and reaction time. The sources of uncertainty due to TMB substrate reaction step were: mixing of TMB and substrate solution, dispensing of TMB substrate into the well and reaction time. Relative standard uncertainty due to reaction steps was calculated as following.
Reading for the absorbance of the sample measured by the ELISA reader was expected to vary due to the non-identical repetitions, so there existed a random error for the measured absorbance forming a rectangular distribution. The main uncertainties were due to the non-homogeneity of the 96 well plates or edge effects. Uncertainty associated with the absorbance measure was quoted by the manufacturer as linearity (0.02) and repeatability (0.05).
Uncertainty due to absorbance measure = (Eqn. 7)
Relative standard uncertainty was then quantified as (Eqn. 8)
Relative standard deviation due to routine analytical imprecision was calculated by using the following equation:
Combined uncertainty and uncertainty budget
The evaluation of uncertainty mainly depends on detailed knowledge of the measurand and the measurement procedure used. The quality and utility of the uncertainty quoted for the result of a measurement did depend on the understanding of the critical analysis and integrity of those who contributed to the ascertainment of its value. GUM proposed a “bottom-up approach” to quantitatively evaluate the ‘combined uncertainty’ of a measurement. This method quantified the MU associated with the individual effects and analytical steps that resulted in the random and systematic errors of the measurements (6). Alternatively, the “top-down” approach was intended to obtain a reliable estimate for the uncertainty, without necessarily knowing any of the sources individually. It relied on the overall reproducibility estimates either from in-house or inter-laboratory measurement trials. Also, a combination of the different approaches were needed often to assess the uncertainty (21). In the present study, the measurement of uncertainty of the method was calculated following a GUM “bottom up approach” that was based on long experience and reflected common practices (4, 22).
As an illustration, the contribution of each component to the total measurement uncertainty of the absorbance measurement in this study is given in Table 3 and Figure 2. We converted all the standard uncertainties into dimensionless relative standard uncertainties and then combined standard uncertainty uC was calculated using propagation principle
Where OD denoted the absorbance of the sample and uC(OD) was combined uncertainty of an ELISA run. RSDsystematic error was obtained from data from 5 different laboratories using reference specimen and RSDRoutine analytical imprecision was obtained from Eqn .9
Combined standard uncertainty and the combined relative standard uncertainty was calculated as 12%.
Expanded uncertainty was calculated by multiplying the combined uncertainty by the coverage factor k, which as usual was chosen to be k=2 to achieve an approximately 95% confidence interval. The relative expanded uncertainty U rel (K=2) was found to be 24%.
In a test, the variations in the reference sample and patient samples were assumed to be equivalent. Thus, this result might be interpreted as the potential variation or MU in the OD following ELISA procedure that was measured as ±24% with 95% confidence interval for a patient sample also. This might have indicated that the result was probably equivocal within the range of ±24% OD value with reference to the cutoff value for qualitative interpretation also.
Estimation of MU is a time consuming and demanding task for laboratories (8). A large number of experiments as well as data are the pre-requisite before all required uncertainties are obtained. The requirement for uncertainty estimation should be considered for the stages of work procedure and measurement method design so that all operations are performed in such a way as to reduce potential uncertainties and simplify their calculation.
Quantification of combined uncertainty by cumulating the contributions arising from routine analytical imprecisions and systematic error using a reference sample may be considered as a practical approach for estimating MU. Where commercial reference material with defined tractability is difficult to obtain, a well-defined in-house reference material tested by different laboratories to obtain the estimation of bias (systematic error) may be an acceptable alternative for resource poor laboratories. The potential for variation or MU in the OD following ELISA procedure, as found in this study, could be ±24% with 95% confidence interval for a patient sample also. This procedure of estimating bias and analytical imprecision may be considered as a generalized scheme for planning a simple and practical method to estimate MU as demonstrated. This method might be able to establish itself, over the years, as a credible time and cost-efficient approach for the determination of MU in ELISA in a serological laboratory.