CLSI EP15
User Verification of Precision and Estimation of Bias
This document provides a protocol for estimating imprecision and bias in clinical laboratory quantitative measurement procedures. The protocol can be completed in as few as five days.
This reaffirmed document has been reviewed and confirmed as suitable to remain published without revision to content, as of September 2019.
{{FormatPrice(currentPrice)}}
Free
{{FormatPrice(nonMemberPrice)}} List PriceClinical and Laboratory Standards Institute document EP15-A3—User Verification of Precision and Estimation of Bias; Approved Guideline—Third Edition describes the verification of precision claims and estimation of relative bias for quantitative methods performed within the laboratory. Included are guidelines for duration, experimental designs, materials, data analysis summarization, and interpretation—techniques adaptable for the widest possible range of analytes and device complexity. A balance is created in the document between the complexity of design and formulae, and the simplicity of operation. The protocol is designed to be completed within five working days based on a uniform experimental design yielding estimates of imprecision and bias.
In this revision of EP15, the experiment to demonstrate trueness using materials with known concentrations was expanded to five days, with encouragement to work with the same sample materials used in the precision verification experiment. The intention of the document development committee was for the user to perform a single experiment to verify precision and trueness simultaneously. This experiment is designed to produce reliable estimates of bias between the mean measurand concentration observed by use of the candidate measurement procedure and the assigned measurand concentration of the material. The degree to which the observed bias is a measurement of trueness depends on the quality of the measurement procedure used to assign the measurand concentrations of the material. As with the precision experiment, complicated calculations were replaced by tables wherever possible.
Similar to previous editions of the document, the document development committee had two principal goals during the development of EP15. One goal was to develop a testing protocol that is suitable for use in the large clinical laboratory, yet simple enough to be applicable in the point-of-care or physician’s office laboratory. The second goal was to develop a protocol that is sufficiently rigorous to provide statistically valid conclusions for verification studies. The bias is assessed by a recovery experiment. Instead of manual worksheets, calculations may be readily performed with CLSI’s StatisPro software or generic spreadsheet software (see recommendation below).
The committee feels that it is important to provide the interested user with an explanation of the statistical procedures that are used in the document. If the user has access to software specifically designed to perform the calculations described in the document, such as StatisPro, a detailed understanding of the statistics is not necessary. Flow charts are included to provide the user with the necessary overview of the experiment and data processing. In any case, the user must follow the protocol described as closely as possible in order to obtain reliable results.
This guideline was developed as a protocol for simultaneously verifying a manufacturer’s claims for precision of a measurement procedure and the trueness of the measurement procedure relative to the assigned values of materials with known concentrations.
The precision verification section of the guideline was developed for situations in which the performance of the procedure has been previously established and documented by experimental protocols with larger scope and duration. It has relatively weak power to reject precision claims with statistical confidence, and should only be used to verify that the procedure is operating in accordance with the manufacturer’s claims. This document is not intended to establish or validate the precision performance of a measurement procedure.
The bias estimation section of the guideline relies on 25 or more measurements by the candidate procedure, made over five or more days, to estimate the measurand concentrations of materials with known concentrations. These estimated measurand concentrations are compared to the assigned measurand concentrations of the materials to estimate bias. The observed bias is a measure of trueness if a high-quality measurement procedure was used to assign the concentrations of the materials.
Because this document’s scope is limited to verification of precision and estimation of bias, other more rigorous CLSI protocols (eg, see CLSI documents EP06, EP17, and EP28) are employed to validate the measurement procedure’s performance against the user’s needs. CLSI documents EP05 and EP09 were developed to assist manufacturers in establishing the performance of a diagnostic device for precision and trueness, respectively. (Also, see CLSI documents EP06, EP17, EP28 and EP10.) CLSI document EP10 is intended for the rapid preliminary evaluation of precision, bias, sample carryover, drift, and nonlinearity.
One may also note that the EP15 protocol has an implicit assumption: Namely, that if the estimated precision and bias are acceptable, then the overall error (eg, total analytical error) of the measurement procedure is acceptable. This implied model can lead to an underestimation of the total analytical error in cases in which other effects are important. Besides conducting more extensive evaluations mentioned above, one could also consider performing the protocol within CLSI document EP21. This protocol is a direct estimation of total analytical error, and does not rely on a model.
The U.S. Food and Drug Administration (FDA) has evaluated and recognized this approved-level consensus standard for use in satisfying a regulatory requirement.
A CLSI-IFCC joint project.
This document is available in electronic format only.
Clinical and Laboratory Standards Institute document EP15-A3—User Verification of Precision and Estimation of Bias; Approved Guideline—Third Edition describes the verification of precision claims and estimation of relative bias for quantitative methods performed within the laboratory. Included are guidelines for duration, experimental designs, materials, data analysis summarization, and interpretation—techniques adaptable for the widest possible range of analytes and device complexity. A balance is created in the document between the complexity of design and formulae, and the simplicity of operation. The protocol is designed to be completed within five working days based on a uniform experimental design yielding estimates of imprecision and bias.
In this revision of EP15, the experiment to demonstrate trueness using materials with known concentrations was expanded to five days, with encouragement to work with the same sample materials used in the precision verification experiment. The intention of the document development committee was for the user to perform a single experiment to verify precision and trueness simultaneously. This experiment is designed to produce reliable estimates of bias between the mean measurand concentration observed by use of the candidate measurement procedure and the assigned measurand concentration of the material. The degree to which the observed bias is a measurement of trueness depends on the quality of the measurement procedure used to assign the measurand concentrations of the material. As with the precision experiment, complicated calculations were replaced by tables wherever possible.
Similar to previous editions of the document, the document development committee had two principal goals during the development of EP15. One goal was to develop a testing protocol that is suitable for use in the large clinical laboratory, yet simple enough to be applicable in the point-of-care or physician’s office laboratory. The second goal was to develop a protocol that is sufficiently rigorous to provide statistically valid conclusions for verification studies. The bias is assessed by a recovery experiment. Instead of manual worksheets, calculations may be readily performed with CLSI’s StatisPro software or generic spreadsheet software (see recommendation below).
The committee feels that it is important to provide the interested user with an explanation of the statistical procedures that are used in the document. If the user has access to software specifically designed to perform the calculations described in the document, such as StatisPro, a detailed understanding of the statistics is not necessary. Flow charts are included to provide the user with the necessary overview of the experiment and data processing. In any case, the user must follow the protocol described as closely as possible in order to obtain reliable results.
This guideline was developed as a protocol for simultaneously verifying a manufacturer’s claims for precision of a measurement procedure and the trueness of the measurement procedure relative to the assigned values of materials with known concentrations.
The precision verification section of the guideline was developed for situations in which the performance of the procedure has been previously established and documented by experimental protocols with larger scope and duration. It has relatively weak power to reject precision claims with statistical confidence, and should only be used to verify that the procedure is operating in accordance with the manufacturer’s claims. This document is not intended to establish or validate the precision performance of a measurement procedure.
The bias estimation section of the guideline relies on 25 or more measurements by the candidate procedure, made over five or more days, to estimate the measurand concentrations of materials with known concentrations. These estimated measurand concentrations are compared to the assigned measurand concentrations of the materials to estimate bias. The observed bias is a measure of trueness if a high-quality measurement procedure was used to assign the concentrations of the materials.
Because this document’s scope is limited to verification of precision and estimation of bias, other more rigorous CLSI protocols (eg, see CLSI documents EP06, EP17, and EP28) are employed to validate the measurement procedure’s performance against the user’s needs. CLSI documents EP05 and EP09 were developed to assist manufacturers in establishing the performance of a diagnostic device for precision and trueness, respectively. (Also, see CLSI documents EP06, EP17, EP28 and EP10.) CLSI document EP10 is intended for the rapid preliminary evaluation of precision, bias, sample carryover, drift, and nonlinearity.
One may also note that the EP15 protocol has an implicit assumption: Namely, that if the estimated precision and bias are acceptable, then the overall error (eg, total analytical error) of the measurement procedure is acceptable. This implied model can lead to an underestimation of the total analytical error in cases in which other effects are important. Besides conducting more extensive evaluations mentioned above, one could also consider performing the protocol within CLSI document EP21. This protocol is a direct estimation of total analytical error, and does not rely on a model.
The U.S. Food and Drug Administration (FDA) has evaluated and recognized this approved-level consensus standard for use in satisfying a regulatory requirement.
A CLSI-IFCC joint project.
This document is available in electronic format only.