Basic QC & Westgard Rules Application Quinn Hsieh Assistant Product Manager, CDG BiO-Rad Laboratories, Taiwan Branch

Internal Quality Control Program External Quality Assurance Program Inter-laboratory Comparison program Precision Patient Samples Analyzer Analyzer Patient Result External Quality Assurance Program Diagnosis Treatment Accuracy

Statistical Quality Control for Quantitative Measurements: Principles and Definitions CLSI Guideline C24-A3 Volume 19 Number 5

Choosing a QC Procedure Determine quality requirements for the test (TEa) Evaluate test performance (Imprecision, Bias) Identify possible QC procedures, which include determining the: Rules to apply Control materials to be used Number of control samples to be analyzed in each run Location in the run where the control samples are tested (e.g. at the beginning, in the middle, at the end, or distributed throughout the run) Predict the performance of the identified QC procedures (Ped, Pfr) (This step requires probability calculations or computer simulations.) Choose goals based on your required quality Select a QC procedure Obviously, this is not a simple process, which may explain why many laboratories fall back on the 1-2s rule. Although the 1-2s rule produces too many false alarms, it feels safe.

Running Quality Controls What control materials are used How many control samples are analyzed How often control samples should be analyzed Where these control are located What quality control rules are applied

What control materials are used 品管液種類: 實驗室自行製備的品管液(Pool serum) 試劑儀器商品管液(In-kit Control) 第三方品管液(Third Party Control) Control materials need to be different from the calibrator materials to ensure that the QC procedure provides an independent assessment of system performance. (CLSI C24-A3: 2006)

How many levels? Analyte concentrations should be at clinically relevant levels to reflect values encountered in patient specimens At least two levels (concentrations) of control materials is recommended. Confirmation of Reportable Range

How many levels? Cut Off Control

How many levels? Normal Range Control

How many levels? Normal Range Control

How often? Concept of Analytical Run An interval which the accuracy and precision of the measuring system is expected to be stable Quality control samples must be analyzed at least once during each user-defined analytical run length (UDRL). For U.S. laboratories, federal and state regulations set the maximum UDRL as 24 hours More frequently if test is unstable. Commonly, done every 8 hour shift. Run a QC sample whenever you expect the process may changed Change in calibration Change in reagent (new or different lot) Change in machinery (switch on)

Location of Control Samples Start End Analytical Run Drift Random Error Shift

What quality control rules are applied

Quality Control -Quantitative Analysis of QC Data

Calculation of Mean The Mean is the laboratory’s best estimate of the analyte’s true value for a specific level of control X = Mean X1 = First result X2 = Second result Xn = Last result in series n – Total number of results

Calculation of Mean: Outliers 192 mg/dL 194 mg/dL 196 mg/dL 160 mg/dL 200 mg/dL 202 mg/dL 255 mg/dL 204 mg/dL 208 mg/dL 212 mg/dL

Calculation of Mean Mean = the calculated average of the values 192 mg/dL 194 mg/dL 196 mg/dL 200 mg/dL 202 mg/dL 204 mg/dL 208 mg/dL 212 mg/dL Sum = 2,200 mg/dL Mean = the calculated average of the values The sum of the values (X1 + X2 + X3 … X11) divided by the number (n) of observations The mean of these 11 observations is (2200  11) = 200 mg/dL

Standard Deviation (SD) It is the square root of the average squared deviation from the mean

Standard Deviation and Probability For a set of data with a normal distribution, a value will fall within a range of: +/- 1 SD 68.0% of the time +/- 2 SD 95.5% of the time +/- 3 SD 99.7% of the time

Standard Deviation (SD) SD is commonly used since it has the same units as the mean and the original observations. SD is the principle calculation used in the laboratory to measure dispersion of a group of values around a mean. When the SD is high, performance is inconsistent and may indicate a problem or malfunction. SD 表示一群受測值的分佈情形，分佈越集中，標準偏差越小。 標準差是一組數值自平均值分散程度的一种測量觀念,量化你的品管數據彼此間關聯性 SD precision and imprecision 可交替使用都是用來評估一制性及再現性 Read this slide as stated. 當sd值太大表示你這個項目的performance不一致性太大,實驗室需進一步調查原因或故障 標準差的應用： （1）表示變數值的離散程度。標準差越大，變數值分佈越散，均數的代表性越差，即S越大， 代表性越小，反之亦然。但當資料的度量單位不同或均數相差較大時，兩組資料的標準差不能直接相比。 （2）結合均數描述正態分佈特徵。根據正態分佈曲線下面積的規律，可以通過 ±S的倍數形式來概括描述變數值的分佈，對這組資料的頻數分佈做出概括性的估計。 （3）標準差還可以用來計算變異係數及結合樣本含量計算標準誤。

Establishing the Value of the Mean and SD on a New Lot Collect a minimum of 20 data points for each level of control data points must be obtained from 20 separate analytical runs( 4 measurement *5 day ) which reflect variables such as calibration frequency, change of reagent or reagent lot,operator technique, temperature/ humidity of testing location,daily/weekly maintenance, etc. Lyophilized controls, it may be appropriate to use 20 bottles Liquid stable quality control products, fewer bottles may be required All new control product should be compared to previously validated controls (parallel testing). Current Lot New Lot overlap

Establishing the Value of the Mean and SD on a New Lot Use product insert ranges only as guide. Ranges are based on reagent lots and materials available at the time of value assignment. During the life of the control lot,manufacturers may reformulate the test or begin using a new source of raw materials for kit/reagent production. Published ranges cannot account for these occurrences or for variables such as instrumentation software updates or performance differences between laboratories.

Establishing the Value of the Mean and SD on a New Lot

Establishing the Value of the Standard Deviation on a New Lot Calculate the mean and SD from the data points collected . With 20 measurements, the estimate of the standard deviation might vary up to 30% from the true standard deviation With 100 measurements, the estimate may vary by as much as 10% 90 data points are recommended before finalizing mean and standard deviation. If there is a history of quality control data from an extended period of stable operation, the established estimate of the standard deviation should be used with the new lot. Laboratories should establish their own means and ranges rather than use product insert ranges. (CLSI C24-A3: 2006)

Coefficient of Variation, CV The coefficient of variation (CV) is the standard deviation (SD) expressed as a percentage of the mean CV 以百分比表示的標準偏差稱之，C.V.值可直接得知一組數群的變異程度。 由於計算變異數（或標準差）時，因為每個值都要減去平均數，因此變異數必然受到平均數的影響。為了避免變異指標受到平均數的影響，將標準差除以平均數，形成變異係數。 平均數M ：一群數值資料的集中趨勢。 標準差S ：一群數值資料的內部變異情形。 變異係數CV ：不同數值資料的相對變異大小。 CV值是用來測量變異程度RANDOM ERROR或IMPRECISION CV值是SD與MEAN的比值,通常以百分比呈現 公式>>> 因為是以百分比呈現所以可以用來比較分析項目在不同濃度時的precision的好壞 舉例 也可以用來作為選擇不同分析方法或評估儀器時的參考 The Coefficient of Variation or CV is the ratio of the standard deviation to the mean and is expressed as a percentage. It is a measure of variability that can be used to standardize results. The CV can be used to make results at low and high concentrations comparable to assess precision at all levels of an assay. It can also be used to compare precision between two different methods. 變異係數（coefficient of variation, 簡記為CV），亦稱離散係數（coefficient of dispersion）。 全距、標準差與變數值的單位相同，而變異係數是相對比的，沒有單位，更便於資料間的分析比較。 常用於： （1）比較均數相差懸殊的幾組資料的變異度，如相同度量衡單位指標的不同時間的縱向比較。 （2）比較度量衡單位不同的多組資料的變異度，即做相同時間不同指標的橫向比較。 （3）變異係數還常用於比較多個樣品重複測定的誤差。

“True” picture? 99.9% of values are within +/-1SD SD used is not “true” This will caused reduce true error detection.

True Chart 5% of points supposed to be more than + 2SD 1 out of 20 points Only 68% of the points supposed to be within +1SD

Why Establishing Your Own Range

Interpreting Multi-Rules Monitoring QC Data Interpreting Multi-Rules (Westgard Rules)

Monitoring Reviewing

12S : A warning rule 1-2s: This is a warning rule which is violated when a single control observation is outside the ±2s limit. If one control measurement exceeds the mean ± 2 standard deviations, you should evaluate other controls in the run (within control material) and in previous runs (across control material) before accepting the run and reporting the results.

13S : Rejection 1-3s: Violation of this rule indicates random error and may also point to systematic error. The run is considered out of control when one control value exceeds the mean ± 3s. This rule is applied within control material only.

2 2S: Rejection 2-2s: This rule detects systematic error and is applied within and across runs. It is violated within the run when two consecutive control values exceed the "same" (mean + 2s or mean - 2s) limit.

2 2S: Rejection The rule is violated across runs when the previous value for a particular control level exceeds the "same" (mean + 2s or mean - 2s) limit.

(2 of 3)-2s: Rejection (2 of 3)-2s: This is a variation of the 2-2s rule and detects systematic errors. It is triggered when any two of all three levels of control in a run exceed 2s on the same side of the mean.

R4S : Rejection R-4s: This rule identifies random error only, and is applied only within the current run. If there is a 4s difference between control values within a single run, the rule is violated for random error.

41S : Reviewing 4-1s: This rule detects systematic bias and is applied both within and across control materials. They are violated within the control material if the last 4 values of the same control level are within the "same" (mean + 1s or mean - 1s) limit.

41S : Reviewing 4-1s: They are violated across the control material if the last 4 consecutive control values for different control levels are within the "same" (mean + 1s or mean - 1s) limit.

10x : Reviewing 10-x: This rule detects systematic bias and is applied both within and across control materials. The rule is violated within the control materials if the last 10 values for the same control level are on the same side of the mean.

10x : Reviewing 10-x: The rule is violated across control materials if the last 10 consecutive values, regardless of control level, are on the same side of the mean.

Which rules tell about which kinds of errors? Random error Rules that look for points that are in the tails or the distribution or measure the width of the distribution 13s, R4s Systematic error Rules that count consecutive measurements beyond a certain limit 22s, 41s, 10x

Why use a multi-rule QC procedure? More complicated than single rule Provide better performance than 12S single-rule 5% of all analytical runs when using one level of control ; 10% of all analytical runs when using two levels of control; 14% of all analytical runs when using three levels of control.

Why bother with multirule QC? Better error detection! Lower false rejections! Additional information about type of error occurring, which will aid in problem-solving and trouble-shooting Rules are logical and make sense to well trained analysts 這些rule雖然繁瑣但是有其規則性及邏輯性,所以在實驗室尚未完成評估sigma或 critical SE前建議這些RULE要持續的使用

Conclusion Define, calculate and apply the following statistics: mean, standard deviation, coefficient of variation Describe, choose and apply each of the Westgard rules Identify which Westgard rules identify random error and which rules identify systematic error.

Thank you