ASTM D6300 Precision and Bias Data for Use in Test Methods for Petroleum Products and Lubricants
ASTM D6300 Standard Practice for Determination of Precision and Bias Data for Use in Test Methods for Petroleum Products and Lubricants
6. Stages in Planning of an Interlaboratory Test Program for the Determination of the Precision of a Test Method
6.1 The stages in planning an interlaboratory test program are: preparing a draft method of test (see 6.2), planning and executing a pilot program with at least two laboratories (optional but recommended for new test methods) (see 6.3), planning the interlaboratory program (see 6.4), and executing the interlaboratory program (see 6.5). The four stages are described in turn.
6.2 Preparing a Draft Method of Test - This shall contain all the necessary details for carrying out the test and reporting the results. Any condition which could alter the results shall be specified. The section on precision will be included at this stage only as a heading.
6.3 Planning and Executing a Pilot Program with at Least Two Laboratories:
6.3.1 A pilot program is recommended to be used with new test methods for the following reasons: (1) to verify the details in the operation of the test; (2) to find out how well operators can follow the instructions of the test method; (3) to check the precautions regarding sample handling and storage; and (4) to estimate roughly the precision of the test.
6.3.2 At least two samples are required, covering the range of results to which the test is intended to apply; however, include at least 12 laboratory-sample combinations. Test each sample twice by each laboratory under repeatability conditions. If any omissions or inaccuracies in the draft method are revealed, they shall now be corrected. Analyze the results for precision, bias, and determinability (if applicable) using this practice. If any are considered to be too large for the technical application, then consider alterations to the test method.
6.4 Planning the Interlaboratory Program:
6.4.1 There shall be at least five participating laboratories, but it is preferable to exceed this number in order to reduce the number of samples required and to make the precision statement as representative as possible of the qualified user population.
6.4.2 The number of samples shall be sufficient to cover the range of the property measured, and to give reliability to the precision estimates. If any variation of precision with level was observed in the results of the pilot program, then at least five samples shall be used in the interlaboratory program. In any case, it is necessary to obtain at least 30 degrees of freedom in both repeatability and reproducibility. For repeatability, this means obtaining a total of at least 30 pairs of results in the program.
6.4.3 For reproducibility, Fig. 1 gives the minimum number of samples required in terms of L, P, and Q, where L is the number of participating laboratories, and P and Q are the ratios of variance component estimates (see 8.3.1) obtained from the pilot program. Specifically, P is the ratio of the interaction component to the repeats component, and Q is the ratio of the laboratories component to the repeats component.
NOTE 1 - Appendix X1 gives the derivation of the equation used. If Q is much larger than P, then 30 degrees of freedom cannot be achieved; the blank entries in Fig. 1 correspond to this situation or the approach of it (that is, when more than 20 samples are required). For these cases, there is likely to be a significant bias between laboratories. The program organizer shall be informed; further standardization of the test method may be necessary.
6.5 Executing the Interlaboratory Program:
6.5.1 One person shall oversee the entire program, from the distribution of the texts and samples to the final appraisal of the results. He or she shall be familiar with the test method, but should not personally take part in the actual running of the tests.
6.5.2 The text of the test method shall be distributed to all the laboratories in time to raise any queries before the tests begin. If any laboratory wants to practice the test method in advance, this shall be done with samples other than those used in the program.
6.5.3 The samples shall be accumulated, subdivided, and distributed by the organizer, who shall also keep a reserve of each sample for emergencies. It is most important that the individual laboratory portions be homogeneous. Instructions to each laboratory shall include the following:
6.5.3.1 The agreed draft method of test;
6.5.3.2 Material Safety Data Sheets, where applicable, and the handling and storage requirements for the samples;
6.5.3.3 The order in which the samples are to be tested (a different random order for each laboratory);
6.5.3.4 The statement that two test results are to be obtained in the shortest practical period of time on each sample by the same operator with the same apparatus. For statistical reasons it is imperative that the two results are obtained independently of each other, that is, that the second result is not biased by knowledge of the first. If this is regarded as impossible to achieve with the operator concerned, then the pairs of results shall be obtained in a blind fashion, but ensuring that they are carried out in a short period of time (preferably the same day). The term blind fashion means that the operator does not know that the sample is a duplicate of any previous run.
6.5.3.5 The period of time during which repeated results are to be obtained and the period of time during which all the samples are to be tested;
6.5.3.6 A blank form for reporting the results. For each sample, there shall be space for the date of testing, the two results, and any unusual occurrences. The unit of accuracy for reporting the results shall be specified. This should be, if possible, more digits reported than will be used in the final test method, in order to avoid having rounding unduly affect the estimated precision values.
6.5.3.7 When it is required to estimate the determinability, the report form must include space for each of the determined values as well as the test results.
6.5.3.8 A statement that the test shall be carried out under normal conditions, using operators with good experience but not exceptional knowledge; and that the duration of the test shall be the same as normal.
6.5.4 The pilot program operators may take part in the interlaboratory program. If their extra experience in testing a few more samples produces a noticeable effect, it will serve as a warning that the test method is not satisfactory. They shall be identified in the report of the results so that any such effect may be noted.
6.5.5 It can not be overemphasized that the statement of precision in the test method is to apply to test results obtained by running the agreed procedure exactly as written. Therefore, the test method must not be significantly altered after its precision statement is written.
