12. Calibration of Test Method
12.1 Requirements - To maintain test consistency and severity levels, engine test stand calibrations are required at regular intervals.
12.2 Reference Oils - The reference oils used to calibrate 1M-PC test stands have been formulated or selected to represent specific chemical types and performance levels. They are available from the ASTM TMC. The TMC will assign reference oils for calibration tests. These oils are supplied under code numbers (blind reference oils).
12.3 Test Numbering:
12.3.1 Number each 1M-PC test to identify the test stand number and the test run number. Number all runs sequentially. Append repeat calibration runs with a letter (also sequential). Maintain the letter suffix sequencing for each test type calibration until the calibration has been accepted. Any test start, regardless of type, increments the run number. Test start is defined in 12.4.
12.3.2 An example of test numbering:
12.4 Definition of Test - A test (or test start) is defined, for purposes of this test method, as any engine test time accumulated in accordance with this test method.
12.5 New Laboratories and New Test Stands:
12.5.1 New Test Stands - A new stand is defined as a test engine and support hardware that has never been previously calibrated under this test method.
12.5.2 New Laboratory - A new laboratory shall have two calibration test passes on approved reference oils to be considered valid.
12.5.3 Special Circumstances - A laboratory not running a 1M-PC test for twelve months from the start of the last test is considered a new laboratory. Under special circumstances (that is, extended downtime due to industry-wide parts shortage or fuel outages) the TMC may extend the lapsed time requirement. Annotate non-reference tests conducted during an extended time allowance in the comments section of the Unscheduled Downtime and Maintenance Summary form of the test report.
12.6 Frequency of Calibration Tests:
12.6.1 A calibration test on a reference oil assigned by the TMC is required after no more than 14 test starts or after six months from the start date of the last acceptable calibration test (whichever comes first). The 1M-PC calibration run is not counted as one of the 14 test starts; however, all other test starts are counted. The TMC is permitted to move up or extend reference tests to enhance reference test program design and test severity monitoring. Ifa reference test calibration period is extended beyond the normal duration, note this fact in the comments section of the Unscheduled Downtime and Maintenance Summary form of the test report for all subsequent non-reference tests. Also attach written confirmation from the TMC to the report.
12.6.2 Start any non-reference test prior to the expiration of the calibration period.
12.7 Guidelines for Adjustments to Calibration Periods - Reference oil test frequency may be adjusted for the following reasons:
12.7.1 Procedural Deviations - On occasions when a laboratory becomes aware of a significant deviation from the test method, such as might arise during an in-house review or a TMC inspection, the laboratory and the TMC shall agree on an appropriate course of action to remedy the deviation. This action may include the shortening of existing reference oil calibration periods.
12.7.2 Parts and Fuel Shortages - Under special
circumstances, such as industry-wide parts or fuel shortages, the surveillance panel may direct the TMC to extend the time intervals between reference oil tests. These extensions shall not exceed one regular calibration period.
12.7.3 Reference Oil Test Data Flow - To ensure continuous severity and precision monitoring, calibration tests are conducted periodically throughout the year. There may be occasions when laboratories conduct a large portion of calibration tests in a short period of time. This could result in an unacceptably large time frame when very few calibration tests are conducted. The TMC can shorten or extend calibration periods as needed to provide a consistent flow of reference oil test data. Adjustments to calibration periods are made such that laboratories incur no net loss (or gain) in calibration status.
12.7.4 Special Use of the Reference Oil Calibration System - The surveillance panel has the option to use the reference oil system to evaluate changes that have potential impact on test severity and precision. This option is only taken when a program of donated tests is not feasible. The surveillance panel and the TMC shall develop a detailed plan for the test program. This plan requires all reference oil tests in the program to be completed as close to the same time as possible, so that no laboratory/stand calibration is left in an excessively long pending status. In order to maintain the integrity of the reference oil monitoring system, each reference oil test is conducted so as to be interpretable for stand calibration. To facilitate the required test scheduling, the surveillance panel may direct the TMC to lengthen and shorten reference oil calibration periods within laboratories such that the laboratories incur no net loss (or gain) in calibration status.
12.8 Donated Reference Oil Test Programs - The Surveillance Panel is charged with maintaining effective reference oil test severity and precision monitoring. During times of new parts introductions, new or re-blended reference oil additions, and procedural revisions, it may be necessary to evaluate the possible effects on severity and precision levels. The surveillance panel may choose to conduct a program of donated reference oil tests in those laboratories participating in the monitoring system, in order to quantify the effect of a particular change on severity and precision. Typically, the surveillance panel requests its panel members to volunteer enough reference oil test results to create a robust data set. Broad laboratory participation is needed to provide a representative sampling of the industry. To ensure the quality of the data obtained, donated tests are conducted on calibrated test stands. The surveillance panel shall arrange an appropriate number of donated tests and ensure completion of the test program in a timely manner.
12.9 Procedural Deviations - On occasions when a laboratory becomes aware of a significant deviation from the test method, such as might arise during an in-house review or a TMC inspection, the laboratory and the TMC shall agree on an appropriate course of action to remedy the deviation. This action may include the shortening of existing reference oil calibration periods.
12.10 Specified Test Parameters - The specified test parameters for determination of test acceptance are:
12.10.1 Top groove fill, percent area.
12.10.2 Weighted total deposits, demerits.
12.11 Acceptance of Calibration Tests - Refer to the TMC's Lubricant Test Monitoring System (LTMS) for calibration test targets and acceptance criteria.
12.12 Failing Reference Oil Calibration Tests:
12.12.1 Failure of a calibration test to meet test acceptance bands can indicate a testing stand problem, testing laboratory problem, or industry-wide problem, or it can be a false alarm. When this occurs, the laboratory, in conjunction with the TMC, must attempt to determine the problem source.
12.12.2 In the determination of the problem, TMC will decide, with input as needed from industry expertise (testing laboratories, test developer, ASTM Technical Guidance Committee, Surveillance Panel, and so forth), if the reason for any unacceptable blind reference oil test is isolated to one particular stand or related to other stands. If it is decided that the problem is isolated to an individual stand, calibrated testing on other stands can continue throughout the laboratory. Alternatively, if it is decided that more than one stand may be involved, the involved stands will not be considered calibrated until the problem is identified and corrected and an acceptable reference oil test is completed in one of the involved stands.
12.13 Non-Standard Tests - If non-standard tests are conducted on the referenced test stand, the stand may, at the discretion of the TMC, be required to be recalibrated prior to running standard tests.
12.14 Severity Adjustments and Control Charting:
12.14.1 Severity Adjustments - This test method incorporates the use of a surveillance panel accepted method of calculating a severity adjustment (SA) for non-reference test results. A control chart technique, described in 12.14.2, has been selected for the purpose of identifying when a bias becomes significant. When a significant bias is identified, an SA is applied to non-reference test results. The SA remains in effect until subsequent calibration test results indicate that the bias is no longer significant. SAs are calculated and applied on a laboratory basis.
12.14.2 Control Chart Technique for SAs - Apply an exponentially weighted moving average (EWMA) technique to standardized calibration test results. Standardize values using Δ/s (result - target)/standard deviation). The targets and standard deviations are published by the TMC. Include all operationally valid calibration tests in a laboratory control chart. Chart tests in order of completion. Record completion of tests by end of test (EOT) date and time. Report EOT as hour and minute in accordance with the 24-h clock (1 a.m. = 1:00, 1 p.m = 13:00). Reporting test completion time enables the TMC to properly order tests that are completed on the same day for industry plotting purposes. Report calibration tests to the TMC in order of test completion. A minimum of two tests is required to initialize a control chart. Calculate EWMA values, using the following equation:
If the absolute value of EWMA, rounded to three places after the decimal, exceeds the alarm limit established in the LTMS document, then apply an SA to subsequent non-reference results.
12.14.3 Calculation of SA - Compute and apply EWMA and SA values as shown in the following example. Please note that test targets are presented for example only.
12.14.3.1 Severity Adjustment Calculation Example:
Since 0.896 > 0.653, apply an SA:SA = -1 * EWMA * STD (in the above example, SA = −14). For TGF, the SA is rounded to a whole percent; for WTD, it is rounded to one decimal place. Enter this number in the appropriate Lab Severity Adjustment box and add it to the Unadjusted Lab Rating on the Test Report Summary form of the test report. An SA will remain in effect until the next calibration test. At that time, calculate a new EWMA and SA.
12.15 Test Reporting:
12.15.1 Report Forms - For reference oil tests, the standardized report forms and data dictionary for reporting test results and for summarizing the operational data are required. All report forms making up the 1M-PC final report are available at the TMC website (http://www.astmtmc.cmu.edu). For calibration tests, attach the control chart summary page sent to the lab from the TMC to the test report. An example of this and other forms are shown in Fig. X2.1 of Appendix X2 and Fig. X3.1 of Appendix X3.
12.15.2 Deviation Percent and Offset Percent Calculation - Offset percent measures how close any given test parameter is run to the target mean. Deviation percent indicates excursions made by any given parameter outside the minimum or maximum limit. Calculate them as follows:
12.15.2.1 Round recorded values, if necessary, in accordance with the specifications listed on the U.S. Customary System (USCS) and SI specifications given in Table 3.
12.15.2.2 Use the test specifications and tolerances listed in Table 3 for the percent calculations.
12.15.2.3 Calculate the percent out and percent off, using the same units as the recorded data. For example, if the test is operated in USCS units, calculate the percentages using USCS units. Do not convert the units before figuring the percentages.
12.15.2.4 The logging frequency used for calculating the percentages will be at the discretion of the laboratory, but shall be at least hourly.
12.15.2.5 Include an explanation for any data used in the calculation of the percentages that are edited. List the data before they are edited, the new value, and the explanation for the change in the comments section of the Unscheduled Downtime and Maintenance Summary form of the test report.
12.15.2.6 Record these values on the Operational Summary - Offset And Deviation form of the test report.
12.15.2.7 Carry each percent out calculation to three significant digits (see Table 9).
12.15.2.8 Round the calculated average used in the percent off calculation to the measurement resolution shown in Table 2. See example in Table 9.
12.15.2.9 Round the percent out summation and percent off results to the minimum measurement resolution listed in Table 2 (see example in Table 9).
NOTE 3 - Use ASTM rounding rules when carrying out 12.15.2.7 through 12.15.2.9.
12.15.2.10 An example of the calculation for percent out for test hour eleven, using the formula shown in 12.15.2.11, is as follows:
12.15.2.11 Use the following formula to calculate the percent out:
where:
A = recorded test measurement of parameter that is beyond test limits prior to any corrective action,
B = upper test spec ifthe measured parameter is out on the high side, and the lower test spec ifit is out on the low side,
C = specification tolerance of the measured parameter,
D = length of deviation in minutes (it cannot be less than the logging frequency),
60 = conversion factor for min/h, and
100 = conversion factor for percentage units.
Calculate the percent out for each measured parameter, based on its logging frequency. Sum the individual percent out's to arrive at the final percent out for judging test validity (see Table 10).
12.15.2.12 Use the following formula to calculate the percent off:
12.15.3 Electronic Data Communication and Data Dictionary - Use the data dictionary and report forms available at http://www.dgaequipment.com for any electronic transmission of data to the TMC. The data dictionary lists all variable names given to all fields as well as the important information about those fields.
12.16 Reporting Reference Results - Transmit the calibration test results by fax or electronic data transmission to the ASTM TMC immediately after completion of test analysis. Send these data within seven days of EOT or the test will be considered invalid.
12.16.1 Though referee ratings are required for all operationally valid tests reported to TMC, statistical validity of the test is computed using the ratings produced by the test lab; referee ratings are used only to reveal rating errors. Report referee results to the TMC on the Referee Rating form of the test report within ten working days of the test completion.
12.16.2 The TMC will review all calibration test results to determine test acceptability. Ifthe test is judged acceptable, the reference oil code along with the industry average for the reference oil, will be disclosed by the TMC. In the event the reference oil test is not acceptable, an explanation of the problem relating to the failure is to be provided by the test laboratory. If the problem is not obvious, recheck all test-related equipment. If no explanation of the problem is presented, it will be assumed that the problem is laboratory-related and another reference oil will be assigned.
12.17 Analysis of Reference Oils - Do not submit reference oils to physical or chemical analyses for identification purposes. Identifying the oils by analyses could undermine the confidentiality required to operate an effective blind reference oil system. Therefore, reference oils are supplied with the explicit understanding that they will not be subjected to analyses other than those specified within this test method unless specifically authorized by the ASTM TMC. In such cases in which analyses are authorized, supply written confirmation of the circumstances involved, the data obtained, and the name of the person authorizing the analysis to the ASTM TMC.