1.1 This practice covers requirements for the validation of measurements made by laboratory, field, or process (online or at-line) infrared (near- or mid-infrared analyzers, or both), and Raman analyzers, used in the calculation of physical, chemical, or quality parameters (that is, properties) of liquid petroleum products and fuels. The properties are calculated from spectroscopic data using multivariate modeling methods. The requirements include verification of adequate instrument performance, verification of the applicability of the calibration model to the spectrum of the sample under test, and verification that the uncertainties associated with the degree of agreement between the results calculated from the infrared or Raman measurements and the results produced by the PTM used for the development of the calibration model meets user-specified requirements. Initially, a limited number of validation samples representative of current production are used to do a local validation. When there is an adequate number of validation samples with sufficient variation in both property level and sample composition to span the model calibration space, the statistical methodology of Practice D6708 can be used to provide general validation of this equivalence over the complete operating range of the analyzer. For cases where adequate property and composition variation is not achieved, local validation shall continue to be used.

1.1.1 For some applications, the analyzer and PTM are applied to the same material. The application of the multivariate model to the analyzer output (spectrum) directly produces a PPTMR for the same material for which the spectrum was measured. The PPTMRs are compared to the PTMRs measured on the same materials to determine the degree of agreement.

1.1.2 For other applications, the material measured by the analyzer system is subjected to a consistent additive treatment prior to being analyzed by the PTM. The application of the multivariate model to the analyzer output (spectrum) produces a PPTMR for the treated material. The PPTMRs based on the analyzer outputs are compared to the PTMRs measured on the treated materials to determine the degree of agreement.

1.1.3 In some cases, a two-step procedure is employed. In the first step, the analyzer and PTM are applied to the measurement of a blendstock material. In a second step, the PPTMRs produced in Step 1 are used as inputs to a second model that predicts the results obtained when the PTM is applied to the analysis of the finished blended product produced by additivation to the blendstock. If the analyzer used in the first step is a multivariate spectroscopic based analyzer, then this practice is used to access the degree of agreement between PPTMRs and PTMRs. Otherwise, Practice D3764 is used to compare the PPTMRs to the PTMRs for this blendstock to determine the degree of agreement. Since this second step does not use spectroscopic data, the validation of the second step is done using Practice D3764. If the first step uses a multivariate spectrophotometric analyzer, then only samples for which the spectra are not outliers relative to the multivariate model are used in the second step. Note that the second model might accommodate variable levels of additive material addition to the blend stock.

1.2 Multiple physical, chemical, or quality properties of the sample under test are typically predicted from a single spectral measurement. In applying this practice, each property prediction is validated separately. The separate validation procedures for each property may share common features, and be affected by common effects, but the performance of each property prediction is evaluated independently. The user will typically have multiple validation procedures running simultaneously in parallel.

1.3 Results used in analyzer validation are for samples that were not used in the development of the multivariate model, and for spectra which are not outliers or nearest neighbor inliers relative to the multivariate model.

1.4 When the number, composition range or property range of available validation samples do not span the model calibration range, a local validation is done using available samples representative of current production. When the number, composition range and property range of available validation samples becomes comparable to those of the model calibration set, a general validation can be done.

1.4.1 Local Validation: 

1.4.1.1 The calibration samples used in developing the multivariate model must show adequate compositional and property variation to enable the development of a meaningful correlation, and must span the compositional range of samples to be analyzed using the model to ensure that such analyses are done via interpolation rather than extrapolation. The Standard Error of Calibration (SEC) is a measure of how well the PTMRs and PPTMRs agree for this set of calibration samples. SEC includes contributions from spectrum measurement error, PTM measurement error, and model error. Sample (type) specific biases are a part of the model error. Typically, spectroscopic analyzers are very precise, so that spectral measurement error is small relative to the other types of error.

1.4.1.2 During initial analyzer validation, the compositional range of available samples may be small relative to the range of the calibration set. Because of the high precision of the spectroscopic measurement, the average difference between the PTMRs and PPTMRs may reflect a sample (type) specific bias which is statistically observable, but which are less than the uncertainty of PPTMR, U(PPTMR). Therefore, the bias and precision of the PTMR/PPTMR differences are not used as the basis for local validation.

1.4.1.3 Based on SEC, and the leverage statistic, the uncertainty of each PPTMR, U(PPTMR) is calculated. During validation, for each non-outlier sample, a determination is made as to whether the absolute difference between PPTMR and PTMR,