化学化工异常数据检测方法研究

论文编号:HG116  论文字数:13446,页数:27

摘 要：在化学化工实验领域，由于不完善的数据采集设备、数据输入有误、传输错误、测量单位混乱、灵敏度不够、手工输入过程中丢失数据等原因，常常导致异常数据的产生，这将给信息的整体应用带来不良后果。为此，本文在调研目前常用的几种异常数据检测方法基础上，选用基于距离的方法中的非线性迭代偏最小二乘法，该法通过提取主成分后，能消除了光谱数据间的严重复共线性，从而使得模型的输入维数得到显著下降。最后，本文将偏最小二乘法实际应用鱼类近红外光谱数据试验，构建光谱数据与脂肪含量的定量预测模型，结果显示，它同多元线性回归方法比较，不仅识别出异常数据准确率高，而且误判率也较低。
 论文还分析了研究工作的不足，并展望了今后的发展。
 关键词：异常数据；偏最小二乘法；多元线性回归；近红外光谱
  
 
 
Study on the Outliers Recognition Method for the Chemistry and Chemical Data
Abstract: In the field of the chemistry and chemical engineering, due to the incomplete of data collection equipment, data input error, error transfers, confusion measure unit, inadequate delicacy, handcraft import course lose data and so on, outliers often be brought. The outliers often bring out the whole information application kickback. So in the article, the partial least squares (PLS) method based on the distance was selected to study and recognize the outliers from the whole sample data. The PLS method derived several components from the independent variables and thus eliminated spectrum data the severity collinearity. Thereby the PLS method brought the low dimension model and the model became concise. Finally, the proposed PLS method application to the analysis of fish data near infrared spectroscopy which dealt with spectrum data to get along with ration forecast model of fatness content . The result of the PLS method was presented with comparison to the MLR. The PLS method not only gets the high right ratio of the outlier, but also holds lower ratio of the right samples.
 The paper has also analyzed the research work insufficiency, and has prospected the next development of the outliers recognition method
 Keywords：Outliers; Partial least squares; Multiple Linear Regression; NIR
 
  
 
 
目  录
中文摘要 I
英文摘要 II
目录 III
1. 绪论 1
 1.1 数据挖掘知识概述 1
 1.2 异常数据发现 1
 1.3 现有异常数据的检测方法 3
 1.4 本章小结 7
2. 基于距离的PLS方法 8
 2.1 PLS简介 8
 2.2 PLS的基本原理 8
     2.3 多元线性回归的原理 13
     2.4 本章小结 15
3. PLS用于鱼类脂肪含量的光谱数据分析 16
 3.1 样本数据说明 16
 3.2 试验结果分析 17
 3.3 结果讨论 18
     3.4 本章小结 19
4. 总结与展望 20
致谢 22
参考文献 23