Research on Bias Detection Methods for Artificial Intelligence Model Training Data
人工智能模型训练数据偏差检测方法研究

摘要

训练数据的样本失衡、历史偏见、标签噪声和代理变量关联会被人工智能模型学习并放大，仅依赖准确 率或单一公平指标难以及时发现偏差。针对该问题，提出一种面向模型训练前审计的多维训练数据偏差检测方法。 该方法从群体代表性、特征分布、标签一致性、群体公平和反事实敏感性五个维度构建指标，经过方向统一、标准 化与权重学习形成综合偏差得分，同时输出偏差群体、关联特征和疑似异常标签。采用包含1200个数据集窗口的 可控仿真进行验证，每个窗口含600个样本，并注入代表性偏差、特征偏移、标签偏差、代理偏差和混合偏差。结 果表明，所提方法的精确率、召回率、F1值和AUC分别达到0.953、0.971、0.962和0.993，对多类型偏差的识别 能力优于样本比例、统计均等和敏感系数等单指标方法。

Abstract

Sample imbalance, historical bias, label noise, and proxy-variable associations in training data can be learned and amplified by artificial intelligence models, and reliance on accuracy or a single fairness metric alone makes it difficult to detect bias in a timely manner. To address this problem, this paper proposes a multidimensional training data bias detection method for pre-training model audits. The method constructs indicators from five dimensions: group representativeness, feature distribution, label consistency, group fairness, and counterfactual sensitivity. Through direction unification, standardization, and weight learning, a comprehensive bias score is formed, while biased groups, associated features, and suspected abnormal labels are also output. Validation is conducted using a controllable simulation containing 1,200 dataset windows, each with 600 samples, with representativeness bias, feature shift, label bias, proxy bias, and mixed bias injected. The results show that the proposed method achieves a precision, recall, F1 score, and AUC of 0.953, 0.971, 0.962, and 0.993, respectively, and outperforms single-indicator methods such as sample proportion, statistical parity, and sensitivity coefficient in identifying multiple types of bias.