Conformal Prediction for Deep Classifier via Label Ranking
ShanghaiTech University · Southern University of Science and Technology · Rutgers University · UNC-Chapel Hill / Microsoft · College of Mathematics and Statistics, Chongqing University and Key Laboratory of Nonlinear Analysis and its Applications (Chongqing University), Ministry of Education, China · Nanyang Technological University
PDF 由论文原始站点提供，PaperCompass 不保存论文文件。
摘要
Conformal prediction is a statistical framework that generates prediction sets containing ground-truth labels with a desired coverage guarantee. The predicted probabilities produced by machine learning models are generally miscalibrated, leading to large prediction sets in conformal prediction. To address this issue, we propose a novel algorithm named $\textit{Sorted Adaptive Prediction Sets}$ (SAPS), which discards all the probability values except for the maximum softmax probability. The key idea behind SAPS is to minimize the dependence of the non-conformity score on the probability values while retaining the uncertainty information. In this manner, SAPS can produce compact prediction sets and communicate instance-wise uncertainty. Extensive experiments validate that SAPS not only lessens the prediction sets but also broadly enhances the conditional coverage rate of prediction sets.