TechRxiv
Dataset_Shift_IEEE_TechRxiv.pdf (237.33 kB)
Download file

Detecting Distributional Shift Responsible for Predictive Model’s Failure*

Download (237.33 kB)
preprint
posted on 27.09.2021, 15:35 by Dipanwita Sinha MukherjeeDipanwita Sinha Mukherjee, Divyanshu Bhandari, Naveen Yeri
Any predictive software deployed with this hypothesis that test data distribution will not differ from training data distribution. Real time scenario does not follow this rule, which results inconsistent and non-transferable observation in various cases. This makes the dataset shift, a growing concern. In this paper, we’ve explored the recent concept of Label shift detection and classifier correction with the help of Black Box shift detection(BBSD), Black Box shift estimation(BBSE) and Black Box shift correction(BBSC). Digits dataset from ”sklearn” and ”LogisticRegression” classifier have been used for this investigation. Knock out shift was clearly detected by applying Kolmogorov–Smirnov test for BBSD. Performance of the classifier got improved after applying BBSE and BBSC from 91% to 97% in terms of overall accuracy.

Funding

Wells Fargo International Solutions Private Limited, Bangalore, India

History

Email Address of Submitting Author

dipanwitas@alum.iisc.ac.in

Submitting Author's Institution

Artificial Intelligence - Center of Excellence, Wells Fargo International Solutions Private Limited, Bangalore, India

Submitting Author's Country

India

Usage metrics

Exports