Rocchetta_mey_Oliehoek_2022.pdf (2.16 MB)
Download file

Exploring the link between scenario theory, complexity- and compression-based learning

Download (2.16 MB)
posted on 2022-12-20, 01:11 authored by Roberto RocchettaRoberto Rocchetta, Mey Alexander, F.A. OliehoekF.A. Oliehoek

This work investigates generalization error bounds that apply to general classes of learning problems and focuses on recently observed parallels between PAC-learning bounds, like compression and complexity-based bounds, and novel error guarantees derived within Scenario theory. Scenario theory renders non-asymptotically and distributional free generalization error bounds for models trained by solving data-driven decision-making problems. Scenario bounds, like complexity-based bounds, can be derived before solving the learning problem from an indicator of the model’s expressiveness and richness or, like compression bounds, can apply to infinite Vapnik–Chervonenkis dimensional problems and on data-dependent hypothesis spaces. Relevant theorems and assumptions are reviewed and discussed, and a comparison of the quality of the bounds proposed for Support Vector Machines (SVMs) classifiers. We analyze the tightness of the bounds numerically for SVM trained on several randomized data sets from thirteen real-life classification problems. This analysis allows for a fair comparison of different approaches from both conceptual and experimental standpoints. Based on the numerical results, we argue that scenario bounds appear to be tighter in many cases and are always bounded in [0,1], hence non-vacuous, and are better in predicting the performance order for different hyper-parameter settings. This work shows that scenario bounds can be a powerful tool to support model selection, structural-risk mini 


Email Address of Submitting Author

ORCID of Submitting Author

Submitting Author's Institution


Submitting Author's Country

  • Switzerland