TechRxiv
release-2.pdf (467.56 kB)

Learning by Teaching, with Application to Neural Architecture Search

Download (467.56 kB)
preprint
posted on 13.03.2021, 15:22 by Parth Sheth, Yueyu Jiang, Pengtao Xie
Learning by teaching is a broadly used methodology in human learning and shows great effectiveness in improving learning outcome: a learner deepens his/her understanding of a topic by teaching this topic to others. We are interested in investigating whether this powerful learning technique can be borrowed from humans to improve the learning abilities of machines. We propose a novel machine learning approach called learning by teaching (LBT). In our approach, the teacher creates a pseudo-labeled dataset and uses it to train a student model. Based on how the student performs on the validation dataset, the teacher re-learns its model and re-teaches the student until the student achieves great validation performance. We propose a multi-level optimization framework to formulate LBT which involves three learning stages: teacher learns; teacher teaches student; teacher and student validate themselves. We develop an efficient algorithm to solve the LBT problem. We apply our approach to neural architecture search on CIFAR-100, CIFAR-10, and ImageNet, where the results demonstrate the effectiveness of our method.

History

Email Address of Submitting Author

p1xie@eng.ucsd.edu

Submitting Author's Institution

UC San Diego

Submitting Author's Country

United States of America

Usage metrics

Licence

Exports