Abstract
Neural architecture search, which aims to automatically search for
architectures (e.g., convolution, max pooling) of neural networks that
maximize validation performance, has achieved remarkable progress
recently. In many application scenarios, several parties would like to
collaboratively search for a shared neural architecture by leveraging
data from all parties. However, due to privacy concerns, no party wants
its data to be seen by other parties. To address this problem, we
propose federated neural architecture search (FNAS), where different
parties collectively search for a differentiable architecture by
exchanging gradients of architecture variables without exposing their
data to other parties. To further preserve privacy, we study
differentially-private FNAS (DP-FNAS), which adds random noise to the
gradients of architecture variables. We provide theoretical guarantees
of DP-FNAS in achieving differential privacy. Experiments show that
DP-FNAS can search highly-performant neural architectures while
protecting the privacy of individual parties. The code is available at
https://github.com/UCSD-AI4H/DP-FNAS