The precise measurement of sleep stages is crucial for evaluating sleep quality. Deep learning has recently been utilized for automatic sleep stage classification, demonstrating exceptional performance. Previous studies on deep learning-based sleep stage classification assumed stationary data generation environments, where samples were drawn from fixed – albeit unknown – unknown distributions and annotated based on predefined criteria. However, this assumption of a stationary distribution is no longer valid in real-world applications due to shifts in data distribution between new and old datasets and changes in classification tasks. Moreover, the unavailability of historical data often poses challenges in training deep learning models. Sleep stage classification faces challenges associated with evolving data acquisition, changing patient demographics, and shifting annotation criteria over time. This paper addresses the classification of sleep stages with varying data distributions, missing historical datasets, and changing label granularity for the first time. We proposed learning strategies for addressing the challenges described above, as well as constructed benchmarks for evaluating the proposed learning strategies. The results demonstrate the effectiveness and performance of the proposed learning strategies. These findings provide compelling evidence for the significance and impact of this work. Furthermore, a comprehensive discussion is presented, highlighting the limitations of our approach, and proposing several avenues for future research.