Sleep stage classification is pivotal for assessing sleep quality and diagnosing sleep disorders. While deep learning has shown promise in automating sleep stage classification, the dynamic nature of sleep research and data collection processes presents unique challenges. These include class-incremental learning, where models must adapt to newly emerging sleep patterns, and domain-incremental learning, necessitating generalization across diverse data collection conditions or varying EEG equipment. Moreover, the unavailability of old datasets, particularly between different institutions, and the lack of annotations in historical data hinder the effective training and adaptability of deep learning models. This study introduces a Dual-Incremental Learning Framework, amalgamating unsupervised domain adaptation, hierarchy-aware feature learning, and diffusion probabilistic model-based generative replay design to improve the generalizability of sleep stage classification models and enhance the re-analysis of historical data with updated classification schemes. We establish two benchmarks using two publicly available datasets to elucidate the challenges posed by evolving sleep datasets. The effectiveness and performance of the proposed framework are evaluated against the established benchmarks, showcasing enhanced generalizability and the potential for unearthing deeper insights from historical patient data. This framework also demonstrates compatibility with existing deep learning models, promoting a versatile solution for advancing sleep stage classification. To the best of our knowledge, this study is the first to address class and domain-incremental learning challenges in sleep stage classification, laying a solid foundation for future research in this domain. Additionally, the proposed Dual-Incremental Learning Framework holds potential for a wide range of healthcare applications.