Harmonic retrieval techniques are the foundation of radio channel sounding, estimation and modeling. This paper introduces a Deep Learning approach for joint delay-and Doppler estimation from frequency and time samples of a radio channel transfer function. Our work estimates the two-dimensional parameters from a signal containing an unknown number of paths. Compared to existing deep learning-based methods, the signal parameters are not estimated via classification but in a quasi-grid-free manner. This alleviates the bias, spectral leakage, and ghost targets that grid-based approaches produce. The proposed architecture also reliably estimates the number of paths in the measurement. Hence, it jointly solves the model order selection and parameter estimation task. Additionally, we propose a multi-channel windowing of the data to increase the estimator's robustness. We also compare the performance to other harmonic retrieval methods and integrate it into an existing maximum likelihood estimator for efficient initialization of a gradient-based iteration.