Machine learning has shown great potential for classifying diverse samples in biomedical applications based on their Raman spectra. However, the acquired spectra typically require several preprocessing steps before standard machine learning algorithms can accurately and reliably classify them. To simplify this workflow and enable future growth of this technology, we present a unified solution for classifying biological Raman spectra without any need of prepossessing, including denoising and baseline establishment. This method is developed based on a custom version of a convolutional neural network (CNN) elicited from ResNet architecture, combined with our proposed data augmentation technique. The superiority of this method compared to conventional classification techniques is shown by applying it to Raman spectra of different grades of bladder cancer tissue and surface enhanced Raman spectroscopy (SERS) spectra of various strains of E. Coli extracellular vesicles (EVs). These results show that our method is far more robust compared to its conventional counterparts when dealing with the various kinds of spectral baselines produced by different Raman spectrometers.