Along with the deployment of the Face Recognition Systems (FRS), concerns were raised related to the vulnerability of those systems towards various attacks including morphed attacks. The morphed face attack involves two different face images in order to obtain via a morphing process a resulting attack image, which is sufficiently similar to both contributing data subjects. The obtained morphed image can successfully be verified against both subjects visually (by a human expert) and by a commercial FRS. The face morphing attack poses a severe security risk to the e-passport issuance process and to applications like border control, unless such attacks are detected and mitigated. In this work, we propose a new method to reliably detect a morphed face attack using a newly designed denoising framework. To this end, we design and introduce a new deep Multi-scale Context Aggregation Network (MS-CAN) to obtain denoised images, which is subsequently used to determine if an image is morphed or not. Extensive experiments are carried out on three different morphed face image datasets. The Morphing Attack Detection (MAD) performance of the proposed method is also benchmarked against 14 different state-of-the-art techniques using the ISO-IEC 30107-3 evaluation metrics. Based on the obtained quantitative results, the proposed method has indicated the best performance on all three datasets and also on cross-dataset experiments.