From their initial days, the fields of computer vision and image processing have been dealing with visual recognition problems. Convolutional Neural Networks (CNNs) in machine learning are deep architectures built as feed-forward neural networks or perceptrons, which are inspired by the research done in fields of visual analysis by the visual cortex of mammals like cats. This work analyzes CNNs for computer vision tasks, natural language processing, fundamental sciences and engineering problems, and other miscellaneous tasks. The general CNN structure, along with its mathematical intuition and working, a brief critical commentary on the advantages and disadvantages, which leads researchers to search for alternatives to CNN, is also mentioned. The paper also serves as an appreciation of the brain-child of past researchers for the existence of such a prolific architecture for handling multidimensional data and approaches to improve their performance further.