Automatic Detection and Analysis of Singing Mistakes for Music Pedagogy

The growing popularity of computer-assisted pedagogy and audio analysis methods based on machine learning stimulates research into developing tools for music pedagogy. Despite the proliferation of commercial tools for teaching music, a lack of systematic research on computer-assisted music pedagogy exists. This paper describes pedagogical aids for music education. Novel methods are proposed for detecting singing mistakes by comparing teacher's and learner's audio. These methods consist of a CNN-based model for comparing pitch and amplitude contours and a CRNN-based model for comparing spectrograms. A new evaluation method is proposed to compare the efficacy of mistake detection systems. Experiments indicate that the proposed learning-based methods are superior to the rule-based baseline. A systematic study of errors and a cross-teacher study reveal insights into music pedagogy that can be utilized when deploying the proposed tool. In addition, a new dataset of teacher and learner audio recordings, annotated for singing mistakes, is presented.