loading page

Meta-iCVIs: Ensemble Validity Metrics for Concise Labeling of Correct, Under- or Over-Partitioning in Streaming Clustering
  • Sasha Petrenko ,
  • Niklas Melton ,
  • Donald Wunsch
Sasha Petrenko
Missouri University of Science and Technology, Missouri University of Science and Technology

Corresponding Author:[email protected]

Author Profile
Niklas Melton
Author Profile
Donald Wunsch
Author Profile

Abstract

Understanding the performance and validity of clustering algorithms is both challenging and crucial, particularly when clustering must be done online. Until recently, most validation methods have relied on batch calculation and have required considerable human expertise in their interpretation. Improving real-time performance and interpretability of cluster validation, therefore, continues to be an important theme in improving unsupervised learning. Building upon previous work on incremental cluster validity indices (iCVIs), this paper introduces the Meta-iCVI as a tool for explainable and concise labeling of partition quality in online clustering. Some iCVIs are better at detecting under-partition; others at over-partition. Combining them was hypothesized to improve cluster validation analysis. Experiments were conducted on generalized synthetic and real-world data sets to demonstrate the efficacy and application of this method.
Results of 100% accuracy were achieved in labeling partition quality on real-world data sets including MNIST and FLIR ADAS, demonstrating that the Meta-iCVI is a powerful and efficient tool for classifying partition quality in a variety of conditions. Its introduction should empower new and more efficient streaming clustering techniques. Additionally, we believe this to be the first implementation of an ensemble iCVI metric and the first time iCVI validation performance has been evaluated on randomized sample presentation.