loading page

Progressive Hard Negative Masking: From Global Uniformity to Local Tolerance
  • Qingqiang Sun ,
  • Wenjie Zhang ,
  • Xuemin Lin
Qingqiang Sun
University of New South Wales

Corresponding Author:[email protected]

Author Profile
Wenjie Zhang
Author Profile
Xuemin Lin
Author Profile

Abstract

Unsupervised contrastive learning has recently become increasingly popular due to its amazing performance without the need for costly annotations. However, indiscriminate sampling of negative pairs is accompanied by the uniformity-tolerance dilemma, which is especially serious in node-level graph contrastive learning due to the smoothing property of graph convolutional operators. Previous negative mining strategies that either overly emphasize hard negatives or rely on precise distribution estimation can make minor improvements or even degrade the performance in such a case. In this paper, we investigate the role of hard negatives in the uniformity-tolerance dilemma and propose a novel contrastive objective with a progressive hard negative masking scheme. The proposed objective, as an asymptotically-tightened lower bound of mutual information, is theoretically and empirically demonstrated to be capable of allowing higher local tolerance and stronger contrastive effects, thus leading to higher-quality embedding distributions and considerable performance improvement in downstream node classification tasks.
2023Published in IEEE Transactions on Knowledge and Data Engineering on pages 1-12. 10.1109/TKDE.2023.3269795