Generalized Category Discovery

transformer Read

Questions What’s the number of operations required to relate signals from two arbitrary input or output positions grows in the distance between positions ? Extended Learning Attention mechanisms with a recurrent network. Bahdanau, Dzmitry. “Neural machine translation by jointly learning to align and translate.” arXiv preprint arXiv:1409.0473 (2014). Use convolutional neural networks to compute hidden representations in parallel. Gehring, Jonas, et al. “Convolutional sequence to sequence learning.” International conference on machine learning. PMLR, 2017.

Thursday, September 12, 2024 | 1 minute Read

DINO Read

Questions What’s the function of centering $ C $ and sharpening parameter $ \tau_t $ ? Extended Learning Multi-crop training. Caron, Mathilde, et al. “Unsupervised learning of visual features by contrasting cluster assignments.” Advances in neural information processing systems 33 (2020): 9912-9924. Use a noise contrastive estimator (NCE) to compare instances instead of classifying them. Wu, Zhirong, et al. “Unsupervised feature learning via non-parametric instance discrimination.” Proceedings of the IEEE conference on computer vision and pattern recognition. 2018.

Friday, September 6, 2024 | 1 minute Read

SimGCD Read

Research Task Generalized Category Discovery (GCD) relaxes the assumption of traditional semi-supervised learning by assuming the unlabelled data can also contain different but related categories from the labelled data. The goal of GCD is to learn a model that is able to classify the already-seen categories in the labelled data, and more importantly, jointly discover the new categories in the unlabelled data and make correct classifications. Limitations of Previous Methods Previous methods can be categorized into two distinct approaches:

Monday, September 2, 2024 | 2 minutes Read