Graduate student JUNWON YOU (advisor Jung Jae-Hun), who is enrolled in the integrated master’s and doctoral program in the Department of Mathematics, will present a poster at the ICML 2024 conference, where their paper has been accepted.
Paper Title: Do Topological Characteristics Help in Knowledge Distillation?
Authors: Jungeun Kim, Junwon You, Dongjin Lee, Ha Young Kim, Jae-Hun Jung
The research focuses on Knowledge Distillation (KD), which aims to transfer knowledge from a large (teacher) network to a smaller (student) network. Previous studies have struggled with effectively conveying the relationships formed by feature vectors in latent space, focusing instead on point-to-point or pairwise relationships with knowledge. To address this issue, this research proposes a new KD method called TopKD, which transmits the global topology of the latent space with knowledge.
The conference will take place from July 21 to July 27, 2024, with the presentation scheduled for July 25 at the Messe Wien Exhibition Congress Center, Vienna, Austria.
