Distilling and Refining Domain-Specific Knowledge for Semi-Supervised Domain Adaptation

Ju Hyun Kim (Dongguk University),* Ba Hung Ngo (Dongguk University), Jae Hyeon Park (Dongguk University), Jung Eun Kwon (Dongguk university), Ho Sub Lee (Daegu University), Sung In Cho (Dongguk University)
The 33rd British Machine Vision Conference


We propose a novel framework, Distilling And Refining domain-specific Knowledge (DARK), for Semi-supervised Domain Adaptation (SSDA) tasks. The proposed method consists of three strategies: Multi-view Learning, Distilling, and Refining. In Multi-view Learning, to acquire domain-specific knowledge, DARK trains a shared generator and two domain-specific classifiers using the labeled source and target data. Then, in Distilling, two classifiers exchange the domain-specific knowledge with each other to exploit a cross-view consistency regularization using soft labels between differently augmented unlabeled target samples. During this, DARK leverages information from low-confidence unlabeled target samples in addition to the high-confidence unlabeled target samples. To prevent a trivial collapse problem caused by the low-confidence samples, we propose the utilization of a sample-wise dynamic weight based on prediction reliability (SDWR). Finally, in Refining, for class alignment, class confusion of the unlabeled target data is minimized considering the model maturity. Simultaneously, to maintain model consistency between the predictions of differently augmented unlabeled target samples, a bridging loss with SDWR is used. Consequently, the experimental results on the SSDA datasets demonstrate that DARK outperforms the state-of-the-art benchmark methods for SSDA tasks.



author    = {Ju Hyun Kim and Ba Hung Ngo and Jae Hyeon Park and Jung Eun Kwon and Ho Sub Lee and Sung In Cho},
title     = {Distilling and Refining Domain-Specific Knowledge for Semi-Supervised Domain Adaptation },
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year      = {2022},
url       = {https://bmvc2022.mpi-inf.mpg.de/0606.pdf}

Copyright © 2022 The British Machine Vision Association and Society for Pattern Recognition
The British Machine Vision Conference is organised by The British Machine Vision Association and Society for Pattern Recognition. The Association is a Company limited by guarantee, No.2543446, and a non-profit-making body, registered in England and Wales as Charity No.1002307 (Registered Office: Dept. of Computer Science, Durham University, South Road, Durham, DH1 3LE, UK).

Imprint | Data Protection