Learning visual representations for transfer learning by suppressing texture

Shlok Kumar Mishra (University of Maryland College Park),* Anshul Shah (Johns Hopkins University), Ankan Bansal (Amazon.com), Janit Anjaria (University of Maryland, College Park), Jonghyun Choi (Yonsei University), Abhinav Shrivastava (University of Maryland), Abhishek Sharma (Cruise LLC), David Jacobs (University of Maryland)
The 33rd British Machine Vision Conference


Recent literature has shown that features obtained from supervised training of CNNs may over-emphasize texture rather than encoding high-level information. In self-supervised learning in particular, texture as a low-level cue may provide shortcuts that prevent the network from learning higher level representations. To address these problems we propose to use classic methods based on anisotropic diffusion to augment training using images with suppressed texture. This simple method helps retain important edge information and suppress texture at the same time. We empirically show that our method achieves state-of-the-art results on object detection and image classification with eight diverse datasets in either supervised or self-supervised learning tasks such as MoCoV2 and Jigsaw. Our method is particularly effective for transfer learning tasks and we observed improved performance on five standard transfer learning datasets. The large improvements (up to 11.49\%) on the Sketch-ImageNet dataset, DTD dataset and additional visual analyses with saliency maps suggest that our approach helps in learning better representations that better transfer.



author    = {Shlok Kumar Mishra and Anshul Shah and Ankan Bansal and Janit Anjaria and Jonghyun Choi and Abhinav Shrivastava and Abhishek Sharma and David Jacobs},
title     = {Learning visual representations for transfer learning by suppressing texture},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year      = {2022},
url       = {https://bmvc2022.mpi-inf.mpg.de/0300.pdf}

Copyright © 2022 The British Machine Vision Association and Society for Pattern Recognition
The British Machine Vision Conference is organised by The British Machine Vision Association and Society for Pattern Recognition. The Association is a Company limited by guarantee, No.2543446, and a non-profit-making body, registered in England and Wales as Charity No.1002307 (Registered Office: Dept. of Computer Science, Durham University, South Road, Durham, DH1 3LE, UK).

Imprint | Data Protection