TsCANet: Three-stream contrastive adaptive network for cross-domain few-shot learning
Authors: 
Yuandong Bi, Hong Zhu, Jing Shi, Bin SongAuthors Info & Claims
Yuandong Bi, Hong Zhu, Jing Shi, Bin SongAuthors Info & ClaimsAbstract
Cross-domain few-shot learning, which aims to solve the problem of domain gap in few-shot learning, has recently received more and more attention. Specifically, when there are great differences between the source domain and the target domain involved in few-shot learning, the performance will fall off a cliff and it is even difficult to train. Therefore, this paper explores a simple, effective and novel method to deal with domain gaps. Firstly, the pre-trained model is obtained by using the labeled data in the source domain. Next, the two-stage adaptive training mainly consists of unlabeled data in the target domain, pseudo-unlabeled data and labeled data in the source domain as the third-stream input, so that the network can gradually adapt to the data in target domain and mitigate the adverse effects caused by the domain gap. Finally, the proposed network can be quickly applied to the tasks to be solved. Through the observation of experimental results, the designed approach can achieve better performance than the existing comparison methods on the standard benchmark of cross-domain few-shot learning. Further analysis reveals the tradeoff between using data in source domain and target domain for cross-domain few-shot learning.
References
[1]
Hassantabar S, Terway P, and Jha NK TUTOR: training neural networks using decision rules as model priors IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems. 2023 42 2 483-496
[2]
Guo Y, Codella N, Karlinsky L, Codella JV, Smith JR, Saenko K, Rosing T, Feris R (2020) A broader study of cross-domain few-shot learning. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J. (eds.) 16th European Conference of Computer Vision, ECCV, Glasgow, UK, August 23-28, 2020, Proceedings, Part XXVII. Lecture Notes in Computer Science, vol. 12372, pp. 124–141.
[3]
Chen H, Li L, Hu F, Lyu F, Zhao L, Huang K, Feng W, and Xia Z Multi-semantic hypergraph neural network for effective few-shot learning Pattern Recognition 2023 142
[4]
Shi B, Li W, Huo J, Zhu P, Wang L, and Gao Y Global- and local-aware feature augmentation with semantic orthogonality for few-shot image classification Pattern Recognition 2023 142
[5]
Xie J, Long F, Lv J, Wang Q, Li P (2022) Joint distribution matters: Deep brownian distance covariance for few-shot classification. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR, New Orleans, LA, USA, June 18-24, 2022, pp. 7962–7971.
[6]
Luo X, Wu H, Zhang J, Gao L, Xu J, Song J (2023) A closer look at few-shot classification again. In: International Conference on Machine Learning, ICML, 23-29 July 2023, Honolulu, Hawaii, USA. Proceedings of Machine Learning Research, vol. 202, pp. 23103–23123. https://proceedings.mlr.press/v202/luo23e.html
[7]
Chen Y, Liu Z, Xu H, Darrell T, Wang X (2021) Meta-baseline: Exploring simple meta-learning for few-shot learning. In: International Conference on Computer Vision, ICCV, Montreal, QC, Canada, October 10-17, 2021, pp. 9042–9051.
[8]
Chen W, Liu Y, Kira Z, Wang YF, Huang J (2019) A closer look at few-shot classification. In: 7th International Conference on Learning Representations, ICLR, New Orleans, LA, USA, May 6-9, 2019. https://openreview.net/forum?id=HkxLXnAcFQ
[9]
Fu Y, Fu Y, Jiang Y (2021) Meta-fdmixup: Cross-domain few-shot learning guided by labeled target data. In: MM ’21: ACM Multimedia Conference, Virtual Event, China, October 20 - 24, 2021, pp. 5326–5334.
[10]
Li P, Gong S, Wang C, Fu Y (2022) Ranking distance calibration for cross-domain few-shot learning. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR, New Orleans, LA, USA, June 18-24, 2022, pp. 9089–9098.
[11]
Tseng H, Lee H, Huang J, Yang M (2020) Cross-domain few-shot classification via learned feature-wise transformation. In: 8th International Conference on Learning Representations, ICLR, Addis Ababa, Ethiopia, April 26-30, 2020. https://openreview.net/forum?id=SJl5Np4tPr
[12]
Wang X, Peng Y, Lu L, Lu Z, Bagheri M, Summers RM (2017) Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR, Honolulu, HI, USA, July 21-26, 2017, pp. 3462–3471.
[13]
Helber P, Bischke B, Dengel A, and Borth D Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification IEEE Journal of Selection Topics in Applied Earth Observations and Remote Sensing 2019 12 7 2217-2226
[14]
Chug A, Bhatia A, Singh AP, and Singh D A novel framework for image-based plant disease detection using hybrid deep learning approach Soft Computing 2023 27 18 13613-13638
[15]
Veronica R, Halpern A, Dusza SW, Codella NCF (2019) The role of public challenges and data sets towards algorithm development, trust, and use in clinical practice. Seminars in Cutaneous Medicine and Surgery 38(1):38–42. 31051022
[16]
Phoo CP, Hariharan B (2021) Self-training for few-shot transfer across extreme ta skdifferences. In: 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. https://openreview.net/forum?id=O3Y56aqpChA
[17]
Islam A, Chen C, Panda R, Karlinsky L, Radke RJ, Feris R (2021) A broad study on the transferability of visual representations with contrastive learning. In: IEEE/CVF International Conference on Computer Vision, ICCV, Montreal, QC, Canada, October 10-17, 2021, pp. 8825–8835.
[18]
Lee K, Maji S, Ravichandran A, Soatto S (2019) Meta-learning with differentiable convex optimization. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR, Long Beach, CA, USA, June 16-20, 2019, pp. 10657–10665. . http://openaccess.thecvf.com/content_CVPR_2019/html Lee_MetaLearning_With_Differentiable_Convex_Optimization_CVPR_2019_paper.html
[19]
Vinyals O, Blundell C, Lillicrap T, Kavukcuoglu K, Wierstra D (2016) Matching networks for one shot learning. In: Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems, December 5-10, 2016, Barcelona, Spain, pp. 3630–3638. https://proceedings.neurips.cc/paper/2016/hash/90e1357833654983612fb05e3ec9148c-Abstract.html
[20]
Snell J, Swersky K, Zemel RS (2017) Prototypical networks for few-shot learning. In: Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems, December 4-9, 2017, Long Beach, CA, USA, pp. 4077–4087. https://proceedings.neurips.cc/paper/2017/hash/cb8da6767461f2812ae4290eac7cbc42-Abstract.html
[21]
Sung F, Yang Y, Zhang L, Xiang T, Torr PHS, Hospedales TM (2018) Learning to compare: Relation network for few-shot learning. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR, Salt Lake City, UT, USA, June 18-22, 2018, pp. 1199–1208. . http://openaccess.thecvf.com/content_cvpr_2018/html/Sung_Learning_to_Compare_CVPR_2018_paper.html
[22]
Finn C, Abbeel P, Levine S (2017) Model-agnostic meta-learning for fast adaptation of deep networks. In: Proceedings of the 34th International Conference on Machine Learning, ICML, Sydney, NSW, Australia, 6-11 August 2017. Proceedings of Machine Learning Research, vol. 70, pp. 1126–1135. http://proceedings.mlr.press/v70/finn17a.html
[23]
Rajasegaran J, Khan S, Hayat M, Khan FS, Shah M (2021) Self-supervised knowledge distillation for few-shot learning. In: 32nd British Machine Vision Conference 2021, BMVC, Online, November 22-25, 2021, p. 179. https://www.bmvc2021-virtualconference.com/assets/papers/0820.pdf
[24]
Yang S, Liu L, Xu M (2021) Free lunch for few-shot learning: Distribution calibration. In: 9th International Conference on Learning Representations, ICLR, Virtual Event, Austria, May 3-7, 2021. https://openreview.net/forum?id=JWOiYxMG92s
[25]
Tian Y, Wang Y, Krishnan D, Tenenbaum JB, Isola P (2020) Rethinking few-shot image classification: A good embedding is all you need? In: Computer Vision - ECCV 2020 - 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XIV, vol. 12359, pp. 266–282.
[26]
Chen Z, Wang C, Wu J, Deng C, and Wang Y Deep convolutional transfer learning-based structural damage detection with domain adaptation Applied Intelligence 2023 53 5 5085-5099
[27]
Karimian M and Beigy H Concept drift handling: A domain adaptation perspective Expert System with Applications 2023 224
[28]
Kumar V, Patil H, Lal R, and Chakraborty A Improving domain adaptation through class aware frequency transformation International Journal of Computer Vision 2023 131 11 2888-2907
[29]
Zhang J, Song J, Gao L, Shen H (2022) Free-lunch for cross-domain few-shot learning: Style-aware episodic training with robust contrastive learning. In: MM ’22: The 30th ACM International Conference on Multimedia, Lisboa, Portugal, October 10 - 14, 2022, pp. 2586–2594.
[30]
Li W, Liu X, Bilen H (2022) Cross-domain few-shot learning with task-specific adapters. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR, New Orleans, LA, USA, June 18-24, 2022, pp. 7151–7160.
[31]
Guan J, Zhang M, Lu Z (2020) Large-scale cross-domain few-shot learning. In: 15th Asian Conference on Computer Vision, ACCV, Kyoto, Japan, November 30 - December 4, 2020, Revised Selected Papers, Part III. Lecture Notes in Computer Science, vol. 12624, pp. 474–491.
[32]
Islam A, Chen CR, Panda R, Karlinsky L, Feris R, Radke RJ (2021) Dynamic distillation network for cross-domain few-shot recognition with unlabeled data. In: Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems, NeurIPS, December 6-14, 2021, Virtual, pp. 3584–3595. https://proceedings.neurips.cc/paper/2021/hash/1d6408264d31d453d556c60fe7d0459e-Abstract.html
[33]
Oh J, Kim S, Ho N, Kim J, Song H, Yun S (2022) Understanding cross-domain few-shot learning based on domain similarity and few-shot difficulty. In: Conference on Neural Information Processing Systems, NeurIPS. http://papers.nips.cc/paper_files/paper/2022/hash/11b3ae28275461741026c46c0c786711-Abstract-Conference.html
[34]
Chen X, Fan H, Girshick RB, He K (2020) Improved baselines with momentum contrastive learning. arXiv arxiv:2003.04297
[35]
Chen T, Kornblith S, Norouzi M, Hinton GE (2020) A simple framework for contrastive learning of visual representations. In: Proceedings of the 37th International Conference on Machine Learning, ICML, 13-18 July 2020, Virtual Event. Proceedings of Machine Learning Research, vol. 119, pp. 1597–1607. http://proceedings.mlr.press/v119/chen20j.html
[36]
Grill J, Strub F, Altché F, Tallec C, Richemond PH, Buchatskaya E, Doersch C, Pires BÁ, Guo Z, Azar MG, Piot B, Kavukcuoglu K, Munos R, Valko M (2020) Bootstrap your own latent - A new approach to self-supervised learning. In: Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, Virtual. https://proceedings.neurips.cc/paper/2020/hash/f3ada80d5c4ee70142b17b8192b2958e-Abstract.html
[37]
Noroozi M, Favaro P (2016) Unsupervised learning of visual representations by solving jigsaw puzzles. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) 14th European Conference of Computer Vision, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part VI. Lecture Notes in Computer Science, vol. 9910, pp. 69–84.
[38]
Borkowski P, Ciesielski K, Klopotek MA (2014) Unsupervised aggregation of categories for document labelling. In: Foundations of Intelligent Systems - 21st International Symposium, ISMIS, Roskilde, Denmark, June 25-27, 2014. Proceedings. Lecture Notes in Computer Science, vol. 8502, pp. 335–344.
[39]
Wang D, Li T, Deng P, Liu J, Huang W, and Zhang F A generalized deep learning algorithm based on NMF for multi-view clustering IEEE Transactions on Big Data 2023 9 1 328-340
[40]
Wang D, Li T, Deng P, Zhang F, Huang W, Zhang P, and Liu J A generalized deep learning clustering algorithm based on non-negative matrix factorization ACM Transactions on Knowledge Discovery from Data 2023 17 7 99-19920
[41]
Wang D, Li T, Huang W, Luo Z, Deng P, Zhang P, and Ma M A multi-view clustering algorithm based on deep semi-nmf Information Fusion 2023 99
[42]
Wang D, Li T, Deng P, Luo Z, Zhang P, Liu K, and Huang W Dnsrf: Deep network-based semi-nmf representation framework ACM Transactions on Intelligent Systems and Technology 2024
[43]
Gidaris S, Singh P, Komodakis N (2018) Unsupervised representation learning by predicting image rotations. In: 6th International Conference on Learning Representations, ICLR, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings. https://openreview.net/forum?id=S1v4N2l0-
[44]
Chen X, He K (2021) Exploring simple siamese representation learning. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR, Virtual, June 19-25, 2021, pp. 15750–15758. . https://openaccess.thecvf.com/content/CVPR2021/html/Chen_Exploring_Simple_Siamese_Representation_Learning_CVPR_2021_paper.html
[45]
Tian Y, Chen X, Ganguli S (2021) Understanding self-supervised learning dynamics without contrastive pairs. In: Proceedings of the 38th International Conference on Machine Learning, ICML, 18-24 July 2021, Virtual Event. Proceedings of Machine Learning Research, vol. 139, pp. 10268–10278. http://proceedings.mlr.press/v139/tian21a.html
[46]
Zbontar J, Jing L, Misra I, LeCun Y, Deny S (2021) Barlow twins: Self-supervised learning via redundancy reduction. In: Proceedings of the 38th International Conference on Machine Learning, ICML, 18-24 July 2021, Virtual Event. Proceedings of Machine Learning Research, vol. 139, pp. 12310–12320. http://proceedings.mlr.press/v139/zbontar21a.html
[47]
Ren M, Triantafillou E, Ravi S, Snell J, Swersky K, Tenenbaum JB, Larochelle H, Zemel RS (2018) Meta-learning for semi-supervised few-shot classification. In: 6th International Conference on Learning Representations, ICLR, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings. https://openreview.net/forum?id=HJcSzz-CZ
[48]
Codella NCF, Rotemberg V, Tschandl P, Celebi ME, Dusza SW, Gutman DA, Helba B, Kalloo A, Liopyris K, Marchetti MA, Kittler H, Halpern A (2019) Skin lesion analysis toward melanoma detection 2018: A challenge hosted by the international skin imaging collaboration (ISIC). arXiv arxiv:1902.03368
[49]
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR, Las Vegas, NV, USA, June 27-30, 2016, pp. 770–778.
Index Terms
- TsCANet: Three-stream contrastive adaptive network for cross-domain few-shot learning
Index terms have been assigned to the content through auto-classification.
Recommendations
Free-Lunch for Cross-Domain Few-Shot Learning: Style-Aware Episodic Training with Robust Contrastive Learning
MM '22: Proceedings of the 30th ACM International Conference on MultimediaCross-Domain Few-Shot Learning (CDFSL) aims for training an adaptable model that can learn out-of-domain classes with a handful of samples. Compared to the well-studied few-shot learning problem, the difficulty for CDFSL lies in that the available ...
HybridPrompt: Domain-Aware Prompting for Cross-Domain Few-Shot Learning
AbstractCross-Domain Few-Shot Learning (CD-FSL) aims at recognizing unseen classes from target domains that vastly differ from training classes from source domains, utilizing only a few labeled samples. However, the substantial domain disparities between ...
ME-D2N: Multi-Expert Domain Decompositional Network for Cross-Domain Few-Shot Learning
MM '22: Proceedings of the 30th ACM International Conference on MultimediaRecently, Cross-Domain Few-Shot Learning (CD-FSL) which aims at addressing the Few-Shot Learning (FSL) problem across different domains has attracted rising attention. The core challenge of CD-FSL lies in the domain gap between the source and novel ...
Comments
Information & Contributors
Information
Published In
© The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2024. Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
Publisher
Kluwer Academic Publishers
United States
Publication History
Published: 06 November 2024
Accepted: 02 October 2024
Author Tags
Qualifiers
- Research-article
Funding Sources
- Natural Science Basic Research Program of Shaanxi Province
- the Key Laboratory of Manufacturing Equipment of Shaanxi Province
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 05 Jan 2025