Ensemble relation network with multi-level measure

doi:10.19682/j.cnki.1005-8885.2022.1015

中国邮电高校学报(英文) ›› 2022, Vol. 29 ›› Issue (3): 15-24.doi: 10.19682/j.cnki.1005-8885.2022.1015

Ensemble relation network with multi-level measure

Li Xiaoxu, Qu Xue, Cao Jie

1. College of Computer and Communication, Lanzhou University of Technology, Lanzhou 730050, China
2. College of Electrical and Information Engineering, Lanzhou University of Technology, Lanzhou 730050, China
3. Engineering Research Center of Urban Railway Transportation of Gansu Province, Lanzhou 730050, China

收稿日期:2022-04-16 修回日期:2022-05-26 出版日期:2022-06-30 发布日期:2022-06-30
通讯作者: 曹洁 E-mail:caoj@lut.edu.cn
基金资助:
This work was supported by the National Natural Science Foundation of China (62176110, 62111530146, 61906080), and Young Doctoral Fund of Education Department of Gansu
Province (2021QB-038)

Ensemble relation network with multi-level measure

Li Xiaoxu, Qu Xue, Cao Jie

1. College of Computer and Communication, Lanzhou University of Technology, Lanzhou 730050, China
2. College of Electrical and Information Engineering, Lanzhou University of Technology, Lanzhou 730050, China
3. Engineering Research Center of Urban Railway Transportation of Gansu Province, Lanzhou 730050, China

Received:2022-04-16 Revised:2022-05-26 Online:2022-06-30 Published:2022-06-30
Supported by:
This work was supported by the National Natural Science Foundation of China (62176110, 62111530146, 61906080), and Young Doctoral Fund of Education Department of Gansu
Province (2021QB-038)

摘要/Abstract

摘要： Fine-grained few-shot learning is a difficult task in image classification. The reason is that the discriminative
features of fine-grained images are often located in local areas of the image, while most of the existing few-shot learning image classification methods only use top-level features and adopt a single measure. In that way, the local features of the sample cannot be learned well. In response to this problem, ensemble relation network with multi-level measure (ERN-MM) is proposed in this paper. It adds the relation modules in the shallow feature space to compare the similarity between the samples in the local features, and finally integrates the similarity scores from the feature spaces to assign the label of the query samples. So the proposed method ERN-MM can use local details and global information of different grains. Experimental results on different fine-grained datasets show that the proposed method achieves good classification performance and also proves its rationality.

关键词: fine-grained image classification, few-shot learning, local feature learning, metric learning

Abstract: Fine-grained few-shot learning is a difficult task in image classification. The reason is that the discriminative
features of fine-grained images are often located in local areas of the image, while most of the existing few-shot learning image classification methods only use top-level features and adopt a single measure. In that way, the local features of the sample cannot be learned well. In response to this problem, ensemble relation network with multi-level measure (ERN-MM) is proposed in this paper. It adds the relation modules in the shallow feature space to compare the similarity between the samples in the local features, and finally integrates the similarity scores from the feature spaces to assign the label of the query samples. So the proposed method ERN-MM can use local details and global information of different grains. Experimental results on different fine-grained datasets show that the proposed method achieves good classification performance and also proves its rationality.

Key words: fine-grained image classification, few-shot learning, local feature learning, metric learning

Li Xiaoxu, Qu Xue, Cao Jie. Ensemble relation network with multi-level measure[J]. The Journal of China Universities of Posts and Telecommunications, 2022, 29(3): 15-24.

参考文献

1. Shu J, Xu Z B, Meng D Y. Small sample learning in big data era. arXiv Preprint, arXiv:1808.04572v1, 2018.

2. WANG Y Q, YAO Q M, Kwok J T, et al. Generalizing from a few examples: A survey on few-shot learning. ACM Computing Surveys , 2020, 53(3): Article 63/1-34.

3. Ratner A, Sa C D, Wu S, et al. Data programming: Creating large training sets, quickly. Proceedings of the 30th Annual Conference on Neural Information Processing Systems (NIPS’16), 2016, Dec 5-10, Barcelona, Spain. Red Hook, NY, USA: Curran Associates Inc, 2016: 3567-3575.

4. Suárez J L, García S, Herrera F. A tutorial on distance metric learning: Mathematical foundations, algorithms, experimental analysis, prospects and challenges. Neurocomputing, 2021, 425: 300-322.

5. Elsken T, Staffler B, Metzen J H, et al. Meta-learning of neural architectures for few-shot learning. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR’20), 2020, Jun 13-19, Seattle, WA, USA. Piscataway, NJ, USA: IEEE, 2020: 12365-12375.

6. Ratner A J, Ehrenberg H R, HUSSAIN Z, et al. Learning to compose domain-specific transformations for data augmentation. Proceeding of the 31st Annual Conference on Neural Information Processing Systems (NIPS’17), 2017, Dec 4-9, Long Beach, CA, USA. Red Hook, NY, USA: Curran Associates Inc, 2017: 3236–3246.

7. Hariharan B, Girshick R. Low-shot visual recognition by shrinking and hallucinating features. Proceeding of the 2017 IEEE International Conference on Computer Vision (ICCV'17), 2017, Oct 22-29, Venice, Italy. Piscataway, NJ, USA: IEEE, 2017: 3018-3027.

8. Li X Z, Sun Q R, Liu Y Y, et al. Learning to self-train for semi-supervised few-shot classification. Proceeding of the 33rd Annual Conference on Neural Information Processing Systems (NIPS’19), 2019, Dec 8-14, Vancouver, Canada. Red Hook, NY, USA: Curran Associates Inc, 2019: 10276-10286.

9. Koch G, ZEMEL R, SALAKHUTDINOV R. Siamese neural networks for one-shot image recognition. Proceeding of the 32nd International Conference on Machine Learning (ICML’15), 2015, Jul 6-11, Lille, France. 2015.

10. Vinyals O, Blundell C, Lillicrap T, et al. Matching networks for one-shot learning. Proceeding of the 30th Annual Conference on Neural Information Processing Systems (NIPS’16), 2016, Dec 5-10, Barcelona, Spain. Red Hook, NY, USA: Curran Associates Inc, 2016: 3630–3638.

11. Li W B, Wang L, XU J L, et al. Revisiting local descriptor based image-to-class measure for few-shot learning. Proceeding of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR'19), 2019, Jun 15-20, Long Beach, CA, USA. Piscataway, NJ, USA: IEEE, 2019: 7253-7260.

12. Zhang C, CAI Y J, LIN G S, et al. DeepEMD: Few-shot image classification with differentiable earth mover’s distance and structured classifiers. Proceeding of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR’20), 2020, Jun 13-19, Seattle, WA, USA. Piscataway, NJ, USA: IEEE, 2020: 12200-12210.

13. Qiao L M, Shi Y M, Li J, et al. Transductive episodic-wise adaptive metric for few-shot learning. Proceeding of the 2019 IEEE International Conference on Computer Vision (ICCV’19), 2019, Oct 27-Nov 2, Seoul, Republic of Korea. Piscataway, NJ, USA: IEEE, 2019: 3602-3611.

14. Snell J, Swersky K, ZEMEL R. Prototypical networks for few-shot learning. Proceeding of the 31st Annual Conference on Neural Information Processing Systems (NIPS’17), 2017, Dec 4-9, Long Beach, CA, USA. Red Hook, NY, USA: Curran Associates Inc, 2017: 4077-4087.

15. Ren M Y, Triantafillou E, RAVI S, et al. Meta-learning for semi-supervised few-shot classification. Proceeding of the 6th International Conference on Learning Representations (ICLR’18), 2018, Apr 30-May 3, Vancouver, Canada. 2018.

16. Das D, Lee C S G. A two-stage approach to few-shot learning for image recognition. IEEE Transactions on Image Processing, 2020, 29: 3336-3350.

17. Sung F, Yang Y X, ZHANG L, et al. Learning to compare: Relation network for few-shot learning. Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition (CVPR'18), 2018, Jun 18-23, Salt Lake City, UT, USA. Piscataway, NJ, USA: IEEE, 2018: 1199-1208.

18. Zhou J, Cui G Q, HU S D, et al. Graph neural networks: A review of methods and applications. Al Open, 2020, 1: 57-81.

19. Garcia V, Bruna J. Few-shot learning with graph neural networks. Proceeding of the 6th International Conference on Learning Representations (ICLR’18), 2018, Apr 30-May 3, Vancouver, Canada. 2018.

20. Chen W Y, Liu Y C, KIRA Z, et al. A closer look at few-shot classification. Proceeding of the 7th International Conference on Learning Representations (ICLR’19), 2019, May 6-9, New Orleans, LA, USA. 2019.

21. Tian Y L, WANG Y, KRISHNAN D, et al. Rethinking few-shot image classification: A good embedding is all you need? Computer Vision: Proceeding of the 16th European Conference on Computer Vision (ECCV’20): Part IX, 2020, Aug 23-28, Glasgow, UK. LNIP 12359. Berlin, Germany: Springer, 2020: 266-282.

22. Finn C, Abbeel P, Levine S. Model-agnostic meta-learning for fast adaptation of deep networks. Proceeding of the 34th International Conference on Machine Learning (ICML’17), 2017, Aug 11-15, Sydney, Australia. 2017: 1126-1135.

23. Raghu A, Raghu M, Bengio S, et al. Rapid learning or feature reuse? towards understanding the effectiveness of MAML. Proceeding of the 8th International Conference on Learning Representations (ICLR’20), 2020, Apr 27-30, Addis Ababa, Ethiopia. 2020.

24. Oh J, Yoo H, Kim C H, et al. Boil: Towards representation change for few-shot learning. Proceeding of the 9th International Conference on Learning Representations (ICLR’21), 2021, May 3-7, Vienna, Austria. 2021.

25. Luo Y D, HUANG Z, ZHANG Z, et al. Learning from the past: Continual meta-learning with Bayesian graph neural networks. Proceeding of the 34th AAAI Conference on Artificial Intelligence (AAAI’20), 2020, Feb 7-12, New York, NY, USA. Menlo Park, CA, USA: American Association for Artificial Intelligence (AAAI), 2020: 5021-5028.

26. Santoro A, Bartunov S, Botvinick M, et al. Meta-learning with memory-augmented neural networks. Proceeding of the 33rd International Conference on Machine Learning (ICML’16), 2016, Jun 19-24, New York, NY, USA. 2016: 1842-1850.

27. Huang H X, ZHANG J J, Zhang J, et al. Low-rank pairwise alignment bilinear network for few-shot fine-grained image classification. IEEE Transactions on Multimedia, 2020, 23: 1666-1680

28. Russakovsky O, Deng J, Su H, et al. ImageNet large scale visual recognition challenge. International Journal of Computer Vision, 2015, 115: 211-252.

29. Welinder P, Branson S, Mita T, et al. Caltech-UCSD Birds 200. CNS-TR-2010-001. Pasadena, CA, USA: California Institute of Technology, 2010.

30. Krause J, Stark M, Deng J, et al. 3D object representations for fine-grained categorization. Proceedings of the 2013 IEEE International Conference on Computer Vision Workshops (ICCV’13), 2013, Dec 2-8, Sydney, Australia. Piscataway, NJ, USA: IEEE, 2013: 554-561.

31. Khosla A, Jayadevaprakash N, Yao B P, et al. Novel dataset for fine-grained image categorization: Stanford dogs. https://people.csail.mit.edu/khosla/papers/fgvc2011.pdf. 2011

32. Zhang X T, Qiang Y T, Sung F, et al. RelationNet2: Deep comparison columns for few-shot learning. Proceedings of the 2020 International Joint Conference on Neural Networks (IJCNN'20), 2020, Jul 19-24, Glasgow, UK. Piscataway, NJ, USA: IEEE, 2020.

33. Zhang H G, Li H D, Koniusz P. Multi-level second-order few-shot learning. IEEE Transactions on Multimedia, 2022, Early Access Article, DOI：10.1109/TMM.2022.3142955.

Ensemble relation network with multi-level measure

Ensemble relation network with multi-level measure

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 0

编辑推荐

Metrics

本文评价