Supervised learning of enhancing convolutional Hash for image retrieval

doi:DOI: 10.19682/j.cnki.1005-8885.2019.1017

中国邮电高校学报(英文) ›› 2019, Vol. 26 ›› Issue (4): 51-61.doi: DOI: 10.19682/j.cnki.1005-8885.2019.1017

• Artificial Intelligence • 上一篇下一篇

Supervised learning of enhancing convolutional Hash for image retrieval

Zhai Qi, Jiang Mingyan

School of Information Science and Engineering, Shandong University, Qingdao 266237, China

收稿日期:2018-06-05 修回日期:2019-08-16 出版日期:2019-08-31 发布日期:2019-10-29
通讯作者: Corresponding author: Jiang Mingyan, E-mail: jiangmingyan@sdu.edu.cn E-mail:jiangmingyan@sdu.edu.cn
作者简介:Corresponding author: Jiang Mingyan, E-mail: jiangmingyan@sdu.edu.cn
基金资助:
This work was supported by the Natural Science Foundation of Shandong Province (ZR2014FM039) and the National Natural Science Foundation of China (61771293).

Supervised learning of enhancing convolutional Hash for image retrieval

Zhai Qi, Jiang Mingyan

School of Information Science and Engineering, Shandong University, Qingdao 266237, China

Received:2018-06-05 Revised:2019-08-16 Online:2019-08-31 Published:2019-10-29
Contact: Corresponding author: Jiang Mingyan, E-mail: jiangmingyan@sdu.edu.cn E-mail:jiangmingyan@sdu.edu.cn
About author:Corresponding author: Jiang Mingyan, E-mail: jiangmingyan@sdu.edu.cn
Supported by:
This work was supported by the Natural Science Foundation of

Shandong Province (ZR2014FM039) and the National Natural

Science Foundation of China (61771293).

摘要/Abstract

摘要： Deep convolutional neural network (CNN) makes great breakthroughs in computer vision. Recently, many works have demonstrated that the performance of the CNN depends on the stacked convolutional layers. It is obvious that the features of the fully connected layers lose the topological structures of images, and the convolutional layer features contain a large amount of redundant information that interferes with the performance of model. Thus, we propose an effective supervised deep Hashing method, enhancing convolutional deep Hashing (ECDH), which learns the binary codes from the strengthened convolutional layer. Specifically, an enhanced convolutional Hash layer is constructed between the top convolutional layer and the output layer, enhancing the local features of the convolutional layer outputs while learning the binary codes by optimizing an objective function. The proposed method works well for existing deep learning models such as Alex neural network (AlexNet), visual geometry group neural network (VGGNet), residual neural network (ResNet), and is easier to be trained. Compared with state-of-the-art methods, extensive experiments show that the proposed method achieves better retrieval performance.

关键词: CNN, Hashing, image retrieval, Hamming distance

Abstract: Deep convolutional neural network (CNN) makes great breakthroughs in computer vision. Recently, many works have demonstrated that the performance of the CNN depends on the stacked convolutional layers. It is obvious that the features of the fully connected layers lose the topological structures of images, and the convolutional layer features contain a large amount of redundant information that interferes with the performance of model. Thus, we propose an effective supervised deep Hashing method, enhancing convolutional deep Hashing (ECDH), which learns the binary codes from the strengthened convolutional layer. Specifically, an enhanced convolutional Hash layer is constructed between the top convolutional layer and the output layer, enhancing the local features of the convolutional layer outputs while learning the binary codes by optimizing an objective function. The proposed method works well for existing deep learning models such as Alex neural network (AlexNet), visual geometry group neural network (VGGNet), residual neural network (ResNet), and is easier to be trained. Compared with state-of-the-art methods, extensive experiments show that the proposed method achieves better retrieval performance.

Key words: CNN, Hashing, image retrieval, Hamming distance

中图分类号:

TP18

Zhai Qi, Jiang Mingyan. Supervised learning of enhancing convolutional Hash for image retrieval[J]. The Journal of China Universities of Posts and Telecommunications, 2019, 26(4): 51-61.

参考文献

References

1. Gong Y, Lazebnik S, Gordo A, et al. Iterative quantization: a Procrustean approach to learning binary codes for large-scale image retrieval. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(12): 2916 -2929

2. He J, Liu W, Chang S F. Scalable similarity search with optimized kernel hashing. Proceedings of ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, July 25 -28, 2010, Washington, USA. New York, NY, USA: ACM, 2010: 1129 -1138

3. Kulis B, Grauman K. Kernelized locality-sensitive hashing. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(6): 1092 -1104

4. Andoni A, Indyk P. Near-optimal hashing algorithms for approximate nearest neighbor in high dimensions. Proceedings of 47th Annual IEEE Symposium on Foundations of Computer Science, Oct 21 -24, 2006, Berkeley, CA, USA. Los Alamitos, CA, USA: IEEE Computer Society, 2006: 459 -468

5. Weiss Y, Torralba A, Fergus R. Spectral hashing. Proceedings of the 21st International Conference on Neural Information Processing Systems, Dec 08 - 10, 2008, Vancouver, British Columbia Canada. USA: Curran Associates Inc, 2008: 1753 -1760

6. Oliva A, Torralba A. Modeling the shape of the scene: a holistic representation of the spatial envelope. International Journal of Computer Vision, 2001, 42(3): 145 -175

7. Dalal N, Triggs B. Histograms of oriented gradients for human detection. Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, June 20 - 25, 2005, San Diego, CA, USA, US. Los Alamitos, CA, USA: IEEE COMPUTER SOC, 2005: 886 -893

8. David G L. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 2004, 60(2): 91 -110

9. Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Jun 07 -12, 2015, Boston, MA. New York, NY, USA: IEEE, 2015: 3431 -3440

10. He K, Zhang X, Ren S, et al. Deep residual learning for image recognition. Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition, Jun 27 - 30, 2016, Las Vegas, NV, USA. New York, NY, USA: IEEE, 2016: 770 -778

11. Redmon J, Divvala S, Girshick R, et al. You Only Look Once: unified, real-time object detection. Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition, Jun 27 -30, 2016, Las Vegas, NV, USA. New York, NY, USA: IEEE, 2016:779 -788

12. Russakovsky O, Deng J, Su H, et al. ImageNet large scale visual recognition challenge. International Journal of Computer Vision, 2014, 115(3):211 -252.

13. Oquab M, Bottou L, Laptev I, et al. Learning and transferring mid-level image representations using convolutional neural networks. Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, June 23 - 28, 2014, Columbus, OH, USA. New York, NY, USA: IEEE, 2014: 1717 -1724

14. Ali S R, Hossein A, Josephine S, et al. CNN features off-the-shelf: an astounding baseline for recognition. Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops, June 23 -28, 2014, Columbus, OH, USA. New York, NY, USA: IEEE, 2014: 512 -519

15. Babenko A, Slesarev A, Chigorin A, et al. Neural codes for image retrieval. Proceedings of 13th European Conference on Computer Vision, Sep 06 -12, 2014, Zurich, Switzerland. Switzerland: Springer Publishing AG, 2014: 584 -599

16. Girshick R, Donahue J, Darrell T, et al. Region-based convolutional networks for accurate object detection and segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(1): 142 -158

17. Chatfield K, Simonyan K, Vedaldi A, et al. Return of the devil in the details: delving deep into convolutional nets. Proceedings of the 25th British Machine Vision Conference, Sep 1st -5th, 2014, Nottingham, UK, 2014: 54 -65

18. Lin K, Yang H F, Hsiao J H, et al. Deep learning of binary hash codes for fast image retrieval. Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops, June 08 -10, 2015, Boston, MA, USA. Piscataway, NJ, USA: IEEE, 2015: 27 -35

19. Razavian A S, Sullivan J, Carlsson S, et al. Visual instance retrieval with deep convolutional networks. ITE Transactions on Media Technology and Applications, 2016, 4(3): 251 -258

20. Wu S, Oerlemans A, Bakker E M, et al. Deep binary codes for large scale image retrieval. Neurocomputing, 2017, 257: 5 -15

21. Babenko A, Lempitsky V. Aggregating local deep features for image retrieval. Proceedings of IEEE International Conference on Computer Vision Computer Science, Dec 11 -18, 2015, Santiago, Chile. New York, NY, USA: IEEE, 2015: 1269 -1277

22. Li Y, Xu Y, Wang J, et al. MS -RMAC: multi-scale regional maximum activation of convolutions for image retrieval. IEEE Signal Processing Letters, 2017, 24(5): 609 -613

23. Zeiler M D, Fergus R. Visualizing and understanding convolutional networks. Proceedings of 13th European Conference on Computer Vision, Sep 06 - 12, 2014 Zurich, Switzerland. Switzerland: Springer International Publishing, 2014: 818 -833

24. Salakhutdinov R, Hinton G. Semantic hashing. International Journal of Approximate Reasoning, 2009, 50(7): 969 -978

25. Xia R, Pan Y, Lai H, et al. Supervised hashing for image retrieval via image representation learning. Proceedings of 28th AAAI Conference on Artificial Intelligence, July 27 -31, 2014, Quebec City, Quebec, Canada, 2014: 2156 -2162

26. Lai H, Pan Y, Liu Y, et al. Simultaneous feature learning and hash coding with deep neural networks. Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition, Jun 07 -12, 2015, Boston, MA, USA. New York, NY USA: IEEE, 2015: 3270 -3278

27. Yang H F, Lin K, Chen C S. Supervised learning of semantics-preserving hash via deep convolutional neural networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(2): 437 -451

28. Zhao W, Luo H, Peng J, et al. Spatial pyramid deep hashing for large-scale image retrieval. Neurocomputing, 2017, 243: 166 -173

29. Jiang Q Y, Cui X, Li W J. Deep discrete supervised hashing. IEEE Transactions on Image Processing, 2018, 27(12): 5996 -6009

30. Liu H, Wang R, Shan S, et al. Deep supervised hashing for fast image retrieval. Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition, Jun 27 -30, 2016, Las Vegas, NV, USA. New York, NY, USA: IEEE, 2016: 2064 -2072

31. Szegedy C, Liu N W, Jia N Y, et al. Going deeper with convolutions. Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition, Jun 07 -12, 2015, Boston, MA, USA. New York, NY USA: IEEE, 2015: 1 -9

32. Glorot X, Bordes A, Bengio Y. Deep sparse rectifier neural networks. Proceedings of the 14th International Conference on Artificial Intelligence and Statistics, April 11 - 13, 2011, Fort Lauderdale, Florida, USA. Cambridge, MA, USA: MIT Press, 2011: 315 -323

33. He K, Zhang X, Ren S, et al. Delving deep into rectifiers: surpassing human-level performance on ImageNet classification. Proceedings of 2015 IEEE International Conference on Computer Vision, Dec 07 - 13, 2015, Santiago, Chile. New York, NY, USA: IEEE, 2015: 1026 -1034

34. Netzer Y, Wang T, Coates A, et al. Reading digits in natural images with unsupervised feature learning. Proceedings of 21st International Conference on Pattern Recognition, Nov 11 -15, 2012, Tsukuba, Japan. New York, NY, USA: IEEE, 2012: 3304 -3308

35. Krizhevsky A, Hinton G. Learning multiple layers of features from tiny images. Technical Report, Computer Science Department, University of Toronto, 2009

36. Nilsback M E, Zisserman A. A visual vocabulary for flower classification. Proceedings of 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, June 17 -22, 2006, New York, NY, USA, USA. New York, NY USA: IEEE, 2006: 1447 -1454

37. Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks. Proceedings of the 25th International Conference on Neural Information Processing Systems, Dec 03 -06, 2012, Lake Tahoe, Nevada. USA: Curran Associates Inc, 2012:1097 -1105

38. Kulis B, Darrell T. Learning to hash with binary reconstructive embeddings. Proceedings of the 22nd International Conference on Neural Information Processing Systems, Dec 07 -10, 2009, Vancouver, British Columbia, Canada. USA: Curran Associates Inc, 2009: 1042 -1050

39. Norouzi M, Fleet D J. Minimal loss hashing for compact binary codes. Proceedings of the 28th International Conference on International Conference on Machine Learning, June 28 -July 02, 2011, Bellevue, Washington, USA. USA: Omnipress, 2011: 353 -360

[1]	Li Hao, Zhang Linghua, Tong Cheng, Zhou Chenyang. Short-term load forecasting model based on gated recurrent unit and multi-head attention [J]. 中国邮电高校学报(英文版), 2023, 30(3): 25-31.
[2]	Du Rong, Chen Shudong, Li Weiwei, Zhang Xueting, Wang Xianhui, Ge Jin. Data augmentation via joint multi-scale CNN and multi-channel attention for bumblebee image generation [J]. 中国邮电高校学报(英文版), 2023, 30(3): 32-40.
[3]	吴青王凡范九伦侯静. L2,1-norm robust regularized extreme learning machine for regression using CCCP method [J]. 中国邮电高校学报(英文版), 2023, 30(2): 61-72.
[4]	Wu Qing, Li Feiyan, Zhang Hengchang, Fan Jiulun, Gao Xiaofeng. Least squares twin support vector machine with asymmetric squared loss[J]. 中国邮电高校学报(英文版), 2023, 30(1): 1-16.
[5]	Zhang Huibin, Li Tianzhu, Liu Haojiang, Li Zhuotong. Deep learning-based symbol detection algorithm in IMDD-OOFDM system [J]. 中国邮电高校学报(英文版), 2022, 29(6): 36-45.
[6]	Jiang Fan, Chen Jiajun, Gao Youjun, Sun Changyin. Research on ECG classification based on transfer learning [J]. 中国邮电高校学报(英文版), 2022, 29(6): 83-96.
[7]	段炼唐贵进. Low-light image enhancement algorithm using a residual network with semantic information[J]. 中国邮电高校学报(英文版), 2022, 29(2): 52-62.
[8]	Wu Qing, Fu Yanlin, Fan Jiulun, Ma Tianlu. Structural regularized twin support vector machine based on within-class scatter and between-class scatter[J]. 中国邮电高校学报(英文版), 2021, 28(4): 39-52.
[9]	李庆华张钊冯超沐雅琪尤越李研强. Human motion prediction using optimized sliding window polynomial fitting and recursive least squares [J]. 中国邮电高校学报(英文版), 2021, 28(3): 76-85.
[10]	焦继超陈新平管孟赵亚鑫. TCL: a taxi trajectory prediction model combining time and space features [J]. 中国邮电高校学报(英文版), 2021, 28(3): 63-75.
[11]	何明枢, 金磊, 王小娟, 李源. Web log classification framework with data augmentation based on GANs[J]. 中国邮电高校学报(英文版), 2020, 27(5): 34-46.
[12]	季一木, 李可, 刘尚东, 刘强, 尧海昌, 李奎. Collaborative filtering recommendation algorithm based on interactive data classification[J]. 中国邮电高校学报(英文版), 2020, 27(5): 1-12.
[13]	杨健健张强王晓林杜毅博王超吴淼. Research on equipment fault diagnosis method based on random stochastic adaptive particle swarm optimization [J]. 中国邮电高校学报(英文版), 2020, 27(4): 17-25.
[14]	Pang Hao, Bu Yunyun, Wang Cong, Xiao Hui. Automatic detection of breast nodule in the ultrasound images using CNN[J]. 中国邮电高校学报(英文版), 2019, 26(2): 9-16.
[15]	Yang Ridong, Zhang Shiyu, Li Lin, Wang Zhe, Zhou Yi. Using boosting tree to learn imbalanced data[J]. 中国邮电高校学报(英文版), 2019, 26(2): 43-51.

Supervised learning of enhancing convolutional Hash for image retrieval

Supervised learning of enhancing convolutional Hash for image retrieval

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价