Exploring the usefulness of light field super-resolution for object detection

doi:10.19682/j.cnki.1005-8885.2021.0023

中国邮电高校学报(英文版) ›› 2021, Vol. 28 ›› Issue (5): 68-81.doi: 10.19682/j.cnki.1005-8885.2021.0023

• Artificial Intelligence • 上一篇下一篇

Exploring the usefulness of light field super-resolution for object detection

张文喆¹,石凡¹,²,赵萌¹,²,陈胜勇¹,²

1. 天津理工大学
2.

收稿日期:2020-09-17 修回日期:2020-12-22 出版日期:2021-10-31 发布日期:2021-10-29
通讯作者: 石凡 E-mail:shifan@email.tjut.edu.cn
基金资助:
国家自然科学基金;国家重点研发计划

Exploring the usefulness of light field super-resolution for object detection

Received:2020-09-17 Revised:2020-12-22 Online:2021-10-31 Published:2021-10-29
Contact: fan .shi E-mail:shifan@email.tjut.edu.cn
Supported by:
National Natural Science Foundation of China;National Key R&D Program of China

摘要/Abstract

摘要：

In order to solve the impact of image degradation on object detection, an object detection method based on light field super-resolution ( LFSR) is proposed. This method takes LFSR as an image enhancement step to provide high- quality images for object detection without using expensive imaging equipment. To evaluate this method, three types of objects: person, bicycle, and car, are chosen and the results are compared from 5 parts: detected object quantity, mean confidence score, detection results in different scenes, error detection, and detection results from different images sizes and detection speed. Experimental results based on the common object in context ( COCO) dataset show that the method incorporated LFSR improves performance of object detection models.

关键词:

light-field ( LF), super-resolution ( SR), object detection, RetinaNet, YOLOv3, TinyYOLOv3

Abstract:

Key words:

light-field ( LF), super-resolution ( SR), object detection, RetinaNet, YOLOv3, TinyYOLOv3

参考文献

1. Girshick R, Donahue J, Darrell T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition ( CVPR’14), 2014, Jun 23 - 28, Columbus, OH, USA. Piscataway, NJ, USA: IEEE, 2014: 580 - 587

2. Girshick R. Fast R-CNN. Proceedings of the 2015 IEEE International Conference on Computer Vision ( ICCV’15 ), 2015, Dec 7 - 13, Santiago, Chile. Piscataway, NJ, USA: IEEE, 2015: 1440 - 1448

3. Ren S Q, He K M, Girshick R, et al. Faster R-CNN: towards real-time object detection with region proposal networks. Advances in Neural Information Processing Systems 28: Proceedings of the 29th Annual Conference on Neural Information Processing Systems: Vol 1, 2015, Dec 7 - 12, Montreal, Canada. Red Hook, NY,USA: Curran Associates Inc, 2015: 91 - 99

4. He K M, Gkioxari G, Doll佗r P, et al. Mask R-CNN. Proceedings of the 2017 IEEE International Conference on Computer Vision ( ICCV’17), 2017, Oct 22 - 29, Venice, Italy. Piscataway, NJ, USA: IEEE, 2017: 2980 - 2988

5. Liu W, Anguelov D, Erhan D, et al. SSD: single shot multibox detector. Proceedings of the 14th European Conference on Computer Vision ( ECCV’16 ), 2016, Oct 8 - 16, Amsterdam, Netherlands. LNCS 9905. Berlin, Germany: Springer, 2016: 21 - 37

6. Redmon J, Divvala S, Girshick R, et al. You only look once: unifled, real-time object detection. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition ( CVPR’16 ), 2016, Jun 27 - 30, Las Vegas, NV, USA. Piscataway, NJ, USA: IEEE, 2016: 779 - 788

7. Rossi M, Frossard P. Geometry-consistent light field super- resolution via graph-based regularization. IEEE Transactions on Image Processing, 2018, 27(9): 4207 - 4218

8. Gershun A. The light field. Journal of Mathematics and Physics, 1939, 18(1 / 2 / 3 / 4): 51 - 151

9. Adelson E H, Bergen J R. The plenoptic function and the elements of early vision. Computational Models of Visual Processing, 1991, 1(2): 3 - 20

10. Levoy M, Hanrahan P. Light field rendering. Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques ( SIGGRAPH’96 ), 1996, Aug 4 - 9, New Orleans, LA, USA. New York, NY, USA: ACM, 1996: 31 - 42

11. Adelson E H, Wang J Y A. Single lens stereo with a plenoptic camera. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1992, 14(2): 99 - 106

12. Ng R. Digital light field photography. Ph D Thesis. Palo Alto, CA, USA: Stanford University, 2006

13. Jeon H G, Park J, Choe G, et al. Accurate depth map estimation from a lenslet light field camera. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition ( CVPR’15), 2015, Jun 7 - 12, Boston, MA, USA. Piscataway, NJ, USA: IEEE, 2015: 1547 - 1555

14. Yang J C, Wright J, Huang T S, et al. Image super-resolution via sparse representation. IEEE Transactions on Image Processing, 2010, 19(11): 2861 - 2873

15. Gao X B, Zhang K B, Tao D C, et al. Joint learning for single-image super-resolution via a coupled constraint. IEEE Transactions on Image Processing, 2012, 21(2): 469 - 480

16. Dong C, Loy C C, He K M, et al. Learning a deep convolutional network for image super-resolution. Proceedings of the 13th European Conference on Computer Vision ( ECCV’14 ): Part IV, 2014, Sept 6 - 12, Zurich, Switzerland. LNCS 8692. Berlin, Germany: Springer, 2014: 184 - 199

17. Bishop T E, Zanetti S, Favaro P. Light field superresolution. Proceedings of the 2009 IEEE International Conference on Computational Photography ( ICCP’09), 2009, Apr 16 - 17, San Francisco, CA, USA. Piscataway, NJ, USA: IEEE, 2009: 1 - 9

18. Bishop T E, Favaro P. Full-resolution depth map estimation from an aliased plenoptic light field. Proceedings of the 10th Asian Conference on Computer Vision ( ACCV’10 ): Part II, 2010, Nov 8 - 12, Queenstown, New Zealand. LNCS 6493. Berlin, Germany: Springer, 2010: 186 - 200

19. Wanner S, Goldluecke B. Spatial and angular variational super-resolution of 4D light fields. Proceedings of the 12th European Conference on Computer Vision ( ECCV’12 ): Part V, 2012, Oct 7 - 13, Florence, Italy. LNCS 7576. Berlin, Germany: Springer, 2012: 608 - 621

20. Wanner S, Goldluecke B. Variational light field analysis for disparity estimation and super-resolution. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 36 ( 3 ): 606 - 619

21. Mitra K, Veeraraghavan A. Light field denoising, light field superresolution and stereo camera based refocussing using a GMM light field patch prior. Proceedings of the 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops ( CVPRW’12), 2012, Jun 16 - 21, Providence, RI, USA. Piscataway, NJ, USA: IEEE, 2012: 22 - 28

22. Cho D, Kim S, Tai Y W. Consistent matting for light field images. Proceedings of the 13th European Conference on Computer Vision

( ECCV’14): Part IV, 2014, Sept 6 - 12, Zurich, Switzerland. LNCS 8692. Berlin, Germany: Springer, 2014: 90 - 104

23. Cui Z, Chang H, Shan S G, et al. Deep network cascade for image super-resolution. Proceedings of the 13th European Conference on

Computer Vision ( ECCV’14 ): Part V, 2014, Sept 6 - 12, Zurich, Switzerland. LNCS 8693. Berlin, Germany: Springer, 2014: 49 - 64

24. Yoon Y, Jeon H G, Yoo D, et al. Learning a deep convolutional network for light-field image super-resolution. Proceedings of the 2015 IEEE International Conference on Computer Vision Workshop ( ICCVW’15 ), 2015, Dec 7 - 13, Santiago, Chile. Piscataway, NJ, USA: IEEE, 2015: 57 - 65

25. Yoon Y, Jeon H G, Yoo D, et al. Light-field image super-resolution using convolutional neural network. IEEE Signal Processing Letters, 2017, 24(6): 848 - 852

26. Gul M S K, Gunturk B K. Spatial and angular resolution enhancement of light fields using convolutional neural networks. IEEE Transactions on Image Processing, 2018, 27 ( 5 ): 2146 -2159

27. Wang Y L, Liu, F, Zhang K B, et al. LFNet: a novel bidirectional recurrent convolutional neural network for light-field image super-resolution. IEEE Transactionss on Image Processing, 2018, 27(9): 4274 - 4286

28. Zhang S, Lin Y F, Sheng H. Residual networks for light fieldimage super-resolution. Proceedings of the 2019 IEEE / CVF Conferenceon Computer Vision and Pattern Recognition ( CVPR’19 ), 2019, Jun 15 - 20, Long Beach, CA, USA. Piscataway, NJ, USA: IEEE, 2019: 11038 - 11047

29. Kim J, Lee J K, Lee K M. Accurate image super-resolution using very deep convolutional networks. Proceedings of the 2016 IEEE Conferenceon on Computer Vision and Pattern Recognition ( CVPR’16 ), 2016, Jun 27 - 30, Las Vegas, NV, USA. Piscataway, NJ, USA: IEEE, 2016: 1646 - 1654

30. Lim B, Son S, Kim H, et al. Enhanced deep residual networks for single image super-resolution. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops ( CVPRW’17 ), 2017, Jul 21 - 26, Honolulu, HI, USA. Piscataway, NJ, USA: IEEE, 2017: 1132 - 1140

31. Zhang Y L, Tian Y P, Kong Y, et al. Residual dense network for image super-resolution. Proceedings of the 2018 IEEE / CVF Conference on Computer Vision and Pattern Recognition, 2018, Jun 18 - 23, Salt Lake City, UT, USA. Piscataway, NJ, USA: IEEE, 2018: 2472 - 2481

32. Viola P, Jones M. Rapid object detection using a boosted cascade of simple features. Proceedings of the 2001 IEEE Computer Society

Conference on Computer Vision and Pattern Recognition ( CVPR’01), 2001, Dec 8 - 14, Kauai, HI, USA. Piscataway, NJ, USA: IEEE, 2001: I. 511 - I. 518

33. Viola P, Jones M J. Robust real-time face detection. International Journal of Computer Vision, 2004, 57(2): 137 - 154

34. Dalal N, Triggs B. Histograms of oriented gradients for human detection. Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition ( CVPR’05): Vol 1, 2005, Jun 20 - 25, San Diego, CA, USA. Piscataway, NJ, USA: IEEE, 2005: 886 - 893

35. Felzenszwalb P, McAllester D, Ramanan D. A discriminatively trained, multiscale, deformable part model. Proceedings of the 2008 IEEE Conference on Computer Vision and Pattern Recognition ( CVPR’08 ), 2008, Jun 23 - 28, Anchorage, AK, USA. Piscataway, NJ, USA: IEEE, 2008: 1 - 8

36. He K M, Zhang X Y, Ren S Q, et al. Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(9): 1904 - 1916

37. Zhao X, Ni Y T, Jia H H. Modifled object detection method based on YOLO’. In: Yang J. et al. ( eds) Computer Vision. CCCV 2017. Communications in Computer and Information Science, 2017, 773: 233 - 244

38. Lin T Y, Gogal P, Girshick R, et al. Focal loss for dense object detection. Proceedings of the 2nd CCF Chinese Conference on Computer Vision ( CCCV’17 ): Part III, 2017, Oct 11 - 14, Tianjin, China. CCIS 773. Berlin, Germany: Springer, 2017: 2999 - 3007

39. Redmon J, Farhadi A. YOLOv3: an incremental improvement. arXiv Preprint, 2018, arXiv:180402767

40. Redmon J, Farhadi A. YOLO9000: better, faster, stronger. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition ( CVPR’17 ), 2017, Jul 21 - 26, Honolulu, HI, USA. Piscataway, NJ, USA: IEEE, 2017: 7263 - 7266

Exploring the usefulness of light field super-resolution for object detection

Exploring the usefulness of light field super-resolution for object detection

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 0

编辑推荐

Metrics

本文评价