Journal of Sensor Science and Technology (센서학회지)

The Korean Sensors Society (한국센서학회)
권호 표지
DOI QR코드

DOI QR Code

Robust architecture search using network adaptation

  • Rana, Amrita (Department of Electronic Engineering, Daegu Universtiy) ;
  • Kim, Kyung Ki (Department of Electronic Engineering, Daegu Universtiy)
  • Received : 2021.09.13
  • Accepted : 2021.09.29
  • Published : 2021.09.30
Download PDF
⟨ Previous Next ⟩

Abstract

Experts have designed popular and successful model architectures, which, however, were not the optimal option for different scenarios. Despite the remarkable performances achieved by deep neural networks, manually designed networks for classification tasks are the backbone of object detection. One major challenge is the ImageNet pre-training of the search space representation; moreover, the searched network incurs huge computational cost. Therefore, to overcome the obstacle of the pre-training process, we introduce a network adaptation technique using a pre-trained backbone model tested on ImageNet. The adaptation method can efficiently adapt the manually designed network on ImageNet to the new object-detection task. Neural architecture search (NAS) is adopted to adapt the architecture of the network. The adaptation is conducted on the MobileNetV2 network. The proposed NAS is tested using SSDLite detector. The results demonstrate increased performance compared to existing network architecture in terms of search cost, total number of adder arithmetics (Madds), and mean Average Precision(mAP). The total computational cost of the proposed NAS is much less than that of the State Of The Art (SOTA) NAS method.

Keywords

Acknowledgement

This research was supported by the Ministry of Science and ICT (MSIT), Korea, under the Information Technology Research Center (ITRC) support program (IITP-2021-0-02052) supervised by the Institute for Information & Communications Technology Planning & Evaluation (IITP). This work was supported by Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government(MSIT) (2020-0-01080, Variable-precision deep learning processor technology for high-speed multiple object tracking).

References

  1. T. Elsken, J. H. Metzen, and F. Hutter, "Neural architecture search: A survey", J. Mach. Learn. Res., Vol. 20, No.1, pp. 1-21, 2019.
  2. B. Zoph and Q. V. Le, "Neural architecture search with reinforcement learning", Proc. of Int. Conf. on Learn. Representations, pp. 1611.01578(1)-1611.01578(16), Toulon, France, 2017.
  3. H. Pham, M. Y. Guan, B. Zoph, Q. V. Le, and J. Dean, "Efficient neural architecture search via parameter sharing", Proc. of Int. Conf. on Mach. Learn., pp. 4095-4104, Stockholm, Sweden, 2018.
  4. B. Zoph, V. Vasudevan, J. Shlens, and Q. V. Le. "Learning transferable architectures for scalable image recognition", Proc. of IEEE Conf. on Comput. Vis. Pattern Recognit., pp. 8697-8710, Salt Lake City, Utah, 2018.
  5. E. Real, A. Aggarwal, Y. Huang, and Q. V. Le. "Regularized evolution for image classifier architecture search", Proc. of AAAI Conf. on artif. intel., Hawaii, United States, pp. 4780-4789, 2019.
  6. H. Liu, K. Simonyan, and Y. Yang, "DARTS: Differentiable architecture search", Proc. of Int. Conf. on Learn. Representations, pp. 1806.09055(1)-1806.09055(13), New Orleans, United States, 2019.
  7. R. J. Wang, X. Li, and C. X. Ling, "Pelee: A real-time object detection system on mobile devices", Proc. of Adv. Neural. Inf. Process. Syst., pp. 184.06882(1)-184.06882(10), Montreal, Canada, 2018.
  8. G. Huang, Z. Liu, L. V. D. Maaten, and K. Q. Weinberger, "Densely connected convolutional networks", Proc. of IEEE Conf. on Comput. Vis. Pattern Recognit., pp.4700-4708, Honolulu, Hawaii, 2017.
  9. Z. Li, C. Peng, G. Yu, X. Zhang, Y. Deng, and J. Sun, "Detnet: Design backbone for object detection", Proc. of Comput. Vis. ECCV, pp. 334-350, Munich, Germany, 2018.
  10. Y. Chen, T. Yang, X. Zhang, G. Meng, X. Xiao, and J. Sun, "DetNAS: Backbone search for object detection", Proc. of Adv. Neural. Inf. Process. Syst., pp. 6642-6652, Montreal, Canada, 2019.
  11. J. Deng, W. Dong, R. Socher, L. J. Li, K. Li, and F. F. Li, "Imagenet: A large-scale hierarchical image database", Proc. IEEE Conf. on Comput. Vis. Pattern Recognit., pp. 248-255, Miami, Florida, 2009.
  12. M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L. C. Chen, "MobilenetV2: Inverted residuals and linear bottlenecks", Proc. of IEEE Conf. on Comput. Vis. Pattern Recognit., pp. 4510-4520, Salt Lake City, Utah, 2018.
  13. W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C. Y. Fu, and A. C. Berg, "SSD: Single shot multibox detector", Proc. of Comput. Vis. ECCV, pp. 21-37, Amsterdam, Netherlands, 2016.
  14. T. Lin, M. Maire, S. J. Belongie, L. Bourdev, R. Girshick, J. Hays, P. Perona, D. Ramanan, C. L. Zitnick, and P. Dollar, "Microsoft coco: Common objects in context", Proc. of Comput. Vis. ECCV, pp. 740-755, Zurich, Switzerland, 2014.