International Journal of Precision Engineering and Manufacturing-Green Technology

Korean Society for Precision Engineering (한국정밀공학회)
권호 표지
DOI QR코드

DOI QR Code

Machine Learning-Based Imaging System for Surface Defect Inspection

  • Received : 2015.04.15
  • Accepted : 2016.06.02
  • Published : 2016.07.01
⟨ Previous Next ⟩

Abstract

Modern inspection systems based on smart sensor technology like image processing and machine vision have been widely spread into several fields of industry such as process control, manufacturing, and robotics applications in factories. Machine learning for smart sensors is a key element for the visual inspection of parts on a product line that has been manually inspected by people. This paper proposes a method for automatic visual inspection of dirties, scratches, burrs, and wears on surface parts. Imaging analysis with CNN (Convolution Neural Network) of training samples is applied to confirm the defect's existence in the target region of an image. In this paper, we have built and tested several types of deep networks of different depths and layer nodes to select adequate structure for surface defect inspection. A single CNN based network is enough to test several types of defects on textured and non-textured surfaces while conventional machine learning methods are separately applied according to type of each surface. Experiments for surface defects in real images prove the possibility for use of imaging sensors for detection of different types of defects. In terms of energy saving, the experiment result shows that proposed method has several advantages in time and cost saving and shows higher performance than traditional manpower inspection system.

Keywords

Acknowledgement

Supported by : NRF

References

  1. Aleixos, N., Blasco, J., Navarron, F., and Molto, E., "Multispectral Inspection of Citrus in Real-Time Using Machine Vision and Digital Signal Processors," Computers and Electronics in Agriculture, Vol. 33, No. 2, pp. 121-137, 2002. https://doi.org/10.1016/S0168-1699(02)00002-9
  2. Espiau, B., Chaumette, F., and Rives, P., "A New Approach to Visual Servoing in Robotics," IEEE Transactions on Robotics and Automation, Vol. 8, No. 3, pp. 313-326, 1992. https://doi.org/10.1109/70.143350
  3. Cheng, Y. and Jafari, M. A., "Vision-Based Online Process Control in Manufacturing Applications," IEEE Transactions on Automation Science and Engineering, Vol. 5, No. 1, pp. 140-153, 2008. https://doi.org/10.1109/TASE.2007.912058
  4. Funck, J., Zhong, Y., Butler, D., Brunner, C., and Forrer, J., "Image Segmentation Algorithms Applied to Wood Defect Detection," Computers and Electronics in Agriculture, Vol. 41, No. 1, pp. 157- 179, 2003. https://doi.org/10.1016/S0168-1699(03)00049-8
  5. Yang, W., Li, D., Zhu, L., Kang, Y., and Li, F., "A New Approach for Image Processing in Foreign Fiber Detection," Computers and Electronics in Agriculture, Vol. 68, No. 1, pp. 68-77, 2009. https://doi.org/10.1016/j.compag.2009.04.005
  6. Kumar, A. and Pang, G. K., "Defect Detection in Textured Materials Using Gabor Filters," IEEE Transactions on Industry Applications, Vol. 38, No. 2, pp. 425-440, 2002. https://doi.org/10.1109/28.993164
  7. Tsai, D.-M. and Lai, S.-C., "Defect Detection in Periodically Patterned Surfaces Using Independent Component Analysis," Pattern Recognition, Vol. 41, No. 9, pp. 2813-2832, 2008.
  8. Latif-Amet, A., Ertüzün, A., and Erçil, A., "An Efficient Method for Texture Defect Detection: Sub-Band Domain Co-Occurrence Matrices," Image and Vision Computing, Vol. 18, No. 6, pp. 543- 553, 2000. https://doi.org/10.1016/S0262-8856(99)00062-1
  9. Cord, A. and Chambon, S., "Automatic Road Defect Detection by Textural Pattern Recognition Based on AdaBoost," Computer-Aided Civil and Infrastructure Engineering, Vol. 27, No. 4, pp. 244-259, 2012. https://doi.org/10.1111/j.1467-8667.2011.00736.x
  10. Shumin, D., Zhoufeng, L., and Chunlei, L., "AdaBoost Learning for Fabric Defect Detection Based on HOG and SVM," Proc. of ICMT on Multimedia Technology, pp. 2903-2906, 2011.
  11. Fukushima, K., "Neocognitron: A Self-Organizing Neural Network Model for a Mechanism of Pattern Recognition Unaffected by Shift in Position," Biological Cybernetics, Vol. 36, No. 4, pp. 193-202, 1980. https://doi.org/10.1007/BF00344251
  12. Le Cun, Y., Bottou, L., Bengio, Y., and Haffner, P., "Gradient-Based Learning Applied to Document Recognition," Proc. of the IEEE, Vol. 86, No. 11, pp. 2278-2324, 1998. https://doi.org/10.1109/5.726791
  13. Hinton, G. E., Osindero, S., and Teh, Y.-W., "A Fast Learning Algorithm for Deep Belief Nets," Neural Computation, Vol. 18, No. 7, pp. 1527-1554, 2006. https://doi.org/10.1162/neco.2006.18.7.1527
  14. Collobert, R. and Weston, J., "A Unified Architecture for Natural Language Processing: Deep Neural Networks with Multitask Learning," Proc. of the 25th ICML, Vol. 25, pp. 160-167, 2008.
  15. Hinton, G., Deng, L., Yu, D., Dahl, G. E., Mohamed, A.-R., et al., "Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups," IEEE Signal Processing Magazine, Vol. 29, No. 6, pp. 82-97, 2012. https://doi.org/10.1109/MSP.2012.2205597
  16. Sonka, M., Hlavac, V., and Boyle, R., "Image Processing, Analysis, and Machine Vision," Cengage Learning, pp. 407-409, 2014.
  17. Soukup, D. and Huber-Mork, R., "Convolutional Neural Networks for Steel Surface Defect Detection from Photometric Stereo Images," in: Advanced is Visual Computing, George, B., Richard, B., (Eds.), Springer, pp. 668-677, 2014.
  18. Yao, S. and Hai-Ru, L., "Detection of Weft Knitting Fabric Defects Based on Windowed Texture Information and Threshold Segmentation by CNN," Proc. of IEEE on Digital Image Processing, pp. 292-296, 2009.
  19. Kwon, B.-K., Won, J.-S., and Kang, D.-J., "Fast Defect Detection for Various Types of Surfaces Using Random Forest with VOV Features," Int. J. Precis. Eng. Manuf., Vol. 16, No. 5, pp. 965-970, 2015. https://doi.org/10.1007/s12541-015-0125-y

Cited by

  1. Multi-task convolutional neural network system for license plate recognition vol.15, pp.6, 2017, https://doi.org/10.1007/s12555-016-0332-z
  2. 표면 결함 검출을 위한 CNN 구조의 비교 vol.66, pp.7, 2016, https://doi.org/10.5370/kiee.2017.66.7.1100
  3. 전자부품의 표면 결함 검출을 위한 CNN 기법 vol.27, pp.3, 2016, https://doi.org/10.5391/jkiis.2017.27.3.195
  4. 표면 결함 검출을 위한 데이터 확장 및 성능분석 vol.67, pp.5, 2016, https://doi.org/10.5370/kiee.2018.66.5.669
  5. Vision-Based Defect Detection for Mobile Phone Cover Glass using Deep Neural Networks vol.19, pp.6, 2016, https://doi.org/10.1007/s12541-018-0096-x
  6. Una Revisión Sistemática de Métodos para Localizar Automáticamente Objetos en Imágenes vol.15, pp.3, 2018, https://doi.org/10.4995/riai.2018.10229
  7. 금속 표면의 결함 검출을 위한 영역 기반 CNN 기법 비교 vol.67, pp.7, 2016, https://doi.org/10.5370/kiee.2018.67.7.865
  8. Research Progress of Visual Inspection Technology of Steel Products-A Review vol.8, pp.11, 2016, https://doi.org/10.3390/app8112195
  9. Convolutional Neural Network Based Surface Inspection System for Non-patterned Welding Defects vol.20, pp.3, 2019, https://doi.org/10.1007/s12541-019-00074-4
  10. Blister Defect Detection Based on Convolutional Neural Network for Polymer Lithium-Ion Battery vol.9, pp.6, 2016, https://doi.org/10.3390/app9061085
  11. SDD-CNN: Small Data-Driven Convolution Neural Networks for Subtle Roller Defect Inspection vol.9, pp.7, 2019, https://doi.org/10.3390/app9071364
  12. Surface Defects Recognition of Wheel Hub Based on Improved Faster R-CNN vol.8, pp.5, 2019, https://doi.org/10.3390/electronics8050481
  13. A Fuzzy Support Vector Machine-Enhanced Convolutional Neural Network for Recognition of Glass Defects vol.21, pp.6, 2016, https://doi.org/10.1007/s40815-019-00697-9
  14. A review of machine learning for the optimization of production processes vol.104, pp.5, 2016, https://doi.org/10.1007/s00170-019-03988-5
  15. Fruit Image Classification Using Convolutional Neural Networks : vol.7, pp.4, 2016, https://doi.org/10.4018/ijsi.2019100103
  16. Fruit Image Classification Using Convolutional Neural Networks : vol.7, pp.4, 2016, https://doi.org/10.4018/ijsi.2019100103
  17. USING METHOD OF MACHINE TRAINING AND ARTIFICIAL INTELLIGENCE IN CHEMICAL TECHNOLOGY. PART II vol.2019, pp.5, 2016, https://doi.org/10.17122/ogbus-2019-5-202-238
  18. Detection of Micro-Defects on Irregular Reflective Surfaces Based on Improved Faster R-CNN vol.19, pp.22, 2016, https://doi.org/10.3390/s19225000
  19. Steel Surface Defect Diagnostics Using Deep Convolutional Neural Network and Class Activation Map vol.9, pp.24, 2016, https://doi.org/10.3390/app9245449
  20. Effective image models for inspecting profile flaws of car mirrors with applications vol.18, pp.1, 2016, https://doi.org/10.5937/jaes18-22825
  21. Strain measurement during tensile testing using deep learning-based digital image correlation vol.31, pp.1, 2016, https://doi.org/10.1088/1361-6501/ab29d5
  22. Artificial Auditory Perception Pattern Recognition System Based on Spatiotemporal Convolutional Neural Network vol.10, pp.1, 2016, https://doi.org/10.3390/app10010139
  23. Deeper and Mixed Supervision for Salient Object Detection in Automated Surface Inspection vol.2020, pp.None, 2020, https://doi.org/10.1155/2020/3751053
  24. Advanced Data Collection and Analysis in Data-Driven Manufacturing Process vol.33, pp.1, 2016, https://doi.org/10.1186/s10033-020-00459-x
  25. 3D Printed Electronics of Non-contact Ink Writing Techniques: Status and Promise vol.7, pp.2, 2016, https://doi.org/10.1007/s40684-019-00139-9
  26. Automatic classification of pavement crack using deep convolutional neural network vol.21, pp.4, 2020, https://doi.org/10.1080/10298436.2018.1485917
  27. Deep Learning Based Defect Inspection Using the Intersection Over Minimum Between Search and Abnormal Regions vol.21, pp.4, 2020, https://doi.org/10.1007/s12541-019-00269-9
  28. Case study: Performance analysis and development of robotized screwing application with integrated vision sensing system for automotive industry vol.17, pp.3, 2016, https://doi.org/10.1177/1729881420923997
  29. Improving the Accuracy of Convolutional Neural Networks by Identifying and Removing Outlier Images in Datasets Using t-SNE vol.8, pp.5, 2020, https://doi.org/10.3390/math8050662
  30. 기계 학습을 활용한 이미지 결함 검출 모델 개발 vol.15, pp.3, 2020, https://doi.org/10.13067/jkiecs.2020.15.3.513
  31. Microstructural inelastic fingerprints and data-rich predictions of plasticity and damage in solids vol.66, pp.1, 2016, https://doi.org/10.1007/s00466-020-01845-x
  32. Surface defect classification and detection on extruded aluminum profiles using convolutional neural networks vol.13, pp.4, 2016, https://doi.org/10.1007/s12289-019-01496-1
  33. Classification of grazing-incidence small-angle X-ray scattering patterns by convolutional neural network vol.27, pp.4, 2020, https://doi.org/10.1107/s1600577520005767
  34. Classification using a three-dimensional sensor in a structured industrial environment vol.29, pp.4, 2016, https://doi.org/10.1117/1.jei.29.4.041008
  35. Defect Detection of Industry Wood Veneer Based on NAS and Multi-Channel Mask R-CNN vol.20, pp.16, 2016, https://doi.org/10.3390/s20164398
  36. An Efficient Network for Surface Defect Detection vol.10, pp.17, 2020, https://doi.org/10.3390/app10176085
  37. Binary classification with ambiguous training data vol.109, pp.12, 2016, https://doi.org/10.1007/s10994-020-05915-2
  38. Image-Based Surface Defect Detection Using Deep Learning: A Review vol.21, pp.4, 2021, https://doi.org/10.1115/1.4049535
  39. Artificial intelligence and internet of things in small and medium-sized enterprises: A survey vol.58, pp.2, 2016, https://doi.org/10.1016/j.jmsy.2020.08.009
  40. Improving Transactional Data System Based on an Edge Computing-Blockchain-Machine Learning Integrated Framework vol.9, pp.1, 2016, https://doi.org/10.3390/pr9010092
  41. Research on Detection Method of Sheet Surface Defects Based on Machine Vision vol.632, pp.None, 2016, https://doi.org/10.1088/1755-1315/632/5/052085
  42. A deep learning-based approach for the automated surface inspection of copper clad laminate images vol.51, pp.3, 2021, https://doi.org/10.1007/s10489-020-01877-z
  43. Machine learning-based system for fault detection on anchor rods of cable-stayed power transmission towers vol.194, pp.None, 2016, https://doi.org/10.1016/j.epsr.2021.107106
  44. Analysis of the Region of Interest According to CNN Structure in Hierarchical Pattern Surface Inspection Using CAM vol.14, pp.9, 2021, https://doi.org/10.3390/ma14092095
  45. Quality prediction for aluminum diecasting process based on shallow neural network and data feature selection technique vol.33, pp.None, 2016, https://doi.org/10.1016/j.cirpj.2021.04.001
  46. A Technology of Surface Defects for the Solar Wafer by Dual Vision vol.769, pp.4, 2021, https://doi.org/10.1088/1755-1315/769/4/042007
  47. Surface defect detection of steel strips based on classification priority YOLOv3-dense network vol.48, pp.5, 2016, https://doi.org/10.1080/03019233.2020.1816806
  48. Printed Circuit Board Defect Detection Using Deep Learning via A Skip-Connected Convolutional Autoencoder vol.21, pp.15, 2021, https://doi.org/10.3390/s21154968
  49. Improved SSD-assisted algorithm for surface defect detection of electromagnetic luminescence vol.235, pp.5, 2016, https://doi.org/10.1177/1748006x21995388
  50. RoadID: A Dedicated Deep Convolutional Neural Network for Multipavement Distress Detection vol.147, pp.4, 2016, https://doi.org/10.1061/jpeodx.0000317
  51. Online recognition of magnetic tile defects based on UPM-DenseNet vol.30, pp.None, 2016, https://doi.org/10.1016/j.mtcomm.2021.103105