Seru üretim kontrolü için gerçek zamanlı görüntü işleme sistemi tasarımı model önerisi = A proposal of real time image processing system design model for seru production control

Abstract

Seru üretim sistemi atölye tipi üretimin esnekliği ile montaj hatlarının yüksek verimliliğini birleştiren yenilikçi bir montaj sistemidir. Bu üretim sistemi, ürün değişimlerine kolaylıkla uyum sağlamakta, yarı mamul stoklarını ve gecikme süresini ortadan kaldırarak ürün israfını azaltmakta ve işletme maliyetlerini, gerekli işgücünü ve alanı azaltarak işletmelere rekabet avantajı sağlamaktadır. Bu avantajlarının yanı sıra, seru tipi üretimin bir ürünün montajı için gerekli tüm görevlerin çapraz eğitimli bir çalışan tarafından bir yatai(hücre)de tamamlanması gibi bir dezavantajı bulunmaktadır. Montaj hatlarında belirli görevler belirli istasyonlarda tamamlanırken, seru üretim sisteminde bahsedilen dezavantaj daha yüksek üretim hatası riskine neden olur. Bu çalışma, seru üretim sistemini Endüstri 4.0 teknolojileri ile bütünleştirerek bu hataların tespiti, önlenmesi ve en küçüklenmesini amaçlamaktadır. Bu amacı gerçekleştirmek için çalışmanın iki temel hedefi bulunmaktadır. İlk hedefi seru üretim sisteminde gerçek zamanlı bir kontrol sistemi modeli geliştirmektir. Bu hedefe yönelik, seru üretim sistemindeki proses ve kalite hatalarının önlenmesi için derin öğrenmeye dayalı gelişmiş analitikler kullanılarak üretim sürecinin çalışan, ortam, montaj araçları, ergonomi, depolama ve envanter gibi çoklu faktörlerinin izlenmesi, kontrol edilmesi ve gerektiğinde uyarı verilmesi için kavramsal bir model önerilmiştir. Önerilen Zeki Seru Üretim Sistemi Modeli, çalışana destek sağlamanın yanı sıra, sistem katılımcılarının üretim süreçleri hakkında veri elde etmelerine ve anlamalarına ve bu bilgilere dayanarak hızlı tepki vermelerine yardımcı olacaktır. İkinci temel hedefi, önerilen kavramsal modelin bünyesinde bulunan montaj eylemlerinin tanınması modelinin geliştirilmesidir. Bu hedefe yönelik ise, incelenen literatürde rastlanmayan iskelet tabanlı derin öğrenmeye dayalı Çift Yönlü Geçitli Tekrarlayan Birim (Bi-directional Gated Recurrent Unit - BiGRU) ve Evrişimsel Sinir Ağları (Convolutional Neural Network - CNN) modelini içeren hibrit CNN-BiGRU-CNN montaj eylemlerinin tanınması modeli bu çalışmada geliştirilmiştir. Çalışmada, çalışan iskelet verilerinden yararlanılarak derin öğrenmeye dayalı sınıflandırıcı modellerin performanslarının iyileştirilmesi için aşağı örnekleme ve ölçeklemeye dayalı iki aşamalı bir veri arttırma yaklaşımı önerilmiştir. Montaj eylemlerinin sınıflandırılması için CNN, Uzun Kısa Süreli Bellek (Long Short Term Memory - LSTM), Geçitli Tekrarlayan Birim (Gated Recurrent Unit - GRU), Çift Yönlü Uzun Kısa Süreli Bellek (Bi-directional Long Short Term Memory - BiLSTM), BiGRU ve bu modellerin kombinasyonları sonucu oluşturulan hibrit modeller olmak üzere derin öğrenmeye dayalı on farklı tahmin modeli geliştirilmiştir. Geliştirilen modellerin her birinin etkinliklerini değerlendirmek için toplanan montaj süreci görüntüleri üzerinde analizler yapılarak bunların performansları karşılaştırılmıştır. Deneyler sonucunda, önerilen veri arttırma yaklaşımının tüm modellerin tahmin performansını iyileştirdiği görülmüştür. En iyi montaj eylemi sınıflandırma doğruluğuna önerilen hibrit CNN-BiGRU-CNN modeli ile ulaşılmıştır. Geliştirilen model ile montaj hattında %97, seru üretim sisteminde ise %96 tahmin doğruluk oranlarına ulaşıldığı belirlenmiştir. Sınıflandırma algoritmalarının sınıf içi varyasyonlardaki farklılıklar azaldığında başarı oranlarının arttığı bilinmektedir. Bu çalışmada montaj hattında tahmin doğruluğunun daha yüksek olmasının nedeni, montaj hattında bulunan görevlerin seru üretim sistemine göre kendi içerisinde daha fazla benzerlik gösterebileceğidir.

As a result of technological developments, companies around the world have to design their production systems in a way that can meet customer demands, make profits and increase their competitiveness. Towards the end of Industry 1.0, the need for higher production volumes was addressed by Henry Ford's conveyor assembly line solution, which became the core innovation of Industry 2.0. Conveyor line systems have become widespread for use in the final assembly steps of mass production systems due to their high efficiency. Low product and labor costs and requiring less technical skills are among the other advantages of conveyor assembly line systems. As a result of the transformation of consumption patterns and the rapid development of digital technology, market demands in the Industry 3.0 process have shown an increasing trend towards high diversity and low volumes. Accordingly, product life cycles have shortened. Fluctuating and variable demand situations encountered due to such dynamic marketing conditions result in significant deterioration in the efficiency of conveyor assembly lines. Due to imbalances between stations on the assembly line, work-in-process are created, thus causing waste. Since it requires high capital costs, it becomes impossible to ensure the flexibility of the process in the production of new products. With Industry 4.0, customization requirements such as shorter delivery times and reconfigurability and flexibility of production processes can not be met in traditional assembly lines. Assembly lines must be reconfigurable to accommodate product changes and changes in the product family of customized products. Companies and organizations can achieve competitive advantage to the extent they can respond to such changes. However, these changes cause difficulties in maintaining balance for traditional assembly lines. It becomes very important for manufacturers to meet the flexibility of their production systems in order to cope with the demands arising from fluctuations in wide range and volumes of product diversity. For this reason, production systems targeting Industry 4.0 must be supported by up-to-date technologies and have high flexibility and fast response features. In order to meet these, the necessity of developing new production systems has emerged. Seru production system, which emerged in Japan in the 1990s as a result of increased product diversity and decreased demand for products, is an innovative assembly system and combines the flexibility of workshop type production with the high efficiency of assembly lines. This production system easily adapts to product changes, reduces product waste by eliminating semi-finished product stocks and delay time, and provides competitive advantage to businesses by reducing operating costs, required labor and space. In addition to these advantages, seru-type production has the disadvantage that all tasks required for the assembly of a product are completed in one yatai by a cross-trained worker. While certain tasks on assembly lines are completed at specific stations, the mentioned disadvantage in seru production causes a higher risk of production errors. This study aims to detect, prevent and eliminate these errors by integrating the seru production system with Industry 4.0 technologies. To achieve this aim, the study has two main objectives. Its first goal is to develop a real-time control system model in the seru production system. Towards this goal, a conceptual system model has been proposed to monitor and control multiple factors of the production process, such as workers, environment, assembly tools, ergonomics, storage and inventory, and to give warnings when necessary, in order to prevent process and quality errors in the seru production system. The proposed Smart Seru Production System Model has six components. These components can be listed as follows: • Monitoring and Controlling of Assembly Operations, • Monitoring and Controlling of Assembly Environment, • Monitoring and Controlling of Product Components and Assembly Tools, • Monitoring and Controlling of Capacity and Performance, • Monitoring and Controlling of Storage / Inventory, • Monitoring and Controlling of Ergonomics and Safety. A deep learning-based system architecture has been developed that supports the use of current technologies such as the Internet of Things (IoT) and augmented reality, which can be applied to each component to monitor and control the Seru system. Components are built in accordance with this architecture and are ensured to operate simultaneously. System components interact with each other. In addition, these components are planned to realize human activity recognition, object detection / tracking, time series prediction, posture analysis, indoor environment quality analysis, prediction / classification, delivery time prediction, worker performance evaluation implementations. In addition to providing support to the worker, the proposed Smart Seru Production System Model will help system participants obtain and understand data about production processes and react quickly based on this information. With the system model, customer, supplier, production environment information network and interaction will be realized, worker performance and distribution system will be monitored in real time, and appropriate workforce and supply chain decisions will be supported. The second main goal is to develop the recognition model of assembly actions included in the proposed conceptual model. In the assembly action recognition model, the basic task of determining the class of assembly actions that are complex and can be performed in different times will be carried out. For this task; • worker pose data obtained from images collected with a single camera during the assembly process, and • the deep learning-based assembly action recognition model developed in the study will be used. For the above-mentioned deep learning-based assembly action recognition model, a skeleton-based hybrid CNN-BiGRU-CNN model is proposed. Assembly process video data was collected from the assembly line and seru production system to cover all assembly operations required to complete a product. MediaPipe Holistic infrastructure, one of the pose estimation approaches, was used to obtain the worker skeleton data used in the model from RGB video images and provide it as input data to the model. With the use of MediaPipe (Hands), the Vhands data set was obtained by estimating only the key points on the hands. The Vhands-body dataset was obtained by combining the predictions of key points on the hands and the body with the hybrid use of MediaPipe (Hands) and MediaPipe (Pose) pose estimation models. A two-stage data augmentation approach based on down-sampling and scaling is proposed to improve the performance of deep learning-based classifier models. Ten different models based on deep learning have been developed for the classification of assembly actions, including CNN, LSTM, GRU, BiLSTM, BiGRU and hybrid models generated as a result of combinations of these models. The ten developed models were evaluated separately on the image data obtained from both the assembly line and seru production system, 80 experimental studies were carried out and compared in three aspects: 1- evaluation of the impact of the data augmentation approach: the impact of applying the data augmentation approach on classification prediction successes. 2- comparison of pose estimation approaches: Effect of Vhands dataset and Vhands-body datasets on classification prediction success. 3- comparison of the performances of deep learning models: evaluation of the classification prediction success of ten classifier deep learning models. When the effect of data augmentation approaches was evaluated, it was seen that there was an improvement in the determined performance criterion values of all assembly action recognition models developed on the Vhands data set and Vhands-body data set, with the application of the data augmentation approach, both in the experiments on the assembly line and in the experiments in the seru production system. In this sense, it has been shown that the data augmentation approach developed is effective in classifying assembly actions. As a result of the comparison of pose estimation approaches, it has been observed that almost all models in assembly line implementation have better classification power in terms of performance criteria determined when they are developed only on the Vhands-body dataset. In the seru production system implementation, it was observed that most of the models showed better performance on the Vhands dataset in experiments where the data augmentation approach was not applied, but the models with the highest performance criterion scores were the models developed on the Vhands-body dataset. As a result of comparing deep learning models with experiments, it has been observed that the hybrid use of the CNN model with LSTM, GRU, BiLSTM and BiGRU models in both assembly line implementation and seru production system implementation generally gives better results in terms of performance criteria (accuracy, precision, recall and F1-Score) than when these models are used individually. Considering all the experiments, better assembly action classification accuracy is observed by applying the data augmentation approach on the proposed hybrid CNN-BiGRU-CNN model Vhands-body dataset. With the developed model, prediction accuracy rates of 97% in the assembly line and 96% in the mass production system were achieved.