Classifying BISINDO Alphabet using TensorFlow Object Detection API


Lilis Nur Hayati(1*); Anik Nur Handayani(2); Wahyu Sakti Gunawan Irianto(3); Rosa Andrie Asmara(4); Dolly Indra(5); Muhammad Fahmi(6);

(1) Universitas Negeri Malang, Universitas Muslim Indonesia
(2) Universitas Negeri Malang
(3) Universitas Negeri Malang
(4) Politeknik Negeri Malang
(5) Universitas Muslim Indonesia
(6) Universitas Muslim Indonesia
(*) Corresponding Author

  

Abstract


Indonesian Sign Language (BISINDO) is one of the sign languages used in Indonesia. The process of classifying BISINDO can be done by utilizing advances in computer technology such as deep learning. The use of the BISINDO letter classification system with the application of the MobileNet V2 FPNLite  SSD model using the TensorFlow object detection API. The purpose of this study is to classify BISINDO letters A-Z and measure the accuracy, precision, recall, and cross-validation performance of the model. The dataset used was 4054 images with a size of  consisting of 26 letter classes, which were taken by researchers by applying several research scenarios and limitations. The steps carried out are: dividing the ratio of the simulation dataset 80:20, and applying cross-validation (k-fold = 5). In this study, a real time testing using 2 scenarios was conducted, namely testing with bright light conditions of 500 lux and dim light of 50 lux with an average processing time of 30 frames per second (fps). With a simulation data set ratio of 80:20, 5 iterations were performed, the first iteration yielded a precision result of 0.758 and a recall result of 0.790, and the second iteration yielded a precision result of 0.635 and a recall result of 0.77, then obtained an accuracy score of 0.712, the third iteration provides a recall score of 0.746, the fourth iteration obtains a precision score of 0.713 and a recall score of 0.751, the fifth iteration gives a precision score of 0.742 for a fit score case and the recall score is 0.773. So, the overall average precision score is 0.712 and the overall average recall score is 0.747, indicating that the model built performs very well.

Keywords


Artificial Intelligence; BISINDO; Computer Vision; Real-time; TensorFlow Object Detection API

  
  

Full Text:

PDF
  

Article Metrics

Abstract view: 587 times
PDF view: 272 times 		     

Digital Object Identifier

doi  https://doi.org/10.33096/ilkom.v15i2.1692.358-364 		  

Cite

How to cite item

References


V. Kissya, “Penggunaan Bahasa Isyarat Dalam Komunikasi Antara Penyandang Tuna Rungu, Guru, Serta Keluarga Di (Sekolah Luar Biasa Pelita Kasih) Rumah Tiga Ambon,” vol. 16, no. 1, 2022.

A. Dicky Saputra and I. B. Artambo Pangaribuan, “Klasifikasi Alfabet Bahasa Isyarat Indonesia (BISINDO) dengan Metode Template Matching dan K-Nearest Neighbors (KNN),” 2020.

A. Wadhawan and P. Kumar, “Sign Language Recognition Systems: A Decade Systematic Literature Review,” Archives of Computational Methods in Engineering, vol. 28, no. 3, pp. 785–813, May 2021, doi: 10.1007/s11831-019-09384-2.

A. Sri Nugraheni, A. Pratiwi Husain, and H. Unayah, “Optimalisasi Penggunaan Bahasa Isyarat dengan SIBI Dan BISINDO pada Mahasiswa Difabel Tunarungu di Prodi PGMI UIN Sunan Kalijaga,” 2021.

T. Handhika, R. I. M. Zen, Murni, D. P. Lestari, and I. Sari, “Gesture recognition for Indonesian Sign Language (BISINDO),” in Journal of Physics: Conference Series, Institute of Physics Publishing, Jun. 2018. doi: 10.1088/1742-6596/1028/1/012173.

R. Z. Fadillah, A. Irawan, M. Susanty, and I. Artikel, “Data Augmentasi Untuk Mengatasi Keterbatasan Data Pada Model Penerjemah Bahasa Isyarat Indonesia (BISINDO),” JURNAL INFORMATIKA, vol. 8, no. 2, 2021, [Online]. Available: http://ejournal.bsi.ac.id/ejurnal/index.php/ji

N. O’ Mahony et al., “Deep Learning vs. Traditional Computer Vision,” 2020.

S. Dong, P. Wang, and K. Abbas, “A survey on deep learning and its applications,” Computer Science Review, vol. 40. Elsevier Ireland Ltd, May 01, 2021. doi: 10.1016/j.cosrev.2021.100379.

D. Indra, H. M. Fadlillah, Kasman, L. B. Ilmawan, and H. Lahuddin, “Classification of good and damaged rice using convolutional neural network,” Bulletin of Electrical Engineering and Informatics, vol. 11, no. 2, pp. 785–792, Apr. 2022, doi: 10.11591/eei.v11i2.3385.

S. E. Basri, D. Indra, H. Darwis, A. W. Mufila, L. B. Ilmawan, and

B. Purwanto, “Recognition of Indonesian Sign Language Alphabets Using Fourier Descriptor Method,” in 3rd 2021 East Indonesia Conference on Computer and Information Technology, EIConCIT 2021, Institute of Electrical and Electronics Engineers Inc., Apr. 2021, pp. 405–409. doi: 10.1109/EIConCIT50028.2021.9431883.

D. Indra, R. Satra, H. Azis, A. R. Manga, and H. L, “Detection System of Strawberry Ripeness Using K-Means,” ILKOM Jurnal Ilmiah, vol. 14, no. 1, pp. 25–31, Apr. 2022, doi: 10.33096/ilkom.v14i1.1054.25-31.

D. Indra and S. Madenda, “Feature Extraction of Bisindo Alphabets Using Chain Code Contour,” Int J Eng Technol, Aug. 2017, doi: 10.21817/ijet/2017/v9i4/170904142.

D. Indra, S. Madenda, and E. P. Wibowo, “Recognition of Bisindo alphabets based on chain code contour and similarity of Euclidean distance,” Int J Adv Sci Eng Inf Technol, vol. 7, no. 5, pp. 1644–1652, 2017, doi: 10.18517/ijaseit.7.5.2746.

D. Indra, H. M. Fadlillah, Kasman, and L. B. Ilmawan, “Rice Texture Analysis Using GLCM Features,” in International Conference on Electrical, Computer, and Energy Technologies, ICECET 2021, Institute of Electrical and Electronics Engineers Inc., 2021. doi: 10.1109/ICECET52533.2021.9698594.

T. V. Janahiraman and M. S. M. Subuhan, “Traffic light detection using tensorflow object detection framework,” in 2019 IEEE 9th International Conference on System Engineering and Technology, ICSET 2019 - Proceeding, Institute of Electrical and Electronics Engineers Inc., Oct. 2019, pp. 108–113. doi: 10.1109/ICSEngT.2019.8906486.

Christiani I, “Deteksi Senjata Genggam Menggunakan Faster R CNN Inception V2,” 2022.

Iskandar D, “Implementasi Framework Mask R-CNN Object Detection API Dalam Mengklasifikasi Jenis Kendaraan Bermotor,” 2022.

D. Indra and F. Shantya Budi, “Implementasi Sistem Penghitung Kendaraan Otomatis Berbasis Computer Vision Implementation Of Automatic Vehicles Counting System Based On Computer Vision,” Jurnal Sistem Komputer, vol. 12, no. 1, p. 2020, 2023, doi: 10.34010/komputika.v12i1.9082.

Ratna P, Sumin A, and Wirawan S, “Pembuatan Aplikasi Deteksi Objek Menggunakan TensorFlow Object Detection API dengan Memanfaatkan SSD MobileNet V2 Sebagai Model Pra - Terlatih,” Jurnal Ilmiah Komputasi, vol. 19, no. 3, Mar. 2020, doi: 10.32409/jikstik.19.3.68.

S. A. Sanchez, H. J. Romero, and A. D. Morales, “A review: Comparison of performance metrics of pretrained models for object detection using the TensorFlow framework,” in IOP Conference Series: Materials Science and Engineering, Institute of Physics Publishing, Jun. 2020. doi: 10.1088/1757-899X/844/1/012024.

J. Koci, A. O. Topal, and M. Ali, “Threat Object Detection in X-ray Images Using SSD, R-FCN and Faster R-CNN,” in Proceedings - 2020 International Conference on Computing, Networking, Telecommunications and Engineering Sciences Applications, CoNTESA 2020, Institute of Electrical and Electronics Engineers Inc., Dec. 2020, pp. 10–15. doi: 10.1109/CoNTESA50436.2020.9302863.

Y.-C. Chiu, C.-Y. Tsai, D. Ruan, G.-Y. Shen, and T.-T. Lee, “Mobilenet-SSDv2: An Improved Object Detection Model for Embedded Systems,” 2020.


Refbacks

  • There are currently no refbacks.


Copyright (c) 2023 Lilis Nur Hayati, Anik Nur Handayani, Wahyu Sakti Gunawan Irianto, Rosa Andrie Asmara, Dolly Indra, Muhammad Fahmi

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.