Smart Verification of High School Student Reports Using Optical Character Recognition and BERT Models


Asep Indra Syahyadi(1*); Nur Afif(2); Ahmad Yusuf(3); Haris Setiaji(4); Ridwang Ridwang(5); Mohammad Irfan(6);

(1) Universitas Islam Negeri Alauddin Makassar
(2) Universitas Islam Negeri Alauddin Makassar
(3) Universitas Islam Negeri Sunan Ampel Surabaya
(4) Institut Agama Islam Negeri Metro Lampung
(5) Universitas Muhammadiyah Makassar
(6) Universitas Islam Negeri Sunan Gunung Djati Bandung
(*) Corresponding Author

  

Abstract


This study proposes an intelligent framework for verifying high school report cards with diverse layouts by integrating Optical Character Recognition (OCR) and a fine-tuned BERT model. While previous works primarily address document formats with uniform structures, this research specifically tackles the heterogeneity of report cards that differ in subject arrangement, naming conventions, and grade presentation across schools. The system was trained and evaluated using 1,000 Indonesian high school report card pages encompassing 20 subjects, both core (e.g., Mathematics, Indonesian History, Religious Education) and non-core (e.g., Arts and Culture, Physical Education). OCR was employed to extract textual content from scanned or image-based report cards, while BERT handled contextual mapping between subjects and corresponding grades. The dataset was divided into 80% for training and 20% for validation, and the model was fine-tuned on the IndoBERT-base architecture. Experimental results showed that the proposed OCR–BERT pipeline achieved an average accuracy of 97.7%, with per-subject accuracies ranging from 96% to 99%. The model exhibited high robustness in handling inconsistent layouts and minimizing deviations between actual and detected grades. Comparative analysis indicated that this hybrid approach outperforms traditional OCR-only or CNN-based methods, which are typically constrained by fixed template assumptions and lack contextual understanding. The proposed system demonstrates practical relevance for large-scale admission platforms such as SPAN-PTKIN, where manual verification of thousands of report cards is laborious and error-prone. By automating the verification process, the framework reduces human workload, enhances accuracy, and supports fairer, data-driven admission decisions. Future research will explore multimodal integration of textual and visual features, expansion to broader datasets, and application to other academic documents such as transcripts and diplomas. Overall, this work contributes a scalable, accurate, and context-aware solution for educational data verification in heterogeneous document environments.


Keywords


Optical Character Recognition, BERT Model, Grade Detection, Educational Data Automation, Educational Data Automation, Report Card Analysis

  
  

Full Text:

PDF
  

Article Metrics

Abstract view: 0 times
PDF view: 0 times 		     

Digital Object Identifier

doi  https://doi.org/10.33096/ilkom.v17i3.2764.252-261 		  

Cite

How to cite item

References


I. Malashin, I. Masich, V. Tynchenko, A. Gantimurov, V. Nelyub, and A. Borodulin, “Image Text Extraction and Natural Language Processing of Unstructured Data from Medical Reports,” Mach Learn Knowl Extr, vol. 6, no. 2, pp. 1361–1377, Jun. 2024, doi: 10.3390/make6020064.

M. Sirajudeen and R. Anitha, “Forgery document detection in information management system using cognitive techniques,” Journal of Intelligent & Fuzzy Systems, vol. 39, pp. 8057–8068, 2020, doi: 10.3233/JIFS-189128.

J. Kefeli and N. Tatonetti, “TCGA-Reports: A machine-readable pathology report resource for benchmarking text-based AI models,” Patterns, vol. 5, no. 3, p. 100933, Mar. 2024, doi: 10.1016/J.PATTER.2024.100933.

G. Zeng et al., “Beyond OCR + VQA: Towards end-to-end reading and reasoning for robust and accurate textvqa,” Pattern Recognit, vol. 138, p. 109337, Jun. 2023, doi: 10.1016/J.PATCOG.2023.109337.

I. Saitov and A. Filchenkov, “CIS Multilingual License Plate Detection and Recognition Based on Convolutional and Transformer Neural Networks,” Procedia Comput Sci, vol. 229, pp. 149–157, Jan. 2023, doi: 10.1016/J.PROCS.2023.12.016.

X. Zhou et al., “Automated data collection tool for real-world cohort studies of chronic hepatitis B: Leveraging OCR and NLP technologies for improved efficiency,” New Microbes New Infect, vol. 62, p. 101469, Dec. 2024, doi: 10.1016/J.NMNI.2024.101469.

Z. Hatala, “Verification Of Semester Grade Transcript Documents Using The Optical Character Recognition Method,” Jurnal Informatika dan Teknik Elektro Terapan, vol. 11, no. 3, Aug. 2023, doi: 10.23960/jitet.v11i3.3277.

Adriani, Ridwang, A. Syamsuddin, Muliadi, and U. Umar, “Improvement In Speech Recognition Of Indonesian Language Using Mel Frequency Cepstral Coefficients And Long Short-Term Memory Method,” ICIC Express Letters, vol. 18, no. 9, pp. 987–995, Sep. 2024, doi: 10.24507/icicel.18.09.987.

Ridwang, A. A. Ilham, I. Nurtanio, and Syafaruddin, “Dynamic Sign Language Recognition Using Mediapipe Library and Modified LSTM Method,” Int J Adv Sci Eng Inf Technol, vol. 13, no. 6, pp. 2171–2180, Dec. 2023, doi: 10.18517/ijaseit.13.6.19401.

R. Oucheikh, T. Pettersson, and T. Löfström, “Product verification using OCR classification and Mondrian conformal prediction,” Expert Syst Appl, vol. 188, Feb. 2022, doi: 10.1016/j.eswa.2021.115942.

Q. Tian, M. Liu, L. Min, J. An, X. Lu, and H. Duan, “An automated data verification approach for improving data quality in a clinical registry,” Comput Methods Programs Biomed, vol. 181, p. 104840, Nov. 2019, doi: 10.1016/J.CMPB.2019.01.012.

X. Zhou et al., “Automated data collection tool for real-world cohort studies of chronic hepatitis B: Leveraging OCR and NLP technologies for improved efficiency,” New Microbes New Infect, vol. 62, p. 101469, Dec. 2024, doi: 10.1016/J.NMNI.2024.101469.

X. Tang, C. Wang, J. Su, and C. Taylor, “An Elevator Button Recognition Method Combining YOLOv5 and OCR,” Computers, Materials and Continua, vol. 75, no. 1, pp. 117–131, Jan. 2023, doi: 10.32604/CMC.2023.033327.

C. A. Rey, J. L. Danguilan, K. P. Mendoza, and M. F. Remolona, “Transformer-based approach to variable typing,” Heliyon, vol. 9, no. 10, p. e20505, Oct. 2023, doi: 10.1016/J.HELIYON.2023.E20505.

S. Carta, A. Giuliani, L. Piano, and S. G. Tiddia, “An End-to-End OCR-Free Solution For Identity Document Information Extraction,” Procedia Comput Sci, vol. 246, pp. 453–462, Jan. 2024, doi: 10.1016/J.PROCS.2024.09.425.

Z. Zhang, A. Yaseen, and H. Wu, “Scholarly recommendation system for NIH funded grants based on biomedical word embedding models,” Natural Language Processing Journal, vol. 8, p. 100095, Dec. 2024, doi: 10.1016/J.NLP.2024.100095.

M. Ponnuru, S. P. Ponmalar, A. Likhitha, T. B. Sree, and G. G. Chaitanya, “Image-Based Extraction of Prescription Information using OCR-Tesseract,” Procedia Comput Sci, vol. 235, pp. 1077–1086, Jan. 2024, doi: 10.1016/J.PROCS.2024.04.102.

I. A. Adeyanju, O. O. Bello, and M. A. Adegboye, “Machine learning methods for sign language recognition: A critical review and analysis,” Intelligent Systems with Applications, vol. 12, p. 200056, 2021, doi: 10.1016/j.iswa.2021.200056.

A. A. Ilham, I. Nurtanio, Ridwang, and Syafaruddin, “Applying LSTM and GRU Methods to Recognize and Interpret Hand Gestures, Poses, and Face-Based Sign Language in Real Time,” Journal of Advanced Computational Intelligence and Intelligent Informatics, vol. 28, no. 2, pp. 265–272, Mar. 2024, doi: 10.20965/jaciii.2024.p0265.

B. Alothman, H. Alazmi, M. Bin Ali, N. Alqallaf, and M. Khan, “Accelerating University Admission System using Machine Learning Techniques,” in 2022 Thirteenth International Conference on Ubiquitous and Future Networks (ICUFN), IEEE, Jul. 2022, pp. 439–443. doi: 10.1109/ICUFN55119.2022.9829611.

V. Seelam, A. K. Penugonda, B. Pavan Kalyan, M. Bindu Priya, and M. Durga Prakash, “Smart attendance using deep learning and computer vision,” Mater Today Proc, vol. 46, pp. 4091–4094, Jan. 2021, doi: 10.1016/J.MATPR.2021.02.625.

A. Wang, Y. Qi, and D. Baiyila, “C-BERT: A Mongolian reverse dictionary based on fused lexical semantic clustering and BERT,” Alexandria Engineering Journal, vol. 111, pp. 385–395, Jan. 2025, doi: 10.1016/J.AEJ.2024.10.041.

X. Chen, B. He, K. Hui, L. Sun, and Y. Sun, “Dealing with textual noise for robust and effective BERT re-ranking,” Inf Process Manag, vol. 60, no. 1, p. 103135, Jan. 2023, doi: 10.1016/J.IPM.2022.103135.

K. Alhanaee, M. Alhammadi, N. Almenhali, and M. Shatnawi, “Face Recognition Smart Attendance System using Deep Transfer Learning,” Procedia Comput Sci, vol. 192, pp. 4093–4102, Jan. 2021, doi: 10.1016/J.PROCS.2021.09.184.

N. H. Imam, V. G. Vassilakis, and D. Kolovos, “OCR post-correction for detecting adversarial text images,” Journal of Information Security and Applications, vol. 66, p. 103170, May 2022, doi: 10.1016/J.JISA.2022.103170.

E. Hsu, I. Malagaris, Y. F. Kuo, R. Sultana, and K. Roberts, “Deep learning-based NLP data pipeline for EHR-scanned document information extraction,” JAMIA Open, vol. 5, no. 2, Jul. 2022, doi: 10.1093/jamiaopen/ooac045.

R. Ridwang, A. Ahmad Ilham, I. Nurtanio, and S. Syafaruddin, “Dynamic Sign Language Recognition Using Mediapipe Library and Modified LSTM Method,” Int J Adv Sci Eng Inf Technol, vol. 13, no. 6, Dec. 2023, doi: 10.1093/jamiaopen/ooac045.

N. Englert, C. Schwab, M. Legnar, and C. A. Weis, “Presenting the framework of the whole slide image file Babel fish: An OCR-based file labeling tool,” J Pathol Inform, vol. 15, p. 100402, Dec. 2024, doi: 10.1016/J.JPI.2024.100402.

bhavatharani, S. Basha, and S. Ahmed, Deep Learning: Practical approach. 2022.

D. Aiqattan, “Towards Sign Language Recognition Using EEG-Based Motor Imagery Brain Computer Interface,” 2017.

Y. Xu, M. Li, L. Cui, S. Huang, F. Wei, and M. Zhou, “LayoutLM: Pre-training of Text and Layout for Document Image Understanding,” in Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Association for Computing Machinery, Aug. 2020, pp. 1192–1200. doi: 10.1145/3394486.3403172.

S. V. Mahadevkar, S. Patil, K. Kotecha, L. W. Soong, and T. Choudhury, “Exploring AI-driven approaches for unstructured document analysis and future horizons,” J Big Data, vol. 11, no. 1, Dec. 2024, doi: 10.1186/s40537-024-00948-z.

J. Zhou, C. Yang, Y. Zhu, and Y. Zhang, “Cross-region feature fusion with geometrical relationship for OCR-based image captioning,” Neurocomputing, vol. 601, p. 128197, Oct. 2024, doi: 10.1016/J.NEUCOM.2024.128197.

J. Zhou, C. Yang, Y. Zhu, and Y. Zhang, “Cross-region feature fusion with geometrical relationship for OCR-based image captioning,” Neurocomputing, vol. 601, p. 128197, Oct. 2024, doi: 10.1016/J.NEUCOM.2024.128197.

M. S. Ghafourian, S. Tarkiani, M. Ghajar, M. Chavooshi, H. Khormaei, and A. Ramezani, “Optimizing warfarin dosing in diabetic patients through BERT model and machine learning techniques,” Comput Biol Med, vol. 186, p. 109755, Mar. 2025, doi: 10.1016/J.COMPBIOMED.2025.109755.

K. I. Roumeliotis, N. D. Tselikas, and D. K. Nasiopoulos, “Optimizing Airline Review Sentiment Analysis: A Comparative Analysis of LLaMA and BERT Models through Fine-Tuning and Few-Shot Learning,” Computers, Materials and Continua, vol. 82, no. 2, pp. 2769–2792, Feb. 2025, doi: 10.32604/CMC.2025.059567.


Refbacks

  • There are currently no refbacks.


Copyright (c) 2025 Asep Indra Syahyadi, Nur Afif, Ahmad Yusuf, Haris Setiaji, Ridwang Ridwang, Mohammad Irfan

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.