Optimizing Seq2Seq LSTM for Regional-to-National language translation on a web platform
Article Sidebar
Main Article Content
Abstract
Machine translation for low-resource languages remains a significant challenge due to the lack of parallel corpora and optimized model configurations. This study developed and optimized a Seq2Seq Long Short-Term Memory (LSTM) model for Tegalan-to-Indonesian translation. A manually curated parallel corpus was constructed to train and evaluate the model. Various hyperparameter configurations were systematically tested, with the best-performing model achieving a BLEU score of 11.7381 using a dropout rate of 0.5, batch size of 64, learning rate of 0.01, and 70 training epochs. The results demonstrated that higher dropout rates, smaller batch sizes, and longer training durations enhanced model generalization and translation accuracy. The optimized model was deployed into a web-based application using Streamlit, ensuring accessibility for real-time translation. The findings highlighted the importance of hyperparameter tuning in neural machine translation for low-resource languages. Future research should explore Transformer-based architectures, larger datasets, and reinforcement learning techniques to further enhance translation quality and generalization.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
References
K. Jiang and X. Lu, “Natural Language Processing and Its Applications in Machine Translation: A Diachronic Review,” Proc. 2020 IEEE 3rd Int. Conf. Safe Prod. Informatiz. IICSPI 2020, pp. 210–214, 2020, doi: 10.1109/IICSPI51290.2020.9332458.
M. D. Okpor, “Machine Translation Approaches: Issues and Challenges,” IJCSI Int. J. Comput. Sci., vol. 11, no. 5, pp. 159–165, 2014, doi: 10.13140/RG.2.2.12055.38561.
A. F. Aji et al., “One Country, 700+ Languages: NLP Challenges for Underrepresented Languages and Dialects in Indonesia,” Proc. Annu. Meet. Assoc. Comput. Linguist., vol. 1, pp. 7226–7249, 2022, doi: 10.18653/v1/2022.acl-long.500.
M. Ramaiah, D. Datta, C. Vanmathi, and R. Agarwal, “Study of neural machine translation with long short-term memory techniques,” Deep Learn. Res. Appl. Nat. Lang. Process., no. January, pp. 65–88, 2022, doi: 10.4018/978-1-6684-6001-6.ch005.
K. Dedes et al., “Neural Machine Translation of Spanish-English Food Recipes UsingLSTM,” Int. J. Informatics Vis., vol. 6, no. June, pp. 290–297, 2022, [Online]. Available: www.joiv.org/index.php/joiv
T. I. Ramadhan, N. G. Ramadhan, and A. Supriatman, “Implementation of Neural Machine Translation for English-Sundanese Language using Long Short Term Memory (LSTM),” Build. Informatics, Technol. Sci., vol. 4, no. 3, pp. 1438–1446, 2022, doi: 10.47065/bits.v4i3.2614.
B. Ren, “The use of machine translation algorithm based on residual and LSTM neural network in translation teaching,” PLoS One, vol. 15, no. 11 November, pp. 1–16, 2020, doi: 10.1371/journal.pone.0240663.
S. Sharma, “A Transformer based approach using LSTM and Paraphrase reference to Translate English Text into Hindi,” 2023.
H. Jain, “Machine Translation Using Seq2Seq Modelling,” 2021. https://www.kaggle.com/code/harshjain123/machine-translation-seq2seq-lstms
Gemma Team et al., “Gemma: Open Models Based on Gemini Research and Technology,” 2024, [Online]. Available: http://arxiv.org/abs/2403.08295
P. Bhuvaneshwari and A. N. Rao, “A comparative study on various pre-processing techniques and deep learning algorithms for text classification,” Int. J. Cloud Comput., vol. 11, no. 1, pp. 61–78, 2022, doi: 10.1504/IJCC.2022.121076.
S. J. Mielke et al., “Between words and characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLP,” 2021, [Online]. Available: http://arxiv.org/abs/2112.10508
M. A. Oladipupo, P. C. Obuzor, B. J. Bamgbade, K. M. Olagunju, A. E. Adeniyi, and S. A. Ajagbe, “An Automated Python Script for Data Cleaning and Labeling using Machine Learning Technique,” Inform., vol. 47, no. 6, pp. 219–232, 2023, doi: 10.31449/inf.v47i6.4474.
K. Imamura and M. Utiyama, “An Empirical Study of Multilingual Vocabulary for Neural Machine Translation Models,” WAT 2024 - 11th Work. Asian Transl. Proc. Work., no. Wat, pp. 22–35, 2024.
Y. Yang, “Application of LSTM Neural Network Technology Embedded in English Intelligent Translation,” Comput. Intell. Neurosci., vol. 2022, 2022, doi: 10.1155/2022/1085577.
G. Tiwari, A. Sharma, A. Sahotra, and R. Kapoor, “English-Hindi Neural Machine Translation-LSTM Seq2Seq and ConvS2S,” in 2020 International Conference on Communication and Signal Processing (ICCSP), IEEE, Jul. 2020, pp. 871–875. doi: 10.1109/ICCSP48568.2020.9182117.
H. Wardhana, I. M. Yadi Dharma, K. Marzuki, and I. Syarif Hidayatullah, “Implementation of Neural Machine Translation in Translating from Indonesian to Sasak Language,” MATRIK J. Manajemen, Tek. Inform. dan Rekayasa Komput., vol. 23, no. 2, pp. 465–476, 2024, doi: 10.30812/matrik.v23i2.3465.
Y. Cui, S. Wang, and J. Li, “LSTM neural reordering feature for statistical machine translation,” 2016 Conf. North Am. Chapter Assoc. Comput. Linguist. Hum. Lang. Technol. NAACL HLT 2016 - Proc. Conf., pp. 977–982, 2016, doi: 10.18653/v1/n16-1112.
B. H. Shekar and G. Dagnew, “Grid search-based hyperparameter tuning and classification of microarray cancer data,” 2019 2nd Int. Conf. Adv. Comput. Commun. Paradig. ICACCP 2019, pp. 1–8, 2019, doi: 10.1109/ICACCP.2019.8882943.
D. Puspitaningrum, “A Study of English-Indonesian Neural Machine Translation with Attention (Seq2Seq, ConvSeq2Seq, RNN, and MHA): A Comparative Study of NMT on English-Indonesian,” ACM Int. Conf. Proceeding Ser., pp. 271–280, 2021, doi: 10.1145/3479645.3479703.
S. Lee et al., “A Survey on Evaluation Metrics for Machine Translation,” Mathematics, vol. 11, no. 4, pp. 1–22, 2023, doi: 10.3390/math11041006.
S. Studer et al., “Towards CRISP-ML(Q): A Machine Learning Process Model with Quality Assurance Methodology,” Mach. Learn. Knowl. Extr., vol. 3, no. 2, pp. 392–413, 2021, doi: 10.3390/make3020020.
A. F. Hidayatullah, S. Cahyaningtyas, and R. D. Pamungkas, “Attention-based CNN-BiLSTM for Dialect Identification on Javanese Text,” Kinet. Game Technol. Inf. Syst. Comput. Network, Comput. Electron. Control, vol. 4, pp. 317–324, 2020, doi: 10.22219/kinetik.v5i4.1121.