Optimizing the Implementation of the BFS and DFS algorithms using the web crawler method on the kumparan site
Article Sidebar
Main Article Content
Abstract
Efficient access to timely information is critical in today's digital era. Web crawlers, automated programs that navigate the Internet, play an important role in collecting data from websites such as Kumparan, a leading news site in Indonesia. This research shows the effectiveness of the Breadth-First Search (BFS) and Depth-First Search (DFS) algorithms in indexing Kumparan content. The results of the research show that BFS consistently indexes more files comprehensively but with longer execution times compared to DFS, which provides faster initial results but with fewer files. For example, at depth 4 BFS indexed 949 files in 886.94 seconds, while DFS indexed 470 files in 233.02 seconds. These findings highlight the balance between precision and speed when selecting a crawling algorithm tailored to the needs of a particular website. This research provides insights into optimizing web crawler technology for complex websites such as Coil and suggests avenues for further research to improve permission efficiency and adaptability across a variety of crawling scenarios.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
References
M. Batari, “‘Web Crawler: Pengertian, Cara Kerja, Fungsi, dan Contohnya,’” Exabytes. .
Stekom, “‘Kumparan (situs web),’” Universitas STEKOM Semarang. https://p2k.stekom.ac.id/ensiklopedia/Kumparan_(situs_web) (diakses Mar 04, 2024).
Sulastri dan E. Zuliarso, “Aplikasi Web crawler Berdasarkan Breadth First Search dan Back-Link,” J. Teknol. Inf. Din., vol. XV, no. 1, hal. 52–56, 2010.
C. Kustanto, R. M. S, dan P. Viqarunnisa, “Penerapan Algoritma Breadth-first Search dan Depth-first Search Pada FTP Search Engine for ITB Network,” Bandung Inst. Teknol. Bandung, hal. 1–3.
D. T. Yuwono dan S. Abdul Fadlil, “Perbandingan Algoritma Breadth First Search dan Depth First Search Sebagai Focused Crawler,” Pros. Annu. Res. Semin. 2016, vol. 2, no. 1, hal. 106–110, 2016.
J. Beel, B. Gipp, dan E. Wilde, “Engine Optimization ( ASEO ): Optimizing Scholarly Literature for Google Scholar & Co .,” J. Sch. Publ., vol. 41, no. May 2014, hal. 2, 2010, doi: 10.1353/scp.0.0082.
A. Muhardono, “Penerapan Algoritma Breadth First Search dan Depth First Search pada Game Angka,” J. Minfo Polgan, vol. 12, hal. 171–182, 2023, doi: https://doi.org/10.33395/jmp.v12i1.12340.
P. Kumari dan G. Kakhani, “Comparative Analysis of Web PageRank Algorithm using DFS and BFS Crawling,” Int. J. Sci. Res., vol. 4, no. 6, hal. 859–863, 2015.
A. E. Wibowo, “Perbandingan Performansi Terhadap Algoritma Breadth First Search (BFS) & Depth First Search (DFS) Pada Web Crawler,” e-Proceeding Eng. Bandung Univ. Telkom, 2019.
M. Parmar dan H. J. Kaur, “Comparative Analysis of Secured Hash Algorithms for Blockchain Technology and Internet of Things,” Int. J. Adv. Comput. Sci. Appl., vol. 12, no. 3, hal. 282–289, 2021.
M. Soulemane dan H. Mahmud, “Crawling the Hidden Web : An Approach to Dynamic Web Indexing Crawling the Hidden Web : An Approach to Dynamic Web Indexing,” Int. J. Comput. Appl., no. October, 2016, doi: 10.5120/8717-7290.
B. Pedroza, J. M. G. Calleros, J. G. García, dan C. A. Collazos, “Continuous Evaluation of the Learning Process of Algebra Through a Semi-Automated Tool,” J. Inf. Technol. Res., vol. 12, no. 3, hal. 1–20, 2019, doi: 10.4018/JITR.2019070101.
Y. Sun, I. Councill, dan C. L. Giles, “The Ethicality of Web Crawlers,” Conf. Web Intell. Intell. Agent Technol. (WI-IAT), 2010, vol. 1, 2010, doi: 10.1109/WI-IAT.2010.316.
N. N. S. Faraj, A. Al Azzawi, S. Darwish, dan H. Al Deeb, “The Balance Between Social Life and Work and its Relationship with Work Stress – An Applied Study on the Ministry of Youth and Sports Affairs In the Kingdom of Bahrain,” Int. J. Data Min. Knowl. Manag. Process, vol. 9, no. January, 2019, doi: 10.5121/ijdkp.2019.9102.
A. Anthony, K. Onasoga, D. U. Ike, dan O. Ajayi, “Council for Innovative Research,” Int. J. Manag. Inf. Technol., vol. 5, no. 3, hal. 598–603, 2013.
K. C. Cox, J. Lortie, D. R. Marshall, dan R. E. Kidwell, “Beyond the balance Sheet: The effects of family influence on social performance,” J. Bus. Res., vol. 143, hal. 318–330, 2022, doi: https://doi.org/10.1016/j.jbusres.2022.01.013.
“‘Kumparan.’” https://kumparan.com/ (diakses Jun 24, 2024).