Open Access
Research Article
A quality exploration of machine learning’s ability to predict predatory dental publications
*Balraj ShuklaCorresponding authorbalrajshukla@hotmail.comDepartment of Pediatric and Preventive Dentistry College of Dental Sciences and Research Centre Gujarat UniversityAhmedabad, Gujarat, India0000-0002-0923-5135View full profile → , Anup Pandadranuppanda76@gmail.comDepartment of Pediatric and Preventive Dentistry College of Dental Sciences and Research Centre Gujarat UniversityAhmedabad, Gujarat, India0000-0002-9229-0097View full profile →
* Corresponding author · click or hover a name for details
- Published Online:
- 02 Jun 2025
- Article type:
- Research Article
- Language:
- EN
- Article no.:
- CJSIM-2024-11002
- Pages:
- 59–70
Abstract
Background: Predatory journals are fraudulent publications that feature unethical and inaccurate research works. Objective: This study aimed to check the ability of a machine learning tool to identify predatory journals when compared to manual methods. Methods and Material: A cross-sectional observational study design was opted for. After data collection and screening of 23 journals, 19 journals were chosen for evaluation. These journals featured research in pediatric dentistry between 2016 and 2023 from three western states of India. Each journal was analyzed through two manual methods [Predatory Rate (PR) and Patwardhan Protocol (PP)] and by a machine learning tool [(Academic Journal Predatory Checking system (AJPC)]. Statistical analysis was carried out using Orange Data Mining Software (3.36.0). A confusion matrix was drawn to calculate accuracy, precision, recall, and F1 score. A Cohen’s Kappa was calculated to evaluate the agreement of results beyond chance, followed by Mcnemar’s test of significance. Results: An F1 value of 0.96 suggests that AJPC nearly replicates the results of PP. However, this cannot be said when compared with PR, where an accuracy of only 68.42% is statistically significant (p<0.05). Conclusion: This is a first-of-its-kind study in the academic domain of pediatric dentistry that explores AJPC’s capability against manual methods to identify predatory journals. Preliminary results indicate a rising trend of predatory publishing in Western India. Further investigations in research malpractice can be explored through larger datasets across expanded geographical regions to overcome the limitations of this study.
Keywords
References
[1] B.-C. Björk, “Open access to scientific articles: a review of benefits and challenges,” Intern. Emerg. Med., vol. 12, no. 2, pp. 247–253 (Mar. 2017), doi: 10.1007/s11739-017-1603-2.[2] K. Takabe, M. Nagahashi, A. Butash, and T. Wakai, “Do Preys Prey on Their Predators? Confusion over Predatory ‘Hage-taka’ Journals,” JMA J., vol. 2, no. 2, pp. 200–201 (2019), doi: 10.31662/jmaj.2019-0011.[3] C. Gallent Torres, “Editorial misconduct: the case of online predatory journals,” Heliyon, vol. 8, no. 3, p. e08999 (Mar. 2022), doi: 10.1016/j.heliyon.2022.e08999.[4] L. Danevska, M. Spiroski, D. Donev, N. Pop-Jordanova, and M. Polenakovic, “How to Recognize and Avoid Potential, Possible, or Probable Predatory Open-Access Publishers, Standalone, and Hijacked Journals,” Maced. Med. Rev., vol. 71, no. 1, pp. 75–81 (Jan. 2017), doi: 10.1515/mmr-2017-0014.[5] R. Jain, “Public Notice on Academic Integrity.” University Grants Commission (2019). [Online]. Available: https://www.ugc.gov.in/pdfnews/6315352_UGC-Public-Notice-CARE.pdf[6] B. Patwardhan, A. Desai, A. Chourasia, S. Nag, and R. Bhatnagar, Guidance Document: Good Academic Research Practices. New Delhi: University Grants Commission (2020). [Online]. Available: https://www.ugc.gov.in/e-book/UGC_GARP_2020_Good%20Academic%20Research%20Practices.pdf[7] A. Grudniewicz, D. Moher, J. D. Cobey, K. Lalu, K. McKee, M. Ravaud, H. Rochon, and D. B. Winker, “Predatory journals: no definition, no defence,” Nature, vol. 576, no. 7786, pp. 210–212 (Dec. 2019), doi: 10.1038/d41586-019-03759-y.[8] B. Shukla and A. Panda, “The Deuce List - A Tool to Evaluate Journals and A Mini-review on Predatory Publishing,” International Journal of Recent Surgical and Medical Sciences, vol. 11, p. e006 (May 2025), doi: https://doi.org/10.25259/ijrsms_11_2025[9] V. M. Pattanshetti, “How to Increase Quality Medical Research Publications in India?,” J. Sci. Soc., vol. 46, no. 1, p. 1 (Apr. 2019), doi: 10.4103/jss.JSS_44_19.[10] A. Aydin, S. E. Yürük, İ. Reisoğlu, and Y. Goktas, “Main barriers and possible enablers of academicians while publishing,” Scientometrics, vol. 128, no. 1, pp. 623–650 (2023), doi: 10.1007/s11192-022-04528-x.[11] S. Huggett, “Journal bibliometrics indicators and citation ethics: a discussion of current issues,” Atherosclerosis, vol. 230, no. 2, pp. 275–277 (Oct. 2013), doi: 10.1016/j.atherosclerosis.2013.07.051.[12] F. Franchignoni, L. Özçakar, and S. Negrini, “Basic bibliometrics for dummies and others: an overview of some journal-level indicators in physical and rehabilitation medicine,” Eur. J. Phys. Rehabil. Med., vol. 54, no. 5, pp. 792–796 (Oct. 2018), doi: 10.23736/S1973-9087.18.05462-X.[13] R. Schulz, R. Shabir, D. C. Reichel, and J. D. West, “Is the future of peer review automated?,” BMC Res. Notes, vol. 15, no. 1, p. 203 (Jun. 2022), doi: 10.1186/s13104-022-06080-6.[14] P. P. Santra and D. Majhi, “Scholarly Communication and Machine-Generated Text: Is it Finally AI vs AI in Plagiarism Detection?,” J. Inf. Knowl., pp. 175–183 (Jul. 2023), doi: 10.17821/srels/2023/v60i3/171028.[15] E. Orel, D. Mavridis, M. Petrisor, and E. Iorio, “An Automated Literature Review Tool (LiteRev) for Streamlining and Accelerating Research Using Natural Language Processing and Machine Learning: Descriptive Performance Evaluation Study,” J. Med. Internet Res., vol. 25, p. e39736 (Sep. 2023), doi: 10.2196/39736.[16] Y. Ma, Y. Han, H. Zeng, and L. Ma, “Study on Contribution of Different Journal Evaluation Indicators to Impact Factor Based on Machine Learning,” Sci. Program., vol. 2023, no. 1, p. 3198385 (2023), doi: 10.1155/2023/3198385.[17] M. Dadkhah, M. H. Oermann, M. Hegedüs, R. Raman, and L. D. Dávid, “Diagnosis Unreliability of ChatGPT for Journal Evaluation,” Adv. Pharm. Bull., vol. 14, no. 1, pp. 1–4 (Mar. 2024), doi: 10.34172/apb.2024.020.[18] L.-X. Chen, S.-W. Su, C.-H. Liao, K.-S. Wong, and S.-M. Yuan, “An open automation system for predatory journal detection,” Sci. Rep., vol. 13, no. 1, p. 2976 (Feb. 2023), doi: 10.1038/s41598-023-30176-z.[19] G. S. Collins, A. J. Cronin, L. Van Calster, M. W. M. J. Schuit, K. Wynants, J. B. Reitsma, D. G. Altman, and B. Van Smeden, “TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods,” BMJ, p. e078378 (Apr. 2024), doi: 10.1136/bmj-2023-078378.[20] “Dental Council of India,” Dental Council of India. Accessed: Jun. 19 (2024). [Online]. Available: https://dciindia.gov.in/CollegeSearch.aspx?ColName=&CourseId=1&SplId=0&StateId=&Hospital=&Type=0&Status=--Select--[21] M. Dadkhah and G. Bianciardi, “Ranking Predatory Journals; Solve the Problem Instead of Removing It!,” Adv. Pharm. Bull., vol. 6, no. 1, pp. 1–4 (Mar. 2016), doi: 10.15171/apb.2016.001.[22] D. Singh, “Academic Integrity and Research Quality.” University Grants Commission (2021). [Online]. Available: https://www.ugc.gov.in/e-book/Academic%20and%20Research%20Book_WEB.pdf[23] B. Patwardhan, S. Nagarkar, S. R. Gadre, S. C. Lakhotia, V. M. Katoch, and D. Moher, “A Critical Analysis of the ‘UGC-Approved List of Journals,’” Curr. Sci., vol. 114, no. 06, p. 1299 (Mar. 2018), doi: 10.18520/cs/v114/i06/1299-1303.[24] Z. Huseynova, N. Pandis, and C. M. Faggion, “PRESUMED PREDATORY JOURNALS ARE ABUNDANT IN ORAL HEALTH,” J. Evid. Based Dent. Pract., vol. 21, no. 2, p. 101539 (Jun. 2021), doi: 10.1016/j.jebdp.2021.101539.[25] A. M. Kummangal, A. Ahsan, V. M. Babu, R. Payambrot, and M. L. Mannakandath, “Mapping of Dental Journal Publishing in India - A Pilot Study on Epistemological Challenges in Southern Scholarly Publishing,” Indian J. Dent. Res. (May 2024), doi: 10.4103/ijdr.ijdr_738_23.[26] N. A. Valente, “Predatory publishing in dentistry: an analysis of its impact and potential risks,” Minerva Dent. Oral Sci., vol. 70, no. 5 (Nov. 2021), doi: 10.23736/S2724-6329.21.04380-6.[27] J. Y. Ng and R. B. Haynes, “‘Evidence-based checklists’ for identifying predatory journals have not been assessed for reliability or validity: An analysis and proposal for moving forward,” J. Clin. Epidemiol., vol. 138, pp. 40–48 (Oct. 2021), doi: 10.1016/j.jclinepi.2021.06.015.[28] S. Cukier, J. Helal, M. Rice, C. Pupkaite, A. Al Dhaheri, D. Moher, and L. Shamseer, “Checklists to detect potential predatory biomedical journals: a systematic review,” BMC Med., vol. 18, no. 1, p. 104 (Dec. 2020), doi: 10.1186/s12916-020-01566-1.[29] F. Schwendicke, T. Krois, W. Gomez, K. Jurianz, P. Koldsland, and A. Krogh, “Artificial intelligence in dental research: Checklist for authors, reviewers, readers,” J. Dent., vol. 107, p. 103610 (Apr. 2021), doi: 10.1016/j.jdent.2021.103610.[30] J. A. Teixeira da Silva and N. Scelles, “An artificial intelligence tool misclassifies sport science journals as predatory,” J. Sci. Med. Sport, vol. 27, no. 4, pp. 266–269 (Apr. 2024), doi: 10.1016/j.jsams.2023.12.006.[31] S. Nasir, R. A. Khan, and S. Bai, “Ethical Framework for Harnessing the Power of AI in Healthcare and Beyond,” IEEE Access, vol. 12, pp. 31014–31035 (2024), doi: 10.1109/ACCESS.2024.3369912.[32] A. R. Memon, “Predatory Journals Spamming for Publications: What Should Researchers Do?,” Sci. Eng. Ethics, vol. 24, no. 5, pp. 1617–1639 (Oct. 2018), doi: 10.1007/s11948-017-9955-6.[33] A. Bucceri and P. Hornung, “Predatory publishing – what medical communicators need to know,” Med. Writ., vol. 28, no. 3, pp. 28–33 (2019).[34] C. Shen and B.-C. Björk, “‘Predatory’ open access: a longitudinal study of article volumes and market characteristics,” BMC Med., vol. 13, no. 1, p. 230 (Dec. 2015), doi: 10.1186/s12916-015-0469-2.[35] D. Moher, M. Altman, D. G. Bossuyt, I. Boutron, M. Clarke, M. Glasziou, J. P. A. Ioannidis, D. Tovey, and P. Glasziou, “Stop this waste of people, animals and money,” Nature, vol. 549, no. 7670, pp. 23–25 (Sep. 2017), doi: 10.1038/549023a.[36] L. Shamseer, D. Moher, K. Maduekwe, K. Turner, D. Barbour, D. B. B. Bouter, and D. J. Shea, “Potential predatory and legitimate biomedical journals: can you tell the difference? A cross-sectional comparison,” BMC Med., vol. 15, no. 1, p. 28 (Dec. 2017), doi: 10.1186/s12916-017-0785-9.[37] S. B. Demir, “Predatory journals: Who publishes in them and why?,” J. Informetr., vol. 12, no. 4, pp. 1296–1311 (Nov. 2018), doi: 10.1016/j.joi.2018.10.008.
Views: 173Downloads: 4Citations: 0
