[go: up one dir, main page]
IDEAS home Printed from https://ideas.repec.org/a/gam/jsusta/v16y2024i12p4929-d1411386.html
   My bibliography  Save this article

Custom-Trained Large Language Models as Open Educational Resources: An Exploratory Research of a Business Management Educational Chatbot in Croatia and Bosnia and Herzegovina

Author

Listed:
  • Nikša Alfirević

    (Faculty of Economics, Business and Tourism, University of Split, 21000 Split, Croatia)

  • Daniela Garbin Praničević

    (Faculty of Economics, Business and Tourism, University of Split, 21000 Split, Croatia)

  • Mirela Mabić

    (Faculty of Economics, University of Mostar, 88000 Mostar, Bosnia and Herzegovina)

Abstract
This paper explores the contribution of custom-trained Large Language Models (LLMs) to developing Open Education Resources (OERs) in higher education. Our empirical analysis is based on the case of a custom LLM specialized for teaching business management in higher education. This custom LLM has been conceptualized as a virtual teaching companion, aimed to serve as an OER, and trained using the authors’ licensed educational materials. It has been designed without coding or specialized machine learning tools using the commercially available ChatGPT Plus tool and a third-party Artificial Intelligence (AI) chatbot delivery service. This new breed of AI tools has the potential for wide implementation, as they can be designed by faculty using only conventional LLM prompting techniques in plain English. This paper focuses on the opportunities for custom-trained LLMs to create Open Educational Resources (OERs) and democratize academic teaching and learning. Our approach to AI chatbot evaluation is based on a mixed-mode approach, combining a qualitative analysis of expert opinions with a subsequent (quantitative) student survey. We have collected and analyzed responses from four subject experts and 204 business students at the Faculty of Economics, Business and Tourism Split (Croatia) and Faculty of Economics Mostar (Bosnia and Herzegovina). We used thematic analysis in the qualitative segment of our research. In the quantitative segment of empirical research, we used statistical methods and the SPSS 25 software package to analyze student responses to the modified BUS-15 questionnaire. Research results show that students positively evaluate the business management learning chatbot and consider it useful and responsive. However, interviewed experts raised concerns about the adequacy of chatbot answers to complex queries. They suggested that the custom-trained LLM lags behind the generic LLMs (such as ChatGPT, Gemini, and others). These findings suggest that custom LLMs might be useful tools for developing OERs in higher education. However, their training data, conversational capabilities, technical execution, and response speed must be monitored and improved. Since this research presents a novelty in the extant literature on AI in education, it requires further research on custom GPTs in education, including their use in multiple academic disciplines and contexts.

Suggested Citation

  • Nikša Alfirević & Daniela Garbin Praničević & Mirela Mabić, 2024. "Custom-Trained Large Language Models as Open Educational Resources: An Exploratory Research of a Business Management Educational Chatbot in Croatia and Bosnia and Herzegovina," Sustainability, MDPI, vol. 16(12), pages 1-18, June.
    • Handle: RePEc:gam:jsusta:v:16:y:2024:i:12:p:4929-:d:1411386
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2071-1050/16/12/4929/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2071-1050/16/12/4929/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Ajay Bandi & Pydi Venkata Satya Ramesh Adapa & Yudu Eswar Vinay Pratap Kumar Kuchi, 2023. "The Power of Generative AI: A Review of Requirements, Models, Input–Output Formats, Evaluation Metrics, and Challenges," Future Internet, MDPI, vol. 15(8), pages 1-60, July.
    2. Thomas K.F. Chiu & Ching-sing Chai, 2020. "Sustainable Curriculum Planning for Artificial Intelligence Education: A Self-Determination Theory Perspective," Sustainability, MDPI, vol. 12(14), pages 1-18, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ramona Simut & Ciprian Simut & Daniel Badulescu & Alina Badulescu, 2024. "Artificial Intelligence and the Modelling of Teachers’ Competencies," The AMFITEATRU ECONOMIC journal, Academy of Economic Studies - Bucharest, Romania, vol. 26(65), pages 181-181, February.
    2. Safa Jameel Al-Kamil & Róbert Szabolcsi, 2024. "Enhancing Mobile Robot Navigation: Optimization of Trajectories through Machine Learning Techniques for Improved Path Planning Efficiency," Mathematics, MDPI, vol. 12(12), pages 1-21, June.
    3. Hua Du & Yanchao Sun & Haozhe Jiang & A. Y. M. Atiquil Islam & Xiaoqing Gu, 2024. "Exploring the effects of AI literacy in teacher learning: an empirical study," Palgrave Communications, Palgrave Macmillan, vol. 11(1), pages 1-10, December.
    4. Shaotong Qi & Yubo Cheng & Zhiyuan Li & Jiaxin Wang & Huaiyi Li & Chunwei Zhang, 2024. "Advanced Deep Learning Techniques for Battery Thermal Management in New Energy Vehicles," Energies, MDPI, vol. 17(16), pages 1-38, August.
    5. Pacala Abayog Frank Angelo, 2023. "Curriculum theory and practice: A comparative literature review," HO CHI MINH CITY OPEN UNIVERSITY JOURNAL OF SCIENCE - SOCIAL SCIENCES, HO CHI MINH CITY OPEN UNIVERSITY JOURNAL OF SCIENCE, HO CHI MINH CITY OPEN UNIVERSITY, vol. 13(2), pages 3-13.
    6. Xiao-Fan Lin & Lu Chen & Kan Kan Chan & Shiqing Peng & Xifan Chen & Siqi Xie & Jiachun Liu & Qintai Hu, 2022. "Teachers’ Perceptions of Teaching Sustainable Artificial Intelligence: A Design Frame Perspective," Sustainability, MDPI, vol. 14(13), pages 1-20, June.
    7. Yun Dai & Ching-Sing Chai & Pei-Yi Lin & Morris Siu-Yung Jong & Yanmei Guo & Jianjun Qin, 2020. "Promoting Students’ Well-Being by Developing Their Readiness for the Artificial Intelligence Age," Sustainability, MDPI, vol. 12(16), pages 1-15, August.
    8. Mihaela-Elena Ulmeanu & Cristian-Vasile Doicin & Paulina Spânu, 2021. "Comparative Evaluation of Sustainable Framework in STEM Intensive Programs for Secondary and Tertiary Education," Sustainability, MDPI, vol. 13(2), pages 1-33, January.
    9. Hyun Yong Ahn, 2024. "AI-Powered E-Learning for Lifelong Learners: Impact on Performance and Knowledge Application," Sustainability, MDPI, vol. 16(20), pages 1-20, October.
    10. Nabil Hasan Al-Kumaim & Abdulsalam K. Alhazmi & Fathey Mohammed & Nadhmi A. Gazem & Muhammad Salman Shabbir & Yousef Fazea, 2021. "Exploring the Impact of the COVID-19 Pandemic on University Students’ Learning Life: An Integrated Conceptual Motivational Model for Sustainable and Healthy Online Learning," Sustainability, MDPI, vol. 13(5), pages 1-20, February.
    11. Theodora Sanida & Maria Vasiliki Sanida & Argyrios Sideris & Minas Dasygenis, 2024. "Enhancing Pulmonary Diagnosis in Chest X-rays through Generative AI Techniques," J, MDPI, vol. 7(3), pages 1-17, August.
    12. Binbin Cai & Yin Chen & Arslan Ayub, 2023. "“Quiet the Mind, and the Soul Will Speak”! Exploring the Boundary Effects of Green Mindfulness and Spiritual Intelligence on University Students’ Green Entrepreneurial Intention–Behavior Link," Sustainability, MDPI, vol. 15(5), pages 1-21, February.
    13. Min Tao & Xiong Wang, 2023. "An Integrated MCDM Model for Sustainable Course Planning: An Empirical Case Study in Accounting Education," Sustainability, MDPI, vol. 15(6), pages 1-25, March.
    14. Po-Kang Shih & Chun-Hung Lin & Leon Yufeng Wu & Chih-Chang Yu, 2021. "Learning Ethics in AI—Teaching Non-Engineering Undergraduates through Situated Learning," Sustainability, MDPI, vol. 13(7), pages 1-16, March.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jsusta:v:16:y:2024:i:12:p:4929-:d:1411386. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.