[go: up one dir, main page]

Towards Understanding Counseling Conversations: Domain Knowledge and Large Language Models

Younghun Lee, Dan Goldwasser, Laura Schwab Reese


Abstract
Understanding the dynamics of counseling conversations is an important task, yet it is a challenging NLP problem regardless of the recent advance of Transformer-based pre-trained language models. This paper proposes a systematic approach to examine the efficacy of domain knowledge and large language models (LLMs) in better representing conversations between a crisis counselor and a help seeker. We empirically show that state-of-the-art language models such as Transformer-based models and GPT models fail to predict the conversation outcome. To provide richer context to conversations, we incorporate human-annotated domain knowledge and LLM-generated features; simple integration of domain knowledge and LLM features improves the model performance by approximately 15%. We argue that both domain knowledge and LLM-generated features can be exploited to better characterize counseling conversations when they are used as an additional context to conversations.
Anthology ID:
2024.findings-eacl.137
Volume:
Findings of the Association for Computational Linguistics: EACL 2024
Month:
March
Year:
2024
Address:
St. Julian’s, Malta
Editors:
Yvette Graham, Matthew Purver
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2032–2047
Language:
URL:
https://aclanthology.org/2024.findings-eacl.137
DOI:
Bibkey:
Cite (ACL):
Younghun Lee, Dan Goldwasser, and Laura Schwab Reese. 2024. Towards Understanding Counseling Conversations: Domain Knowledge and Large Language Models. In Findings of the Association for Computational Linguistics: EACL 2024, pages 2032–2047, St. Julian’s, Malta. Association for Computational Linguistics.
Cite (Informal):
Towards Understanding Counseling Conversations: Domain Knowledge and Large Language Models (Lee et al., Findings 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.findings-eacl.137.pdf
Software:
 2024.findings-eacl.137.software.zip