license: apache-2.0
language:
- en
base_model:
- meta-llama/Meta-Llama-3.1-8B
Empathetic teacher model
Overview
This is a LLM fine-tuned with real-life, ideally-empathetic teacher-student conversations. This model processes the recent conversation history and provides guidance on how a teacher might respond to the student's utterance.
To fine-tune an open-weighted LLM to act as this generic teacher, we have used the following datasets: the Teacher-Student Chatroom Corpus, TSCCv2 Caines et al., 2022, CIMA Stasaski et al., 2020, the Multicultural Classroom Discourse Dataset Rapanta et al., 2021, MathDial Macina et al., 2023, and Conversational Uptake [Demszky et al., 2021].
We are evaluating LLaMa-3 for this task. Instead of using programmable fine-tuning libraries such as Axolotl (link) or Huggingface TRL (link), we are employing the more general command-line LLaMA-Factory (link) toolkit that facilitates the fine-tuning of various well-known LLMs on custom data. Parameter-efficient fine-tuning is achieved via the QLoRA method Dettmers et al., 2023.