Socratic artificial mind (SAM). Lessons from the evaluation of a GPT-3 based chatbot for Socratic dialogue
Are you already subscribed?
Login to check
whether this content is already included on your personal or institutional subscription.
Abstract
This contribution describes the design and evaluation of SAM, a GPT3-based chatbot trained to hold conversations using Socratic dialogue rules. The system was tested by twenty users and two types of analysis were carried out, one from the user’s point of view, the other from a technical point of view, to verify compliance with the principles of the Socratic dialogue. The results show a satisfactory performance concerning the non-violation of the basic tenets of the approach and a highly positive user experience, especially regarding the perceived understanding abilities and competence of SAM and its effect on provoking reflection
Keywords
- socratic dialogue
- fine-tuning
- GPT-3
- evaluation
- user experience