Gianfranco Caneva Salvatore Calcagno Daniela Giordano

Socratic artificial mind (SAM). Lessons from the evaluation of a GPT-3 based chatbot for Socratic dialogue

Are you already subscribed?
Login to check whether this content is already included on your personal or institutional subscription.

Abstract

This contribution describes the design and evaluation of SAM, a GPT3-based chatbot trained to hold conversations using Socratic dialogue rules. The system was tested by twenty users and two types of analysis were carried out, one from the user’s point of view, the other from a technical point of view, to verify compliance with the principles of the Socratic dialogue. The results show a satisfactory performance concerning the non-violation of the basic tenets of the approach and a highly positive user experience, especially regarding the perceived understanding abilities and competence of SAM and its effect on provoking reflection

Keywords

  • socratic dialogue
  • fine-tuning
  • GPT-3
  • evaluation
  • user experience

Preview

Article first page

What do you think about the recent suggestion?

Trova nel catalogo di Worldcat