The English Companion

Users sometimes want to talk to their Companion, but not necessarily as a part of a finalized, task-oriented dialogue. The Companion has to be able both to listen to long utterances and to respond appropriately to them relating to the user's day or past events. In a similar way, the Companion has to be able to entertain the user with long utterances, which are more suitable when expressing interest and empathy.

Since the conversation with the Companion differs from small talk, it goes beyond the state-of-the-art. The challenge is met by the partners' experience in some key technologies such as Information Extraction, and the ability of Automatic Speech Recognition (ASR) to process long utterances when using dictation systems with user-specific tuning.

The 'How was your day?' scenario

The 'How was your day?' (HWD) scenario is based on an overarching conversational system, which establishes a relationship with the user and supports the user emotionally. When appropriate, the conversational application calls task-oriented, self-contained sub-dialogues. The scenario allows both the user and the Companion to have conversations about the daily life events mainly related to work. The Companion recognizes the key events in the life of the user as well as the associated emotional context.

The emotional conversation deals with the appraisal of the affective state of the user. It relies on the analysis of what is said and how it is said. It also manages the generation of the Companion's responses in terms of emotional strategies and behaviour e.g. speech, gestures, facial expressions, etc.

The conversation is based on turn taking, where shifting of topic is motivated both by the user and by the Companion. Both the topic and the perceived emotional state can play a role in controlling the conversation. Such a scenario creates several opportunities for the Companion to express empathy with a user. The Companion has to guess the emotional state of the user and discover and learn what is pleasant or unpleasant for the user.