CPSC 4170/6170: �Milestone 1 Presentation�

Nafisa Sabnam Shama

CPSC 6170

Game description

Name title: ConvoTA

Genre:

ConvoTA is an educational application which helps learners to engage in a certain topic deeply with the help of Embodied Conversational Agent and AI chatbot.

​

​

Game description

Game Objectives:

​

Students interact with an AI-powered embodied conversational agent to learn Python programming. Ask natural language questions, access lesson materials, complete quizzes with instant feedback, and utilize AI-generated suggestions to guide their learning journey.

Game information

Skills required: Knowledge of Computer Science

​

Age: 18 and older

​

Target Audience: College students

​

Platform: Unity WebGL

Game References

Color palette

Entities – Intro Video

Entities – Creator Tool

1.Quiz and Lessons Designer

• A separate software for designing the quizzes and lessons
• Live preview of the contents
• Options for adding open ended questions, configuring texts in the contents and scoring

​

Entities – Video

2. Video

• A Pygame-based video introduces core concepts before students begin their quizzes.
• Learners have the flexibility to watch the full introduction or skip the video at any time to start learning.

Entities – Games

2. Games

• A drag-and-drop mini-game where students match programming terms with their correct definitions.
• Each round shuffles terms and definitions to improve recall
• Sound effects in the game provide cues for correct or incorrect matches

Entities – Quizzes with images

1. Quiz
1. There are quizzes on a certain topic
2. Users analyze images and answer related questions to test their knowledge.
3. They can ask AI after answering quiz to clear out their confusions

Entities – ECA

2. Embodied Conversational Agent

• Reads aloud question and gives feedback to students
• Provides expression after answering quizzes. Such as showing a smiling face if the answer is correct or a sad face if the answer is incorrect
• Rocketbox character is used for the ECA
• Mixamo animation is used for the expression
• For correct/incorrect answer the animation as triggered
• DoTween is used to blend the reactions smoothly

​

Entities – ECA

Entities – Chat Option

3. Chat Option

• User can interact with chatbot after answering to the quiz
• GPT 5 mini is used for the chatbot
• They can clear out their confusion regarding the topic provided in the platform
• The AI is designed to answer the questions the system intended for

​

​

Entities – AI Generated Suggestion

4. AI Generated Suggestions

• User can press AI generated suggestions to ask AI teaching assistant
• 3 suggestions are generated for users to ask the AI teaching assistant

​

Entities – Lesson Materials

6. Lesson Materials

• Students can acces lessons on Python 2D Game Programming
• A read aloud option is added including ECA so users can engage with it better

Entities – Feedback

5. Feedback

• Users can give feedback after completing their quizzes
• Users can also provide open ended feedback in an input box

Entities – Check Answers

5. Check Answers and Timing

• Users can check which are the correct answers
• They can also check how much time they spent while answering the quizzes

​

Game mechanics

• Homework Designer tool for authoring quiz items and lessons
• Content exported in JSON format for web delivery
• Active quiz item provides conversational context
• Generative, item-specific feedback
• Text converted to SSML with timing markers
• Google Cloud Text-to-Speech API for audio synthesis
• Real-time audio rendering in Unity
• All events logged to cloud database for analysis
• SALSA LipSync Suite for facial sync in ECA
• Rocketbox library used for ECA

Technical Description-Assets

• Character from Rocketbox library
• UI components: task canvas, chat window, agent panel
• Google Cloud Text-to-Speech API with SSML support for natural speech generation
• SALSA LipSync Suite for real-time facial synchronization and animation
• Cloud database infrastructure for comprehensive event logging

Software architecture