Team: PersonifyIt
Team Participants:
- Angela Wayne
- Matthew Loh
- Reuben Nogueira
- Anthony Lam
“What if there’s a video chatbot that doesn’t sleep or eat or take a break, that can help [a student] at any time that [he or she] needs?”
Problem
Students and staff needing IT support with password resets, account access, and technical questions during evenings and weekends have no access to live assistance when IT staff are unavailable. The existing IT chatbot provides text-based help but lacks the personal, conversational experience of interacting with a human support representative. This gap particularly affects online students and the approximately 80% Hispanic student population who would benefit from Spanish-language support. The text-only interface limits engagement and accessibility for users who prefer visual, video-based interaction. With IT demand extending well beyond business hours, the existing workflow creates a support bottleneck that delays resolution times and places additional pressure on staff during peak hours. The reliance on a text-only, English-primary chatbot means a significant portion of the student population receives a diminished support experience, consuming time during business hours that could be better spent on complex, high-touch IT issues.
Technical Solution
The Hartnell College team developed a bilingual video-based IT support assistant that delivers personalized help through an interactive talking-head avatar. When a student, faculty member, or staff person speaks a question in English or Spanish, the system queries Amazon Bedrock Knowledge Bases (managed RAG) preloaded with Hartnell’s IT documentation to retrieve grounded answers with source citations, then rewrites the structured response into natural conversational speech using Google Gemini (generative AI), and streams the result back to the Tavus video avatar which speaks it aloud with synchronized lip movement. The architecture consists of a React web frontend that embeds the Tavus Conversational Video Interface, an Express backend serving an OpenAI-compatible proxy endpoint that orchestrates the Bedrock retrieval and Gemini rewrite steps, and session data storage in Amazon DynamoDB (NoSQL database) for transcripts, user ratings, and feedback. After a conversation ends, users can rate their experience and optionally receive a full transcript via Amazon SES (email delivery), with long-term archival to Amazon S3 (cloud storage) for compliance and analytics. This solution eliminates the need for students to navigate complex help documentation or wait in service-desk queues, providing instant, grounded, and accessible IT support in the user’s preferred language while capturing session data to help the college identify common issues and improve support resources over time.
Showcase
| Presentation Recording | A recording of AI Summer Camp students presenting their project. |
| Slideshow | The accompanying slides shown in the video. |
| Source Code | All of the code and assets developed during the course of the AI Summer Camp. |
About the DxHub
The Cal Poly Digital Transformation Hub (DxHub) is a strategic relationship with Amazon Web Services (AWS) and is the world’s first cloud innovation center supported by AWS on a University campus. The primary goal of the DxHub is to provide students with real-world problem-solving experiences by immersing them in the application of proven innovation methods in combination with the latest technologies to solve important challenges in the public sector. The challenges being addressed cover a wide variety of topics including homelessness, evidence-based policing, digital literacy, virtual cybersecurity laboratories and many others. The DxHub leverages the deep subject matter expertise of government, education, and non-profit organizations to clearly understand the customers affected by public sector challenges and develop solutions that meet the customer needs.