Fungi Hackathon · February 2025
AI-Driven Virtual Classroom
Immersive classroom prototype which incorporates real-time gestures and OCR with multi-persona agents in a chat interface.
Problem framing
Most AI learning companions still feel like chatbots bolted onto courseware. They rarely support the tactile workflows that make instruction stick, such as handwriting formulas, sketching free-body diagrams, or gathering around a shared whiteboard. Our challenge was to deliver an AI-first classroom that stayed human-centered, augmenting teaching rather than replacing it.
Build highlights
- Created a text-extraction pipeline using OpenCV and EasyOCR through Firebase Functions to turn handwriting into structured study cards within seconds.
- Imported classroom context into Fungi's (now Sail's) Agent Development Environment to author persona-based tutors—ranging from succinct drill sergeants to Socratic coaches.
- Connected Leap Motion gesture recognition so instructors could trigger lesson modes, advance scenes, and summon assistants hands-free.
Capturing handwritten notes that stream into the classroom knowledge graph.
Hackathon team wiring sensors and calibrating agent prompts in the lab space.
Demo results
The immersive classroom earned the Persona-lity Plus Award for the most inventive AI persona and was recognized for its potential to transform STEM outreach. Students were able to move from handwritten problems to multi-modal AI feedback loops without leaving the workspace.
Live demo showcasing gesture-triggered persona switches for student coaching.