AI-Driven Virtual Classroom

Problem framing

Most AI learning tools are chat interfaces added onto courseware, without support for handwritten formulas, free-body diagram sketches, or shared whiteboards. Our goal was a classroom system that integrates AI feedback with those physical workflows.

Build highlights

• Created a text-extraction pipeline using OpenCV and EasyOCR through Firebase Functions to turn handwriting into structured study cards within seconds.
• Imported classroom context into Fungi's (now Sail's) Agent Development Environment to author persona-based tutors, from succinct drill sergeants to Socratic coaches.
• Connected Leap Motion gesture recognition so instructors could trigger lesson modes, advance scenes, and summon assistants hands-free.

Capturing handwritten notes for the OCR pipeline.

Hackathon team wiring sensors and calibrating agent prompts in the lab space.

Demo results

We won the Persona-lity Plus Award for the most inventive AI persona. In the demo, students could go from a handwritten problem to AI feedback without switching tools or leaving the workspace.

Live demo: switching tutor personas with a hand gesture.