Xinke Coco Wu · CLIENT: The Multiple Sclerosis Center of Georgia
Upgrove is a mobile application for upper limb rehabilitation assessment at home. The application uses camera-based motion capture on standard phones, tablets and televisions with front-facing cameras, so no wearable sensors are needed. Standardized clinical motion tests are translated into farm-themed mini-games. Computer vision analyzes movement to estimate range of motion, accuracy and stability, and a score is produced after each activity. Progress reports allow patients to follow long-term recovery and allow clinicians to review performance remotely.
Press Kit
- Unique Properties
- UpGrove addresses a key gap in upper-limb rehabilitation, where accurate assessments are typically limited to clinical settings and recovery progress remains largely invisible between visits. The system translates standardized clinical motion tests into intuitive farm-themed activities that patients can perform at home. Camera-based motion capture analyzes mobility and coordination while gameplay scores objectively quantify performance. Continuous self-tracking, progress visualization, and rewarding feedback help patients stay motivated while enabling clinicians to monitor long-term recovery beyond the clinic.
- Dimensions & Packaging
- The system operates on standard cross-platform devices such as mobile phones, tablets, and TVs equipped with front-facing cameras. Computer vision models analyze skeletal movement patterns to estimate joint mobility and motion trajectories without requiring wearable sensors. The application provides real-time visual guidance, scoring mechanisms, and progress-tracking dashboards. Motion data are translated into measurable indicators such as range of motion, accuracy, and movement stability, enabling scalable and sensor-free rehabilitation assessment.
- Tags
- Digital Health, Upper-Limb Recovery, Sensor-Free Motion Capture, Computer Vision, Gamified Therapy, Gamified Assessment, Gesture Interaction, Motion Games
- Production Technology
- UpGrove is implemented as a mobile application using camera-based motion capture powered by computer vision algorithms. The system analyzes upper-limb movements in real time to estimate range of motion, stability, and control. Gesture recognition maps clinical motion patterns to interactive game mechanics, while data processing generates performance scores and actionable feedback. The design integrates motion tracking, gamification, and data visualization to support continuous rehabilitation assessment.
- Design Challenge
- The main challenge was translating clinically standardized upper-limb assessments into engaging digital interactions without compromising medical validity. Clinical tests are designed for accuracy rather than motivation, while game experiences prioritize engagement. The project required balancing these two perspectives by mapping therapeutic motions to intuitive gameplay mechanics. Another challenge was enabling reliable motion evaluation using device cameras without wearable sensors. Through close collaboration with clinicians and iterative prototyping, the design aligned clinical requirements with accessible interaction design.
- Project Duration
- The project was initiated in 2023 as part of human-computer interaction research on digital rehabilitation. Concept development, interaction design, and prototype implementation were completed between 2024 and 2025. The project was developed through interdisciplinary collaboration among designers, engineers, clinicians, and patients to explore accessible at-home rehabilitation assessment. The design is currently being implemented by the Georgia Tech Contextual Computing Lab in collaboration with the Georgia Multiple Sclerosis Center.
- Operation Flow
- Users launch UpGrove on a personal mobile device and create a personalized profile to receive tailored self-assessment tasks. On a regular schedule, patients perform guided upper-limb motion tests designed as farm-themed mini-games in their home environment. The front-facing camera captures body movements while the system analyzes gestures in real time. Visual cues guide users through therapeutic motions, and after each activity the system generates scores and progress feedback. Over time, the application builds longitudinal recovery reports that allow patients to understand their rehabilitation progress while also enabling clinicians to remotely review performance data and monitor patient recovery outside clinical visits.
- Research
- Mixed-methods design research was conducted to explore how clinical upper-limb assessments could become engaging gamified experiences. The study combined literature review, interviews with clinicians, domain experts, and patients, concept testing, usability testing, and technical validation. Data were gathered through clinical protocol analysis and motion interaction experiments using mobile cameras. Participants included 10+ clinicians and physical therapists and 3 patients. User testing results indicated that UpGrove significantly improved patients’ visibility of recovery progress, motivation to perform self-assessments, and accessibility of upper-limb rehabilitation beyond clinical environments.
- Inspiration
- UpGrove was inspired by the gap between clinic-based upper-limb assessments and patients’ everyday rehabilitation experiences at home. Many patients lack visibility into their long-term recovery progress and often feel discouraged during extended rehabilitation periods. In addition, traditional assessments typically require clinical visits, making frequent monitoring inconvenient for patients and clinicians alike. Research with clinicians, patients, and engineers inspired us to explore how clinical assessments could be translated into intuitive motion games and objective performance scores using device cameras. By transforming an often invisible and clinic-isolated recovery process into a visible and accessible experience, UpGrove aims to improve engagement, accessibility, and continuity in long-term rehabilitation.
- Image Credits
- Images and visual materials were created by Xinke Wu (Designer), Jhih Rou (Iris) Sun (Designer) with AI-assisted generation tools like ChatGPT and Gemini, UpGrove Project, 2025.
Press releases are available in 1 languages.