Upgrove Gamified App for Upper Limb Assessment by Xinke Coco Wu Is Granted the A' Design Award in Mobile Technologies, Applications and Software Design

The Work

Project

Upgrove Gamified App for Upper Limb Assessment

Function

Gamified App for Upper Limb Assessment

Category

Mobile Technologies, Applications and Software Design

Discipline

Communication Design

Industry

Software and Electronics & Appliances

Colors

Sienna, Navajo White, Medium Sea Green, Dark Orange and Gray

Images

5

Client

The Multiple Sclerosis Center of Georgia

The Recognition

Laureate

Xinke Coco Wu

Country

United States of America

Recognition

A' Design Award

Edition

2025 - 2026

Design ID

#182818

Credits

Team

Xinke (Coco) Wu, Jhih Rou (Iris) Sun and Yameng Li

Image Credits

Images and visual materials were created by Xinke Wu (Designer), Jhih Rou (Iris) Sun (Designer) with AI-assisted generation tools like ChatGPT and Gemini, UpGrove Project, 2025.

Design Details

Unique Properties

UpGrove addresses a key gap in upper-limb rehabilitation, where accurate assessments are typically limited to clinical settings and recovery progress remains largely invisible between visits. The system translates standardized clinical motion tests into intuitive farm-themed activities that patients can perform at home. Camera-based motion capture analyzes mobility and coordination while gameplay scores objectively quantify performance. Continuous self-tracking, progress visualization, and rewarding feedback help patients stay motivated while enabling clinicians to monitor long-term recovery beyond the clinic.

Inspiration

UpGrove was inspired by the gap between clinic-based upper-limb assessments and patients’ everyday rehabilitation experiences at home. Many patients lack visibility into their long-term recovery progress and often feel discouraged during extended rehabilitation periods. In addition, traditional assessments typically require clinical visits, making frequent monitoring inconvenient for patients and clinicians alike. Research with clinicians, patients, and engineers inspired us to explore how clinical assessments could be translated into intuitive motion games and objective performance scores using device cameras.

Research

Mixed-methods design research was conducted to explore how clinical upper-limb assessments could become engaging gamified experiences. The study combined literature review, interviews with clinicians, domain experts, and patients, concept testing, usability testing, and technical validation. Data were gathered through clinical protocol analysis and motion interaction experiments using mobile cameras. Participants included 10+ clinicians and physical therapists and 3 patients.

Challenge

The main challenge was translating clinically standardized upper-limb assessments into engaging digital interactions without compromising medical validity. Clinical tests are designed for accuracy rather than motivation, while game experiences prioritize engagement. The project required balancing these two perspectives by mapping therapeutic motions to intuitive gameplay mechanics. Another challenge was enabling reliable motion evaluation using device cameras without wearable sensors. Through close collaboration with clinicians and iterative prototyping, the design aligned clinical requirements with accessible interaction design.

Realization and Technology

UpGrove is implemented as a mobile application using camera-based motion capture powered by computer vision algorithms. The system analyzes upper-limb movements in real time to estimate range of motion, stability, and control. Gesture recognition maps clinical motion patterns to interactive game mechanics, while data processing generates performance scores and actionable feedback. The design integrates motion tracking, gamification, and data visualization to support continuous rehabilitation assessment.

Operation and Interaction

Users launch UpGrove on a personal mobile device and create a personalized profile to receive tailored self-assessment tasks. On a regular schedule, patients perform guided upper-limb motion tests designed as farm-themed mini-games in their home environment. The front-facing camera captures body movements while the system analyzes gestures in real time. Visual cues guide users through therapeutic motions, and after each activity the system generates scores and progress feedback.

Project Duration and Location

The project was initiated in 2023 as part of human-computer interaction research on digital rehabilitation. Concept development, interaction design, and prototype implementation were completed between 2024 and 2025. The project was developed through interdisciplinary collaboration among designers, engineers, clinicians, and patients to explore accessible at-home rehabilitation assessment. The design is currently being implemented by the Georgia Tech Contextual Computing Lab in collaboration with the Georgia Multiple Sclerosis Center.

Specifications

The system operates on standard cross-platform devices such as mobile phones, tablets, and TVs equipped with front-facing cameras. Computer vision models analyze skeletal movement patterns to estimate joint mobility and motion trajectories without requiring wearable sensors. The application provides real-time visual guidance, scoring mechanisms, and progress-tracking dashboards. Motion data are translated into measurable indicators such as range of motion, accuracy, and movement stability, enabling scalable and sensor-free rehabilitation assessment.

Keywords

Digital Health, Upper-Limb Recovery, Sensor-Free Motion Capture, Computer Vision, Gamified Therapy, Gamified Assessment, Gesture Interaction, Motion Games