PaperTracker
A low-tech educational platform that teaches programming and music through gamification. Users program agents by drawing shapes on paper, placing 3D objects on a table surface, and compiling code by photographing the board — layered sound output provides real-time feedback on solutions. Delivered as interactive workshops for educators and young learners.
The Thesis
Affordable, accessible technology for any community. PaperTracker teaches music, technology, and game design through gamification — promoting problem-solving, collaborative work, and programming without screens or keyboards. Originally designed as interactive workshops for educators and children ages 10–14 from disadvantaged areas of New Zealand.
The Challenge
How to make programming tangible and accessible for young learners without requiring computers or expensive equipment. Traditional coding education creates barriers — abstract syntax, screen fatigue, and cost. The challenge was building a system where the interface is paper, the feedback is sound, and the learning is play.
Solution: Programmable agents as 3D objects placed on a table surface, similar to Reactable. Programming occurs through drawing shapes on paper and using mobile devices with shape recognition. Users program new agent behaviour using existing agents, then compile by photographing the board. Layered sound output provides real-time feedback on solutions.
Design Decisions
- —Paper as interface — no screens, no keyboards, maximum accessibility
- —Sound as feedback — immediate, intuitive response to programmed behaviours
- —Mobile devices for recognition — leveraging hardware everyone already owns
- —Modular agent design — learners build complexity by combining simple functions
Presentations & Publications
SUAC University, Hamamatsu, Japan
Workshop at SUAC University of Art and Culture
Tokyo Festival of Modular 2019
Installation at Tokyo Festival of Modular
Wellington Festival 2019
Presentation on Wellington Festival
University of York, UK 2019
Lecture and Workshops
SIGGRAPH Asia 2020
Paper: R. Rudnicki et al. "A Gamified Music & Technology Teaching Tool." (peer-reviewed)
Team
| Name | Role |
|---|---|
| Tristan Bunn | Lead Programmer, UX Design |
| Jon He | Programmer |
| Andre Mūrnieks | Interface & Board Design |
| Radek Rudnicki | Project Lead, Sound Design & Workshop Delivery |
Links
Stack
JavaScript • CSS • Python • OpenCV • Shape Recognition • Web Audio • Mobile Integration • Physical Computing