Engineers at Northwestern University have developed a revolutionary new system that allows anyone to capture full-body motion without needing expensive equipment or special cameras.
Called MobilePoser, this app uses just a smartphone, smartwatch, or even wireless earbuds to track a person’s body movements in real time.
This breakthrough makes motion capture technology more accessible, opening the door for exciting possibilities in gaming, fitness, and health care.
Traditionally, motion capture (or “mocap”) has been a complex and costly process, often requiring actors to wear special suits covered in sensors and move in rooms filled with high-tech cameras.
This method, used in movies like Avatar and Lord of the Rings, costs tens of thousands of dollars.
But the team behind MobilePoser, led by Karan Ahuja, wanted to develop a more affordable and portable version that anyone could use with devices they already own.
MobilePoser works by taking advantage of sensors that are already built into many everyday devices, such as smartphones and smartwatches.
These sensors, called inertial measurement units (IMUs), track movement using accelerometers, gyroscopes, and magnetometers.
While these sensors are not as accurate as the ones used in professional motion capture systems, Ahuja’s team improved their performance by creating a powerful artificial intelligence (AI) algorithm.
This custom-built AI uses a large database of high-quality motion capture data to “train” MobilePoser on how to predict body movements.
By feeding sensor data through the AI, the app can estimate joint positions, walking speed, and the orientation of the body. MobilePoser also uses a physics-based optimizer to ensure that the movements captured are realistic.
For example, the app knows that joints can’t bend backward and heads can’t spin all the way around, so it corrects any impossible movements.
The result is a highly accurate system with a tracking error of just 8 to 10 centimeters. For comparison, Microsoft’s Kinect motion capture system has a tracking error of 4 to 5 centimeters but requires a person to stay within the camera’s view.
With MobilePoser, users can move freely, and the app will still track their full-body movements.
MobilePoser adapts to different devices. If a person is wearing both a smartphone and a smartwatch, the system becomes even more accurate. But even if they only have one device, such as a phone in their pocket, the app can still estimate their full-body pose.
While MobilePoser can make gaming more immersive, the app also has potential uses in health and fitness. It goes beyond simple step counting by allowing users to track their posture and form during exercises.
This could help people avoid injuries by ensuring they are using the correct technique. Additionally, the app could be valuable for doctors to assess patients’ mobility and activity levels, or even for indoor navigation where GPS is not effective.
Ahuja explained, “Right now, phones know more about the outside world than about our own bodies. We want phones to go beyond step counting and help us understand our movements and activities better.”
To encourage further development, Ahuja’s team has made their pre-trained models and code open-source, so other researchers can build on their work.
The MobilePoser app will soon be available for iPhone, Apple Watch, and AirPods, making full-body motion capture easier and more accessible than ever before.
