A group of computer science students from the University of Toronto has developed a cool robotic device that could make creating DIY tutorial videos a breeze.
They’ve named this camera-wielding robot “Stargazer.”
Its job? To help teachers and other content creators film engaging instructional videos.
Especially for those who don’t have a cameraperson at hand, Stargazer could be a game-changer. Instead of working with a stationary camera, this dynamic robot captures videos with a lot more flexibility.
Jiannan Li, the lead researcher and a Ph.D. student at U of T, emphasizes, “The robot is there to help humans, but not to replace humans. The instructors are here to teach. The robot’s role is to help with filming—the heavy-lifting work.”
The team’s work on Stargazer was presented at an international conference on human-computer interaction in Germany.
Stargazer uses a single camera mounted on a robot arm with seven motors that can move along with the subject in the video. It identifies important regions in the frame and can adjust its movement based on cues from the instructor.
This includes body movements, gestures, and speech picked up by Stargazer’s sensors.
Interestingly, the instructor’s voice is captured by a wireless microphone and then translated into text using Microsoft Azure’s speech-recognition software. This text, along with a custom prompt, is sent to GPT-3, a language model program, which helps determine the instructor’s intent for the camera. Whether it’s a standard or high angle, or normal or tight framing, Stargazer adapts accordingly.
For instance, if an instructor points to various tools they’re using during a tutorial, Stargazer pans its camera to each tool. Or if the instructor mentions a top view of a certain action, the robot adjusts its framing to give a high angle shot. These commands don’t disrupt the teaching process, making them an ideal integration into the lesson.
“We want these vocabularies to be non-disruptive. It should feel like they fit into the tutorial,” Li explains. The team’s objective is to have Stargazer understand in real-time the kind of shot the instructor wants without interrupting the flow of the lesson.
In a test involving six instructors teaching different skills, Stargazer proved its worth. With its help, instructors created videos on a range of topics, from maintaining skateboards to setting up virtual-reality headsets. The robot effectively tracked the subject and adjusted camera framing and angles.
While Stargazer excels at tabletop activities, the team is interested in exploring camera drones and robots on wheels for filming larger spaces from varied angles. Additionally, they’re looking into ways Stargazer could recognize more subtle cues, like an instructor’s gaze, posture, or speech.
Despite using expensive equipment, Li believes that the concept of Stargazer isn’t necessarily limited by cost. He sees potential in the consumer market for robotic filming equipment and hopes that Stargazer could inspire more advanced and interactive devices.
The team is enthusiastic about the potential Stargazer offers for human-robot collaboration. “For robots to work together with humans, the key is for robots to understand humans better,” Li explains.
He hopes to inspire others to consider how robots can better interpret human communication for improved collaborative behaviors.
Source: University of Toronto.
