Scientists at UCLA and UC Berkeley have created a groundbreaking type of image sensor that can recognize materials and objects the moment light hits it—without needing a computer to process the data afterward.
This new technology, described in Science, could transform machine vision, environmental sensing, robotics, and mobile imaging by making them dramatically faster and more energy efficient.
Normally, spectral cameras collect huge amounts of information by capturing many separate images, each at a different wavelength of light.
These stacks of data must then be sent to processors to analyze what the scene contains.
That process is powerful but slow, energy-hungry, and limited by the amount of data that must be moved around inside the hardware.
The new sensors, called spectral kernel machines (SKMs), avoid this problem entirely.
Instead of storing or analyzing large volumes of data, they perform the analysis inside the detector itself.
When light hits the device, the sensor instantly decides what material or object is present and turns that result into a simple electric signal.
This approach copies the behavior of machine-learning systems known as kernel machines, but the computation happens physically, not digitally.
According to UCLA Professor Aydogan Ozcan, one of the lead researchers, this is the first time photodetectors have been designed to learn and compute directly as they detect light. The device doesn’t just sense—it interprets.
To teach the sensor, the team displayed images to it during a training phase. For example, they showed scenes of colorful birds in forests. The SKM device randomly sampled pixels while receiving simple instructions like “find the bird.”
Over many examples, it learned how to tune its electrical controls to highlight pixels belonging to the bird and ignore everything else.
Later, when shown new images, the device responded only to the parts containing the bird, generating a positive signal for those pixels. This behavior is what the researchers describe as a “sniff and seek” process, similar to how a trained retriever dog searches for a target.
Because the SKM can be tuned to detect specific spectral features, it works across both visible and infrared light. In tests, silicon-based versions performed tasks such as inspecting semiconductor wafers and identifying features at high speed. Mid-infrared versions detected chemicals and analyzed mixtures—all without needing the traditional hyperspectral data cubes used in industry.
The researchers also demonstrated practical uses, such as measuring the hydration of plant leaves and separating objects in complex scenes, all based solely on the device’s output photocurrent.
By reducing data movement and eliminating the need for digital post-processing, SKMs offer a new model for machine vision—one that is ultrafast, compact, and energy efficient. This could benefit everything from smartphones and autonomous robots to satellites and environmental monitoring systems.
As co-author Yuhang Li notes, the technology redefines photodetection as a form of automatic physical computation, allowing complex spectral analysis to happen instantly, right where the photons first arrive.
