Artificial intelligence is transforming many industries in positive ways, but new research warns that the same technology could also be used for dangerous purposes.
A team from NYU Tandon School of Engineering has found that widely available AI tools can help hackers secretly insert hard-to-detect flaws into computer chips.
These hidden vulnerabilities, sometimes called “hardware Trojans,” could allow attackers to steal sensitive data, disable systems, or gain unauthorized control.
The study, published in IEEE Security & Privacy, showed how large language models (LLMs) like ChatGPT can be misused in chip design.
To test the risks, the researchers launched the AI Hardware Attack Challenge, a two-year competition held as part of CSAW, a student-run cybersecurity event at NYU.
Teams were tasked with using generative AI to add security weaknesses into open-source hardware designs such as RISC-V processors and cryptographic systems, then demonstrate how those weaknesses could be exploited.
Jason Blocklove, a Ph.D. candidate in NYU Tandon’s Electrical and Computer Engineering Department and lead author of the study, explained that AI significantly lowered the barrier for creating these attacks.
“AI tools definitely simplify the process of adding these vulnerabilities,” he said. Some teams managed to fully automate the process, while others used AI to better understand the hardware design and write malicious code.
The most successful entries came from teams that built automated tools capable of analyzing chip code, identifying weak spots, and inserting malicious logic without much human input.
The AI-generated flaws included hidden backdoors granting access to memory, methods to leak encryption keys, and code designed to crash systems under certain conditions.
Perhaps most worrying, even participants with little background in hardware were able to create damaging vulnerabilities.
Undergraduate teams with limited knowledge of chip design still produced medium- to high-severity flaws, according to industry scoring systems. This suggests that AI could allow attackers without advanced expertise to develop sophisticated hardware-level threats.
Although large language models include safety filters to block malicious use, participants discovered ways to bypass them. Some asked the AI to generate harmful code by framing it as a research problem, while others avoided safeguards by requesting outputs in less common languages.
What makes hardware flaws particularly dangerous is that they cannot be fixed after a chip has been manufactured. Unlike software bugs, which can often be patched with an update, hardware Trojans are permanent unless the chips are physically replaced. “Once a chip has been manufactured, there is no way to fix anything in it without replacing the components,” Blocklove said.
The research highlights the double-edged nature of AI in chip design. While previous projects showed AI could accelerate the creation of new processors, this study reveals the same tools could be weaponized.
The team concluded that stronger AI safeguards and better chip verification systems will be essential to prevent such threats in the future.
