Artificial intelligence, or AI, has made significant advancements in recent years. One of the main players is OpenAI with their AI chatbot, ChatGPT, which has received much praise for its capabilities.
This article offers a simple look at ChatGPT’s abilities, mixed reviews, and concerns raised.
What Is ChatGPT and How Good Is It?
ChatGPT is a creation of OpenAI, a company that builds machines that can talk just like humans. People have said it’s the best chatbot ever, and it’s been considered a big achievement in computing.
It can answer many types of questions like a person would, so much so that some people think it has passed the Turing test. The Turing test is like a yardstick for seeing if a machine can think like a human.
ChatGPT is pretty smart, too. It scored high in various tests in subjects like math and law. In July 2023, a report showed that its health care advice was almost as good as that from actual medical staff.
But There Are Some Doubts
But not everyone is ready to put all their trust in ChatGPT just yet. Some researchers at two big universities, Stanford and UC Berkeley, have found some issues with it.
They say that it doesn’t always perform consistently, and sometimes it seems to be getting worse.
The researchers wrote about this in a paper in July 2023. They found that both GPT-3.5 and GPT-4, two versions of ChatGPT, were not stable in their performance. Some tasks that it was previously good at, it started doing worse in.
For example, in March 2023, GPT-4 could solve math problems about prime numbers with a 97.6% accuracy.
But by June 2023, this fell to just 2.4%. In programming help, it went from giving accurate answers half the time in March to just 10% of the time in June.
Another version, GPT-3.5, had opposite results. Its math skills improved from 7.4% to 86.8% between March and June. But it wasn’t very clear why these changes were happening.
Understanding the Changes and Addressing Concerns
James Zhu, one of the researchers, said they didn’t completely understand why these changes were happening. He said that tweaking the system to get better at certain things might have accidentally made it worse at others.
Some people who’ve noticed these changes have come up with their own theories.
Some think OpenAI is trying to save money by using smaller versions of their models. Others think OpenAI is making GPT-4 worse so people will pay for another product they have.
But OpenAI says that’s not true. They said they’re making each new version smarter, not dumber. They said if people are finding more issues, it might be because they’re using it more.
Some people are asking OpenAI to share more about how they make and train ChatGPT.
Sasha Luccioni from AI company Hugging Face said results from models that are kept secret can’t be checked or repeated, so it’s like comparing apples to oranges. She thinks it’s on the creators to allow access to their models, so people can check them for any issues.
The study was posted in arViv.
Copyright © 2023 Knowridge Science Report. All rights reserved.
