ChatGPT Outperforms US College Students in SAT Problem-Solving, Study Finds

ChatGPT problem-solving capabilities match or exceed those of undergraduate students, a recent study has found.

A group of psychologists from the University of California, Los Angeles (UCLA) decided to test the capacity of GPT-3 in predicting the next image in a complex array of shades.

According to researchers, GPT-3, a large language model released by OpenAI in 2020, has performed almost as effectively as undergraduates at colleges in the United States, accurately solving 80 percent of reasoning problems, Erudera.com reports.

The research published in the Nature Human Behaviour journal showed that OpenAI's GPT-3 solved most problems in intelligence exams such as SAT, a test used for US college admission.

As the Guardian reports, the same tasks were given to about 40 UCLA undergraduate students, and according to the researchers, while GPT-3 successfully solved 80 percent of the problems, undergraduates solved below 60 percent.

Nonetheless, the model was less successful when researchers asked it to connect a passage of prose with a different short story that conveyed the same meaning. On the other hand, students performed better in this task.

Taylor Webb, the study's main author, said that the model appeared to struggle more with social interactions, mathematical logic, and problem-solving skills, such as placing sweets from one bowl into a different one.

“It’s definitely not fully general human-level intelligence. But it has definitely made progress in a particular area,” said Webb, a postdoctoral researcher in psychology at UCLA,” he told the newspaper.

The launch of ChatGPT triggered concerns within the education sector worldwide, with many universities deciding to restrict access to it to prevent potential student plagiarism. Some institutions issued warnings to their students, telling them to refrain from using the tool to cheat.

A previous survey involving thousands of undergraduates and graduates revealed that 50 percent of students in the US use artificial intelligence tools to complete their college assignments, with 17 percent saying they didn't edit their assignments after using AI.

Despite the panic over student cheating, experts in the sector claimed that artificial intelligence will not affect critical thinking and problem-solving skills.

In early January, Jenna Lyle, a spokeswoman for the New York City Department of Education, told the Washington Post that although ChatGPT could provide prompt and easy responses to different questions, it fails to build critical thinking and problem-solving abilities, which are very important skills for academic achievements.

Last month, Harvard announced that starting this fall, computer science students will be able to use AI to find errors in their codes, as its CS50 course will be taught entirely by a chatbot.

>> A.I. Versus Humans: Subjects That Teach You Skills Machines Can’t Replicate