ChatGPT passed tests on a model of the human mental state

Michal Kosinski, a computational psychologist at Stanford University, has suggested that Open AI’s ChatGPT AI chatbot is capable of creating a 9-year-old child’s mental state model. The results of the study are published in the arXiv repository .

The Theory of Mind is the ability to understand and describe what another person is thinking based on available information. Previous research has shown that these abilities appear and develop during childhood and into adulthood. The study of the model led to the development of tests to determine their presence and measure them.

Kosinski tested the pre-2022 version of ChatGPT and found that it failed the mental state model tests. He then tested a version of GPT-3 (davinci-002), released shortly after, and found that it could solve 70 percent of the tests, roughly equivalent to a 7-year-old child. Then, last November, he tested the latest version of davinci-003 and found that it was able to solve 93 percent of the problems, which is about the equivalent of a 9-year-old child.