For the theory test, medical students usually have to cram for years. The text robot ChatGPT answers the difficult questions off the cuff – and achieves a surprisingly good result, as researchers report. It is not the first university exam that the artificial intelligent masters.
The text robot ChatGPT would do remarkably well in an exam for prospective medical students in the USA. Under certain conditions, the AI ??software was able to achieve the required minimum number of points in the three theoretical parts of the United States Medical Licensing Exam (USMLE). This is reported by US experts in the journal “PLOS Digital Health”. However, the team led by Victor Tseng from the Californian startup AnsibleHealth left out a few questions. The USMLE is a standardized three-part exam that medical students must pass in order to practice medicine in the United States. Knowledge from most medical disciplines – from biochemistry to diagnostic thinking to bioethics – is evaluated.
However, when testing with ChatGPT, the authors of the study had to take into account that the OpenAI AI software can only accept text input. Image-based questions have been removed. The authors then tested the software on 350 publicly available questions that were part of the USMLE exam last summer.
The result: After ambiguous answers from the AI ??were removed, ChatGPT achieved 52.4 to 75 percent of the points that were still achievable in the three USMLE exam parts. The results varied depending on the part of the test and the type of task – such as tick tests or open questions. The threshold for passing is around 60 percent and changes slightly depending on the year. If the ambiguous answers were included in the result, ChatGPT achieved 36.1 to 61.5 percent of the possible points. According to the authors, ChatGPT outperformed PubMedGPT, a counter-model trained exclusively on biomedical literature.
The team concluded that ChatGPT had the potential to improve medical education and, in turn, clinical practice. “Achieving the minimum pass mark for this notoriously difficult expert exam, and doing so without any human assistance, is a remarkable milestone in the development of clinical AI,” the authors wrote.
The text robot ChatGPT had previously asserted itself in other university subjects, even if it did not achieve top marks in the exams. According to professors at the University of Minnesota, the AI ??chatbot tool passed the law exams in four courses and another course at the Wharton School of Business at the University of Pennsylvania.
After the public launch of ChatGPT by the Californian start-up company OpenAI in November 2022, a number of players in the tech industry were in a real race for the favor of users. OpenAI is supported by Microsoft with billions of dollars. In return, Microsoft may integrate the technology into its Bing search engine and other products.
The big opponent of OpenAI and Microsoft is Google. The search engine giant launched its own chat bot Bard on Tuesday. Smaller players such as You.com by German AI researcher Richard Socher are also taking part in the race. Socher comes from Dresden, but has been living and working in the USA for years.
