GPT-4 shines in Microsoft radiology study, outperforming human experts on some tasks



summary
Summary

Microsoft recently published a study that explores the capabilities and limitations of GPT-4 in radiology.

Working with a radiologist and Nuance, a Microsoft company whose PowerScribe solution is used by more than 80 percent of radiologists in the U.S., the research team created a comprehensive evaluation and defect analysis framework.

Within this framework, the team evaluated GPT-4’s ability to process radiology reports, including understanding natural language and generating radiology tasks such as classifying diseases and summarizing findings. For the tasks, the team was careful to emphasize complex and challenging real-world radiology scenarios.

GPT-4 outperforms even human radiologists

The study found that GPT-4 demonstrated new state-of-the-art performance on some tasks, outperforming existing systems by up to ten percent. Although GPT-4 occasionally fails to surface domain knowledge, it has “substantial capability in processing and analysis of radiology text, achieving near-ceiling performance in many tasks,” the paper states.

Ad

Ad

In einer Studie untersuchte Microsoft die Möglichkeiten und Grenzen von GPT-4 in der Radiologie.
GPT-4 can outperform existing systems on radiological text tasks by up to ten percent. | Picture: Microsoft

In some cases, the radiology report summaries generated by GPT-4 were even more accurate and provided more detailed findings than the reports generated by experienced radiologists.

The radiology reports generated by GPT-4 were preferred over those written by humans. | Image: Microsoft

Another promising aspect of GPT-4 is its ability to automatically structure radiology reports, which are often complex and unstructured. Studies have shown that structured reports can improve standardization and consistency in the description of disease.

This facilitates interpretation by other healthcare providers and makes them more searchable for research and quality improvement initiatives.

GPT-4 could help improve real-world data (RWD) and its use for real-world evidence (RWE) to complement clinical trials and accelerate the translation of research into clinical practice.

Are large language models better specialists?

The results are encouraging, but need to be confirmed by further research and clinical trials, the researchers wrote. “When used with human oversight, GPT-4 also has the potential to transform radiology by assisting professionals in their day-to-day tasks.”

Recommendation

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top