The Clinical Pulse of Generative AI: An Expert Evaluation of Publicly Available LLMs in Medical Diagnosis and Advice (August 2025)

The 2025 Frontier of General-Purpose LLMs: A Comparative Technical Overview

As of August 2025, the landscape of publicly available large language models (LLMs) is dominated by three state-of-the-art (SOTA) systems: OpenAI's GPT-5, Google's Gemini 2.5 Pro, and xAI's Grok 4. A critical examination of their underlying architecture reveals a significant industry-wide pivot away from monolithic models toward multi-component systems engineered for nuanced reasoning. This architectural evolution is not merely an incremental upgrade but a direct response to the documented failures of previous generations in tasks requiring complex, multi-step logic—the very essence of clinical reasoning. While earlier models demonstrated impressive knowledge recall by passing standardized medical exams , their practical application was hindered by a brittleness in dynamic, real-world scenarios. The new generation of models explicitly addresses this shortcoming by separating rapid, general-purpose response generation from slower, more computationally intensive "thinking" processes.

GPT-5 exemplifies this trend with a unified system that employs a real-time router to delegate queries. Simpler requests are handled by a "smart, efficient model," while more complex problems are passed to a "deeper reasoning model" known as GPT-5 Thinking. This specialized reasoning mode is explicitly reserved for high-stakes queries, such as those related to mental health, and is trained with an emphasis on caution and safety. Similarly, Google markets Gemini 2.5 Pro as a "thinking model" that reasons through its thoughts before responding, a capability built upon techniques like reinforcement learning and chain-of-thought prompting. Google has further advanced this concept with "Deep Think," a mode that uses parallel thinking techniques to explore multiple reasoning paths simultaneously, mimicking a more robust form of human cognition. Grok 4 distinguishes itself by prioritizing deep reasoning over rote memory recall, a focus embedded in its training through a multi-agent approach.

Beyond their reasoning architectures, these models are differentiated by key capabilities with direct relevance to medical applications. Multimodality is now a standard feature, with all three models capable of processing text, images, and files. This is crucial for future clinical tools that may analyze diagnostic images, procedural videos, or patient-recorded audio symptoms alongside text-based records. Gemini 2.5 Pro has a distinct advantage in this area with native support for audio and video inputs, while Grok 4 is unique in its ability to generate video outputs. Another key differentiator is the context window—the amount of information a model can process in a single prompt. Gemini 2.5 Pro leads the industry with a 1 million token context window, enabling the analysis of entire electronic health records (EHRs) or extensive medical literature at once. GPT-5 offers a substantial 400k token window, followed by Grok 4 at 256k tokens.

Each model also possesses unique features that hint at their potential clinical roles. Both Gemini and GPT-5 offer a "Deep Research" function that autonomously plans, searches the web, and synthesizes information into comprehensive reports—a powerful tool for evidence-based medicine. Grok's real-time integration with the social media platform X provides an unparalleled, though potentially noisy, tool for monitoring public health sentiment and the spread of medical information.

The Digital House Call: LLMs as a First Port of Call for Patients

The widespread availability of SOTA LLMs has positioned them as a new front door to the healthcare system for many patients. These tools offer the promise of empowerment, providing 24/7 access to medical information that can enhance health literacy, help patients prepare for clinical appointments, and enable them to better advocate for their own care. By translating complex medical jargon into understandable language, LLMs can demystify medicine for a lay audience. Testimonials from patients, such as a cancer patient who used ChatGPT to understand biopsy results and weigh treatment options, underscore this potential for positive impact.

However, when used as frontline symptom checkers, the performance of general-purpose LLMs is concerning. Their capabilities mirror those of specialized symptom checker apps, which continue to demonstrate diagnostic accuracy far below that of human physicians. A 2025 study in orthopedics and traumatology found that physicians provided a correct diagnosis in 84.4% of cases, whereas the apps averaged only 35.8%. While modern LLMs offer a more intuitive, natural language interface for describing symptoms , they are prone to generating broad differential diagnoses that can cause undue patient anxiety, such as suggesting a brain tumor for a common headache.

The risks associated with direct patient use are substantial and multifaceted. The most immediate danger is the propagation of factually incorrect, incomplete, or outdated information, which can lead to patients delaying necessary care or attempting harmful self-treatments. Studies consistently show that the accuracy of LLM responses declines as the complexity of the medical question increases, particularly for specific queries about diagnosis and treatment side effects.

A more subtle but equally profound risk emerges from the intersection of the models' design and user psychology. LLMs are engineered to be conversational and empathetic, which can lead users to project "quasi-humanness" onto them and develop an unwarranted level of trust. This creates a dangerous paradox. The very design choices meant to improve user experience and build rapport are fostering a false sense of reliability in systems that are technically fragile. Research from MIT has revealed that the clinical logic of LLMs can be significantly skewed by non-clinical variations in a patient's query, such as typos, informal language, or even extra whitespace. The study found that these minor alterations made the model more likely to erroneously recommend self-management instead of a clinical visit, a bias that disproportionately affected recommendations for female patients. This puts the most vulnerable users—those with lower health or digital literacy who are more likely to use informal language—at the greatest risk of receiving compromised medical guidance.

Finally, a critical and often overlooked risk is data privacy. Patients interacting with these public platforms may be inputting highly sensitive personal health information into systems that are not governed by healthcare privacy regulations like the Health Insurance Portability and Accountability Act (HIPAA), creating a significant risk of data misuse.

The Clinician's "Co-pilot": Assessing LLM Utility in Professional Healthcare Settings

Within professional healthcare settings, the adoption of general-purpose LLMs has followed a clear, risk-stratified pattern. The most widespread and successful integration has been in automating low-risk administrative workflows, where the consequences of an error are primarily operational or financial and can be remediated. These tasks include summarizing clinical notes, processing insurance claims, managing patient correspondence, and assisting with clinical coding. By reducing the significant administrative burden on clinicians, these tools free up valuable time for direct patient care. As of 2025, adoption for these purposes is already notable, with 21% of U.S. healthcare organizations using LLMs to answer patient questions and 20% operating medical chatbots.

In contrast, the embrace of LLMs for high-risk clinical decision support (CDS) is far more cautious. The potential is tantalizing: LLMs can rapidly synthesize patient data, generate differential diagnoses, and retrieve the latest evidence from biomedical literature to support clinical judgment. Some clinicians are already seeing this potential realized. A physician specializing in neurodevelopmental conditions described GPT-5 as the first model with medical reasoning "on par with mine," viewing it as a potential real-time "buddy" for collaborating on complex cases. However, the current reality is that off-the-shelf models often fail to provide sufficiently relevant or evidence-based answers to clinical questions and, critically, do not reliably adhere to established diagnostic or treatment guidelines.

This bifurcation in adoption reflects an evolution in physician sentiment, which has shifted from early skepticism to a more nuanced and critical perspective. For clinicians, reliability is the paramount virtue. A "super intelligent" model that hallucinates or makes basic errors even 5% of the time is considered functionally useless for high-stakes clinical work. Consequently, the significant reduction in factual errors and hallucinations in the "thinking" modes of the latest models like GPT-5 is seen as a major step toward genuine clinical utility. While overall organizational adoption of LLMs is high at 67% globally, most of this activity remains experimental; only 23% of companies have deployed these models in production-level systems, citing persistent privacy and ethical concerns.

A promising and less contentious clinical application is in medical education. LLMs are emerging as powerful pedagogical tools, capable of acting as tireless tutors for case-based learning, providing instant feedback on students' clinical reasoning, and simulating a vast range of patient encounters. This can help standardize and democratize medical training, especially in resource-constrained settings.

From Theory to Practice: A Critical Analysis of LLM Performance in Medical Benchmarks

The evaluation of LLM performance in medicine reveals a stark and critical disconnect between their capabilities in standardized, academic tests and their competence in realistic clinical scenarios. Since 2023, LLMs have demonstrated the ability to pass notoriously difficult medical licensing exams. By August 2025, SOTA models are achieving near-perfect scores that far exceed the passing threshold. On the MedQA benchmark, which consists of U.S. Medical Licensing Exam (USMLE)-style questions, GPT-5 scores 96.3%, with other OpenAI models close behind. This represents a dramatic improvement over previous generations.

OpenAI's proprietary HealthBench, developed with physicians to reflect realistic scenarios, further highlights these advances. On the benchmark's most challenging tier, "HealthBench Hard," GPT-5 achieves a score of 46.2%, a significant leap from GPT-4o's 31.6%. Crucially, the reliability of these models has also improved. When using its "thinking" mode, GPT-5's hallucination rate on difficult medical cases is just 1.6%, compared to a clinically unacceptable 15.8% for GPT-4o.

However, this superhuman performance on multiple-choice exams masks deep-seated flaws in practical clinical reasoning. A growing body of research exposes this performance gap. Evaluation frameworks like CRAFT-MD, which simulate realistic doctor-patient conversations, reveal "critical limitations" in the ability of current LLMs to conduct a comprehensive clinical history or engage in sound conversational reasoning. When tested on their ability to adhere to established clinical guidelines for diagnosis and treatment, SOTA models perform poorly, with one study finding only 60% agreement between LLM advice and guidelines. These failures are not just theoretical; an ablation test of GPT models showed they repeatedly failed to include pneumonia in the differential diagnosis for three clear-cut clinical cases.

This discrepancy has led to the realization that high scores on benchmarks like MedQA are becoming "vanity metrics"—impressive for marketing but increasingly detached from true clinical utility. These exams primarily test for declarative knowledge recall and pattern recognition within a highly structured format. They fail to assess the dynamic, iterative, and communicative skills that are the foundation of safe medical practice. The recognition of this gap is fueling a necessary shift in the research community toward developing more robust, performance-based evaluation frameworks that prioritize procedural competence over knowledge regurgitation—a sign of the field maturing from asking "Can it pass the test?" to "Can it practice safely?".

The Unseen Risks: Navigating the Ethical, Regulatory, and Practical Challenges

Beyond technical performance, the integration of LLMs into healthcare is constrained by a formidable set of ethical, regulatory, and practical challenges. Healthcare organizations face immense pressure to adopt these tools, yet their traditional governance frameworks are ill-equipped to manage technologies that are flexible, rapidly evolving, and updated unpredictably by external companies. This creates a fundamental mismatch between the dynamic nature of LLMs and the static nature of medical device regulation.

The U.S. Food and Drug Administration (FDA) is attempting to adapt, issuing draft guidance for AI-enabled medical devices that emphasizes a "Total Product Life Cycle" (TPLC) approach, requiring risk management from design through post-market monitoring. However, this framework is predicated on discrete, versioned products where changes can be validated before deployment. General-purpose LLMs, in contrast, are centrally controlled services that undergo continuous, often unannounced, updates. A clinically validated workflow built on the GPT-5 API could be rendered unsafe or unreliable overnight by a backend model update from OpenAI. This reality breaks the TPLC model, creating a "governance vacuum" where the most widely used AI tools in healthcare operate largely outside of intended safety and oversight mechanisms.

This regulatory ambiguity exacerbates the unresolved issue of liability. A primary concern for physicians is who bears responsibility when an AI system errs and causes patient harm. The American Medical Association (AMA) strongly advocates for policies that apportion liability appropriately and limit physician liability for the failure of a tool they did not create and cannot control. Until this question is legally and regulatorily settled, it will remain a significant barrier to the adoption of LLMs in high-risk clinical roles.

These practical challenges are underpinned by foundational ethical considerations. The core principles of bioethics—beneficence (doing good), non-maleficence (not doing harm), autonomy (respecting patient choice), and justice (fairness)—must guide the deployment of these technologies. The AMA insists that physicians must be integral partners at every stage of the AI lifecycle to ensure tools are clinically valid, ethically sound, and supportive of the patient-physician relationship. Furthermore, LLMs risk perpetuating and amplifying existing societal biases. The MIT study that found different recommendations for female patients is a stark warning of this danger. Biases embedded in training data can lead to responses that are not generalizable across diverse populations, while subscription-based access models threaten to create a new vector of health inequity.

Conclusion and Strategic Recommendations for Stakeholders

As of August 2025, general-purpose LLMs such as GPT-5, Gemini 2.5 Pro, and Grok 4 represent a profound technological leap, offering remarkable capabilities for knowledge synthesis and administrative automation. However, they are not yet reliable or safe for autonomous medical diagnosis or for providing definitive medical advice. Their impressive performance on standardized exams masks critical deficiencies in real-world clinical reasoning, safety, and adherence to evidence-based guidelines. At present, they are best conceptualized as sophisticated yet fallible "co-pilots" that require constant, vigilant human oversight in any clinical context. Navigating this landscape requires a nuanced and stakeholder-specific approach.

Recommendations for Healthcare Providers

• Verify, Never Trust: Treat all LLM outputs, particularly diagnostic suggestions or treatment plans, as unverified hypotheses that must be rigorously cross-referenced with authoritative, evidence-based sources and established clinical guidelines.
• Prioritize Administrative Use: Focus the application of LLMs on tasks with lower clinical risk, such as drafting documentation, summarizing patient records, and generating patient communication templates. This can significantly reduce administrative burden without compromising patient safety.
• Facilitate Patient Education: Proactively discuss the use of LLMs with patients. Counsel them on the significant risks of self-diagnosis and guide them toward reliable sources of medical information. Frame these tools as a way for patients to formulate better questions for their clinical encounters, not as a source of definitive answers.

Recommendations for Healthcare Organizations

• Develop Robust Governance: Establish clear, dynamic institutional policies for the use of generative AI. These policies must define acceptable use cases, mandate transparency in application, and create frameworks for ongoing risk management and performance monitoring that account for the continuous evolution of the underlying models.
• Invest in "Human-in-the-Loop" Systems: Prioritize the implementation of AI tools that augment and enhance, rather than replace, clinical judgment. Focus on technologies that integrate seamlessly into existing clinical workflows and are designed to support, not circumvent, clinician oversight.
• Educate the Workforce: Implement comprehensive training programs to upskill clinicians and staff on both the capabilities and, more importantly, the limitations of LLMs. Fostering a culture of critical and responsible AI use is essential for safe adoption.

Recommendations for Patients

• Use as a Learning Tool, Not a Doctor: Employ LLMs to learn about medical conditions, understand complex terminology, or prepare questions for your healthcare provider. Under no circumstances should these tools be used to self-diagnose or make independent treatment decisions.
• Protect Your Privacy: Exercise extreme caution when entering personal health information into public LLM interfaces. Assume that any information provided can be stored and used by the technology company and is not protected by healthcare privacy laws.
• Always Consult a Professional: Information generated by an LLM is not a substitute for professional medical advice, diagnosis, or treatment. Always seek the guidance of your physician or another qualified health provider for any questions regarding a medical condition.

Recommendations for Policymakers and Regulators

• Address the Governance Gap: Develop new regulatory frameworks specifically tailored to the dynamic, service-based nature of general-purpose LLMs. The traditional medical device model, designed for static software, is insufficient for technologies that are continuously and centrally updated.
• Clarify Liability: Establish clear legal and regulatory frameworks for apportioning liability when AI systems cause patient harm. This ambiguity is a key barrier to physician adoption and must be resolved to ensure accountability.
• Mandate Transparency and Auditing: Require developers to provide greater transparency regarding their training data, methodologies, and model limitations. Support the creation of independent, third-party auditing mechanisms to validate the safety, efficacy, and equity of these powerful systems before and during their deployment in the sensitive domain of healthcare.

Works cited

1. An evaluation framework for clinical use of large language models in patient interaction tasks | Request PDF - ResearchGate, https://www.researchgate.net/publication/387673328_An_evaluation_framework_for_clinical_use_of_large_language_models_in_patient_interaction_tasks 2. Evaluating and Mitigating Limitations of Large Language Models in Clinical Decision Making | medRxiv, https://www.medrxiv.org/content/10.1101/2024.01.26.24301810v1.full 3. Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models - PMC, https://pmc.ncbi.nlm.nih.gov/articles/PMC9931230/ 4. Introducing GPT-5 - OpenAI, https://openai.com/index/introducing-gpt-5/ 5. OpenAI highlights GPT-5's leap in handling complex health queries, https://firstwordhealthtech.com/story/5987830 6. GPT-5: A Technical Breakdown - Encord, https://encord.com/blog/gpt-5-a-technical-breakdown/ 7. Gemini 2.5: Our most intelligent AI model - Google Blog, https://blog.google/technology/google-deepmind/gemini-model-thinking-updates-march-2025/ 8. Gemini 2.5: Deep Think is now rolling out - Google Blog, https://blog.google/products/gemini/gemini-2-5-deep-think/ 9. What to know about Gemini 2.5 Deep Think - TechTalks, https://bdtechtalks.com/2025/08/03/gemini-2-5-deep-think/ 10. How Grok 4 AI Model Redefined Benchmark Standards in 2025, https://www.nitromediagroup.com/grok-4-ail-benchmark-leader-2025/ 11. GPT 5 Compared to Gemini and Claude & Grok - Nitro Media Group, https://www.nitromediagroup.com/gpt-5-vs-gemini-claude-grok-differences-comparison/ 12. Ultimate Comparison of GPT-5 vs Grok 4 vs Claude Opus 4.1 vs Gemini 2.5 Pro [August 2025] | Fello AI, https://felloai.com/2025/08/ultimate-comparison-of-gpt-5-vs-grok-4-vs-claude-opus-4-1-vs-gemini-2-5-pro-august-2025/ 13. Revolutionizing Health Care: The Transformative Impact of Large Language Models in Medicine - Journal of Medical Internet Research, https://www.jmir.org/2025/1/e59069 14. Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities. - Googleapis.com, https://storage.googleapis.com/deepmind-media/gemini/gemini_v2_5_report.pdf 15. Gemini 2.5 Updates: Flash/Pro GA, SFT, Flash-Lite on Vertex AI | Google Cloud Blog, https://cloud.google.com/blog/products/ai-machine-learning/gemini-2-5-flash-lite-flash-pro-ga-vertex-ai/ 16. Gemini Deep Research — your personal research assistant, https://gemini.google/overview/deep-research/ 17. Generative AI/LLMs for Plain Language Medical Information for ..., https://pmc.ncbi.nlm.nih.gov/articles/PMC12325106/ 18. Current applications and challenges in large language models for patient care: a systematic review - PMC - PubMed Central, https://pmc.ncbi.nlm.nih.gov/articles/PMC11751060/ 19. OpenAI's Sam Altman touts benefit of GPT-5 for healthcare, https://www.fiercehealthcare.com/ai-and-machine-learning/altman-touts-benefit-gpt-5-healthcare 20. Accuracy of Artificial Intelligence Based Chatbots in Analyzing Orthopedic Pathologies: An Experimental Multi-Observer Analysis, https://pmc.ncbi.nlm.nih.gov/articles/PMC11764310/ 21. 8 Best Medical Symptom Checkers of 2025 - Docus.ai, https://docus.ai/blog/best-symptom-checkers 22. Medical Framework utilising Gemini 2.5 Pro : r/Bard - Reddit, https://www.reddit.com/r/Bard/comments/1l5p4jp/medical_framework_utilising_gemini_25_pro/ 23. 6 Potential Medical Use Cases For ChatGPT, https://medicalfuturist.com/6-potential-medical-use-cases-for-chatgpt/ 24. LLMs factor in unrelated information when recommending medical ..., https://imes.mit.edu/news-events/llms-factor-unrelated-information-when-recommending-medical-treatments-0 25. LLMs factor in unrelated information when recommending medical treatments - MIT EECS, https://www.eecs.mit.edu/llms-factor-in-unrelated-information-when-recommending-medical-treatments/ 26. The Future of AI in Healthcare – 2025 | SS&C Blue Prism, https://www.blueprism.com/resources/blog/the-future-of-ai-in-healthcare/ 27. AI, ML & NLP in Healthcare: What to Know for 2025 - IMO Health, https://www.imohealth.com/resources/ai-in-healthcare-101-the-role-of-clinical-ai-ml-and-nlp-in-2025-and-beyond/ 28. LLMs in Healthcare: A Measured Path to Impact, https://www.healthcareittoday.com/2025/01/29/llms-in-healthcare-a-measured-path-to-impact/ 29. ChatGPT: Transforming Healthcare with AI - MDPI, https://www.mdpi.com/2673-2688/5/4/126 30. LLM statistics 2025: Comprehensive insights into market trends and integration - Hostinger, https://www.hostinger.com/tutorials/llm-statistics 31. Towards Metacognitive Clinical Reasoning: Benchmarking MD-PIE Against State-of-the-Art LLMs in Medical Decision-Making - medRxiv, https://www.medrxiv.org/content/10.1101/2025.01.28.25321282v1.full.pdf 32. Physicians GPT-5 Review. Finally trades IQ flexing for reliability..... : r/OpenAI - Reddit, https://www.reddit.com/r/OpenAI/comments/1mm6cvz/physicians_gpt5_review_finally_trades_iq_flexing/ 33. 7 ways AI is transforming healthcare - The World Economic Forum, https://www.weforum.org/stories/2025/08/ai-transforming-global-health/ 34. Evaluating General-Purpose LLMs for Patient-Facing Use: Dermatology-Centered Systematic Review and Meta-Analysis | medRxiv, https://www.medrxiv.org/content/10.1101/2025.08.11.25333149v1.full-text 35. An Overview of GPT-5 in Biotechnology and Healthcare - IntuitionLabs, https://intuitionlabs.ai/articles/gpt-5-biotechnology-healthcare-overview 36. Large Language Model Statistics And Numbers (2025) - Springs - Custom AI Compliance Solutions For Enterprises, https://springsapps.com/knowledge/large-language-model-statistics-and-numbers-2024 37. Building the AI-enabled medical school of the future: A new era in ..., https://oncologynews.com.au/commentary/building-the-ai-enabled-medical-school-of-the-future-a-new-era-in-clinician-education/ 38. MedQA Benchmark - Vals AI, https://www.vals.ai/benchmarks/medqa-02-25-2025 39. Capabilities of GPT-4 on Medical Challenge Problems - Microsoft, https://www.microsoft.com/en-us/research/wp-content/uploads/2023/03/GPT-4_medical_benchmarks.pdf 40. GPT-5 Benchmarks - Vellum AI, https://www.vellum.ai/blog/gpt-5-benchmarks 41. An evaluation framework for clinical use of large language models in ..., https://pubmed.ncbi.nlm.nih.gov/39747685/ 42. Reliability of Medical Information Provided by ChatGPT: Assessment Against Clinical Guidelines and Patient Information Quality Instrument - PMC, https://pmc.ncbi.nlm.nih.gov/articles/PMC10365578/ 43. Preliminary evaluation of ChatGPT model iterations in emergency ..., https://pmc.ncbi.nlm.nih.gov/articles/PMC11947261/ 44. Evaluating large language models and agents in healthcare: key challenges in clinical applications | Intelligent Medicine - MedNexus, https://mednexus.org/doi/10.1016/j.imed.2025.03.002 45. Why LLMs demand a rethink of healthcare AI governance | Viewpoint, https://www.chiefhealthcareexecutive.com/view/why-llms-demand-a-rethink-of-healthcare-ai-governance-viewpoint 46. FDA Issues Comprehensive Draft Guidance for Developers of ..., https://www.fda.gov/news-events/press-announcements/fda-issues-comprehensive-draft-guidance-developers-artificial-intelligence-enabled-medical-devices 47. FDA Releases Draft Guidance on AI-Enabled Medical Devices | Insights, https://www.gtlaw.com/en/insights/2025/1/fda-releases-draft-guidance-on-ai-enabled-medical-devices 48. AMA position on the 2025 federal government AI action plan ..., https://www.ama-assn.org/practice-management/digital-health/ama-position-2025-federal-government-ai-action-plan 49. Large Language Models in Medicine: Addressing Ethical Challenges - IOVS, https://iovs.arvojournals.org/article.aspx?articleid=2796252 50. Implementing Large Language Models in Health Care: Clinician-Focused Review With Interactive Guideline - Journal of Medical Internet Research, https://www.jmir.org/2025/1/e71916 51. How AI is Reshaping Clinical Decision-Making in 2025, https://www.himssconference.com/how-ai-is-reshaping-clinical-decision-making-in-2025/