AI and the Soul of Medicine

Authors

  • Thomas F Heston Department of Family Medicine, University of Washington, Seattle, USA; Department of Medical Education and Clinical Sciences, Elson S. Floyd College of Medicine, Washington State University, Spokane, USA. https://orcid.org/0000-0002-5655-2512

DOI:

https://doi.org/10.5281/zenodo.19544014

Keywords:

artificial intelligence, large language models, medical ethics, physician-patient relationship, healthcare AI, alignment problem, clinical decision support, empathy, human dignity, responsible AI

Abstract

The rapid integration of large language models into clinical medicine raises ethical, philosophical, and existential questions that extend beyond traditional concerns about patient safety and data privacy. These artificial intelligence systems now perform clinical tasks such as diagnostic reasoning, protocol selection, and patient communication at levels comparable to trained physicians, yet the frameworks governing their deployment remain anchored in an era when machines could not convincingly simulate human understanding. The ethical implications of incorporating large language models into healthcare are examined through the lens of historical precedent, the physician-patient relationship, and the evolving meaning of human agency in clinical care. Drawing on examples from organ transplantation, intensive care medicine, and genetic engineering, medicine has repeatedly absorbed technologies that challenged foundational ethical assumptions, and each such absorption required the development of new governance frameworks rather than the rejection of the technology itself. Three critical tensions specific to large language models emerge: the gap between linguistic competence and genuine understanding, the redistribution of clinical authority from physicians to algorithms, and the erosion of empathy as a uniquely human contribution to healing. Ethical integration of large language models into medicine requires not only the application of existing bioethical principles but also the development of new frameworks that address the unprecedented capacity of these systems to inhabit the communicative and relational spaces previously reserved for human clinicians.

References

1. Gottlieb S, Silvis L. How to Safely Integrate Large Language Models Into Health Care. JAMA Health Forum. 2023 Sep 21;4(9):e233909. doi:10.1001/jamahealthforum.2023.3909

2. Biever C. ChatGPT broke the Turing test — the race is on for new ways to assess AI. Nature. 2023 Jul 27;619(7971):686–9. doi:10.1038/d41586-023-02361-7

3. Jones CR, Bergen BK. Large Language Models Pass the Turing Test [Internet]. arXiv; 2025 [cited 2026 Apr 12]. Available from: http://arxiv.org/abs/2503.23674 doi:10.48550/arXiv.2503.23674

4. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, et al. Attention is All you Need. In: Guyon I, Luxburg UV, Bengio S, Wallach H, Fergus R, Vishwanathan S, et al., editors. Advances in Neural Information Processing Systems [Internet]. Curran Associates, Inc.; 2017. Available from: https://proceedings.neurips.cc/paper_files/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf doi:10.48550/arXiv.1706.03762

5. Tu T, Palepu A, Schaekermann M, Saab K, Freyberg J, Tanno R, et al. Towards Conversational Diagnostic AI [Internet]. arXiv; 2024 [cited 2026 Apr 11]. Available from: https://arxiv.org/abs/2401.05654 doi:10.48550/ARXIV.2401.05654

6. Ayers JW, Poliak A, Dredze M, Leas EC, Zhu Z, Kelley JB, et al. Comparing Physician and Artificial Intelligence Chatbot Responses to Patient Questions Posted to a Public Social Media Forum. JAMA Intern Med. 2023 Jun 1;183(6):589. doi:10.1001/jamainternmed.2023.1838

7. Harrer S. Attention is not all you need: the complicated case of ethically using large language models in healthcare and medicine. eBioMedicine. 2023 Apr;90:104512. doi:10.1016/j.ebiom.2023.104512

8. Gabriel I. Artificial Intelligence, Values, and Alignment. Minds Mach. 2020 Sep;30(3):411–37. doi:10.1007/s11023-020-09539-2

9. Bostrom N. Superintelligence: paths, dangers, strategies. First edition. Oxford, United Kingdom: Oxford University Press; 2014. 1 p.

10. Heston TF. Safety of large language models in addressing depression. Cureus. 2023 Dec 18;15(12):e50729. doi:10.7759/cureus.50729

11. Heston TF, Lewis LM. ChatGPT provides inconsistent risk-stratification of patients with atraumatic chest pain. PloS One. 2024;19(4):e0301854. doi:10.1371/journal.pone.0301854 PubMed PMID: 38626142; PubMed Central PMCID: PMC11020975.

12. Shah NH, Entwistle D, Pfeffer MA. Creation and Adoption of Large Language Models in Medicine. JAMA. 2023 Sep 5;330(9):866. doi:10.1001/jama.2023.14217

13. Mumford L. The myth of the machine. 1: Technics and human development. In. New York, NY: Harcourt, Brace, Jovanovich; 1991. (A Harvest HBJ book).

14. Giacomini M. A change of heart and a change of mind? Technology and the redefinition of death in 1968. Soc Sci Med. 1997 May;44(10):1465–82. doi:10.1016/S0277-9536(96)00266-3

15. Rothman DJ. Strangers at the bedside: a history of how law and bioethics transformed medical decision making. 2nd pbk. ed. New York: Aldine de Gruyter; 2003. 313 p. (Social institutions and social change).

16. Berg P, Baltimore D, Brenner S, Roblin RO, Singer MF. Summary statement of the Asilomar conference on recombinant DNA molecules. Proc Natl Acad Sci. 1975 Jun;72(6):1981–4. doi:10.1073/pnas.72.6.1981

17. Berg P. Asilomar 1975: DNA modification secured. Nature. 2008 Sep;455(7211):290–1. doi:10.1038/455290a

18. Baltimore D, Berg P, Botchan M, Carroll D, Charo RA, Church G, et al. A prudent path forward for genomic engineering and germline gene modification. Science. 2015 Apr 3;348(6230):36–8. doi:10.1126/science.aab1028

19. Chen Q, Ma Y, Labude M, Schaefer GO, Xafis V, Mills P. Making sense of it all: Ethical reflections on the conditions surrounding the first genome-edited babies. Wellcome Open Res. 2021 Jun 23;5:216. doi:10.12688/wellcomeopenres.16295.2 PubMed PMID: 34395922; PubMed Central PMCID: PMC8340653.

20. Hallinan JTPD, Leow NW, Low YX, Lee A, Ong W, Chan MDZ, et al. Initial Insights Into an Institutional Secure Large Language Model for Magnetic Resonance Imaging Examination Requests: Retrospective Study. J Med Internet Res. 2026 Apr 7;28:e82579–e82579. doi:10.2196/82579

21. Asimov I. Runaround. Astounding Sci-Fict. 1942;9(1):94–103.

22. Russ P, Bedenbender S, Einloft J, Meyer HL, Wenzel LT, Ganser A, et al. Potential of large language models for rapid clinical information support: evidence from acute kidney injury knowledge testing. Sci Rep. 2026 Apr 2;16(1):11224. doi:10.1038/s41598-026-46846-7

23. Fabi A, Egli CE, Wendelspiess SR, Griewing S, Haas Y, De Pellegrin L, et al. Exploring the Role of AI in Managing Treatment Recommendations for Lymphedema: International, Multidisciplinary, Multiprofessional Survey Study of Trust, Reliability, and Impact on Decision-Making. JMIR Med Inform. 2026 Apr 8;14:e80553–e80553. doi:10.2196/80553

24. Cassel EJ. The Nature of Suffering and the Goals of Medicine. N Engl J Med. 1982 Mar 18;306(11):639–45. doi:10.1056/NEJM198203183061104

25. Mesinovic M, Watkinson P, Zhu T. Explainability in the age of large language models for healthcare. Commun Eng. 2025 Jul 17;4(1):128. doi:10.1038/s44172-025-00453-y

26. Lu M, Gillette JJ, Heston TF. Ethical Hazards of Large Language Models in Primary Care: A Clinician-Focused Update. In: Heston DTF, editor. Medical Ethics - Navigating Complex Decisions in Contemporary Healthcare [Internet]. London: IntechOpen; 2025. Available from: https://doi.org/10.5772/intechopen.1012864 doi:10.5772/intechopen.1012864

27. Hurley ME, Lang BH, Kostick-Quenet KM, Smith JN, Blumenthal-Barby J. Patient Consent and The Right to Notice and Explanation of AI Systems Used in Health Care. Am J Bioeth AJOB. 2025 Mar;25(3):102–14. doi:10.1080/15265161.2024.2399828 PubMed PMID: 39288291; PubMed Central PMCID: PMC12143229.

28. Iftikhar Z, Xiao A, Ransom S, Huang J, Suresh H. How LLM Counselors Violate Ethical Standards in Mental Health Practice: A Practitioner-Informed Framework. Proc AAAIACM Conf AI Ethics Soc. 2025 Oct 15;8(2):1311–23. doi:10.1609/aies.v8i2.36632

29. Poulain R, Adiba FI, Fayyaz H, Beheshti R. Bias Patterns in the Application of LLMs for Clinical Decision Support. Del J Public Health. 2026 Mar;12(1):54–67. doi:10.32481/djph.2026.03.10

Downloads

Published

2026-04-13

How to Cite

Heston, T. F. (2026). AI and the Soul of Medicine. Internet Medical Journal, 1(1), e1954401. https://doi.org/10.5281/zenodo.19544014

Issue

Section

Articles

Similar Articles

You may also start an advanced similarity search for this article.