Optimizing ChatGPT

Applying prompt engineering to frequently asked questions in urologic oncology

Authors

  • Mark N. Alshak The James Buchanan Brady Urological Institute, Johns Hopkins University School of Medicine
  • Michelle I. Higgins The James Buchanan Brady Urological Institute, Johns Hopkins University School of Medicine
  • Craig Cronin The James Buchanan Brady Urological Institute, Johns Hopkins University School of Medicine
  • William S. Azar The James Buchanan Brady Urological Institute, Johns Hopkins University School of Medicine
  • Joseph G. Cheaib The James Buchanan Brady Urological Institute, Johns Hopkins University School of Medicine
  • Max Kates The James Buchanan Brady Urological Institute, Johns Hopkins University School of Medicine
  • Sunil H. Patel The James Buchanan Brady Urological Institute, Johns Hopkins University School of Medicine

DOI:

https://doi.org/10.5489/cuaj.9578

Keywords:

urologic oncology, prostate cancer, bladder cancer, kidney cancer, ChatGPT, articial intelligence, prompt engineering

Abstract

Introduction: Patients rely on online searches for patient education materials (PEMs). PEMs are recommended to be written at or below a sixth-grade reading level but are regularly written at a college reading level. Using prompt engineering, we assess the information, misinformation, and readability of ChatGPT responses to urologic oncology questions.

Methods: Forty-five questions relating to prostate, bladder, and kidney cancer were presented to ChatGPT (version 4o, OpenAI). Quality of health information was assessed using DISCERN (1 [low] to 5 [high]). Understandability and actionability were assessed using PEMAT-P (0 [low] -100% [high]). Misinformation was scored from 1 [no misinformation] to 5 [high misinformation]. Grade and reading level were calculated using the Flesch-Kincaid scale [5 (easy) to 16 (difficult), and 100-90 (5th grade level) to 10-0 (professional level), respectively]. Prompt engineering was then applied to responses and evaluated.

Results: ChatGPT answers are highly accurate but too advanced of a reading level and lacked explanations of benefits, risks, visual aids, actionability, and citations. Using prompt engineering, DISCERN (3.42-4.47, p<0.0001), PEMAT-P understandability (88.4-95.5%, p<0.0001), and actionability (25.6-84.2%, p<0.0001), grade reading level (10.55.3, p<0.0001), and reading level (42 [college level] to 71.7 [7th grade], p<0.0001), all significantly improved. Misinformation did not change significantly.

Conclusions: Using prompt engineering, ChatGPT provides highly accurate and understandable PEMs at a patient appropriate reading level and provides concrete resources for patient action. Urologists should understand prompt engineering and be involved in the development of artificial chatbots to optimize results.

Downloads

Download data is not yet available.

Downloads

Published

2026-03-30

How to Cite

Alshak, M. N., Higgins, M. I., Cronin, C., Azar, W. S., Cheaib, J. G., Kates, M., & Patel, S. H. (2026). Optimizing ChatGPT: Applying prompt engineering to frequently asked questions in urologic oncology. Canadian Urological Association Journal, 20(8). https://doi.org/10.5489/cuaj.9578

Issue

Vol. 20 No. 8 (2026): CUAJ August

Section

Original Research

License

You, the Author(s), assign your copyright in and to the Article to the Canadian Urological Association. This means that you may not, without the prior written permission of the CUA:

  • Post the Article on any Web site
  • Translate or authorize a translation of the Article
  • Copy or otherwise reproduce the Article, in any format, beyond what is permitted under Canadian copyright law, or authorize others to do so
  • Copy or otherwise reproduce portions of the Article, including tables and figures, beyond what is permitted under Canadian copyright law, or authorize others to do so.

The CUA encourages use for non-commercial educational purposes and will not unreasonably deny any such permission request.

You retain your moral rights in and to the Article. This means that the CUA may not assert its copyright in such a way that would negatively reflect on your reputation or your right to be associated with the Article.

The CUA also requires you to warrant the following:

  • That you are the Author(s) and sole owner(s), that the Article is original and unpublished and that you have not previously assigned copyright or granted a licence to any other third party;
  • That all individuals who have made a substantive contribution to the article are acknowledged;
  • That the Article does not infringe any proprietary right of any third party and that you have received the permissions necessary to include the work of others in the Article; and
  • That the Article does not libel or violate the privacy rights of any third party.