Comparative assessment of AI models in addressing questions on priapism

An evaluation of response quality and clinical utility

Authors

  • Ahmet Halis Yedikule Chest Diseases and Thoracic Surgery Training and Research Hospital
  • Hacibey Ibrahim Bagcilar Training and Research Hospital, Istanbul, Turkey

DOI:

https://doi.org/10.5489/cuaj.9302

Keywords:

Artificial Intelligence, Priapism, clinical application, ChatGPT

Abstract

INTRODUCTION: This study aimed to evaluate the performance of three artificial intelligence (AI) models — ChatGPT, Gemini, and Copilot — in addressing priapism-related inquiries. The accuracy, comprehensiveness, and clinical applicability of AI-generated responses were systematically analyzed.

METHODS: Frequently asked questions (FAQs) regarding priapism were collected from medical guidelines, literature, and online health platforms. Each AI model generated responses, which were independently assessed by two experts based on accuracy, fluency, and clinical relevance. The Global Quality Score (GQS) was used for evaluation. Statistical analysis was performed using one-way ANOVA, with a significance threshold of p<0.05.

RESULTS: ChatGPT and Gemini demonstrated comparable performance across all thematic categories, with mean scores ranging from 4.5-4.9, while Copilot showed significantly lower scores (3.2–4.2, p<0.001). Both ChatGPT and Gemini provided clinically relevant and accurate information, whereas Copilot’s responses frequently lacked guideline-based recommendations.

CONCLUSIONS: ChatGPT and Gemini were statistically comparable in generating reliable, clinically useful responses, making them valuable tools for medical education and patient counseling. Copilot, however, exhibited lower accuracy and applicability. These findings highlight the need for continuous refinement of AI models to enhance their role in clinical decision-making while ensuring human expertise remains central to patient care.

Downloads

Download data is not yet available.

Downloads

Published

2025-11-25

How to Cite

Halis, A., & Ibrahim, H. (2025). Comparative assessment of AI models in addressing questions on priapism: An evaluation of response quality and clinical utility. Canadian Urological Association Journal, 20(3), E89–92. https://doi.org/10.5489/cuaj.9302

Issue

Vol. 20 No. 3 (2026): CUAJ March

Section

Original Research

License

You, the Author(s), assign your copyright in and to the Article to the Canadian Urological Association. This means that you may not, without the prior written permission of the CUA:

  • Post the Article on any Web site
  • Translate or authorize a translation of the Article
  • Copy or otherwise reproduce the Article, in any format, beyond what is permitted under Canadian copyright law, or authorize others to do so
  • Copy or otherwise reproduce portions of the Article, including tables and figures, beyond what is permitted under Canadian copyright law, or authorize others to do so.

The CUA encourages use for non-commercial educational purposes and will not unreasonably deny any such permission request.

You retain your moral rights in and to the Article. This means that the CUA may not assert its copyright in such a way that would negatively reflect on your reputation or your right to be associated with the Article.

The CUA also requires you to warrant the following:

  • That you are the Author(s) and sole owner(s), that the Article is original and unpublished and that you have not previously assigned copyright or granted a licence to any other third party;
  • That all individuals who have made a substantive contribution to the article are acknowledged;
  • That the Article does not infringe any proprietary right of any third party and that you have received the permissions necessary to include the work of others in the Article; and
  • That the Article does not libel or violate the privacy rights of any third party.