Chat.OpenAI is an AI-powered platform that provides a natural language processing (NLP) interface for building conversational agents, also known as chatbots. It is a cloud-based service that allows developers to quickly create chatbots that can understand and respond to user queries, allowing for a more natural and engaging user experience. Chat.OpenAI is based on cutting-edge AI research, including deep learning and NLP, and is designed to constantly improve its abilities through machine learning. The platform is highly flexible and customizable, allowing developers to create chatbots for a wide variety of use cases, such as customer service, personal assistants, and more. Since its introduction in March 2023, clinicians have been exploring its possible use in different areas of research. Nowadays, 36 articles have been published, and several studies are ongoing.

Cocci et al. have the merit of publishing the first urological study on ChatGPT [1]. The authors evaluated the capacity of ChatGPT to solve 100 urological cases comprising several different clinical scenarios. The accuracy of ChatGPT responses was evaluated by an experienced board-certified consultant urologist with 5 years of experience. According to their results, only 52% of all responses were deemed appropriate. Interestingly, only 11% of emergency urology cases were correctly managed by ChatGPT. This first experience in urology clearly highlights some of the expected limitations of ChatGPT technology and opens several questions and doubts. In urology, we probably have to learn from other specialties.

Suchman et al. evaluated the ability of ChatGPT to pass the American College of Gastroenterology self-assessment test [2]. According to their results, ChatGPT-3 scored 65.1% on 455 included questions, and GPT-4 scored 62.4%. Also, Gilson et al. evaluated the performance of ChatGPT on questions within the scope of the United States Medical Licensing Examination Step 1 and Step 2 exams, as well as analyzed responses for user interpretability [3]. According to their results, of the four data sets, AMBOSS-Step1, AMBOSS-Step2, NBME-Free-Step1, and NBME-Free-Step2, ChatGPT achieved accuracies of 44% (44/100), 42% (42/100), 64.4% (56/87), and 57.8% (59/102), respectively. Estomba et al. evaluated the potential of ChatGPT as a supportive tool for sialendoscopy clinical decision making. According to their results, the mean level of agreement was 3.4 (SD: 0.69; Min: 2, Max: 4) for ChatGPT’s answers, while it was 4.1 (SD: 0.56; Min: 3, Max: 5) for the group of EESS (p < 0.015). Standing to the available evidence, ChatGPT should be used with caution given the suboptimal medical performances [4].

The ability of ChatGPT in scientific writing has also been tested. Manohar and Prasads made ChatGPT write a paper on the association between lupus and HIV, and the paper was fairly accurate. When reading the paper, it appears as a consensus paper; it was written in simple language and may not apply to specialist doctors. The style is rather robotic; however, such a style is often present in scientific literature and, therefore, may be undistinguishable from human writing. Also, Majovski et al. evaluated the ability of artificial intelligence to generate fraudulent but authentic-looking scientific medical articles. Their study clearly highlights that the AI language model can create a highly convincing fraudulent article that resembles a genuine scientific paper in terms of word usage, sentence structure, and overall composition [5]. Last but not least, the process took no more than one hour.

Currently, this software lacks the ability for advanced critical thinking. However, in its present state, it can be valuable for recognizing common ideas from various data and literature sources. It can then present this information to skilled users who can formulate testable hypotheses. Nonetheless, the AI’s frequent training may hinder innovation. The publicly accessible versions of this technology were constructed using outdated resources from 5 years ago, and its training only extends until 2021, which means that recently published data are not currently integrated. Thus far, software such as ChatGPT appears to be an intelligent tool, metaphorically speaking, assisting in the dissemination of relatively straightforward and easily understandable information. It serves as a helpful starting point. However, like conducting an immunohistochemistry experiment using an antibody with uncertain specificity, the outputs may hold little value without the involvement of inquisitive, integrative, and non-linear minds to evaluate the content. Science has always been a human endeavor, and instead of being fearful, we should embrace new technologies while preserving the human element in science. It is important to contemplate how we can prepare for the next conceptual phase, where a critically thinking, analytical AI contributes to scientific progress in the future.

An important limitation of the actual versions of ChatGPT is related to privacy and ethical issues. The output of ChatGPT is based on a probabilistic basis and may not consider these issues. The use of ChatGPT by non-expert users may easily lead to wrong information especially if the input is not correct. Unfortunately, ChatGPT suffers the same limitations as “Dr Google;” however, it has at least the ability to give balanced answers to important questions such as: “Should I treat my BPH? As an AI language model, I cannot provide personal medical advice. However, I can offer you some general information about benign prostatic hyperplasia (BPH) that may help you make an informed decision.”

The possible future applications of ChatGPT in urology are infinite, especially if used as an assistant and not as a substitute for the clinician. The possible applications include patient information, radiological and histological analysis, prediction models, and therapeutic and prognostic assistance [6, 7]. As with every technological innovation, ChatGPT raised skepticism among scientists and fear of being replaced by technology. The abovementioned experiences clearly tear away these fears, considering the suboptimal results of ChatGPT alone. However, if adequately trained, ChatGPT may become an excellent assistant. It is important to underline that ChatGPT answers should take into account social context and physician’s preferences and experience. Further studies will better define the role of ChatGPT in Urological clinical and academic practice and if it can overcome some unmet needs in urology. However, our personal impression, as well stated by the authors, is that “ChatGPT will not replace Urologists however Urologists who use ChatGPT will probably replace those who don’t.”