More and more people are turning to Dr AI for advice
Dr AI has become a colleague of Dr Google. Researchers see both benefits and risks in this development.
Text: Annika Lund, first published in Medicinsk Vetenskap nr 2 2026
According to the report ‘The Swedes and the Internet’, published in September 2025, four in ten Swedes use a public AI tool. ChatGPT is by far the most common, followed by tools such as Claude, Copilot, Gemini, Grok and Perplexity. One in five AI users say they have asked questions about their health. Young adults in particular are using AI as a ‘doctor’.
“This development cannot be reversed. A pragmatic approach is to accept it and spread knowledge about how to ask health-related questions safely, as well as how healthcare professionals can make use of the large language models,” says speech and language therapist Fredrik Sand Aronsson, a doctoral student at the Department of Clinical Science, Intervention and Technology – CLINTEC, at Karolinska Institutet.
There are both risks and benefits
He has attempted to apply the concept of harm reduction in this context. The term originates from addiction medicine and involves reducing risks, for example by providing access to clean syringes, rather than pursuing a zero-tolerance approach. Using this framework, Fredrik Sand Aronsson has examined the risks and benefits of using large language models for medical and health-related questions.
“We invite others to share their views on our article and engage in a wider discussion,” says Fredrik Sand Aronsson.
Among the benefits mentioned is that AI systems are always available. In the report ‘The Swedes and the Internet’, nearly one in four people report they have refrained from booking a doctor’s appointment due to difficulties with the healthcare system’s digital booking system.
If chatbots are used as an accessible way to obtain general knowledge, then AI has made a positive contribution, argues Fredrik Sand Aronsson. Such queries might include learning more about an illness, having terms and concepts explained, or preparing for a doctor’s appointment.
Can worsen mental health problems
However, it is risky to ask AI to make diagnoses or suggest treatments. There are cases in which interactions with AI have exacerbated mental health problems in people in crisis. In a widely reported case from Belgium, a man took his own life after chatting with an AI, which is said to have reinforced his thoughts rather than urging him to seek emergency care. There are more similar cases.
“With rare conditions, such as psychosis or acute suicide risk, the likelihood increases that chatbots will not respond appropriately. Given that they have millions of users, this is a serious matter,” says Julian Striegl, who researches how AI systems can be used in mental health at, among other places, the Department of Clinical Neuroscience at Karolinska Institutet.
The explanation lies in the architecture of large language models. The algorithms consistently provide the most “probable” answer, that is, the one that occurs most frequently in the texts they have been trained on. When a situation is more unusual in the training data, there is an increased risk that the answers will be incorrect or that the model will fill in with fabricated information.
A medically safe chatbot is being tested
One attempt to create a medically safe chatbot is Sundi. All responses are based on studies selected by medical experts. A question such as “What is HbA1C?” provides an explanation and a follow-up question: “Would you like to know more about why it’s important?” A ‘yes’ yields an easy-to-understand answer, which concludes with the question of whether you would like tips on how to keep your blood sugar stable. Another ‘yes’ provides advice on diet and exercise, as well as an offer of a prescription.
And so it continues. Sundi responds in a personal tone, prompting the user on how they would like to proceed, just as ChatGPT or other major chatbots do. The difference is that Sundi’s answers are evidence-based and there is an editorial team in the background responsible for ensuring the lifestyle advice is accurate. Questions about personal diagnoses or treatments are referred to healthcare providers or advisory services, such as 1177.
One of the features offers support for setting personal goals for sleep, diet, exercise, stress and relationships. Sundi asks questions to gauge personal motivation and what is perceived as achievable. This leads to a plan with realistic milestones, in line with what research shows about how lifestyle changes can be achieved.
“As a first step, we want to investigate what users think of the platform. We want it to be clear and engaging. The next step is to see whether it actually helps people, for example, to sleep better or change their eating habits. The aim is to make it possible to get this kind of help via Sundi, but we want to know that it works before we claim it does,” says Carl Johan Sundberg, professor at the Department of Physiology and Pharmacology at Karolinska Institutet.
Footnote: Sundi is led by the German foundation Bosch Health Campus, in collaboration with researchers from institutions including the Karolinska Institutet.