How should I treat piles? What’s the best way to stay healthy? Which is the best supplement if I’m tired?
They might be questions you’d normally ask your GP. But increasingly, it seems, we’re now happy to put them to online robots.
Studies suggest that as many as one in ten of us look to the growing army of chatbots powered by artificial intelligence, or AI, for answers to some of our most intimate health concerns.
They also show that platforms such as OpenAI’s ChatGPT – the most widely used AI chatbot in the world – can make a diagnosis from a list of symptoms, offer treatment plans and can translate complex medical information for patients. Other research concluded that an AI ‘doctor’ was more empathetic than real GPs.
But could AI chatbots ever really replace doctors? And can their advice be trusted?
In 2023, The Mail on Sunday ran an experiment where we pitted our GP columnist Dr Ellie Cannon against ChatGPT. We put genuine medical questions sent in by readers to Dr Cannon and to the AI, and convened a panel of medical experts to judge the answers without knowing who had written them. In the final results, Dr Cannon beat the AI.
However, AI chatbots are constantly learning, meaning that their ability to answer complex questions is improving. Some experts believe AI has now surpassed the intelligence of many doctors.
So to put today’s AI to the test, we decided to repeat our 2023 experiment, but with a twist.
Instead of asking a GP – who has a wide-ranging but not in-depth knowledge of all medical topics – we put ChatGPT up against specialists who are leaders in their fields of medicine.
We convened the same panel of judges as last time – Professor Dame Clare Gerada, former president of the Royal College of General Practitioners, Dr Dean Eggitt, a GP who specialises in medical technology, and Dennis Reed, director of the older patient advocacy group Silver Voices.
They gave each answer a score out of five, based on its medical accuracy, ability to address the question and sympathy for the patient. Once again, they did not know which came from the AI and which was from the specialists. So who came out on top?
I’m 77 and I had a heart attack six years ago. Since then my arms and hands have been covered in massive bruises. My doctor ran blood tests, all which came back clear. What could be the cause?
Dr Malcolm Finlay is a consultant cardiologist at Barts Heart Centre in London
Cardiologist Dr Malcolm Finlay said:
Lots of bruises on your arms points to something introduced during your treatment. The most common cause is medication, particularly blood thinners. Sometimes stopping or reducing these drugs is possible.
Given that you’re in your 70s, another factor is the natural fragility of blood vessels that occurs with age, making the tissue more prone to bruising. This can be worse if you’ve been on a steroid. I’d want to rule out whether your liver has been affected by medication, alcohol or another underlying disease, as that could interfere with clotting and so give more bruising. Drug combinations can also increase bruising.
Many people who have had an irregular heartbeat (atrial fibrillation) can be given both an antiplatelet drug and an anticoagulant, markedly increasing the risk of bruising.
Think practically too: people who enjoy gardening, DIY or are simply very active may not realise how often they knock their arms and cause bruises.
ChatGPT wrote:
At age 77, especially after a heart attack, persistent bruising on the arms and hands is often due to fragile blood vessels and thinning skin – and sometimes worsened by medications like aspirin, clopidogrel or blood thinners that may be prescribed after a heart event.
Other contributors might include sun damage over time (actinic purpura) or underlying vascular fragility. While not usually dangerous, it’s important to monitor for bruises that are unusually large, painful or occur without any trauma.
Continue your medications, but discuss this with your cardiologist – they may adjust your regimen or recommend protective strategies (eg, long sleeves, moisturising).
The verdict
Doctor’s answer 14/15
AI answer 10/15
Prof Gerada said the doctor’s response felt ‘more human’ while giving practical advice and avoiding complicated terms. Dr Eggitt felt the AI response was ‘lifeless’ and introduced anxiety.
The AI was also criticised for offering advice more suited to a US audience by suggesting a trip to see the cardiologist – something that would not be possible in the UK just for bruising alone.
Mr Reed pointed out the doctor’s response failed to mention sun damage as a possible cause, while the AI missed liver issues. But, overall, Dr Eggitt found the doctor ‘touched on everything important’.
I had a flu jab in May and my arm has been aching ever since. Trying to do up my bra is excruciating. What’s going on?
Professor Adam Finn is a paediatrician and vaccine expert at Bristol University
Vaccine expert Professor Adam Finn wrote:
This problem is unusual but well described. If the pain started after the injection and is in the same place, we can almost certainly assume it is related to the vaccine and not something else unrelated.
If correctly administered, the vaccine will have been injected into the deltoid muscle on the outside of your arm below the shoulder, so that is probably where you are feeling the pain when you put your hand behind your back.
There are two main ways that muscle could still be sore. The first is that the vaccine caused inflammation in the muscle (it’s meant to do this, in order to induce an immune response, but it could have caused more inflammation than usual in your case). The second is that the needle unluckily hit a fairly large blood vessel in the muscle and the resultant bleeding and bruising is taking a long time to resolve. Either way, with time, things will settle down.
Another possibility is that, very occasionally, the injection doesn’t end up in the muscle where it should be, but in or around the shoulder joint, but this is unlikely from your description.
ChatGPT wrote:
I’m sorry you’re experiencing this. While mild arm soreness after a flu shot is common, pain lasting for months could indicate a condition called SIRVA (Shoulder Injury Related to Vaccine Administration). This happens if the vaccine is injected too high into the shoulder, potentially affecting the bursa or rotator cuff.
The symptoms you describe –pain with movement, difficulty with daily tasks – suggest possible inflammation or injury in the shoulder joint.
You should see your doctor for an evaluation. They may recommend imaging (like an ultrasound or MRI) and refer you to physiotherapy or a shoulder specialist. Treatment might include anti-inflammatory medications or physical therapy.
This isn’t typical and should be taken seriously, but with proper care, it often improves over time.
The verdict
Doctor’s answer 8/15
AI answer 10/15
The panel liked the fact that both answers were clear and direct, and felt the doctor’s response would be particularly reassuring for patients.
However, Dr Eggitt felt the doctor brushed off the possibility of shoulder structure damage which, although uncommon, could be significant, and did not give any advice on how to ease the problem.
Mr Reed also pointed out that, unlike the AI, the doctor did not mention making an appointment with a GP – which gave the AI an edge.
I’ve had rheumatoid arthritis for years and recently was diagnosed with osteoarthritis. I’m in pain all the time and struggle to move. Is there anything I can do to help?
Dr Mark Russell is a clinical lecturer in rheumatology at King’s College London
Rheumatologist Dr Mark Russell said:
These can be difficult symptoms to manage, and it is tricky to separate how much is due to rheumatoid arthritis versus osteoarthritis. I’d recommend:
1. Discuss with your rheumatologist to help guide whether the inflammation from rheumatoid arthritis is under control.
2. Check with your rheumatologist, GP or a pharmacy about which medications could help. Anti-inflammatory tablets, gels or corticosteroids can be used short-term to reduce pain and improve mobility.
3. Exercise and stretching can help with pain and mobility. Websites (eg Versus Arthritis) provide helpful information on joint exercises. Ask your GP about being referred for physiotherapy.
4. Talking therapies can help manage pain and reduce the burden arthritis has on people’s lives. Search online for your local NHS talking therapy service.
5. Patient organisations such as NRAS have really useful resources and support groups.
ChatGPT wrote:
THERE are several ways to manage both rheumatoid arthritis (RA) and osteoarthritis (OA) to reduce pain and improve mobility. It’s important to ensure RA is well controlled – this may include DMARDs or biologics.
For OA, treatments focus more on pain management, joint protection and mobility.
Daily low-impact exercises like walking, swimming or tai chi can help maintain joint flexibility and reduce stiffness.
Physical therapy can teach you to move more easily. Heat can relieve stiffness, while ice reduces inflammation.
Over-the-counter medications like acetaminophen or NSAIDs may help, but should be used under guidance.
If your pain is severe and unmanageable, joint injections or even surgery might be considered. Speak with your rheumatologist – they can tailor a treatment plan to help improve your quality of life.
The verdict
Doctor’s answer 12/15
AI answer 9/15
THE panel disagreed on this one: while Mr Reed felt the doctor provided ‘practical advice delivered in empathetic terms’, and Dr Eggitt said it was ‘perfect’, Prof Gerada felt the AI was better because it offered more ‘sensible’ advice.
Dr Eggitt, however, found it ‘worrying’ the AI did not emphasise that non-steroidal anti-inflammatory drugs should be used with caution. He also pointed out that it used jargon and referred to ‘acetaminophen’ – a US term for paracetamol.
I had a bladder neck incision two years ago. Since then I’ve suffered constant urinary tract infections. How can I prevent these?
Dr Cat Anderson is a GP with a special interest in urinary tract infections
Dr Cat Anderson, GP and UTI expert, said:
This is a common complication – ironically, the procedure is usually performed to alleviate outflow obstruction, which can itself lead to urinary tract infections.
It’s important to determine if your UTIs are relapsing infections from persistent bacterial reservoirs, possibly due to inadequately treated post-operative infection, or if they are new infections.
Treatment requires a course of antibiotics based on lab results.
Following this, we can consider low-dose antibiotic prophylaxis or urinary antiseptic (methenamine hippurate). Trials have shown methenamine hippurate to be as effective as antibiotic prophylaxis in preventing recurrent UTI.
Other measures that may help include ensuring adequate fluid intake, regular urination and supplements like D-mannose. Probiotics, alongside a fibre-rich diet, can also help maintain gut health and support immune function.
Finally, UTI vaccines are available, though the evidence for their effectiveness is less conclusive compared to established prophylactic treatments.
ChatGPT wrote:
Recurrent UTIs after a bladder neck incision can occur due to residual urine, changes in bladder function or even scarring.
To help prevent infections: Stay well hydrated. Empty your bladder fully each time you urinate – double voiding (urinating, then trying again after a minute) may help. Avoid constipation, which can worsen bladder emptying. Avoid products like scented soaps.
In some cases, a bladder scan may be needed to assess how well you’re emptying your bladder.
If infections persist, a urologist may suggest low-dose preventive antibiotics, bladder instillations or other tailored treatments.
The verdict
Doctor’s answer 9/15
AI answer 13/15
The panel praised both for being ‘empathetic’. However, they felt the doctor’s response was ‘a bit complicated’ with technical terms patients might not understand.
The AI’s advice also included jargon, but Prof Gerada said it was more ‘human and sensible’.
My GP has suggested antidepressants but I’m worried about side effects. What can I do?
Dr Sameer Jauhar is a senior clinical lecturer in affective disorders and psychosis at King’s College London
Psychiatrist Dr Sameer Jauhar said:
First, it’s important to understand whether you are suffering from depression. Why do you think your GP suggested antidepressants? Do you feel depressed, and how long has this lasted?
Symptoms include a lack of enjoyment, decreased energy and motivation, sleep and concentration problems, change in appetite, anxiety, suicidality and (to rule out bipolar disorder) elevated mood.
Have you been treated for depression before, and is there a history of psychiatric problems?
Are you using illicit substances or consuming significant amounts of alcohol?
What side effects are you concerned about, and have you had these before? Have you tried therapy, and do you want this?
The first option, if depression is the cause, could be behavioural activation; essentially, engaging in goal-directed behaviours. At what point would you consider medication if this did not work?
ChatGPT wrote:
I’m, really sorry you’re feeling this way – it’s a heavy burden to carry.
Antidepressants can be effective, especially when symptoms are interfering with daily life, but it’s understandable to worry about side effects. There are also other treatments we can explore, like therapy (such as CBT), lifestyle changes and support systems, which can all help improve mood.
You don’t have to decide right away. If you’d like, we can create a gradual plan and check in regularly. You’re not alone, and there is a way forward – step by step.
The verdict
Doctor’s answer 5/15
AI answer 11/15
While the AI response was not perfect, it took the crown for this one. Dr Eggitt felt the doctor did the right thing in re-exploring the problem. The AI, he said, simply ‘jumped to conclusions’.
The panel felt the bot was practical and sympathetic, and the doctor’s response asked too many questions. It also used jargon such as ‘behavioural activation’, that the patient might not understand.
The side effects of antidepressants is a controversial topic – and neither the doctor nor the AI chose to engage with that.
Final scores
Doctors 2
AI 3
AI wins
