What is the danger?
AI is spreading through healthcare — from insurance decisions and hospital triage to chatbots that give health advice. Used well, it can help doctors. Used carelessly, it can deny people care they need, give dangerous advice to vulnerable patients, and make errors that are hard to see and hard to challenge.
How it happens
- Insurers use algorithms to predict how much care a patient “should” need — and cut coverage accordingly.
- Health chatbots answer vulnerable people without the judgement of a trained professional.
- Diagnostic tools trained on narrow data can perform worse for some groups of patients.
- Doctors and patients may trust a confident machine over their own judgement.
Real cases
A 2023 class-action lawsuit alleges that UnitedHealth used an AI tool called nH Predict to cut off care for elderly Medicare Advantage patients, claiming the tool has a 90% error rate. The company says the tool only guides decisions and does not make them.
Source: CBS News ↗In 2023 the US National Eating Disorders Association took its chatbot “Tessa” offline after it gave weight-loss and calorie-counting advice to people seeking help — just as the organisation planned to rely on it instead of its human helpline.
Source: Psychiatrist.com ↗
Warning signs
- A coverage or treatment decision nobody can explain in plain words.
- Health advice from a chatbot that contradicts your doctor.
What you can do
- Ask whether an algorithm was involved in a decision about your care — and appeal denials.
- Use AI health tools for general information only; decisions belong with qualified professionals.
- Support rules that require human review of automated medical and insurance decisions.
Frequently asked questions
What is the danger of health in AI?
AI is spreading through healthcare — from insurance decisions and hospital triage to chatbots that give health advice. Used well, it can help doctors. Used carelessly, it can deny people care they need, give dangerous advice to vulnerable patients, and make errors that are hard to see and hard to challenge.
How does it happen?
Insurers use algorithms to predict how much care a patient “should” need — and cut coverage accordingly. Health chatbots answer vulnerable people without the judgement of a trained professional. Diagnostic tools trained on narrow data can perform worse for some groups of patients. Doctors and patients may trust a confident machine over their own judgement.
How can I protect myself?
Ask whether an algorithm was involved in a decision about your care — and appeal denials. Use AI health tools for general information only; decisions belong with qualified professionals. Support rules that require human review of automated medical and insurance decisions.
Latest updates on health
No updates in this category yet — check back tomorrow.
What do you think?
Seen this danger up close? Worried about it? Share it with the community.
Share your thoughts on health