AI Chatbots Chose Kidney Recipients Differently From People in Penn State Study

A study led by Penn State researchers tested how large language models respond to a stark hypothetical: two people need a kidney transplant, but only one organ is available. The researchers compared chatbot choices with responses from people in earlier studies, using scenarios that varied traits including age, dependents, health, and drinking habits.

The models often diverged from human judgments by placing too much weight on a single attribute. They also tended to choose decisively even when offered a coin flip as a way to express uncertainty. Human participants were more likely to acknowledge that the scenario had no obvious right answer.

Why confidence is part of the problem

The study does not show that hospitals are currently handing transplant decisions to chatbots. The scenarios were designed to study moral reasoning, and the researchers caution against using AI as a substitute for professional judgment. Its value is diagnostic: a system that gives a crisp answer can conceal how it has simplified competing considerations.

That matters as organizations explore AI decision support in settings involving scarce resources. Explanations and model accuracy alone may not be enough. A responsible process would need to show which criteria shaped a recommendation, test those criteria against community and clinical standards, and make room for human review when values conflict.

The research was presented at the 2026 ACM Conference on Fairness, Accountability, and Transparency. Its central lesson is narrow but important: confident language is not evidence that an AI system has resolved an ethical dilemma.

Sources: Penn State’s report on the study; ACM FAccT 2026 paper.