The Ethics of Medical AI — Fairness, Explainability, Consent— Bias, accountability, informed consent, and the distribution of responsibility

01The Problem of Bias — Skew the Data, and the AI Skews Too

An AI's judgments are built from the data it was trained on. If that data is skewed toward a particular group, the skew shows up directly in the AI's output. This is not a flaw in the algorithm (= the computational procedure); it is the plain fact that the composition of the data you feed in is reflected straight into the result.

Several kinds of bias tend to cause trouble in medicine.

A well-known example: an algorithm widely used in the United States to allocate medical resources was found, in a 2019 report in Science, to be rating a particular racial group as "less in need" even at the same level of illness. The cause was that it used "past medical spending" as a proxy for health status. The group whose access to care had been restricted spent less, and as a result was wrongly assessed as "less in need." Even when the designers had no intent to discriminate, a single choice of metric skews the outcome.

The same problem arises in the pharmaceutical context. When AI is used to narrow the target patients for a therapy, a training set skewed toward a particular group will drop patients who should have qualified. This is a point to check constantly — in clinical trial subject selection and in post-market real-world data analysis alike.

02Explainability — Can You Say "Why It Judged That"?

Many high-performing AIs are built so that their internal reasoning is hard for humans to follow. This is called the "black box problem." However accurate the output, if you cannot explain why it came out that way, the physician cannot give the patient a reason and cannot take responsibility for the judgment.

Explainability (= Explainability, showing the basis of an AI's judgment in a form humans can understand) comes at two levels. They are easily confused, so we separate them.

What medicine really needs, in most cases, is the local explanation. Not "in general it works like this," but "regarding your test results, the AI focused here and judged this way" — without that, dialogue with the patient does not hold together. A caution is in order, though. The "apparent basis" an AI produces is not necessarily its true reason for judging. Sometimes a plausible-looking explanation is simply generated after the fact, so the correctness of the explanation itself has to be examined.

03Informed Consent — How to Convey the Involvement of AI

Informed consent (= consent given after receiving an explanation) is a pillar of medical ethics. It is the procedure by which a patient understands the care they will receive and chooses it willingly. Once AI participates in care, new questions are added to what that consent must contain.

The points that have to be decided in practice are these.

• Whether to disclose at all — How far do you explain to the patient that AI was used to support the diagnosis? Should it be treated like not explaining the blood-test analyzer, or, if the degree of involvement in the judgment is high, should it be conveyed separately?
• What to convey — What scope does the AI handle, and who makes the final decision? How do you communicate the limits of accuracy and the possibility of error?
• Secondary use of data — Might this patient's clinical data be used to train or improve the AI? If so, does that not require separate consent?
• The right to refuse — When a patient says "please don't use AI," can you offer an alternative?

What to avoid here is the formalism of settling the matter with one line in the consent form: "AI may be used." Consent is not obtaining a signature on a document; it is creating a state in which the patient understands and can choose. If the information conveyed is so voluminous that it cannot be understood, that is not consent. Reconciling accuracy with clarity is the hard part of designing consent in the age of AI.

04Distributing Responsibility — In Care Involving AI, Who Bears It?

When a patient is harmed in care that used AI, the locus of responsibility becomes the question, because several actors are involved and each plays a different role.

Under the current general view, the principle holds that the final judgment is made by a human, and accountability for that judgment is also borne by a human. AI is a tool that supports judgment, not a subject of responsibility. But as AI performance rises and humans come, in effect, to merely rubber-stamp the AI's output, there is a danger that the premise "a human judged" becomes an empty formality. Keeping responsibility with humans requires a design in which humans stay meaningfully involved in the judgment. Internationally, this is discussed under the term "meaningful human control" (= meaningful human control).

05Designing for Fairness — "Fix It Later" Is Too Late for Bias

Bias is not something you can catch by inspecting a finished AI after the fact. Unless you build checkpoints into every stage of development, the skew reaches the field unseen. Fairness (= Fairness) is something to design for from the entrance of development through to its exit.

• The data-gathering stage — Is the diversity of the target patient population reflected in the training data? Confirm in numbers whether any group is underrepresented.
• The building stage — Measure accuracy separately by group. Even at 95% overall, if one group sits at 80%, that AI is not fair.
• The use stage — After deployment, keep monitoring group-by-group performance and watch continuously for degradation or new bias.

There is one point here you cannot escape. "Fairness" does not have a single definition. Equalizing accuracy across all groups and equalizing miss rates across all groups can be mathematically impossible to satisfy at the same time. In other words, what counts as fair is not a technical matter but a value judgment, and it cannot be decided by developers alone. It is a question that healthcare professionals, patients, and regulators should be involved in deciding. What is needed is the disposition to ask which definition of fairness the words "we made it fair" actually refer to.

06The Patient's Perspective — AI as Seen by the "Used-Upon" Side

Ethical discussion tends to take the developer's or the physician's point of view. But the one most directly affected by medical AI is the patient. Seen from the patient's side, there are a few plain wishes.

• To be treated as an individual — Not a statistical average, but my own specific condition, seen. The unease of an AI settling things with "patients like you turned out this way."
• To know the reason — To hear, in words that convince, why this diagnosis or treatment. "Because the AI said so" does not convince.
• To have room to choose — To keep the sense that I can choose for myself, including whether to use AI or not.
• To know where the data goes — To know where my clinical data goes and what it is used for.

Even if introducing AI raises the efficiency of care, if the patient feels "processed on a machine assembly line," trust in medicine falls. Medicine is a technology and, at the same time, a relationship between people. If AI taking over clerical and analytical work gives physicians more time to face the patient, that is an ethically desirable direction. Conversely, if the time AI frees up is merely rerouted into other efficiencies, the meaning for the patient thins out. Whom the time freed by AI is used for — this too is an ethical question.

07Norms — The Principles and Guidance to Stand On

The ethics of medical AI need not be thought up from scratch. Medical ethics has a long accumulation, and principles applicable to AI have already been organized. First, take the four classical principles of bioethics as the foundation.

On top of these four principles, AI-specific guidance is layered. Here are the main references usable in practice.

• The World Medical Association (WMA) statement — The "WMA Statement on Augmented Intelligence in Medical Care" confirms that final judgment and responsibility to the patient rest with the physician, and that AI supports rather than replaces the physician.
• The World Health Organization (WHO) guidance — "Ethics and Governance of Artificial Intelligence for Health" (2021) sets out principles including protecting autonomy, human oversight, transparency, accountability, and inclusiveness and equity.
• Japan's guidance — The Cabinet Office's "Social Principles of Human-Centric AI" (2019) and the "AI Guidelines for Business" from the Ministry of Internal Affairs and Communications and METI (2024) organize the thinking for each role in development, provision, and use.
• The Japan Medical Association's "Principles of Medical Ethics" — The pre-AI foundation of physicians' professional ethics; the stance of putting the patient's benefit and trust first does not change when AI is used.

These guidelines do not contradict one another. Different in wording, they share a common core: "keep human judgment," "avoid bias," "maintain transparency," "make responsibility clear." Individual technologies change; this core does not move.

08Connections to Other Chapters on This Site

The ethics of medical AI is not a self-contained topic. Reading it alongside other chapters on this site sharpens the resolution of your judgment.

• The installments of the AI Medical series — Diagnostic support, drug discovery, patient communication, and more; in each application area you can see, in concrete situations, how the four principles of this installment play out.
• The advertising-regulation chapters — When AI-generated information is delivered to healthcare professionals or patients, the advertising regulations of the Pharmaceuticals and Medical Devices Act cannot be set aside (exaggerated advertising under Article 66, advertising of unapproved drugs under Article 68, information provision for proper use under Article 68-2). The accuracy and evidence of the wording are guaranteed by humans, even when AI is used.
• The material-review chapters — Materials made with AI do not change the discipline of review. Being able to produce fast and confirming correctly are separate steps.
• The chapters on social trust and governance — Medicine is a "trust good." A single ethical failure damages trust built up over a long time. Reconciling AI's efficiency with trust becomes a management-level question.

The ethics of medical AI is not a binary of "use AI or don't." It is the work of designing, on the premise of using it, how to use it so that the patient's benefit and trust are protected. Check bias from the entrance of development; prepare explainability in a form that can be spoken about the patient in front of you; obtain consent as understanding rather than paperwork; and distribute responsibility so that the final judgment stays with a human. Decide the definition of fairness among the stakeholders as a value judgment, and place the patient's perspective at the center of the design. None of this is newly invented ethics; it is a restatement, fitted to the new tool of AI, of what medicine has always held dear.

Seen from a pharmaceutical vantage point, the ethics of AI is not a cost of regulatory compliance but an investment that builds trust. Next time, we shift the focus to the very actor who puts this ethics into practice in the field — the physician. We will describe what the power to wield AI as a tool — AI literacy — will demand of the physicians to come.