Thanks for the Advice. We’ll Still Decide.

There is a rather comfortable model for our relationship with AI: the machine advises, the human decides.

It works beautifully while the adviser is still wrong on a regular basis.

Now make the adviser genuinely good. Good enough that, statistically, it spots a diagnosis better than a doctor, forecasts demand better than a director, evaluates risk better than an engineer and estimates the consequences of a planning decision better than a city planner.

Formally, the button is still in human hands.

But what does “the human decides” actually mean now?

Advice that becomes difficult to refuse

If a machine’s recommendation is right more often than ours, disagreement gradually becomes an event.

While an algorithm is often wrong, a person can say, “I think differently”, and move on. Once the algorithm has demonstrated better results for years, the next question becomes: “On what grounds did you reject the recommendation?”

The advice is still advice. It is simply that every refusal now needs an explanation.

Power can move towards a system without a single law formally transferring that power. It is enough for human judgement to look, more and more often, like a less well-founded departure from the machine norm.

Competence has never been the only source of authority

There is an obvious temptation here: if AI knows better, let it decide.

But society does not work that way even between people.

The country’s best economist does not acquire the right to manage every household budget. A brilliant surgeon cannot force a patient to consent to an operation. The most talented architect does not get to choose the location of a city simply because they understand construction better than the people who will live there.

We distinguish competence from authority. The decision does not always belong to the person best able to predict the outcome.

AI simply makes that distinction painfully visible.

Competence and the right to decide are different things. AI may simply make the difference too obvious to ignore.

Optimal for whom?

Every recommendation has an objective, even when that objective is buried deep inside a formula.

Transport can be optimised for journey time, cost, emissions, safety or neighbourhood convenience. A budget can be optimised for growth, resilience, equality or political acceptability. Treatment can be optimised for life expectancy, quality of life, risk or the preferences of the individual patient.

AI may be extraordinarily good at finding the best route towards a chosen objective. Choosing the objective itself remains a conflict of values.

When two people want different futures, another billion parameters do not turn the disagreement into a mathematical problem.

We can calculate the route with remarkable precision. The satnav still cannot decide why we got into the car in the first place.

Power likes to hide behind objectivity

There is a less comfortable possibility too. A person formally makes the decision, but it becomes convenient to say: “the system recommended it”.

That removes some of the psychological burden and quickly creates a new form of bureaucratic magic. Once it was “that is the procedure”. Then it becomes “that is what the model calculated”.

But someone still chose the data, the objective, the acceptable level of risk and the rules for using the recommendation.

The danger may not be that AI seizes power. It may be much easier for people to use AI as an extremely convincing way of not noticing their own power.

The algorithm becomes not a dictator but an ideal screen: it looks objective, speaks confidently and never turns up at the committee meeting to argue.

The right to be wrong may become a human privilege

There is a strange moral question here: does a person have the right to make a worse decision when the consequences primarily affect that person?

A patient may refuse the best treatment. Someone may choose the less lucrative job. Parents may prefer a school that a model scores lower. Voters may support a decision that is not economically optimal.

We allow this not because mistakes are useful, but because human autonomy includes the right not to maximise a metric every time.

If AI becomes much more competent, human choice will have to be defended with something other than “the person understands better”. Sometimes the person will understand less.

The remaining argument is different: it is their life, their risk, their values — and therefore the decision belongs to them.

We will probably still decide. But we will have to understand why

The phrase “a human must always remain in the loop” sounds sensible today, and reassuring too. It does not answer the important question: why, exactly?

If the human is needed because the machine still makes mistakes, that is a temporary reason. If the human is needed because someone has to sign the document, that is a legal arrangement. If the human is needed because the decision expresses human values and a right to shape one’s own life, that is a different level altogether.

Perhaps the future will force us to separate three things we currently find convenient to mix together: who knows best, who is responsible, and who has the right to choose.

AI may well win the first category.

That is precisely when we will need to decide, very carefully, whether we are prepared to hand it the other two by default.