In artificial intelligence, ‘trust’ is a tricky but crucial topic

Artificial Intelligence (AI) is everywhere, and we interact with it more often than we care to think on. Social media such as X and Facebook, online services such as Google Maps, the camera apps in our smart phones and something as ‘simple’ as predictive text all deploy AI to help us decide what films to watch, what food to eat, how to spend our money and the best route to take.

While we often marvel or laugh at the decisions and recommendations made for us by AI, very few users understand how these decisions are arrived at. Even developers might find themselves unable to analyse statistical calculations that are no longer humanly possible to track.

Do you ‘trust’?

When people work together, we understand the concept of ‘trust’, and what it means to ‘trust’ our colleagues. Working with Artificial Intelligence presents a very different set of issues when thinking about ’trust’.

Trust is defined as a strong belief in the reliability, honesty or ability of a person, group or institution.  It is a psychological feeling one party has for another, usually the vulnerable side extending trust to an individual who wishes to be trusted. The word has a slightly different meaning when applied to machines. The growing prevalence of Artificial Intelligence does underline issues around trust, transparency and the tolerance of errors. A team group of humans cannot carry out tasks if they do not trust each other. Similarly, collaboration within a Human AI (HAI) Team pivots around trust in algorithmic systems. Just as in human society, if trust between persons and an algorithmic system is broken, repairing it is critical to the coherence of the HAI Team.

The topic encompasses more than whether an individual ‘trusts’ (has faith in the reliability of) AI. In this context trust can form a spectrum, from ‘algorithm opposition’ (outright distrust or aversion) to loafing (extreme complacency or bias toward automation). The ideal median would be algorithmic vigilance.

With humanity’s natural mistrust of anything it doesn’t understand, transparency is necessary to grow trust in AI and its results.

Explaining explanations

On its own, explainability may not be enough to create the ideal mid-point in the human trust of AI. Rather than simply explaining how an algorithm completes a task and produces results, there are other techniques. Analytics to measure performance and confidence levels in AI, alongside dynamic task allocation, could be combined to produce a suitable level of trust in artificial intelligence.

The danger of explanation is that it produces automation complacency and even automation bias in favour of everything the AI does. Among the most likely to induce this state is ‘feature importance explanation’, which measure what features most greatly affect a model’s outputs.

Providing explanations before a user has experienced the decisions of an AI algorithm can build a tendency to fixate on the first data the AI produces, which can affect how they respond to future results from the AI.

If an explanation goes too well, it can produce user trust in the AI’s results even when it is making a mistake. Similarly, poor / confusing explanations can produce AI algorithm aversion in users.

What we see is that explanation must strike a fine balance to achieve its aims.

The power of control

The degree to which humans can trust AI may depend on the degree of control a user has, or feels they have, over the AI itself. It has been found that users will accept the forecasts of an AI if they can alter or adjust those outputs. The user is willing to accept the AI’s work, even in the face of the occasional mistake.

One way of creating a sense of control is by using prompts to move the user to request more information, should it be required. Tasks where control moves flexibly between AI and User are better at fostering a sense of algorithm vigilance in the User.

It’s all difficulty

Another key is difficulty, and one outcome might be creating vigilance in users.

Inevitably humans and machines differ in what they find ‘difficult’.  Most human trust in AI is to be found in tasks involving objective calculation, even in the face of a machine’s mistakes. That trust decreases when tasks involve social and emotional skills.

Some models of dynamic task allocation would allow humans to decide what tasks to reserve for themselves and what to leave for AI. Other models would allow the machine itself to divide the labour and reserve for itself the most difficult of tasks, passing on moderately or easy tasks to humans. This can help make human users of AI more vigilant.

Trust – variations on a theme of AI

The question of human trust in AI depends on which AI a person is supposed to trust.  It has been found that human trust in robotic AI develops in a manner similar to trust between humans. Indeed, this trust increases after direct engagement.

It has been found that people who have driven a semi-autonomous vehicle have greater trust in the abilities of the AI than people who lack this experience.  Similarly, trust in a robotic pet increased after interaction between children / adults and the pet.

Researchers have found users place a higher level of trust in robotic AI when they can make adjustments to the AI output and control those adjustments by pushing a button.  They found their experience positive to the extent they reported higher trust in potential future robots.

In contrast, it has been found that trust in virtual AI (ie artificial intelligence that lacks a physical presence, but is represented by an avatar or a chatbot) decreases from high initial hopes after interaction between user and bot.  Initially, the trust in a bot or avatar is higher than in human advice.  Over time, this trust declines more than trust in a human advisor, so much so that use of bots on commercial websites decreased over the years, to the point where they became virtually redundant.

Tellingly, this is an issue unique to virtual agents, because the difference between the assumed performance and the actual performance of the bot creates frustration leading to abandonment. The problem appears to surround human-like representation of virtual AIs, which create high expectations of performance that the technology cannot meet.

AI bots can be mistaken, something that can be costly for customers and organisations that set them up and use them.Similarly, users’ trust in embedded AI (where the AI is active within an app and the user may not be aware of its existence) results from its reliability and transparency.  Similar to responses to robotic AI, perceptions of expertise and machine intelligence drive a user’s trust.  The type of task also plays a role in trust, with algorithms being seen as better at calculations than social tasks.

It appears that people set a higher standard for trusting AI than for people or other technology.  One misstep, or even not living up to overly optimistic expectations, leads to people abandoning AI. What Ella Glikson and Anita Williams Woolley have found is that trust appears to arise when users find AI useful and relatively reliable (Google Maps, for instance). Perhaps widespread acceptance of Artificial Intelligence is a matter of time and improvements in the technology, which make it more useful and (importantly) more reliable.

Humanity has a preponderance to distrust anything it does not understand. How do you think we can overcome this to make artificial intelligence a more widespread, trusted technology than it already is? What are the ethics of ‘pushing’ or ‘nudging’ people toward accepting a new technology such as AI?