We have alien minds among us. Not the little green men of science fiction, but alien minds that add facial recognition to your smartphone, determine your credit worthiness, and write poetry and computer code. These alien minds are artificial intelligence systems, the machine ghosts you encounter every day.
But AI systems have a significant limitation: many of their internal workings are impenetrable, making them fundamentally imprecise and unpredictable. Moreover, creating AI systems that behave as people expect them to is a significant challenge.
If you don’t fundamentally understand something as unpredictable as AI, how can you trust it?
Why AI is unpredictable
Trust is based on guesswork. It depends on your ability to predict the behavior of others. If you trust someone and they don’t do what you expect, your perception of their trustworthiness diminishes.
Wiso via Wikimedia Commons
Many AI systems are built on deep learning neural networks, which in some ways mimic the human brain. These networks consist of interconnected “neurons” with variables or “parameters” that affect the strength of connections between neurons. As a simple network is presented with training data, it “learns” how to classify the data by adjusting these parameters. In this way, the AI ​​system learns to classify data it has never seen before. It does not remember what each data point is, but instead predicts what the data point might be.
Most powerful AI systems have trillions of parameters. Because of this, the reasons why AI systems make decisions are often opaque. This is the problem of AI interpretation – the impenetrable black box of AI decision making.
Consider a variation of the “trolley problem”. Imagine you are a passenger in a self-driving vehicle controlled by AI. A child is running down the road, and the AI ​​must now decide: run over the child or swerve and crash, potentially injuring passengers. It is difficult for a human to make this choice, but the human has the advantage of being able to explain his decision. Their rationalizations—moral norms, perceptions of others, and expected behavior—support beliefs.
In contrast, AI cannot rationalize its decision-making. You can’t look at the trillion parameters of a self-driving vehicle to explain why you made this decision. AI meets the predictive requirement for trust.
AI behavior and human expectations
Trust depends not only on predictability, but also on common or moral motivations. You generally expect people to behave as they should, not as you assume. Human values ​​are influenced by common experience, and moral reasoning is a dynamic process, shaped by moral standards and perceptions of others.
Unlike humans, AI does not adjust its behavior based on how its behavior is perceived by others or by following moral rules. An AI’s internal representation of the world is largely static, set by its training data. His decision-making process is based on an immutable model of the world, not at all surprised by the dynamic, subtle social interactions that constantly influence human behavior. Researchers are working on programming AI to incorporate ethics, but that remains challenging.
The situation with self-driving cars illustrates this problem. How can you ensure that a car’s AI makes decisions that match human expectations? For example, a car may decide that hitting a child is the optimal course of action, something most human drivers would instinctively avoid. This problem is the AI ​​alignment problem and is another source of uncertainty that creates trust barriers.
Trust critical systems and AI
One way to reduce uncertainty and increase trust is to ensure that people trust the decisions AI systems make. This is the approach taken by the US Department of Defense, which requires that for all AI decision-making, a human be either in the loop or on the loop. In the loop means the AI ​​system makes recommendations but requires a human to initiate the action. On the loop means that an AI system can initiate an action on its own, while a human monitor can interrupt or change it.
While engaging humans is a great first step, I don’t believe it will last long. As companies and governments continue to adopt AI, the future will involve nested AI systems, where rapid decision-making limits opportunities for human intervention. It is important to resolve clarification and alignment issues before reaching a critical point where human intervention is impossible. In that case, there will be no option but to trust AI.
Avoiding that threshold is especially important as AI becomes increasingly integrated into critical systems, including things like electric grids, the Internet, and military systems. In critical systems, trust is paramount and undesirable behavior can have dire consequences. As AI integration becomes more complex, addressing issues that limit reliability becomes more important.
Can people ever trust AI?
AI is alien – an intelligent system that humans have little insight into. Humans are largely predictable to other humans because we share similar human experiences, but this does not extend to artificial intelligence, even if humans created it.
If trustworthiness has inherently predictable and normative elements, then AI fundamentally lacks the qualities that would merit trust. Hopefully, more research in this area will shed light on this issue, ensuring that future AI systems are worthy of our trust.