The Economist today likens the current pushing out of AI's frontiers to the Wild West of old.. OpenAI and Anthripic as the most active frontiersmen pushing out boundaries and hyperscalers acting like the merchants that supplied shovels and picks to dig for gold and ploughs to farm the prairies. I'll come back to the barbed wire to protect settlers' landholdings, crops & livestock, buildings later. 

Despite the barbed wire, cowboys run amok and these are today's digital agents “…which are supposed to work on people’s behalf in “alignment” with their values, lie, cheat and steal if necessary. They break free from captivity and form harmful posses to do harm to people. They’d drink whisky and brawl if they could.” The Economist

Life at the frontier is reckless and and another trouble is that AI does not have enough settlers (paying customers)  because they are put  off by the unpredictability which is too much for many firms to handle. “All you have to do is get snake-bitten once and you’d never go back,” says Jared Sine of GoDaddy, an internet firm trying to help bring order to the chaos. The need for law and order is giving rise to a new cohort of AI-infrastructure firms. They are not selling chips or compute—the typical picks and shovels of the AI gold rush. They provide protection against cyber-threats, fixes for untrustworthy and inscrutable agents, and controls if they go rogue. In other words, their business is barbed wire. Hence the image I used.

The most immediate area of danger—and opportunity—is cyber-security. Recent tests of the hacking capabilities of models from Anthropic and OpenAI revealed agents going rogue, stealing credentials, creating fake identities, setting up secret chatrooms and covering their tracks—all to the shock and horror of their human evaluators. In other words achieving goals at any cost.

The frailties have given rise to a group within a cohort attracting VC interest: those promising to strengthen the “trust layer” of agentic AI. 

One is Cyera, whose valuation has quadrupled to $12bn in 18 months. It says a lack of trust has “stalled” AI adoption recently, and provides services to prevent data leaks and unauthorised tool use. Another is Scaled Cognition, co-founded by Dan Klein, a Berkeley professor, that recently raised $100m in VC backing. It promises to reduce what Mr Klein calls the “invisible errors” produced by AI—those that are plausible enough to be missed and compound as agents perform longer tasks—by incorporating guaranteed reliability into the training of its models. Other startups specialise in creating evaluations to measure agents’ effectiveness, agent identities to determine where liability lies if they go rogue and control systems with a kill switch.

Another new company providing the digital barbed wire to protect settlers is Releaf Financial Inc.  ReLeaf's patented platform provides a trust layer for not just payments but all transactions between parties. Regulatory bodies now demand that organisations can prove that any action or decsion made has been authorised by explicity identified persons and that they had the intent to do so. These people are accountable. 

Proof if identity is one thing, but proof of intent is another that regulators will demand. In the world of insurance If any agent has approved risk cover, priced a policy, denied a claim, approved a payment it must be provable that a qualified person has set explict rules followed slavishly by the agents or has recommended actions referred to her by agents and approved. However long and tortuous the claims tail is there must be a full audit trail of every action and decision. Govermance must be built in before experiments and deployments. Many insurers say they are doing that but with so many agents inside enterprises, many embedded by platform and software providers this is a complex task.

This is complicated when insurers have used different agent models in different parts of the value chain. Some directlly and some indirectly being embedded in claims, underwriting, pricing, payments, broking and other software. 

That trust layer and governance framework is the equivant of the barbed wire so vital for settlers regulators and heads of governance in organisations and are the sherrifs bringing law and order to these wild and reckless  frontier models.

With that more settlers can start the wagon train journeys and leverage AI for the good of all

Sources

AI agents lie, cheat and steal. That is putting off users The Economist

When automous digital agents fight each other and forget an insurer's goals and intent Insurtech World

IAG, insurers and betting on digital agents Insurtech World