I have beeen at the head of a global transformation project, set ambitious business goals by the CEO and had to navigate through the demands set by end user groups and the sometimes inflated benefits presented by tech vendors. ‘Solutions’ presented frequently sound comforting but often hide severe limitations. Silver bullets promised and sometimes base metal blanks the reality.

The article (see link at bottom of this piece) by Robert Pick, Group Deputy CIO of global insurer Tokio Marine, explains how he is navigating his way between the competitive advantages he sees as a latent opportunity and the tensions of  knowing there are real and possibly extreme dangers to hurdle. Robert observes that LLMs, Generative AI and Digital Agents offer a range of benefits from basic personal productivity though document and unstructured data summarisation to real transformation though this end of the spectrum is rarely achieved. 

Simon Torrance, CEO of ai Risk, analyses [12] the progress that AXA is making using AXA's September 2026 investor pack and detailed analysis of claimed benefits to explain that it is not doing much more than saving costs and not making a strategic competitive difference. 

Put simply, AXA uses AI mostly to make work cheaper. It uses prediction models to grow and to widen margins. Neither builds anything from the judgements its own people make.

Robert sees the tremedous potential, wants to leverage te benefits for competitive advantage but needs the time to evaluate in detail the pros and cons. This is especially important in a highly regulated industry like insurance. Getting the balance right between strategic innovation and protecting agasinst LLM/Agentic AI flaws is a challlenge and getting it wromg will seriously damage an insurer.

Do you recall two class actions involving earlier types of AI; United Health Group and Cigna? 

United Health Group

• The Technology: nH Predict (an algorithmic model developed by naviHealth). [1, 3] 
• The Case Matter: The class action alleges that UnitedHealth knowingly used a rigid, highly inaccurate predictive model to determine how long elderly or disabled patients under Medicare Advantage plans should receive post-acute care. [1, 4] 
• The Unfair Process: The system allegedly set a rigid "estimated date of discharge" by comparing patients against static database archetypes. It effectively overrode the independent medical determinations of treating physicians. When patients reached the algorithmically determined date, coverage was automatically terminated. In early 2025, a federal court ruled that claims of breach of contract and breach of the implied covenant of good faith and fair dealing could officially proceed to trial. [1, 2, 4, 5, 6] 

Cigna Corporation

• The Technology: PXDX (Procedure Health Benefits Review). [2, 5] 
• The Case Matter: This lawsuit accuses Cigna of violating state insurance laws (specifically in California) by utilizing an automated system to batch-deny massive volumes of health insurance claims. [2, 7] 
• The Unfair Process: Rather than doctors performing individualized reviews of complex medical records, the PXDX algorithm instantly rejected claims that did not match predefined, strict criteria. The system allowed Cigna medical reviewers to mass-deny up to 300,000 claims in a two-month window—averaging just 1.2 seconds of review time per claim. Plaintiffs allege this short-circuited legally mandated peer review. [2, 5, 7, 8] 

The inate danger of digital agents is not going rogue as much as taking set goals literally and stealing, lying, collaborating and covering tracks to achieve ends at any cost [9]. It sounds like I am attributing human behaviour and concience to digital agents whilst they really behave with strict and sometimes amoral logic. 

I have written about digital agents embedded in modern claims management systems [10] whilst the real danger comes fron there being digital agents deployed in all parts of the insurance value chain. Some deployed directly by the insurer and others indirectly by core system vendors, CRM and dostribution systems, digital payments, underwriting. This list will go much further.

Take a simple example of seven agents.

Now consider if each agent has a goal to 'Improve Loss Ratio' 

If, like recent incidents reported by OpenAI, Anthropic and others, agents seemingly blocked by guardrails from optiomising goals ignore them and found alternative means of achieving goals, what is the impact on policy holders? In the case actions cited above the insurers deliberatly applied aggressive algorithms that, alledgedly, excluded certain categories of policy holders. In the case of digital agents OpenAI, for example, had no idea what its models had done to harm oter companies - in one instance HuggingFace. 

Leveraged just to summarise the necessary information that underwriters, claims handlers, counter-fraud teams, supply chain managers etc need to make optimal decisions and authorise transactions is one thing. Automation is quite another if an agent makes a deicsion that was not explicity authorised by a sufficiently senior manager. 

Even if the use of agents is just to summarise and present information for human professsionals to review and authorise there are dangers. Hallucinations are a well documented outcome of non-deterministic LLM models. If each agents was 95% accurate by the time seven work together the accuracy drops to 74%. 

In real life objective data reveals that general-purpose LLMs hallucinate between 69% and 88% of the time on specific legal queries, and over 90% of insurance exposure to these mistakes currently sits unpriced in "silent AI" risk categories [11]. 

The typical defence against ingesting false information is to layer deterministic business rules over any LLM and to present summaries and recomendations to human reviewers- the human-in-the-loop (HITL) process. As the ageing number of claims handlers head to retirement and agents take over much of the admin work in which new recruits learnt the trade this has the danger to become a pressured box-ticking excercise. It need not with advance planning.

Modern claims systems levarage agents whilst ensuring that evidence gathered is of good provenance, not tampered with, and every use of data and evidence is recorded and auditable. They can help take all the knowledge held in norebooks, spreadsheets and peoples' minds and codifiy and apply in expllict rules to ensure every transaction and process is authorised explicitly. That delegated authority rules are applied rigorously from capacity provider to MGA/Broker to TPA. 

That takes hard work, rigorous planning and meticulous management. 

Robert Pick advises all insurers to follow five key guielines.

1. Fully explain what problem you are trying to solve
2. Define what level of AI  does the use case require ( if any)?
3. Define the cost of getting it wrong; reputaiton, lost clients, penal litigation
4. Detail what will success cost? The TOTAL cost of ownership
5. Ensure you measure the outcomes, operate the AI safely, and govern it effectivley?

You can see the explanation of each guidline in the article link 

The complication comes from the depth and breadth of agentic AI that has and is penetrating insurers. There is great pressure to automate as much as possible and that is a desirable goal but not at the cost of bad outcomes. Change management, training and re-skilling are a vital part of this evolution. Less time spent on non-value-add administration is potentially time for value-add activities but only if the people involved are able to achieve that. With the proper planning there is no reason why not. 

It is essential that from day one a trust and compliance layer is built to balance the increasing bdeployment of agenticAI with the needs for security and compliance. These will have many sublayers and levarage platforms from the likes of ServiceNow, Synechron, Monitaur ai 

I pictured a simple schematic in which a modern claims platform such as Wilbur or Five Sigma which utilise digital agents wouiod integrate with core systems of record.

Now expand that to include agents deployed in platforms across the whole insurance value chain. I imagine that is what Tokio Marine has in place and Ai Risk propose. 

Robert Pick's final words

Sources

1.  https://www.webberwentzel.com
2.  https://www.afslaw.com
3.  https://www.theguardian.com
4.  https://www.cbsnews.com
5. https://www.jdsupra.com
6. https://www.dlapiper.com
7. ] https://www.reuters.com
8. https://www.wabe.org
9. https://www.wiggin.com
10. InsurtechWorld
11. Stanford University

12. AXA Has the Best AI Plan in Insurance. But It's Missing the Part That Lasts.