That was something I heard a few times during conversations about AI at the Consumer Duty Alliance conference, and on the face of it, it makes complete sense. The adviser knows the client, they’re responsible for the advice and ultimately they’re the person signing it off.

But it did get me thinking about what we actually mean when we say the adviser has “checked” something. Are they checking that the report reads well, that the figures are correct and that the client’s circumstances have been captured accurately? Or are they stepping back from the document and checking that the recommendation itself is suitable? Because those are slightly different things.

The FCA’s recent publication, “AI in financial services: good practice and areas for improvement” gives some useful context around this. Rather than creating a new set of AI rules, it looks at how existing regulatory expectations apply when firms are using AI and shares examples of good practice alongside areas where firms could improve.  

For me, one of the most interesting areas was human oversight. ‍

‍

A human in the loop isn’t the same as oversight

Having a human involved in the process is obviously important, particularly where AI is being used anywhere around the advice process. But simply having an adviser at the end of the process doesn’t necessarily mean that meaningful oversight has happened.

And actually, I think it’s quite easy to see how that could happen in practice!

An adviser opens a suitability report that is well structured, clearly written and sounds reassuringly like the sort of report they’re used to reading. The client details are right, the recommendation is there, the figures look sensible and all the usual sections have been included. They make a few amendments, tidy up some wording and it’s ready to go.  

‍

‍Convincing isn’t the same as correct

The problem is that a report can look completely convincing and still be wrong.

The client’s objective might have changed slightly between the meeting notes and the finished report. An important qualification might have disappeared as the wording has been simplified. Perhaps a benefit of the existing arrangement hasn’t been properly considered, or the reasoning sounds perfectly plausible but doesn’t actually reflect why the adviser reached that recommendation in the first place.

Those are much harder things to spot than a typo or an incorrect figure, and they’re exactly the sort of things that a meaningful review needs to pick up.

We’re naturally quite good at spotting things that look wrong. An unfinished sentence, a strange bit of wording or a figure that doesn’t add up makes us stop and take another look.  

I think the bigger risk with AI is potentially the opposite. If something is well written, polished and sounds like something we might have written ourselves, are we as likely to challenge it? ‍

‍

What does a meaningful check look like?

So, if a firm tells me that “the adviser checks it”, I’d probably want to understand a little more:

• What are they checking?  

• What are they checking it against?  

• What happens if something doesn’t make sense?  

• And, importantly, how do they know whether that checking process is actually working?

That’s why I think the review needs to go beyond whether the report itself looks right and back to the underlying evidence. Does it reflect what the client actually told us? Have the alternatives genuinely been considered? Are the costs, risks and disadvantages accurate? Does the recommendation meet the client’s actual objectives?

There may also be a temptation to ask the same AI tool to check the report it has just produced.  

That might be a useful additional sense-check, but I’d be cautious about treating it as independent assurance. ‍

‍

The most useful test of all

Perhaps the most useful test of all is this:

If you took the suitability report away, could the adviser still explain why this recommendation is right for this particular client?

If the answer is yes, the report is documenting the advice.  

If the answer is no, there is probably a bigger question to ask.  ‍

‍

Keeping it practical for small firms

None of this means a small advice firm needs to create an enormous AI governance project before it can use the technology.

In fact, one of the useful messages from the FCA’s work is that the approach can be proportionate.  

For a firm starting with something relatively defined, such as using AI to help draft suitability reports, I think there are some quite practical questions that can sit behind the process:

• What exactly are we asking the tool to do?  

• What information can it access?  

• What does the adviser need to check back against the original client evidence?  

• What sorts of errors are we seeing and are we keeping track of them?  

• At what point would those errors cause us to change the process or reconsider how the tool is being used?

I’d also want to know what happens if the technology suddenly isn’t available.  

If a firm becomes heavily reliant on AI to produce reports or complete parts of its advice process, there still needs to be a way of continuing to look after clients if that technology isn’t there. That feels much more useful to me than simply having a policy which says that “all AI-generated content must be reviewed by a human.” ‍

‍

Is the benefit actually there?

There were some really positive examples at the conference of technology helping advisers spend less time taking notes and completing admin, and more time actually listening to their clients. If AI can help an adviser get from meeting notes to a first draft more quickly, produce clearer explanations and free up time to think properly about the client’s circumstances, then there is potentially a real benefit. But we still need to understand whether that benefit is actually happening in practice:

• How much editing does the first draft need?  

• Is important client detail being preserved as information moves through the process?  

• Does the finished report actually help the client understand what is being recommended and why?

• And, perhaps most importantly, is the time saved creating more space for the conversations and judgement that really add value?

AI might be able to produce the first draft in seconds, but understanding the client, challenging the recommendation and being able to explain why the advice is suitable still deserve proper time and attention.

If you’re looking for expert input on using AI tools in your advice firm, our Compliance team is always on hand to help and the team love to chat – so feel free to say hi. And don’t forget about our AI hot topic section where you can find loads of insights about AI tools available to advice firms.  

‍