What AI Tools Can and Cannot Do Reliably
In Level 1 you separated evidence from hype and fear. This level explains how an AI assistant such as ChatGPT, Gemini, Claude or Copilot actually produces its answers. Once you understand that, you can decide which jobs to hand it and which answers to check.
How an AI assistant writes its answers
A generative AI assistant is not a database or a search engine. It learned patterns from a very large amount of text. When you ask it something, it predicts the most likely next words, one after another, until the answer is finished.
It works a bit like a very well-read new employee who answers every question at once and never says "let me check". Most of the time the answer is fluent and useful. But the assistant is built to produce plausible text, and plausible is not always the same as true.
What this means in practice:
- It sounds equally confident whether it is right or wrong. A good writing style tells you nothing about whether the facts are accurate.
- It can invent details. These invented details are called "hallucinations". In a paper published on 5 September 2025, OpenAI researchers defined them as cases where a model confidently gives an answer that isn't true. They argued that standard training and testing reward a model for guessing rather than for admitting it is unsure.
- Its knowledge can be out of date. A model learns from data collected up to a certain date. Unless it searches the web or reads a document you give it, it may not know about recent price changes, new rules or your latest opening hours.
- It does not know your business. It cannot see your price list, stock or diary unless you supply them.
The vendors say so themselves. ChatGPT and Gemini both show a notice that the assistant can make mistakes and that you should check what it says. Anthropic's help pages warn against relying on Claude as a single source of truth for high-stakes advice.
Why this matters: two real cases
- Moffatt v Air Canada (February 2024). Air Canada's website chatbot told a grieving customer he could claim a bereavement discount after he had booked. The airline's actual policy said otherwise. A Canadian tribunal held the airline responsible for what its chatbot said and ordered it to pay compensation. Lesson: your business is answerable for AI answers given in its name.
- Ayinde v Haringey (High Court, June 2025). Legal documents put before the court cited cases that did not exist. In its judgment the court warned lawyers that AI tools can produce convincing but false references. Lesson: names, references and quotations need checking against the original source.
Task 1: Rate the risk of each job
Before using AI for a task, ask two questions:
- How accurate does it have to be?
- How much harm could an error cause? Think about money, safety, the law, your reputation and customers' trust.
Then place the task in one of three bands.
| Risk | Typical tasks | How to use AI |
|---|---|---|
| Low | Brainstorming post ideas, first drafts of a friendly email, rewording a notice, summarising your own meeting notes | Use freely, then read it through |
| Medium | Customer replies that mention prices or dates, product descriptions, quotes drafted from your notes | Use, but check every fact and figure |
| High | Allergen information, safety instructions, tax or VAT figures, legal or employment wording, anything sent automatically without a person reading it | Use only as a rough starting point. A qualified person checks it against the original source, or you don't use AI at all |
Worked examples from our three sample firms:
- Rosie's Café asks AI for five ideas for autumn specials. That is low risk. Asking it to write allergen labels for those specials is high risk, because a wrong label could harm a customer. Rosie writes the allergen labels herself from the supplier's ingredient sheets.
- Patel Plumbing drafts a reply to a customer who has asked for a quote. That is medium risk. The tone is easy to fix, but the price, call-out charge and dates must match Raj's real rates.
- Bright Hair Studio asks AI to explain new employment rules for its staff contract. That is high risk. The AI may be out of date or simply wrong, so the owner checks official guidance or takes professional advice.
Do this now: list five tasks you are considering for AI and mark each one Low, Medium or High.
Task 2: Use the four-point check before anything goes out
Run this routine on every piece of AI output before you use it. It takes a minute or two.
- Facts. Is every claim true and current? Check anything you are unsure of against a reliable source, such as your own records, the supplier or an official website.
- Figures. Check prices, dates, times, quantities, percentages and totals line by line against your own figures. Never trust a calculation without checking it.
- Names. Are the names of people, products, businesses, laws and references real and spelled correctly? Look up any reference or quotation before using it.
- Tone. Does it sound like your business? Is it polite, clear and free of promises you can't keep, such as "guaranteed" or "same-day"?
If something fails a check, correct it yourself or ask the AI to redo it. Then check the new version again. The higher the risk band, the more carefully you check.
Do this now: print or save the four checks (Facts, Figures, Names, Tone) and keep them next to your screen.
Checklist
- I understand that AI predicts likely text and does not look up the truth
- I have rated my planned AI tasks Low, Medium or High
- High-risk tasks are checked by a qualified person against the original source, or not done with AI
- Every output passes Facts, Figures, Names and Tone before use
- Nothing AI-written reaches a customer without a person reading it
Key takeaways
- AI assistants write the most likely words, so they can sound confident and still be wrong or out of date.
- Your business is responsible for AI-written content sent in its name (Moffatt v Air Canada, 2024).
- Rate each task by how accurate it must be and how much harm an error could cause: Low, Medium or High.
- Check Facts, Figures, Names and Tone every time, before anything is used.