HR Chatbot Accuracy: Controls Against Hallucinated Policy Answers

Photo: Syced, CC0 (public domain), via Wikimedia Commons.
In February 2024 a Canadian tribunal ordered an airline to honor a refund policy that existed only in its chatbot's answer. The airline argued the chatbot was a separate entity responsible for its own actions. The tribunal called that "a remarkable submission." Now picture the same chatbot as your HR assistant, and the invented rule is about leave, severance or final pay. The employee on the other end will not read the fine print either.
An HR chatbot gives wrong policy answers (confabulation) when it answers from memory instead of from your approved documents. The fix is layered controls: ground every answer in a versioned policy library, require citations, refuse when there is no source, escalate sensitive topics to humans, and test with a fixed question set before and after each change. The Air Canada ruling (Moffatt v. Air Canada, 2024 BCCRT 149) shows that a chatbot's answers can be treated as the organization's own.
By the Decision Makers Hub editorial team. Last updated November 2026. This article is general information, not legal advice. Have counsel and your HR leadership review any chatbot that answers questions about pay, leave, benefits or termination.
Key takeaways
- NIST defines confabulation as "the production of confidently stated but erroneous or false content."
- In Moffatt v. Air Canada the tribunal held the airline responsible for what its website chatbot said and ordered $812.02 in total.
- The strongest control is grounding: answer only from approved, current policy documents, with a visible citation.
- A chatbot that says "I do not have that answer, contact HR" is working correctly.
- Test with a fixed question set (a "golden set") before every release, and review real conversations after.
What we keep hearing. A vendor demo where the bot nails ten hand-picked questions, followed by "it is accurate and compliant."
Our take. A demo proves nothing. Ask for test results on your own policies, including questions the bot should refuse to answer.
What happened in Moffatt v. Air Canada, and why does HR care?
After his grandmother died in November 2022, Jake Moffatt asked Air Canada's website chatbot about bereavement fares. The chatbot said he could apply for a reduced fare after booking, within 90 days of ticket issue. Air Canada's separate bereavement page said retroactive requests were not allowed. The B.C. Civil Resolution Tribunal decided the case on February 14, 2024. It held that Air Canada owed a duty of care, did not take reasonable care to ensure the chatbot was accurate, and that Moffatt reasonably relied on it. It rejected the argument that the chatbot was a separate entity and said it was part of Air Canada's website. Damages were $650.88, plus $36.14 interest and $125 in fees, for $812.02 total. You can read the full tribunal decision.
The amount is small. The principle is not. An employee who relies on a wrong answer about PTO payout, notice periods or eligibility can make the same argument, and the stakes are higher.
Why do chatbots invent policy answers?
NIST's Generative AI Profile (NIST AI 600-1) lists confabulation among twelve generative AI risks and defines it as "the production of confidently stated but erroneous or false content," colloquially called hallucinations. It lists "Human-AI Configuration" as another, covering over-reliance and automation bias. Both show up in HR chat: the model writes fluent answers whether or not it has the right document, and employees trust fluent answers.
Which controls reduce wrong answers?
| Control | What it prevents | How to test it |
|---|---|---|
| Ground answers in approved documents only | Answers from the model's general memory | Ask questions your policies do not cover and expect a refusal |
| Show the source (section and date) | Unverifiable claims, stale policies | Click every citation in a sample of 50 answers |
| Refuse when no source is found | Confident guessing | Include trick questions in the test set |
| Escalate sensitive topics | Wrong answers on leave, discipline, termination, immigration, accommodations | Confirm a human contact appears every time |
| Handle location and eligibility | One-size-fits-all answers where rules vary by state or country | Test the same question for several jurisdictions |
| Version control | Old handbooks still in the library | Check last-updated dates monthly |
| Logging and review | Errors nobody notices | Review thumbs-down and low-confidence chats weekly |
How do you build the controls in order?
- Clean the library first. Retire duplicates, expired policies and informal FAQs. A chatbot that reads conflicting documents will repeat the conflict. Your company-wide AI policy should say who owns each document.
- Set the scope in writing. Decide what the bot answers (benefits enrollment steps, holiday calendar) and what it routes to a person (terminations, discipline, accommodations). The same approach appears in our AI governance framework and our generative AI acceptable use policy.
- Make location a first-class input. Rules change by place. Final pay deadlines differ across states, as we show in final paycheck laws by state, and pay posting rules differ too, as in pay transparency laws by state. A chatbot that answers both without asking where the employee works is guessing.
- Require citations and refusals. Configure the bot to quote or link the policy section, and to say it does not know when retrieval returns nothing relevant.
- Build a golden test set. Write 100 or more real employee questions with approved answers, including questions the bot must refuse. Run it before launch, after every document or model change, and on a schedule.
- Keep a person in the loop. Give employees an easy way to reach HR, and give HR a review queue. Our guide to human in the loop AI explains how to design oversight that people actually use.
- Protect the data. Chat logs contain personal information. Apply the controls in protecting employee data in HR systems.
What should you measure?
Track a handful of numbers and keep them honest: the share of golden-set questions answered correctly with a valid citation, the refusal rate on out-of-scope questions, the rate of escalation to HR, the age of the oldest document cited, and the share of employee feedback marked unhelpful. Set your own thresholds, and treat any wrong answer on a high-risk topic (pay, leave, termination) as an incident to investigate, not a statistic to average away. NIST's suggested actions include after-action reviews of AI incident response (GV-1.5-002), a useful model for how to review these.
Frequently asked questions
Why do HR chatbots give wrong policy answers?
Language models generate fluent text from patterns. Without grounding in your approved documents, they can produce confidently stated but false answers, which NIST calls confabulation.
Is a company responsible for its chatbot's answers?
In Moffatt v. Air Canada, a B.C. tribunal held that the airline was responsible for its website chatbot's content and rejected the idea that the chatbot was a separate entity. Other jurisdictions may differ, so ask counsel.
What is the best way to reduce hallucinated answers?
Ground every answer in a versioned library of approved policies, require citations, refuse when no source is found, and escalate sensitive topics to people.
How often should we test the chatbot?
Before launch, after every model or document change, and on a regular schedule such as weekly, using a fixed set of real employee questions.
Should the chatbot answer questions about termination or discipline?
Most organizations route these to a person. Decide in writing which topics the bot may answer and which it must escalate.
Comments
Post a Comment