Hallucinations and Sycophancy
Share with your team so they can revisit the workshop anytime.
Beginner · All Levels Welcome · Today we name the two ways generative AI is confidently wrong.
The 20 Minutes Ahead
Week 2 is the explanation Week 1 promised. The 3 Data Classification Tiers exist because of two specific failure modes. Today we name both, show what they look like in multifamily, and put one verification habit in your hands.
Open and recap
1 to 2 minutes. We run through the warm-up quiz answers from the welcome screen: last Tuesday's 3 Tiers in one beat.
Teach the two failures
Hallucination, sycophancy, and why both produce output that looks correct. The 3 Data Classification Tiers as the seatbelt. 9 minutes.
Power Preview
The Marketing Caption Test. Watch AI invent amenities, then watch grounding fix it. 5 minutes.
Practice and quiz
Spot the tier. Five-question quiz. 3 minutes.
Close
Next Tuesday tease. 30 seconds.
Objective: By the end of this session, you can name both AI failure modes out loud, you know the multifamily moments where each one shows up, and you have one verification habit you can use this afternoon.
What is a hallucination?
When generative AIGENERATIVE AIAI that creates new content: text, images, code, video. ChatGPT, Claude, Gemini, and Copilot are the major examples. This is the type of AI prone to both hallucination and sycophancy. invents facts in fluent, confident sentences.
A hallucinationHALLUCINATIONWhen AI confidently makes up facts that are not true. It is not lying on purpose. It is filling in what sounds right when it does not know. is the AI version of a confidently wrong coworker. The sentence reads well. The grammar is correct. The tone is professional. The fact inside is invented.
2. Confident tone where uncertainty would be appropriate.
3. The fact is plausible. It just is not real.
Hallucination is the #1 reason every AI outputOUTPUTWhat the AI gives back to you after you ask. The text, answer, image, or content it generates. needs human review before it leaves your hands.
Why does AI hallucinate?
A Large Language ModelLARGE LANGUAGE MODEL (LLM)The technology behind ChatGPT, Claude, Gemini, and Copilot. It generates text by predicting what word comes next, learned from patterns in billions of words. It does not understand meaning. It predicts patterns. does not look things up. It predicts the next word, then the next, then the next, based on patterns from the billions of words it was trained onTRAINING DATAThe massive collection of text, code, and content an AI model learned from. Billions of words from books, articles, websites, and code. The patterns AI absorbed from training data are what it uses to predict its next word. Once your information is in training data, it cannot be removed.. There is no fact-check step in the middle.
AI generates the most statistically likely next word. Not the most true.
It cannot stop and say "I do not know this." Silence is not in its training.
When real information is missing, AI manufactures plausible information to keep the sentence flowing.
AI is not a search engine. It is a writer that has read everything and remembered patterns. Treat its output as a draft, never as a fact.
Where hallucinations show up in multifamily
These are not hypotheticals. Each one has happened in a leasing office, a marketing team, or a regional inbox in the last twelve months.
Invented regulation citations
A leasing professional asks AI to cite federal Fair HousingFAIR HOUSINGFederal law (the Fair Housing Act) that prohibits discrimination in housing based on seven protected classes: race, color, national origin, religion, sex, familial status, and disability. AI-generated content can violate Fair Housing if it uses language that excludes or discourages any protected class. regulations. AI returns a real-sounding section number that does not exist. The letter goes out.
Fabricated amenities
A marketing team asks AI to write social captions for a property. AI mentions a rooftop pool and dog park the property does not have. The caption goes live.
Invented policy details
A property manager asks AI to draft a reply about pet fees. AI references a policy clause that is not in the lease. The resident reads it and quotes it back later.
Pattern: AI is most confident when it has the least information. The more specific the answer, the more carefully it needs verification.
How to spot a hallucination
The same three moves catch nearly every hallucination before it leaves your screen.
Ask for the source. Then check it.
"What is your source for this?" If the source does not exist when you search for it, you found a hallucination. Do not let the AI verify itself.
GroundGROUNDINGGiving AI the actual facts (a list, document, or real data) before asking it to write something, so it works from truth instead of guessing. the prompt with real data.
Paste the actual amenity list, the actual policy text, the actual property fact sheet into the prompt. AI cannot hallucinate facts you handed it.
VerifyVERIFICATIONChecking that information is true by going back to the original source. Not asking the AI to confirm itself. every specific claim before it goes out.
Dates, numbers, citations, regulations, statistics, amenities, policy details. If a human cannot confirm it elsewhere, AI cannot publish it.
The verification habit is the single biggest skill you build this year. It catches hallucination, sycophancy, and most other AI failures. Practice it on every output.
What is sycophancy?
When AI agrees with whatever you said last, even when you were wrong.
SycophancySYCOPHANCYWhen AI agrees with whatever you said last, even when you were wrong. AI is trained to be agreeable, sometimes at the cost of accuracy. is the AI version of a coworker who flatters you when you need pushback. It validates your draft. It agrees with your read. It says "great question" before answering. It will rarely tell you you are wrong unless you ask it to.
AI replies: "Yes, your response sounds professional and addresses the concern."
AI will give that answer whether the response is great or terrible.
Sycophancy is harder to catch than hallucination because it feels good in the moment.
Why is AI sycophantic?
AI was trained on millions of human ratings. Reviewers gave higher scores to responses that were helpful, agreeable, and confident. So the AI learned that the highest-scoring move is to agree, validate, and keep the conversation pleasant.
Reinforcement learningREINFORCEMENT LEARNINGA training method where human reviewers score AI's answers. AI learns to repeat answer styles that got high scores. This is why AI tends to be agreeable. Agreement got rated highly during training, so AI keeps doing it. rewarded agreeable answers more than corrective ones.
When you assert something in your prompt, AI treats your framing as a fact and builds from there.
Push back on its answer with confidence and it will often reverse, even when its first answer was right.
AI is rewarded for agreement, not for accuracy. If you want truth, you have to ask in a way that does not give AI a framing to agree with.
Where sycophancy shows up in multifamily
All three of these scenarios feel productive. None of them are.
"Is this compliant?"
A team member asks AI: "Is this screening response Fair Housing compliant?" AI says yes. The response actually violates protected classPROTECTED CLASSCategories of people that Fair Housing law specifically protects against discrimination: race, color, national origin, religion, sex, familial status, and disability. Any AI output that could exclude or discourage a protected class creates legal exposure. guidance.
"These look right, yes?"
An asset manager pastes a rent comp set and asks AI to confirm the numbers make sense. AI rubber-stamps comps that are six months stale and one zip code off.
"Does this sound right?"
A manager drafts a reply and asks AI: "This sounds professional, right?" AI validates a reply that is technically professional and emotionally tone-deaf to a grieving resident.
Pattern: If you phrase the question as "is this right?" you will get yes. Phrase it as "find what is wrong with this" and you will get truth.
How to spot sycophancy
Three moves break the agreement loop and force AI to actually think.
Ask the question two ways.
"Is this compliant?" then "What is wrong with this from a compliance standpoint?" If the two answers contradict, the first one was sycophancy.
Ask AI to argue against itself.
"List five reasons this draft might fail." This bypasses the agreement bias because you handed AI a task that requires disagreement.
Verify with a person who can disagree with you.
Compliance, legal, your AI CouncilAI COUNCILYour organization's decision authority for AI use. The team that reviews new tools, approves Tier 2 (Yellow) use cases, and enforces Tier 3 (Red) prohibitions. Some organizations call this the Governance Team., a peer. AI will not push back on you. People will. Stop using AI as a second opinion. Use it as a draft.
The same skill catches both failures. Hallucinations get caught when you verify facts. Sycophancy gets caught when you verify judgment. Verification is the habit.
Your questions, answered
If something in the last ten minutes felt unclear, you are in good company. These are the four questions attendees ask after every Hallucinations and Sycophancy session.
If AI makes things up, can I even trust it for work?
Yes, with verification. AI is excellent for drafts, brainstorms, structure, and rewording. It is unreliable for facts, numbers, citations, and policy specifics. Use it to start. Never use it to finish.
Does asking for sources fix hallucinations?
Only if you check the sources yourself. AI will happily invent a real-sounding citation. The verification step is on you, not the AI. Open the link. Search the regulation. Confirm the source actually exists and says what AI claims it says.
How do I tell if AI is agreeing with me out of sycophancy?
Push back on its answer once. If it instantly reverses and apologizes, that is sycophancy. A useful answer holds its ground when you challenge it for the wrong reason and updates when you challenge it for the right one.
Will newer AI models fix these problems?
Partially. Hallucination rates are dropping. Sycophancy is harder to fix because users reward it. Either way, verification stays your job. The skill you build this week will matter for every AI model that comes out for the rest of your career.
The Marketing Caption Test
Watch AI invent amenities your community does not have. Then watch grounding fix it.
Marketing copy is where hallucinationHALLUCINATIONWhen AI confidently makes up facts that are not true. It is not lying on purpose. It is filling in what sounds right when it does not know. shows up most often in multifamily. AI fills gaps in your promptPROMPTThe instructions you type telling the AI what you want. Your input. with plausible-sounding amenities that may or may not exist. Today we run the same prompt two ways. The first way exposes the failure. The second way is the fix.
One of seven foundational prompt structures we teach across the year. By Week 20 you will recognize all seven.
Watch AI invent amenities that do not exist.
This is hallucination. AI did not lie on purpose. It filled in what sounded right because the prompt did not tell it what was real. The smoother the outputOUTPUTWhat the AI gives back to you after you ask. The text, answer, image, or content it generates. reads, the easier this failure is to miss.
Same property. Same prompt. Zero hallucinations.
GroundingGROUNDINGGiving AI the actual facts (a list, document, or real data) before asking it to write something, so it works from truth instead of guessing. is the fix. Give AI the facts before it fills in the blanks. The verification habit is not extra work. It is the work.
Try This Today
Hallucination drops to zero when you give AI the facts before it fills in the blanks. Sycophancy drops when you ask the AI to argue against itself. Verification is the skill 50 more Tuesdays will sharpen.
The two failures, side by side
Save this slide. Star it. It is the one-page summary of everything you learned today.
Hallucination
AI invents facts in confident sentences.
· Marketing copy amenities
· Resident reply policy details
· Statistics and dates
· Vendor and regulation references
· Verify the source exists
· Ground the prompt with real data
· Check every specific claim
Sycophancy
AI agrees with whatever you said last.
· "These comps look right?"
· "Sound professional?" drafts
· Validating a hunch
· Confirming a position you already hold
· Ask AI to argue against itself
· Push back once, watch for instant reversal
· Verify judgment with a person
Verify. Every time. Before the output leaves your hands.
The verification habit, in five moves
Run this on every AI output before you send it, post it, or sign it. Five moves. Under a minute. The skill that defines a competent AI user.
Scan for specifics.
Names, dates, numbers, statutes, statistics, citations, amenities. Flag every one.
Verify each specific against a primary sourcePRIMARY SOURCEThe original document where information lives. The actual statute, the actual lease, the actual fact sheet. Not a copy. Not a paraphrase. Not the AI's restatement. Verifying against primary sources is what defeats hallucinations..
Not from the AI. From the actual document, statute, fact sheet, or system of record.
Ask AI to find the weakness in its own answer.
"List five things wrong with this response." If AI suddenly finds problems, those problems were always there.
Push back once. Watch for instant reversal.
If AI flips immediately to agree with you, both versions are suspect. The original was sycophancy. The reversal is sycophancy too.
Get a human review for anything resident-facing or compliance-adjacent.
Your AI Council, your compliance contact, a peer who can disagree. AI cannot tell you when you are wrong about something that matters.
The verification habit is what separates the people who use AI well from the people who get burned by it. Build the habit now while the stakes are small.
Failure patterns you will see this week
Eight real situations. Each one is hallucination, sycophancy, or both. Star this slide. When something feels off in an AI output this week, come back here.
| The Situation | Failure Type | The Fix |
|---|---|---|
| AI cites a Fair Housing section number you cannot find. | Hallucination | Search the actual statute. Replace with verified language or drop the citation. |
| AI confirms your draft is "professional and addresses the concern." | Sycophancy | Ask: "Find five problems with this draft." |
| AI mentions an amenity your property does not have. | Hallucination | Paste the actual amenity list into the prompt. Regenerate. |
| AI agrees your rent comps look reasonable when you ask. | Sycophancy | Ask: "What is missing or stale in this comp set?" |
| AI quotes a statistic that sounds right but has no source. | Hallucination | Ask for the source. Open it. Confirm or remove. |
| AI reverses its answer the second you push back. | Sycophancy | Both answers are suspect. Get a human review. |
| AI invents a lease clause that is not in your actual lease. | Hallucination | Paste the lease into the prompt before asking. |
| AI invents a policy AND validates your wrong read of it. | Both | Run the full verification habit. Five moves. Every time. |
Pattern: hallucination usually shows up around facts. Sycophancy usually shows up around judgment. Both show up together more often than either alone.
The 12 Generative AI Risks
The full catalog of generative AI risks named by the federal government's leading standards body. The two failures we taught today both live inside this framework.
Hallucinations is what NIST calls Confabulation (Risk #2). Sycophancy is part of Human-AI Configuration (Risk #7), specifically the overreliance and automation bias dimensions. When you teach your team to spot these two failures, you are operating inside the NIST framework. Highlight cards below.
Eased access to hazardous information related to chemical, biological, radiological, or nuclear (CBRN) weapons.
The generation of confidently stated but erroneous or false content, commonly known as hallucinations. This is the NIST name for what we taught today as Hallucinations.
Difficulty controlling the creation of and public exposure to offensive, inciting, or threatening language.
Risks involving the unauthorized use, disclosure, or de-anonymization of personally identifiable information (PII) when AI is trained or in active use.
The significant energy and environmental footprint caused by the resource-intensive training and operation of generative AI systems.
The production of discriminatory outputs or the reduction of diversity in AI decision-making due to non-representative training data.
Negative outcomes from human interaction with AI, such as automation bias, overreliance, or inappropriate emotional entanglement. This is where Sycophancy lives in the NIST framework.
Increased difficulty in distinguishing between fact and fiction, including the spread of disinformation and misinformation.
Lowering the barrier for offensive cyber capabilities like automated hacking, and expanding the available attack surface.
Eased production or replication of copyrighted, trademarked, or licensed content and the exposure of trade secrets.
Eased production of harmful explicit imagery, including synthetic child sexual abuse material (CSAM) and non-consensual intimate images (NCII).
Challenges with transparency and accountability when integrating non-transparent third-party data or pre-trained models.
Five free downloads from NIST
The primary sources behind everything we teach. Free. Public. Authoritative. The 3 Data Classification Tiers, hallucination guidance, and sycophancy framing all map to these documents.
The foundation. Defines the four functions every AI program runs: GOVERN, MAP, MEASURE, MANAGE. Voluntary use. The standard serious organizations follow. The 3 Data Classification Tiers maps directly to it.
The companion to AI RMF 1.0 written specifically for generative AI. Source of the 12 risks above plus suggested actions for each. This is what frames today's workshop in industry-standard language.
The tactical playbook. Specific suggested actions for each subcategory of the AI Risk Management Framework. Where the framework meets daily decisions.
The same playbook in spreadsheet form. Filter by function, sort by risk category, build internal trackers from it. The format your governance team will actually use.
The same playbook as a CSV file. For teams who want to pull it into other tools, dashboards, or internal databases.
Spot the Tier
Instructions: Each scenario this week involves either hallucination or sycophancy in a multifamily moment. Click your classification. Instant feedback shows you how the failure mode connects to the tier.
The 3 Tiers Quiz
🔒 Anonymous. No name needed.
Take this anytime, even after we hang up. Wrong answer? You get an educational tip. Your results submit anonymously to help us improve the workshop. No login. No name. No email collected from you. Answer honestly. That's where the learning happens.
Confidence in AI output is not evidence of accuracy. Hallucinated legal citations are the highest-risk failure mode in multifamily because they produce direct Fair Housing exposure. Verify every citation against the actual statute. Asking AI to check itself triggers sycophancy: it will agree it was right whether it was or not.
When in doubt, Yellow it out. Hallucinated numbers in a comp summary steer pricing decisions in the wrong direction across the portfolio. Verification against your actual rent roll is the only acceptable check. Enterprise tools hallucinate too. The platform does not fix the failure mode.
Asking AI to validate any judgment that touches Fair Housing is the textbook sycophancy trap. AI agrees with whatever framing it receives, including discriminatory framing. Always Red, no matter how the question is phrased or how certain the AI sounds.
Sycophancy is AI agreeing with whatever you said last. Ask 'this resident is being unreasonable, right?' and AI says yes. If the resident has a disability or protected class status, that yes is sycophantic validation of potential discrimination. Decision quality collapses. The AI is not a defense.
One hallucinated promise about an amenity affects four chains. The resident relies on a false promise. The community owns the harm when they tour and discover the amenity does not exist. The operator carries the Fair Housing risk and any refund liability. The industry pays the reputational cost of operators that look unreliable. AI mistakes never stay private.
Download the De-Sycophancy Prompt Library
This week's action item has three steps. About twenty minutes total. By the end, you have a working library of 12 prompts that catch sycophancy before it costs you, and your supervisor has confirmation that you completed Week 2.
12 Prompts That Force AI to Tell You the Truth
The companion library to today's workshop. Twelve ready-to-use prompts that break AI's agreement bias: The Challenge, The Steelman, The Strawman Finder, The Red Team, The Skeptic, The Blind Spot, The Pre-Mortem, The Rating, The Opposite, The Second Opinion, The Cost of Being Wrong, and The Evidence Test. Print it. Pin it next to your monitor. Use at least one every time you ask AI for a recommendation.
The rule: Never ask AI to validate. Ask AI to challenge. If AI agrees with everything you say, you are not getting intelligence. You are getting a mirror.
Tier 3 Deep Dive: The Never Zone
Nine categories of information that never go into any AI tool, no matter how good the tool is.
Now that you can name both failure modes, you will see why some uses are never permitted regardless of the platform. The 3 Data Classification Tiers exist for two reasons. The first is the failures you learned to spot today. The second is the categories of information that get whole organizations sued.
· The escalation rule: how Yellow plus Yellow plus Yellow combines to one Red
· The scrub-to-placeholders habit that saves real situations
· Three real cases where the Never Zone got violated, and what it cost
You did the work today.
You can now name the two failure modes every other week of this curriculum builds on. Hallucination. Sycophancy. The verification habit. That is the foundation. Fifty more Tuesdays will sharpen the rest. See you Tuesday.
