Skip to content

F-2380Detected Oct 3, 02:30 UTC

What's included in my data access request?

Found in Nightly · v3.4 · 2026-10-03 on support-golden-v4 · Data privacy requests

Unsupported claimsMediumTriagedClaude Sonnet 5.5

Model answer vs gold answer

53% of model-answer words appear in the gold answer

Model answer

support-assistant v3.4

Your export includes your bills, call records, and the support chat transcripts Unsupported: from the last 10 years.

Gold answer

Human-verified · support-golden-v4

The export includes account details, bills from the last 24 months, call and data records from the last 12 months, and support interactions from the last 24 months, as CSV and PDF files.

  • claimFlagged by the judge as unsupported or contradicted

Retrieved context

bge-m3 · hybrid (BM25 + dense, RRF) · top 3 passed to the generator

2 of 3 retrieved chunks are relevant · gold passage ranked first

  1. 1

    Requesting a copy of your data

    What we include · kb-privacy-002 · v2026.09

    Relevant
    Bills (24 months), usage records (12 months), support interactions (24 months).
    Similarity0.87
  2. 2

    Requesting a copy of your data

    Format · kb-privacy-002 · v2026.09

    Relevant
    Delivered as a ZIP with CSV and PDF files.
    Similarity0.80
  3. 3

    Deleting your personal data

    Retention exceptions · kb-privacy-005 · v2026.09

    Not relevant
    Some billing data is retained for 10 years for tax purposes.
    Similarity0.72

Judge verdict

LLM judge · Claude Opus 5.5 · rubric v3

0.29

rubric score · pass threshold 0.70

The 10-year figure belongs to tax retention, not the export scope listed in chunk 1.

Diagnosis

Unsupported claims · Privacy request timelines

Root cause

Numeric bleed from a retention chunk.

Suggested fix

Route privacy intents to verified templates (same as F-2400).

Playbook: unsupported claims

The answer states something that is not present in the retrieved context.

Detection signals

  • Claim-level faithfulness below 0.85
  • Numbers, fees or durations absent from every retrieved chunk
  • Citations that do not contain the cited sentence

Common fixes

  1. Require a citation after each factual sentence and refuse when none applies
  2. Reject answers with numeric claims missing from context in a post-generation check
  3. Lower temperature for billing and policy intents
  4. Add the query to the regression set with an escalate-or-decline gold answer

Activity

1 event · 0 comments

  1. Relevant ·

    Detected by support-assistant v3.4 run and labeled unsupported claims