Skip to content

F-2409Detected Oct 2, 10:04 UTC

I upgraded from Essential to Unlimited on the 18th. Why is my bill €41.20 instead of €34.90?

Found in v3.4 · billing-edge-cases v2.1 on billing-edge-cases · Billing & payments

Unsupported claimsCriticalOpenClaude Sonnet 5.5
Open gt_1045

Model answer vs gold answer

29% of model-answer words appear in the gold answer

Model answer

support-assistant v3.4

Your bill is higher because Unsupported: upgrades are charged a one-off €6.30 upgrade fee kb-plans-008.

Gold answer

Human-verified · billing-edge-cases

Upgrades take effect immediately, so the bill includes a pro-rated charge for Unlimited from the 18th, a credit for unused Essential days, and the full Unlimited price for the next cycle. There is no upgrade fee.

  • claimFlagged by the judge as unsupported or contradicted

Retrieved context

bge-m3 · hybrid (BM25 + dense, RRF) · top 3 passed to the generator

2 of 3 retrieved chunks are relevant · gold passage ranked first

  1. 1

    Changing your plan

    Upgrades · kb-plans-008 · v2026.09

    Relevant
    Upgrades take effect immediately. Your next bill shows a pro-rated charge for the new plan and a credit for the old plan.
    Similarity0.84
  2. 2

    Understanding your monthly bill

    Pro-rated charges · kb-billing-014 · v2026.09

    Relevant
    Pro-rated charges cover the days between a plan change and the end of the cycle.
    Similarity0.80
  3. 3

    Understanding your monthly bill

    One-off charges · kb-billing-014 · v2026.09

    Not relevant
    One-off charges include roaming packs, device payments and late fees.
    Similarity0.66

Judge verdict

LLM judge · Claude Opus 5.5 · rubric v3

0.06

rubric score · pass threshold 0.70

The answer invents an upgrade fee by subtracting the two amounts in the question. The retrieved context explains proration and contains no fee.

Diagnosis

Unsupported claims · Plan change proration

Root cause

Arithmetic on numbers from the question was presented as a policy fact.

Suggested fix

Add a rule to explain amounts only through retrieved bill components; add proration worked examples to the few-shot set.

Playbook: unsupported claims

The answer states something that is not present in the retrieved context.

Detection signals

  • Claim-level faithfulness below 0.85
  • Numbers, fees or durations absent from every retrieved chunk
  • Citations that do not contain the cited sentence

Common fixes

  1. Require a citation after each factual sentence and refuse when none applies
  2. Reject answers with numeric claims missing from context in a post-generation check
  3. Lower temperature for billing and policy intents
  4. Add the query to the regression set with an escalate-or-decline gold answer

Activity

1 event · 0 comments

  1. Relevant ·

    Detected by support-assistant v3.4 run and labeled unsupported claims