Skip to content

F-2388Detected Oct 2, 10:20 UTC

Why is my bill higher this month after I downgraded on the 12th?

Found in v3.4 · billing-edge-cases v2.1 on billing-edge-cases · Billing & payments

Retrieval mismatchHighOpenClaude Sonnet 5.5

Model answer vs gold answer

30% of model-answer words appear in the gold answer

Model answer

support-assistant v3.4

Unsupported: Downgrades take effect immediately, so Unsupported: you should see a credit for the rest of this month on your next bill.

Gold answer

Human-verified · billing-edge-cases

Downgrades take effect at the start of your next billing cycle, so this month you were billed for your current plan in full. The lower price appears from the next cycle.

  • claimFlagged by the judge as unsupported or contradicted

Retrieved context

bge-m3 · hybrid (BM25 + dense, RRF) · top 3 passed to the generator

1 of 3 retrieved chunks are relevant · gold passage at rank 3

  1. 1

    Changing your plan

    Upgrades · kb-plans-008 · v2026.09

    Not relevant
    Upgrades take effect immediately with a pro-rated charge.
    Similarity0.85
  2. 2

    Understanding your monthly bill

    Pro-rated charges · kb-billing-014 · v2026.09

    Not relevant
    Pro-rated charges and credits apply to immediate plan changes.
    Similarity0.79
  3. 3

    Changing your plan

    Downgrades · kb-plans-008 · v2026.09

    Relevant
    Downgrades take effect at the start of the next billing cycle.
    Similarity0.78

Judge verdict

LLM judge · Claude Opus 5.5 · rubric v3

0.05

rubric score · pass threshold 0.70

The upgrade section outranks the downgrade section; the answer follows the upgrade rule.

Diagnosis

Retrieval mismatch · Plan change proration

Root cause

Direction of plan change not used as a retrieval signal.

Suggested fix

Add query intent classification (upgrade vs downgrade) and use it as a section filter.

Playbook: retrieval mismatch

The retriever returned the wrong article, the wrong section or a stale version.

Detection signals

  • High similarity on the wrong article title
  • Stale-version flag on top-ranked chunks
  • Exception clause split from its rule across chunk boundaries

Common fixes

  1. Filter or boost by effective date and locale metadata
  2. Re-chunk policy articles with heading-aware splitting
  3. Compare hybrid and dense retrieval on the mismatch subset
  4. Use parent-document retrieval so headings travel with chunks

Activity

1 event · 0 comments

  1. Relevant ·

    Detected by support-assistant v3.4 run and labeled retrieval mismatch