Skip to content

F-2412Detected Oct 8, 02:31 UTC

How much data can I use in Germany on Unlimited Plus before I get charged?

Found in Nightly · v3.4 · 2026-10-08 on support-golden-v4 · Roaming & travel

Retrieval mismatchHighOpenClaude Sonnet 5.5
Open gt_1042

Model answer vs gold answer

61% of model-answer words appear in the gold answer

Model answer

support-assistant v3.4

Unlimited Plus includes Unsupported: 25 GB of roaming data per month in the EU kb-roaming-011. After that, data is charged at €0.0024 per MB.

Gold answer

Human-verified · support-golden-v4

Unlimited Plus includes 35 GB of roaming data per month in Zone 1 (EU and EEA). Above 35 GB, data costs €0.0024 per MB until the end of the billing cycle.

  • claimFlagged by the judge as unsupported or contradicted

Retrieved context

bge-m3 · hybrid (BM25 + dense, RRF) · top 3 passed to the generator

1 of 3 retrieved chunks are relevant · gold passage at rank 2

  1. 1

    Roaming fair-use (2025 edition)

    Fair-use allowance · kb-roaming-011-2025 · v2025.11

    Superseded versionNot relevant
    Unlimited Plus customers can use up to 25 GB per month in Zone 1 before fair-use charges apply.
    Similarity0.83
  2. 2

    Roaming fair-use

    Fair-use allowance · kb-roaming-011 · v2026.09

    Relevant
    From 1 September 2026, Unlimited Plus includes 35 GB of Zone 1 roaming data per month.
    Similarity0.81
  3. 3

    Roaming zones and countries

    Zone 1: EU and EEA · kb-roaming-003 · v2026.09

    Not relevant
    Zone 1 covers the 27 EU member states plus Iceland, Liechtenstein and Norway.
    Similarity0.74

Judge verdict

LLM judge · Claude Opus 5.5 · rubric v3

0.18

rubric score · pass threshold 0.70

The 25 GB figure comes from the superseded 2025 article ranked first. The current article (rank 2) states 35 GB. Citation points to kb-roaming-011 but the number does not match it.

Diagnosis

Retrieval mismatch · Stale roaming articles

Root cause

The 2025 edition of the fair-use article is still indexed and outranks the current version by 0.02.

Suggested fix

Filter retrieval to effective-dated articles (version >= v2026.09) and remove kb-roaming-011-2025 from the index.

Playbook: retrieval mismatch

The retriever returned the wrong article, the wrong section or a stale version.

Detection signals

  • High similarity on the wrong article title
  • Stale-version flag on top-ranked chunks
  • Exception clause split from its rule across chunk boundaries

Common fixes

  1. Filter or boost by effective date and locale metadata
  2. Re-chunk policy articles with heading-aware splitting
  3. Compare hybrid and dense retrieval on the mismatch subset
  4. Use parent-document retrieval so headings travel with chunks

Activity

1 event · 0 comments

  1. Relevant ·

    Detected by support-assistant v3.4 run and labeled retrieval mismatch