Skip to content

F-2394Detected Sep 28, 11:30 UTC

I'm going on a cruise in the Mediterranean. Will my phone work?

Found in v3.4 · roaming-and-travel v1.4 on roaming-and-travel · Roaming & travel

Missing contextHighOpenClaude Sonnet 5.5
Open gt_1054

Model answer vs gold answer

5% of model-answer words appear in the gold answer

Model answer

support-assistant v3.4

Yes. Unsupported: Your Zone 1 roaming allowance covers Greek waters, so you can use your phone as normal on the cruise.

Gold answer

Human-verified · roaming-and-travel

On board, ships use maritime satellite networks outside every roaming zone, charged at €9.99 per MB. Turn on Block satellite roaming in My Harbor before boarding. In port, Zone 1 rates apply.

  • claimFlagged by the judge as unsupported or contradicted

Retrieved context

bge-m3 · hybrid (BM25 + dense, RRF) · top 3 passed to the generator

0 of 3 retrieved chunks are relevant · gold passage at rank 11

  1. 1

    Roaming zones and countries

    Zone 1: EU and EEA · kb-roaming-003 · v2026.09

    Not relevant
    Greece is in Zone 1.
    Similarity0.83
  2. 2

    Roaming packs

    Daily packs · kb-roaming-007 · v2026.09

    Not relevant
    Zone 1 daily packs cost €3 for 1 GB.
    Similarity0.72
  3. 3

    Roaming fair-use

    Long stays abroad · kb-roaming-011 · v2026.09

    Not relevant
    Fair-use checks apply after 4 months abroad.
    Similarity0.70

Judge verdict

LLM judge · Claude Opus 5.5 · rubric v3

0.06

rubric score · pass threshold 0.70

The maritime networks article (kb-roaming-015) is the gold source and ranked 11th. The answer could lead to large unexpected charges.

Diagnosis

Missing context · Maritime and in-flight roaming

Root cause

'Cruise' is not in the article body; only 'maritime' and 'ship' are.

Suggested fix

Add synonyms (cruise, ferry, ship, at sea) to the article's search keywords.

Playbook: missing context

A relevant article exists but was not retrieved, or was only partially retrieved.

Detection signals

  • Context recall below 0.7
  • Gold passage outside the top 10
  • Generic or hedged answer where gold is specific

Common fixes

  1. Add customer-phrased synonyms and regional terms to query expansion
  2. Increase k for long-tail intents or add a second retrieval pass
  3. Fill knowledge-base gaps, then re-index
  4. Escalate when no chunk passes the relevance threshold

Activity

1 event · 0 comments

  1. Relevant ·

    Detected by support-assistant v3.4 run and labeled missing context