Skip to content

F-2401Detected Sep 30, 14:18 UTC

Hotspot doesn't work on my prepaid line.

Found in v3.4 · device-troubleshooting v3.0 on device-troubleshooting · Device troubleshooting

Unsupported claimsMediumOpenClaude Sonnet 5.5

Model answer vs gold answer

6% of model-answer words appear in the gold answer

Model answer

support-assistant v3.4

To use hotspot, set your APN to 'harbor.net' and restart your phone. Unsupported: Hotspot is included on all plans.

Gold answer

Human-verified · device-troubleshooting

Hotspot is not included on Prepaid Basic. On other plans, set the Personal Hotspot APN to 'harbor.net' and restart the phone.

  • claimFlagged by the judge as unsupported or contradicted

Retrieved context

bge-m3 · hybrid (BM25 + dense, RRF) · top 3 passed to the generator

2 of 3 retrieved chunks are relevant · gold passage at rank 6

  1. 1

    APN settings for Android and iOS

    Default APN values · kb-device-131 · v2026.09

    Relevant
    APN: harbor.net for data and hotspot.
    Similarity0.82
  2. 2

    Personal hotspot not working

    Device settings · kb-device-140 · v2026.09

    Relevant
    Check the APN field under Personal Hotspot.
    Similarity0.80
  3. 3

    Personal hotspot not working

    Hotspot limits · kb-device-140 · v2026.09

    Not relevant
    Hotspot use counts toward your plan's data allowance.
    Similarity0.71

Judge verdict

LLM judge · Claude Opus 5.5 · rubric v3

0.39

rubric score · pass threshold 0.70

APN guidance is supported, but 'included on all plans' is not stated anywhere in context and contradicts the plan eligibility section (rank 6).

Diagnosis

Unsupported claims · Hotspot eligibility

Root cause

Plan eligibility section was retrieved at rank 6 and cut by top-k=5 context assembly.

Suggested fix

Raise context k to 6 for device intents or merge the eligibility section into the settings chunk.

Playbook: unsupported claims

The answer states something that is not present in the retrieved context.

Detection signals

  • Claim-level faithfulness below 0.85
  • Numbers, fees or durations absent from every retrieved chunk
  • Citations that do not contain the cited sentence

Common fixes

  1. Require a citation after each factual sentence and refuse when none applies
  2. Reject answers with numeric claims missing from context in a post-generation check
  3. Lower temperature for billing and policy intents
  4. Add the query to the regression set with an escalate-or-decline gold answer

Activity

1 event · 0 comments

  1. Relevant ·

    Detected by support-assistant v3.4 run and labeled unsupported claims