Skip to content

F-2368Detected Oct 4, 10:48 UTC

I'm moving my UK number from another operator to Harbor. How long does it take?

Found in v3.4 · locale-formatting on locale-formatting · Number porting

Retrieval mismatchHighOpenClaude Sonnet 5.5

Model answer vs gold answer

13% of model-answer words appear in the gold answer

Model answer

support-assistant v3.4

Moving your number takes Unsupported: 3 to 5 business days once your porting code is received.

Gold answer

Human-verified · locale-formatting

Once we receive a valid porting code, the transfer completes within 1 business day. Keep your old line active until your Harbor SIM shows signal.

  • claimFlagged by the judge as unsupported or contradicted

Retrieved context

bge-m3 · hybrid (BM25 + dense, RRF) · top 3 passed to the generator

2 of 3 retrieved chunks are relevant · gold passage at rank 2

  1. 1

    Porting out to another operator

    Final bill · kb-porting-006 · v2026.09

    Not relevant
    Final bill within 5 business days of porting out.
    Similarity0.76
  2. 2

    Bringing your number to Harbor

    Timeline · kb-porting-002 · v2026.09

    Relevant
    Transfers complete within 1 business day.
    Similarity0.74
  3. 3

    Bringing your number to Harbor

    What to keep active · kb-porting-002 · v2026.09

    Relevant
    Keep your old SIM active.
    Similarity0.70

Judge verdict

LLM judge · Claude Opus 5.5 · rubric v3

0.12

rubric score · pass threshold 0.70

Port-out timeline used for a port-in question (same pattern as F-2405).

Diagnosis

Retrieval mismatch · Porting direction confusion

Root cause

The phrase 'moving my number from' matches the port-out article, which ranks first; porting direction is not in the metadata.

Suggested fix

Add port-in and port-out direction metadata and customer-phrased aliases to the porting sections.

Playbook: retrieval mismatch

The retriever returned the wrong article, the wrong section or a stale version.

Detection signals

  • High similarity on the wrong article title
  • Stale-version flag on top-ranked chunks
  • Exception clause split from its rule across chunk boundaries

Common fixes

  1. Filter or boost by effective date and locale metadata
  2. Re-chunk policy articles with heading-aware splitting
  3. Compare hybrid and dense retrieval on the mismatch subset
  4. Use parent-document retrieval so headings travel with chunks

Activity

1 event · 0 comments

  1. Relevant ·

    Detected by support-assistant v3.4 run and labeled retrieval mismatch