Skip to content

F-2379Detected Oct 6, 17:22 UTC

Give me the steps as a numbered list, please: how do I reset my network settings on Android?

Found in v3.4 · GPT-5 comparison on support-golden-v4 · Device troubleshooting

Instruction-following failuresMediumOpenGPT-5

Model answer vs gold answer

89% of model-answer words appear in the gold answer

Model answer

v3.4 · GPT-5

Unsupported: Open Settings > System > Reset options > Reset Wi-Fi, mobile & Bluetooth, then confirm. Your phone will restart.

Gold answer

Human-verified · support-golden-v4

1. Open Settings > System > Reset options. 2. Tap Reset Wi-Fi, mobile & Bluetooth. 3. Confirm the reset. Your phone will restart.

  • claimFlagged by the judge as unsupported or contradicted

Retrieved context

bge-m3 · hybrid (BM25 + dense, RRF) · top 3 passed to the generator

1 of 3 retrieved chunks are relevant · gold passage ranked first

  1. 1

    No mobile data after a software update

    Reset network settings · kb-device-102 · v2026.09

    Relevant
    Android: Settings > System > Reset options > Reset Wi-Fi, mobile & Bluetooth.
    Similarity0.89
  2. 2

    Calls drop or will not connect

    SIM reseat · kb-device-124 · v2026.09

    Not relevant
    Remove and reinsert the SIM.
    Similarity0.70
  3. 3

    APN settings for Android and iOS

    Restoring defaults · kb-device-131 · v2026.09

    Not relevant
    Restore default APN settings.
    Similarity0.68

Judge verdict

LLM judge · Claude Opus 5.5 · rubric v3

0.40

rubric score · pass threshold 0.70

Steps are correct but the explicit request for a numbered list was ignored.

Diagnosis

Instruction-following failures · Customer format rules ignored

Root cause

Formatting instruction in the user turn not enforced.

Suggested fix

Same as F-2410: enforce explicit format requests and the customer's regional conventions with a post-generation rule check.

Playbook: instruction-following failures

The answer ignored format, tone, policy or language constraints from the system prompt.

Detection signals

  • Rule check fails: length, language, required closing line
  • Promises that policy reserves for agents, such as guaranteed refunds
  • Judge flags tone or format

Common fixes

  1. Move hard constraints into automated rule checks
  2. Detect the customer's language and enforce it
  3. Maintain a policy-violation phrase list with support leads
  4. Gate prompt versions on the instruction-adherence subset

Activity

1 event · 0 comments

  1. Relevant ·

    Detected by v3.4 · GPT-5 run and labeled instruction-following failures