Skip to content

F-2410Detected Oct 4, 10:29 UTC

My account is billed in pounds. What is the late fee if I pay after the due date?

Found in v3.4 · locale-formatting on locale-formatting · Billing & payments

Instruction-following failuresHighTriagedClaude Sonnet 5.5
Open gt_1044

Model answer vs gold answer

53% of model-answer words appear in the gold answer

Model answer

support-assistant v3.4

If you pay after the due date, Unsupported: a late fee of €8.50 is added after a 10-day grace period.

Gold answer

Human-verified · locale-formatting

There is a 10-day grace period after the due date. After that, a late fee of £7.50 is added to your next bill, because your account is billed in pounds sterling. Outgoing calls are restricted on bills that stay unpaid for 30 days.

  • claimFlagged by the judge as unsupported or contradicted

Retrieved context

bge-m3 · hybrid (BM25 + dense, RRF) · top 3 passed to the generator

2 of 3 retrieved chunks are relevant · gold passage ranked first

  1. 1

    Late payment fees and service restrictions

    Late fee amount · kb-billing-031 · v2026.09

    Relevant
    After the 10-day grace period a late fee of €8.50 (£7.50 for UK-billed accounts) is added.
    Similarity0.86
  2. 2

    Currency, VAT and date formats on your bill

    Date and number formats · kb-billing-016 · v2026.09

    Not relevant
    Bills for accounts registered in Germany, France and Spain show dates as DD.MM.YYYY or DD/MM/YYYY and use a decimal comma.
    Similarity0.71
  3. 3

    Late payment fees and service restrictions

    Restriction and reconnection · kb-billing-031 · v2026.09

    Relevant
    Outgoing services are restricted after 30 days of non-payment.
    Similarity0.69

Judge verdict

LLM judge · Claude Opus 5.5 · rubric v3

0.41

rubric score · pass threshold 0.70

The grace period is correct, but the fee is quoted in euros although the customer said the account is billed in pounds, violating the billing-currency rule. The 30-day restriction is also omitted.

Diagnosis

Instruction-following failures · Customer format rules ignored

Root cause

The account's billing currency is not passed to the generator, so it repeats the first amount in the top-ranked chunk.

Suggested fix

Pass the customer's billing country and currency explicitly to the generator and add a post-generation currency check.

Playbook: instruction-following failures

The answer ignored format, tone, policy or language constraints from the system prompt.

Detection signals

  • Rule check fails: length, language, required closing line
  • Promises that policy reserves for agents, such as guaranteed refunds
  • Judge flags tone or format

Common fixes

  1. Move hard constraints into automated rule checks
  2. Detect the customer's language and enforce it
  3. Maintain a policy-violation phrase list with support leads
  4. Gate prompt versions on the instruction-adherence subset

Activity

2 events · 0 comments

  1. Relevant ·

    Detected by support-assistant v3.4 run and labeled instruction-following failures

  2. Priya Nair ·

    Triaged: billing-currency rule, assigned to self