Claude Sonnet 5.5 (effort default (no --effort flag) · thinking on) on Refactor nested legacy pricing code + add a bulk-discount rule with exact behaviour preservation

AClaude Sonnet 5.5default · thinking on · Claude Code 2.1.299 (headless -p) · cloud · 2026-10-16 · by @bobsiopener

Trust 80%Fully autonomoussuccess · first tryCompare

Estimated cost: $0.11
Wall time: 49s
Human step-ins: 0
Outcome: Pass

Compare with other models on this task →

Exact prompt

# Task: refactor the legacy pricing engine and add one rule `pricing.py` has a deeply nested `price(order)` function. 1. Refactor it so it is readable: no function may have nesting depth greater than 3 (if/for/while/with/try blocks), and the category/tier discounts must live in a data table, not if-chains. Behaviour must stay IDENTICAL for every order (outputs are compared to the old version on hundreds of random orders, rounded to cents). 2. Add a new rule: **bulk discount** — any line item with qty >= 10 gets an extra 5% off that line, applied after the tier discount and before coupons. The bulk discount does NOT combine with coupon "SAVE20" (if SAVE20 is present, no bulk discount anywhere). Keep the signature `price(order) -> float`. Add tests in `test_pricing.py`. No network, no new dependencies. Order shape: {"customer":{"tier":"gold|silver|none"},"region":"EU|US|ROW","coupon":null|"SAVE10"|"SAVE20"|"FREESHIP","items":[{"category":"books|electronics|toys|garden","unit":float,"qty":int}]}

Refactor a deeply nested legacy pricing function into readable code with a discount table, then add a bulk-discount rule. Behaviour must stay identical to the cent on 800 random orders, including several odd regional tax and shipping quirks that a careless refactor would 'fix'. Nesting depth and the data-table requirement are checked from the code's syntax tree. Original benchmark task written for usecaserank.com (not based on a specific public post). Headless, offline run on a reproducible fixture. Each run's notes say how its cost was measured (estimates are marked).

Success criteria: 1. price(order) is called on 800 seeded random orders (tiers, regions, coupons, quantities incl. >=10) and must match the reference implementation within half a cent on every order (0 mismatches); exceptions count as mismatches. 2. Maximum nesting depth of if/for/while/with/try inside any function must be <= 3 (elif chains count as one level), measured with the AST. 3. pricing.py must contain a dict literal mentioning 'gold' (tier discounts kept in a data table).

Setup notes: Use Case Scout batch 3 case c02-refactor-pricing (offline fixture). Prepared with batch3/prepare.sh; hidden key/tests kept outside the agent's work dir. Prompt = TASK.md verbatim. Scored with the scout's scorer: `python score.py WORKDIR`. Tools: Read/Write/Edit/Glob/Grep/Bash, web disabled. Single run, no retries. Claude Code's own reported cost for claude-sonnet-5.5: $0.108572. Score: 1/1 checks. Cost is an ESTIMATE: measured tokens x pricing.json list price (Claude Pro subscription, nothing billed); input_tokens include cache reads and writes. Single run.

Estimated cost List-price estimate; not actually billed.

Metrics

Estimated cost: $0.11
Wall time: 49s
Human step-ins: 0
Turns: 5

Model usage
Model	Input tok	Output tok	Cost
AClaude Sonnet 5.5anthropic/claude-sonnet-5.5effort default (no --effort flag) · thinking on	85,621	5,330	$0.11

Reproduce this run

Use the use case prompt on the use case page.
Same setup: Claude Sonnet 5.5 (anthropic/claude-sonnet-5.5, effort default (no --effort flag) · thinking on) via Claude Code 2.1.299 (headless -p), cloud.
It counts as a success only if it meets every success criterion: 1. price(order) is called on 800 seeded random orders (tiers, regions, coupons, quantities incl. >=10) and must match the reference implementation within half a cent on every order (0 mismatches); exceptions count as mismatches. 2. Maximum nesting depth of if/for/while/with/try inside any function must be <= 3 (elif chains count as one level), measured with the AST. 3. pricing.py must contain a dict literal mentioning 'gold' (tier discounts kept in a data table).
Log model, tokens, wall time, human step-ins and the evidence log, then submit it as a reproduction. A different model? Submit a head-to-head rerun.

How runs are scored: Methodology.

Was this useful?

⇄ Rerun on another model

Automated checks

Trust 80% comes from these checks (how Trust % works).

Run date is not in the future : passed
Model is in the dated pricing table : passed
Cost recomputed from tokens matches : passed
Cached tokens are consistent : passed
Output throughput is plausible : passed
Autonomy matches step-ins : passed
Step-ins do not exceed turns : passed
Retries agree with first-try result : passed
Evidence log SHA-256 matches : passed
Evidence log is substantial : passed

Evidence log

[
 {
  "duration_api_ms": 48186,
  "stop_reason": "end_turn",
  "session_id": "[REDACTED]",
  "total_cost_usd": 0.10857199999999999,
  "usage": {
   "input_tokens": 8,
   "cache_creation_input_tokens": 11973,
   "cache_read_input_tokens": 73640,
   "output_tokens": 5330,
   "output_tokens_details": {
    "thinking_tokens": 478
   },
   "server_tool_use": {
    "web_search_requests": 0,
    "web_fetch_requests": 0
   },
   "service_tier": "standard",
   "cache_creation": {
    "ephemeral_1h_input_tokens": 11973,
    "ephemeral_5m_input_tokens": 0
   },
   "inference_geo": "global",
   "iterations": [
    {
     "input_tokens": 2,
     "output_tokens": 514,
     "cache_read_input_tokens": 23464,
     "cache_creation_input_tokens": 1048,
     "cache_creation": {
      "ephemeral_5m_input_tokens": 0,
      "ephemeral_1h_input_tokens": 1048
     },
     "type": "message"
    }
   ],
   "speed": "standard",
   "fallback_credit": null
  },
  "modelUsage": {
   "claude-sonnet-5-5": {
    "inputTokens": 8,
    "outputTokens": 5330,
    "cacheReadInputTokens": 73640,
    "cacheCreationInputTokens": 11973,
    "webSearchRequests": 0,
    "costUSD": 0.10857199999999999,
    "contextWindow": 1000000,
    "maxOutputTokens": 128000,
    "thinkingTokens": 478,
    "canonicalModel": "claude-sonnet-5-5",
    "provider": "firstParty",
    "costBasis": "list"
   }
  },
  "permission_denials": [],
  "terminal_reason": "completed",
  "fast_mode_state": "off",
  "fast_mode_disabled_reason": "sdk_opt_in_required",
  "subagent_stats": {
   "spawned": 0,
   "requested": {
    "background": 0,
    "foreground": 0,
    "unset": 0
   },
   "started_in_background": 0,
   "max_depth": 0,
   "spawned_by_subagents": 0,
   "completed": 0,
   "failed": 0,
   "killed": {
    "parent": 0,
    "user": 0,
    "system": 0
   },
   "refused": {
    "depth_limit": 0,
    "concurrency_limit": 0,
    "budget": 0
   },
   "by_type": {}
  },
  "safety_stops": 0,
  "is_error": false,
  "num_turns": 5,
  "subtype": "success",
  "api_error_status": null,
  "result": "I refactored `pricing.py` and added the bulk discount rule. All 13 tests in `test_pricing.py` pass, and the deepest nesting in any function is 2.\n\n**Refactor**\n- Tier discounts now live in a `TIER_DISCOUNTS` table. Books and electronics have their own rows; toys, garden and any other category use `DEFAULT_CATEGORY`. A tier that isn't listed pays full price.\n- Coupons are in a `COUPON_RULES` table, and shipping and tax are in a separate helper.\n- The arithmetic keeps the same order of operations as the old code, including US tax being charged on the total without shipping. I kept it that way so results stay identical to the cent.\n\n**Bulk discount**\n- A line with qty >= 10 gets an extra 5% off after the tier discount and before coupons.\n- If the coupon is SAVE20, no line gets the bulk discount. FREESHIP and SAVE10 keep it.\n\n**Tests**\n- There are cases for each tier and category, the bulk boundary at qty 9 and 10, bulk combined with a tier discount, the SAVE20 exclusion, coupon thresholds, and EU and US shipping and tax.\n- The test file also holds a verbatim copy of the old `price` function and checks 1000 random orders against it. Those orders use qty 1\u20139, so the bulk rule never fires, and every result matched.\n- Orders with qty >= 10 aren't checked against the old version, because the new rule changes those prices on purpose. Only the hand-written cases cover them.",
  "ttft_ms": 1431,
  "type": "result",
  "duration_ms": 48653,
  "uuid": "[REDACTED]",
  "ttft_stream_ms": 1047,
  "time_to_request_ms": 222,
  "first_content_frame_ms": 1047,
  "queued_turn_count": 0,
  "result_index": 0
 }
]