Skip to content

Claude Haiku 5.5 (effort default (no --effort flag) · thinking on) on Model a game-ready low-poly 3D prop from a one-line brief, within a tight triangle budget

AClaude Haiku 5.5default · thinking on · Claude Code 2.1.299 (headless -p) · cloud · 2026-10-20 · by @bobsiopener

Trust 80%Fully autonomoussuccess · first try
Estimated cost
$0.02
Wall time
2m
Human step-ins
0
Outcome
Pass

Compare with other models on this task →

Rescored on 2026-10-20. Previous score: 90.0 / 100, outcome: success. See the previous run.

Summary

Exact prompt

# Game-ready low-poly 3D asset from a one-line brief `asset.json` holds a one-line `description` of a game prop plus its art brief: `triangle_budget`, `min_segments` and `parts`. Write `solve.py` (Python 3 standard library only; no Blender or other 3D tools are needed or assumed). `python solve.py input.json answer.json` (first argument unused; read `asset.json` from the current directory) writes `asset.obj` and `answer.json`. Fresh briefs (other props) are generated when scoring. ## Geometry rules (metres, +Y up, ground plane y = 0) 1. Shapes: `box` with `size` [x, y, z]; `cylinder` (`radius`, `height`); `frustum` (`radius_bottom`, `radius_top`, `height`); `cone` (`radius`, `height`, apex at the top centre). Round shapes have their axis along +Y. 2. Placement: a part with `"on": "ground"` has its bottom at y = 0 and its x/z centre at (0, 0) + `offset` [dx, dz] (default [0, 0]). A part `"on": "<name>"` has its bottom at the top (max y) of that part and its x/z centre at the centre of that part's bounding box + `offset`. If the named part is an array, use the bounding box of all its instances. 3. `array` `{"count": 4, "layout": "corners", "inset": s}`: four instances standing on the parent's top, centred at the parent's bounding-box x/z corners moved inward by `s` on both axes, in the order (-x,-z), (+x,-z), (+x,+z), (-x,+z). 4. Each part's bounding box must equal the ideal shape's bounding box (within 1e-4), so round shapes need a ring vertex on each of the +x, -x, +z and -z axes. 5. Round shapes use one ring of N segments per circle, N >= `min_segments` and a multiple of 4. The whole asset must use at most `triangle_budget` triangles (it is tight: spend triangles carefully). 6. Every part is a separate closed, manifold mesh: every edge shared by exactly two of that part's triangles, consistent winding, counter-clockwise when seen from outside (outward normals, positive signed volume), no degenerate or zero-area triangles. Vertices may be shared or duplicated; edges are compared by position. ## asset.obj Wavefront OBJ, `v x y z` and triangle-only `f` lines. One `o <name>` per part; array instances are `o <name>.1` .. `o <name>.4`. No other objects. ## answer.json `{"triangles": total, "parts": {object name: {"triangles": n, "bounds": [[minx, miny, minz], [maxx, maxy, maxz]]}}}`; counts must match asset.obj. ## Scoring Hidden checks (partial credit; all must pass), computed by parsing your OBJ: `obj_parses_named_parts_triangles`, `triangle_budget_and_ring_segments`, `part_bounds_and_stacking`, `closed_manifold_consistent_winding`, `outward_normals_positive_volume`, `answer_json_matches_mesh` on the given brief; `fresh_run_1`, `fresh_run_2` (the first five checks on new briefs). 30-second limit per run.

Getting a mesh that renders is easy; getting every part closed, outward-facing, exactly sized and under a budget with 2 spare triangles needs real geometry reasoning. Original benchmark task written for usecaserank.com (not based on a specific public post). Headless, offline run on a reproducible fixture. Each run's notes say how its cost was measured (estimates are marked). Oct 20, 2026: Scorer mode: file-based. This case's checks read the output file directly by design; no browser is used in scoring. Measured data unchanged.

Success criteria: 1. obj_parses_named_parts_triangles 2. triangle_budget_and_ring_segments 3. part_bounds_and_stacking 4. closed_manifold_consistent_winding 5. outward_normals_positive_volume 6. answer_json_matches_mesh 7. fresh_run_1 8. fresh_run_2

Setup notes: Oct 20, 2026: Scorer mode: file-based. This case's checks read the output file directly by design; no browser is used in scoring. Measured data unchanged. Use Case Scout batch 13 case n02-game-asset-mesh (offline, synthetic fixture). Prepared with batch13/prepare.sh; hidden key/checker kept outside the agent's work dir (temp dir during the run). Prompt = TASK.md verbatim. Scored with the case's scorer: `python score.py WORKDIR KEYDIR` (python3, run from the case dir). Tools: Read/Write/Edit/Glob/Grep/Bash, web disabled. Single run, no retries. 1-min load average at scoring 0.18 (8 cores). asset.obj preview rendered offline with matplotlib (flat shading, painter's sort): PNG 1280x720, MP4 6 s 1280x720 24 fps H.264 turntable. Claude Code's own reported cost for claude-haiku-5.5: $0.020014. Score: 8/8. Cost is an ESTIMATE: measured tokens x pricing.json list price (Claude Pro subscription, nothing billed); input_tokens include cache reads and writes. Single run.

Estimated cost List-price estimate; not actually billed.

Output

Metrics

Estimated cost
$0.02
Wall time
2m
Human step-ins
0
Turns
7
Model usage
ModelInput tokOutput tokCost
AClaude Haiku 5.5anthropic/claude-haiku-5.5effort default (no --effort flag) · thinking on232,36922,884$0.02

Reproduce this run

  1. Use the use case prompt on the use case page.
  2. Same setup: Claude Haiku 5.5 (anthropic/claude-haiku-5.5, effort default (no --effort flag) · thinking on) via Claude Code 2.1.299 (headless -p), cloud.
  3. It counts as a success only if it meets every success criterion: 1. obj_parses_named_parts_triangles 2. triangle_budget_and_ring_segments 3. part_bounds_and_stacking 4. closed_manifold_consistent_winding 5. outward_normals_positive_volume 6. answer_json_matches_mesh 7. fresh_run_1 8. fresh_run_2
  4. Log model, tokens, wall time, human step-ins and the evidence log, then submit it as a reproduction. A different model? Submit a head-to-head rerun.

How runs are scored: Methodology.

Was this useful?

⇄ Rerun on another model

Automated checks

Trust 80% comes from these checks (how Trust % works).

Evidence log
[
 {
  "duration_api_ms": 97918,
  "stop_reason": "end_turn",
  "session_id": "[REDACTED]",
  "total_cost_usd": 0.020013770000000004,
  "usage": {
   "input_tokens": 14,
   "cache_creation_input_tokens": 32878,
   "cache_read_input_tokens": 199477,
   "output_tokens": 22884,
   "output_tokens_details": {
    "thinking_tokens": 16031
   },
   "server_tool_use": {
    "web_search_requests": 0,
    "web_fetch_requests": 0
   },
   "service_tier": "standard",
   "cache_creation": {
    "ephemeral_1h_input_tokens": 32878,
    "ephemeral_5m_input_tokens": 0
   },
   "inference_geo": "global",
   "iterations": [
    {
     "input_tokens": 2,
     "output_tokens": 1131,
     "cache_read_input_tokens": 43451,
     "cache_creation_input_tokens": 1966,
     "cache_creation": {
      "ephemeral_5m_input_tokens": 0,
      "ephemeral_1h_input_tokens": 1966
     },
     "type": "message"
    }
   ],
   "speed": "standard",
   "fallback_credit": null
  },
  "modelUsage": {
   "claude-haiku-5-5": {
    "inputTokens": 14,
    "outputTokens": 22884,
    "cacheReadInputTokens": 199477,
    "cacheCreationInputTokens": 32878,
    "webSearchRequests": 0,
    "costUSD": 0.020013770000000004,
    "contextWindow": 1000000,
    "maxOutputTokens": 128000,
    "thinkingTokens": 16031,
    "canonicalModel": "claude-haiku-5-5",
    "provider": "firstParty",
    "costBasis": "list"
   }
  },
  "permission_denials": [],
  "terminal_reason": "completed",
  "fast_mode_state": "off",
  "fast_mode_disabled_reason": "sdk_opt_in_required",
  "subagent_stats": {
   "spawned": 0,
   "requested": {
    "background": 0,
    "foreground": 0,
    "unset": 0
   },
   "started_in_background": 0,
   "max_depth": 0,
   "spawned_by_subagents": 0,
   "completed": 0,
   "failed": 0,
   "killed": {
    "parent": 0,
    "user": 0,
    "system": 0
   },
   "refused": {
    "depth_limit": 0,
    "concurrency_limit": 0,
    "budget": 0
   },
   "by_type": {}
  },
  "safety_stops": 0,
  "is_error": false,
  "num_turns": 7,
  "subtype": "success",
  "api_error_status": null,
  "result": "`solve.py` is written and runs in well under a second. On the given brief it produces 90 triangles against a budget of 92, and an independent checker I wrote passes every rule for all four parts.\n\n**What it does**\n- Each round shape uses N = the smallest multiple of 4 that is at least `min_segments`, so N = 12 here.\n- Triangle counts are the minimum for closed meshes: box 12, cylinder/frustum 4N\u22124, cone 2N\u22122. Caps are fan-triangulated convex rings with no centre vertex, which is what keeps the budget at 90.\n- Placement follows the brief's rules: `ground`, named parents (by bounding box), and corner arrays.\n- Output is `asset.obj` in the current directory and `answer.json` at the path given as the second argument. The bounds in `answer.json` come from the same rounded vertices written to the OBJ.\n\n**Checks run**\n- Given brief: base 12, post 44, lamp 12, cap 22 triangles. Bounds match the expected stacking, every edge pairs with its opposite, and every part has positive volume. The post and cap volumes agree with hand calculation.\n- A test brief in `/tmp/t1` with a corner array of four legs, a frustum, a cone on an array instance, and an inverted cone (zero bottom radius) with 13 requested segments, which rounds up to 16. All parts pass the same checks, and the array instances land on the correct corners.\n\n**Assumptions to be aware of**\n- Array parts are assumed to be written as a part with an `\"array\"` key alongside `\"on\"` naming the parent. The brief doesn't pin down that JSON layout, so a fresh brief in a different shape would need this adjusted.\n- A part's `offset` is applied to each array instance's centre.\n- Instances are keyed as `name.1` to `name.4` in `answer.json`, with no entry for the bare array name.\n- If a brief's budget can't be met at the required segment count, the script still writes the files and prints a warning to stderr.\n\nThis is not a git repository, so I did not commit anything.",
  "ttft_ms": 1888,
  "type": "result",
  "duration_ms": 98630,
  "uuid": "[REDACTED]",
  "ttft_stream_ms": 1783,
  "time_to_request_ms": 272,
  "first_content_frame_ms": 1784,
  "queued_turn_count": 0,
  "result_index": 0
 }
]

Get new runs and ranking changes weekly