Skip to content

Fix cache invalidation in an incremental static-site builder (transitive partials, config, deletions, same-second edits)

Exact prompt

# Task: fix the incremental builder's cache invalidation `minibuild.py` builds `site/out/*.html` from `site/src/*.md` and caches what it built in `site/.cache.json`. Users report stale pages: editing a partial (even one included indirectly) doesn't update pages, changing `config.json` doesn't either, quick successive edits within the same second are missed, deleted pages leave their HTML behind, and deleting or hand-editing an output file isn't repaired. Fix the caching in `minibuild.py` so that after **any** sequence of changes, `python3 minibuild.py site` leaves `out/` exactly equal to a clean full build (same files, same bytes, no extra files), while still being incremental: a page is rewritten only if its output would change or its output file is missing/modified. Pages unaffected by a change must not be rewritten at all (the grader checks output file timestamps). Note: the grader resets the timestamps of all output files to an old value before each build, so detect missing/modified outputs by their content (e.g. a hash), not by their timestamps. Only config keys a page actually uses (`site_title` is used by every page; `{{var KEY}}` by the pages containing it, directly or through partials) should cause it to rebuild. Keep the rendering output and the command line unchanged; `build()` should still return the list of rebuilt page names. Standard library only. No network.

Over-invalidating fails the minimal-rebuild checks; under-invalidating fails correctness. Mtime-based caches miss same-second edits, dependency tracking must be transitive and config-key aware, and outputs must be repaired and pruned. Original benchmark task written for usecaserank.com (not based on a specific public post). Headless, offline run on a reproducible fixture. Each run's notes say how its cost was measured (estimates are marked).

Success criteria: 1. clean_build_correct: a first build produces out/ identical to a clean build by the original renderer. 2. For each of 11 change scenarios (indirect partial edit, used config var, site_title, same-mtime source edit, deleted page, deleted output, hand-edited output, new page, no change, touch with identical content, unrelated config key) <scenario>_correct: out/ is byte-identical to a clean full build with no extra files. 3. For each of those 11 scenarios <scenario>_minimal: exactly the expected set of output files is rewritten (others keep their timestamps).

Best model for this task so far: Claude Haiku 5.5 (effort default (no --effort flag) · thinking on) (success, $0.0095, 44s).

Run this on another model

All runs (4)

100% pass rate across 4 eligible runs: 4 pass, 0 partial, 0 fail. Partial and failed runs are listed below with everything else.ⓘ

Runs for this use case
ModelCostTimeStep-insⓘAutonomyOutcomeTrustⓘDateCompare
OGPT-6.1 Soldefault · thinking on$0.071m0Fully autonomouspassTrust 80%2026-10-22
AClaude Haiku 5.5default · thinking on$0.009544s0Fully autonomouspassTrust 80%2026-10-17
AClaude Opus 5.5default · thinking on$0.2355s0Fully autonomouspassTrust 80%2026-10-17
AClaude Sonnet 5.5default · thinking on$0.0628s0Fully autonomouspassTrust 80%2026-10-17

Embed this result

A live badge for this use case. It links back here and updates as new runs are published.

UseCaseRank badge for Fix cache invalidation in an incremental static-site builder (transitive partials, config, deletions, same-second edits)

Get new runs and ranking changes weekly

Which models got better or cheaper on real tasks like this one.