Skip to content

Find vendor contracts whose non-renewal notice deadline is near, through amendments, month-end dates and business-day notice periods

Exact prompt

# Which vendor contracts need a non-renewal decision soon? Inputs: `contracts.csv`, `amendments.csv` and `calendar.json` (today's date and public holidays). Today is the date in `calendar.json`. For every contract that auto-renews, find the current term's end date and the last day on which a non-renewal notice can still be given. - A term of N months starting on date S ends the day before S plus N months. If that month is shorter, S plus N months falls on its last day (so 31 January plus 1 month is 28 or 29 February). - A contract renews for its renewal term each time a term ends, so the current term is the one that contains today. Each renewal term starts on the day after the previous term ends. - Notice must arrive at least the notice period before the term ends: the deadline is the end date minus the notice period. Periods are in days, months (same month rule as above) or business days (Monday to Friday, not holidays; count back from the end date, not counting the end date itself). - Only signed amendments change a contract; a later signed amendment to the same field replaces an earlier one, by effective date. - If the current term's deadline has already passed, the contract will renew: report the next term instead (its end date and deadline). Write `out/renewals.json`: `{"contracts": {"<contract_id>": {"term_end": "YYYY-MM-DD", "notice_deadline": "YYYY-MM-DD"}, ...}, "decide_within_30_days": [<ids whose reported deadline is from today to today + 30 days inclusive, sorted>], "missed_current_term": [<ids whose current-term deadline has passed, sorted>]}`. Leave contracts that don't auto-renew out of `contracts`. ## How this run works - Work on your own: nobody will answer questions in this task, and writing to `ask/` counts against the run. - Write only under `out/`. Create `out/` if it is missing. Inputs are synthetic; don't fetch anything from the network. - Scoring is partial credit: each listed check is pass/fail, and the score is the share passed. Numbers are compared to within 0.005 unless the task says otherwise.

Work out the current term end and notice deadline for 18-24 auto-renewing contracts as of a fixed date. Notice periods mix days, months and business days with holidays, several start on the 29th-31st, amendments include drafts that must not apply, and some deadlines have already passed so the next term is the one to report. Thirty-day months, applying drafts, or treating business days as calendar days each change the answer. Original benchmark task written for usecaserank.com (not based on a specific public post). Headless, offline run on a reproducible fixture. Each run's notes say how its cost was measured (estimates are marked). Folder-restore note: this run's output files under work/out/ were lost in a 1:22 AM ET restore on Oct 21, 2026. The trace, step-in exchanges and output-hash list were checked against the recorded SHA-256 values and match; the hashes of the individual output files inside that list can no longer be recomputed. Measured data unchanged.

Success criteria: deadlines decide_soon missed amendments

Best model for this task so far: Claude Haiku 5.5 (effort default (no --effort flag) · thinking on) (success, $0.0073, 38s).

Run this on another model

All runs (3)

100% pass rate across 3 eligible runs: 3 pass, 0 partial, 0 fail. Partial and failed runs are listed below with everything else.ⓘ

Runs for this use case
ModelCostTimeStep-insⓘAutonomyOutcomeTrustⓘDateCompare
AClaude Opus 5.5default · thinking on$0.1327s0Fully autonomouspassTrust 80%2026-10-21
AClaude Haiku 5.5default · thinking on$0.007338s0Fully autonomouspassTrust 80%2026-10-21
AClaude Sonnet 5.5default · thinking on$0.0718s0Fully autonomouspassTrust 80%2026-10-21

Embed this result

A live badge for this use case. It links back here and updates as new runs are published.

UseCaseRank badge for Find vendor contracts whose non-renewal notice deadline is near, through amendments, month-end dates and business-day notice periods

Get new runs and ranking changes weekly

Which models got better or cheaper on real tasks like this one.