Skip to content

Read a synthetic annual-report excerpt and compute financial ratios

Exact prompt

# Read an annual-report excerpt and compute financial ratios `annual_report_excerpt.md` is an excerpt from a **fictional** company's annual report (statements plus notes). Read it and write `ratios.json` with the 16 values below. You may compute by hand or with a script (Python 3 standard library only). Use only this document. Watch the units and the notes. ## Rules 1. Dollar amounts in the statements are in thousands unless a line says otherwise. Notes can state amounts in millions; convert them to thousands. 2. Where a note restates a prior-year amount, use the restated amount everywhere that amount (or a total that includes it) is used. "Average X" = (FY2025 year-end X + FY2024 year-end X) / 2. 3. Total debt = current portion of long-term debt + long-term debt (net of current portion) + finance lease obligations (Note 7). Operating lease liabilities are not debt. 4. Free cash flow = net cash provided by operating activities - purchases of property and equipment - capitalized internal-use software. 5. Definitions (FY2025 unless stated): - `current_ratio` = total current assets / total current liabilities - `quick_ratio` = (cash and equivalents + short-term investments + accounts receivable) / total current liabilities - `gross_margin`, `operating_margin`, `net_margin` = gross profit, operating income, net income / net sales - `effective_tax_rate` = provision for income taxes / income before income taxes - `roe` = net income / average stockholders' equity - `total_debt_thousands` (rule 3, at Dec 31, 2025), `debt_to_equity` = total debt / Dec 31, 2025 equity - `interest_coverage` = operating income / interest expense - `fcf_thousands` (rule 4), `fcf_margin` = free cash flow / net sales - `inventory_turnover` = cost of sales / average inventory - `dso_days` = average accounts receivable / net sales * 365 - `revenue_growth` = FY2025 net sales / FY2024 net sales - 1 - `eps_diluted` = net income / diluted weighted-average shares (dollars per share) 6. Ratios and margins are plain fractions (0.25, not 25%). ## Output `ratios.json`: one JSON object with exactly these keys: `current_ratio`, `quick_ratio`, `gross_margin`, `operating_margin`, `net_margin`, `effective_tax_rate`, `roe`, `total_debt_thousands`, `debt_to_equity`, `interest_coverage`, `fcf_thousands`, `fcf_margin`, `inventory_turnover`, `dso_days`, `revenue_growth`, `eps_diluted`. (That is 16 keys: the 15 ratios plus total debt.) ## Scoring Hidden checks (partial credit = fraction passed; all must pass). Tolerances: fractions and ratios within 0.0005; `total_debt_thousands` and `fcf_thousands` within 0.5; `dso_days` within 0.05; `eps_diluted` within 0.005. - `liquidity`: current_ratio, quick_ratio - `margins_and_tax`: gross_margin, operating_margin, net_margin, effective_tax_rate - `return_on_equity`: roe (needs the restated equity) - `debt_and_coverage`: total_debt_thousands, debt_to_equity, interest_coverage (needs the note in millions) - `free_cash_flow`: fcf_thousands, fcf_margin - `efficiency`: inventory_turnover (needs the restated inventory), dso_days - `growth_and_eps`: revenue_growth, eps_diluted

A restated prior-year inventory and equity in a note, finance-lease debt stated in millions inside a note, a non-GAAP distractor, and averages that need the restated column. Original benchmark task written for usecaserank.com (not based on a specific public post). Headless, offline run on a reproducible fixture. Each run's notes say how its cost was measured (estimates are marked).

These are synthetic test cases for comparing AI models, not financial advice.

Success criteria: 1. liquidity 2. margins_and_tax 3. return_on_equity 4. debt_and_coverage 5. free_cash_flow 6. efficiency 7. growth_and_eps

Best model for this task so far: Claude Haiku 5.5 (effort default (no --effort flag) · thinking on) (success, $0.0041, 12s).

Run this on another model

All runs (13)

85% pass rate across 13 eligible runs: 11 pass, 1 partial, 1 fail. Partial and failed runs are listed below with everything else.ⓘ

Runs for this use case
ModelCostTimeStep-insⓘAutonomyOutcomeTrustⓘDateCompare
?MiniMax M2.5 (Venice)thinking on$0.00612m0Fully autonomouspassTrust 80%2026-10-26
ZGLM 4.7 (Venice)thinking on$0.023m0Fully autonomouspassTrust 80%2026-10-25
MMistral Small 3.2 24B Instruct (Venice)thinking on$0.001140s0Fully autonomouspartialTrust 80%2026-10-25
KKimi K3.1 Pro (Venice)thinking on$0.0428s0Fully autonomouspassTrust 80%2026-10-25
GGemma 4 31B Instruct (Venice)thinking on$0.00663m0Fully autonomouspassTrust 80%2026-10-24
DDeepSeek V3.2 (Venice)thinking on$0.00822m0Fully autonomouspassTrust 80%2026-10-24
QQwen3 Coder 480B Turbo (Venice)thinking on$0.003110s0Fully autonomousfailTrust 80%2026-10-23
DDeepSeek V4.1 Flash (Venice)thinking on$0.004214s0Fully autonomouspassTrust 80%2026-10-23
OGPT-6.1 Soldefault · thinking off$0.0217s0Fully autonomouspassTrust 80%2026-10-22
KKimi K2.5 (Venice)thinking on$0.021m0Fully autonomouspassTrust 80%2026-10-22
AClaude Haiku 5.5default · thinking on$0.004112s0Fully autonomouspassTrust 80%2026-10-20
AClaude Opus 5.5default · thinking off$0.1112s0Fully autonomouspassTrust 80%2026-10-20
AClaude Sonnet 5.5default · thinking off$0.059s0Fully autonomouspassTrust 80%2026-10-20

Embed this result

A live badge for this use case. It links back here and updates as new runs are published.

UseCaseRank badge for Read a synthetic annual-report excerpt and compute financial ratios

Get new runs and ranking changes weekly

Which models got better or cheaper on real tasks like this one.