Extract 40 fields from a 125k-token RFP into JSON
Exact prompt
CHUNK PROMPT (sent once per half, n=1,2): Standard input contains PART {n} OF 2 of a long Request for Proposals (RFP); the other part is processed separately and the results are merged afterwards. For each of the 40 fields below, reply with ONLY a JSON object (no prose, no code fence) mapping each field name to {{"value": ..., "quote": "<verbatim supporting quote, max 25 words>"}}, or to null if this part does not state it. Use YYYY-MM-DD for dates, plain numbers without $ signs or commas for numeric fields, and true/false for booleans. If the document later amends a value (for example in an addendum), report the final amended value. If a field is not stated in the text you were given, use null. Do not use any tools. Fields: - issuing_agency: name of the agency issuing the RFP - rfp_number: RFP/solicitation number - project_title: official project title - procurement_officer_name: name of the single point of contact / procurement officer - procurement_officer_email: procurement officer email - procurement_officer_phone: procurement officer phone - rfp_issue_date: date the RFP was issued (YYYY-MM-DD) - questions_deadline: final deadline for written questions (YYYY-MM-DD) - pre_proposal_conference_date: pre-proposal conference date (YYYY-MM-DD) - pre_proposal_conference_mandatory: is attendance at the pre-proposal conference mandatory (true/false) - proposal_due_date: final proposal due date (YYYY-MM-DD) - proposal_due_time: proposal due time incl. time zone - submission_method: how proposals must be submitted - number_of_hard_copies: number of printed hard copies required (integer, 0 if none) - technical_proposal_page_limit: page limit for the technical proposal (integer) - minimum_font_size_pt: minimum font size in points (number) - proposal_validity_days: days the proposal must remain valid (integer) - contract_term_years: initial contract term in years (integer) - renewal_options_count: number of optional renewal periods (integer) - not_to_exceed_budget_usd: not-to-exceed budget for THIS procurement in USD (number) - bid_bond_percent: bid bond as % of proposed price (number) - performance_bond_percent: performance bond as % of contract value (number) - general_liability_per_occurrence_usd: prime contractor's commercial general liability per-occurrence limit in USD (number) - cyber_liability_usd: cyber/network security liability limit in USD (number) - liquidated_damages_per_day_usd: liquidated damages per calendar day in USD (number) - go_live_date: required production go-live date (YYYY-MM-DD) - uptime_sla_percent: monthly availability SLA in percent (number) - p1_response_time_hours: vendor response time for Priority 1 / critical incidents in hours (number) - data_retention_years: years records must be retained (integer) - eval_weight_technical: evaluation weight (points or %) for technical approach - eval_weight_cost: evaluation weight for cost/price - eval_weight_experience: evaluation weight for experience/qualifications - eval_weight_local_business: evaluation weight for local business preference - min_reference_projects: minimum number of comparable reference projects required (integer) - min_years_in_business: minimum years the proposer must have been in business (integer) - dbe_goal_percent: DBE/disadvantaged business participation goal in % (number) - retainage_percent: retainage withheld from each payment in % (number) - payment_terms_days: days after approved invoice that payment is made (integer) - max_concurrent_users: number of concurrent users the system must support (integer) - asset_records_to_migrate: number of asset records to be migrated (integer) MERGE PROMPT (template; {a},{b} = the two chunk replies): Two partial extractions of the same RFP are below, one per half of the document (part 2 is the later half and includes any attachments/addenda). Each maps field -> {{value, quote}} or null. Merge them into ONE final JSON object with exactly the 40 field names as keys and plain values (no quotes objects). When the parts conflict, use the quotes to decide which value is final (later amendments supersede original values). Reply with ONLY the JSON object. Do not use any tools. PART 1 EXTRACTION: {a} PART 2 EXTRACTION: {b}
Pulling 40 fields out of a roughly 125k-token request-for-proposals document, either in one prompt or in two halves merged afterwards. The document hides three addendum overrides and several near-miss values. It measures accuracy and how the price changes when a single prompt crosses a 100k-token long-context pricing threshold. Original benchmark task written for usecaserank.com (not based on a specific public post). Headless, offline run on a reproducible fixture. Each run's notes say how its cost was measured (estimates are marked).
Success criteria: Field-level accuracy ≥ 95% against a hand-labelled key.
Best model for this task so far: Claude Haiku 5.5 (effort default (no --effort flag) · thinking on) (success, $0.03, 43s). Best AI model for Software & DevOps →
All runs (4)
100% pass rate across 4 eligible runs: 4 pass, 0 partial, 0 fail. Partial and failed runs are listed below with everything else.ⓘ
| Model | Cost | Time | Step-insⓘ | Autonomy | Outcome | Trustⓘ | Date | Compare |
|---|---|---|---|---|---|---|---|---|
| AClaude Sonnet 5.5default · thinking on | $0.51 | 7s | 0 | Fully autonomous | pass | Trust 80% | 2026-10-16 | |
| AClaude Sonnet 5.5default · thinking on | $0.58 | 23s | 0 | Fully autonomous | pass | Trust 80% | 2026-10-16 | |
| AClaude Haiku 5.5default · thinking on | $0.13 | 10s | 0 | Fully autonomous | pass | Trust 80% | 2026-10-16 | |
| AClaude Haiku 5.5default · thinking on | $0.03 | 43s | 0 | Fully autonomous | pass | Trust 80% | 2026-10-16 |
Embed this result
A live badge for this use case. It links back here and updates as new runs are published.
Get new runs and ranking changes weekly
Which models got better or cheaper on real tasks like this one.