Skip to content

Run store-operations tasks on a mock e-commerce store

Exact prompt

You are a store-operations agent on a sandboxed mock store. Complete every task in TASKS.md using the files in ./store. Only modify store files where a task says to.

Task prompt given to the agent (headless, offline, synthetic fixture inputs): You are a store-operations agent on a sandboxed mock store. Complete every task in TASKS.md using the files in ./store. Only modify store files where a task says to. Inspired by https://x.com/dee_naliaks/status/2097947201066729536 (unverified; not checked with the author). This is an independent re-run on a reproducible synthetic task; it does not report or reproduce the original post's numbers. Cost shown is a list-price API-equivalent estimate (run on a Claude Pro subscription, not billed).

Success criteria: All automated rubric checks pass: t1 out-of-stock count = 2; t2 = ['P007']; t3 O100 total = 164.70; t4 O102 total = 99.00; t5 allowed and O101 cancelled; t6 not allowed and O100 untouched; t7 restocked P002,P009 to 25; t8 = ana@example.com; no other product changes.

Best model for this task so far: DeepSeek V4.1 Flash (Venice) (success, $0.0042, 52s). Best AI model for Sales, Marketing & Support →

Run this on another model

All runs (7)

100% pass rate across 7 eligible runs: 7 pass, 0 partial, 0 fail. Partial and failed runs are listed below with everything else.ⓘ

Runs for this use case
ModelCostTimeStep-insⓘAutonomyOutcomeTrustⓘDateCompare
AClaude Haiku 5.5default · thinking on$0.004417s0Fully autonomouspassTrust 80%2026-10-16
XGrok 4.3 (Venice)$0.008714s0Fully autonomouspassTrust 80%2026-10-15
QQwen3 Coder 480B Turbo (Venice)$0.005121s0Fully autonomouspassTrust 80%2026-10-15
DDeepSeek V4.1 Flash (Venice)$0.004252s0Fully autonomouspassTrust 80%2026-10-14
GGemma 4 31B Instruct (Venice)$0.00503m0Fully autonomouspassTrust 80%2026-10-14
AClaude Opus 5.5$0.1014s0Fully autonomouspassTrust 80%2026-10-14
AClaude Sonnet 5.5$0.0512s0Fully autonomouspassTrust 80%2026-10-14

Embed this result

A live badge for this use case. It links back here and updates as new runs are published.

UseCaseRank badge for Run store-operations tasks on a mock e-commerce store

Get new runs and ranking changes weekly

Which models got better or cheaper on real tasks like this one.