Claude Sonnet 5.5 on Reverse engineer a security patch for an exploited CVE
AClaude Sonnet 5.5 · Claude Code 2.1.299 (headless -p) · cloud · 2026-10-14 · by @bobsiopener
- Estimated cost
- $0.06
- Wall time
- 17s
- Human step-ins
- 0
- Outcome
- Pass
Compare with other models on this task →
Summary
Exact prompt
patch.diff is a security fix from a widely used TLS library for a vulnerability that was exploited in the wild. Analyze only the diff (no web). Write ANALYSIS.md covering: (1) the vulnerability class, (2) the exact function and the line(s) that were dangerous before the patch, (3) what an attacker could send and what they would get back, (4) why the added checks fix it, (5) the CVE id if you can identify it. Do not write exploit code.
Task prompt given to the agent (headless, offline, synthetic fixture inputs): patch.diff is a security fix from a widely used TLS library for a vulnerability that was exploited in the wild. Analyze only the diff (no web). Write ANALYSIS.md covering: (1) the vulnerability class, (2) the exact function and the line(s) that were dangerous before the patch, (3) what an attacker could send and what they would get back, (4) why the added checks fix it, (5) the CVE id if you can identify it. Do not write exploit code. Inspired by https://x.com/KevTheHermit/status/2094160406935638494 (unverified; not checked with the author). This is an independent re-run on a reproducible synthetic task; it does not report or reproduce the original post's numbers. Cost shown is a list-price API-equivalent estimate (run on a Claude Pro subscription, not billed).
Success criteria: All automated rubric checks pass: identifies buffer over-read / out-of-bounds read; names tls1_process_heartbeat; points at unchecked payload length + memcpy; explains memory disclosure (up to ~64KB of process memory); explains length checks vs rrec.length; identifies CVE-2014-0160 / Heartbleed; no exploit code (no socket/send code blocks).
Setup notes: Cost is an ESTIMATE: Claude Code's list-price API-equivalent figure. The run used a Claude Pro subscription, so nothing was billed. input_tokens = all input incl. cache-creation and cache-read tokens; cached_tokens = cache reads (subset of input). Claude Code's cost bills cache writes at the 1h cache-write premium, so it can differ from a flat input-price recompute. Web disabled; tools Read/Write/Edit/Glob/Grep/Bash; single run; scored automatically (7/7 checks).
Estimated cost List-price estimate; not actually billed.
Metrics
- Estimated cost
- $0.06
- Wall time
- 17s
- Human step-ins
- 0
- Turns
- 3
| Model | Input tok | Output tok | Cost |
|---|---|---|---|
| AClaude Sonnet 5.5anthropic/claude-sonnet-5.5 | 56,781 | 1,983 | $0.06 |
Reproduce this run
- Use the use case prompt on the use case page.
- Same setup: Claude Sonnet 5.5 (anthropic/claude-sonnet-5.5) via
Claude Code 2.1.299 (headless -p), cloud. - It counts as a success only if it meets every success criterion: All automated rubric checks pass: identifies buffer over-read / out-of-bounds read; names tls1_process_heartbeat; points at unchecked payload length + memcpy; explains memory disclosure (up to ~64KB of process memory); explains length checks vs rrec.length; identifies CVE-2014-0160 / Heartbleed; no exploit code (no socket/send code blocks).
- Log model, tokens, wall time, human step-ins and the evidence log, then submit it as a reproduction. A different model? Submit a head-to-head rerun.
How runs are scored: Methodology.
Was this useful?
Automated checks
Trust 80% comes from these checks (how Trust % works).
- Run date is not in the future : passed
- Model is in the dated pricing table : passed
- Cost recomputed from tokens matches : passed
- Cached tokens are consistent : passed
- Output throughput is plausible : passed
- Autonomy matches step-ins : passed
- Step-ins do not exceed turns : passed
- Retries agree with first-try result : passed
- Evidence log SHA-256 matches : passed
- Evidence log is substantial : passed
Evidence log
{
"duration_api_ms": 16078,
"stop_reason": "end_turn",
"session_id": "[REDACTED]",
"total_cost_usd": 0.0551556,
"usage": {
"input_tokens": 6,
"cache_creation_input_tokens": 7599,
"cache_read_input_tokens": 49176,
"output_tokens": 1983,
"output_tokens_details": {
"thinking_tokens": 0
},
"server_tool_use": {
"web_search_requests": 0,
"web_fetch_requests": 0
},
"service_tier": "standard",
"cache_creation": {
"ephemeral_1h_input_tokens": 7599,
"ephemeral_5m_input_tokens": 0
},
"inference_geo": "global",
"iterations": [
{
"input_tokens": 2,
"output_tokens": 506,
"cache_read_input_tokens": 18652,
"cache_creation_input_tokens": 1486,
"cache_creation": {
"ephemeral_5m_input_tokens": 0,
"ephemeral_1h_input_tokens": 1486
},
"type": "message"
}
],
"speed": "standard",
"fallback_credit": null
},
"modelUsage": {
"claude-sonnet-5-5": {
"inputTokens": 6,
"outputTokens": 1983,
"cacheReadInputTokens": 49176,
"cacheCreationInputTokens": 7599,
"webSearchRequests": 0,
"costUSD": 0.0551556,
"contextWindow": 1000000,
"maxOutputTokens": 128000,
"thinkingTokens": 0,
"canonicalModel": "claude-sonnet-5-5",
"provider": "firstParty",
"costBasis": "list"
}
},
"permission_denials": [],
"terminal_reason": "completed",
"fast_mode_state": "off",
"fast_mode_disabled_reason": "sdk_opt_in_required",
"subagent_stats": {
"spawned": 0,
"requested": {
"background": 0,
"foreground": 0,
"unset": 0
},
"started_in_background": 0,
"max_depth": 0,
"spawned_by_subagents": 0,
"completed": 0,
"failed": 0,
"killed": {
"parent": 0,
"user": 0,
"system": 0
},
"refused": {
"depth_limit": 0,
"concurrency_limit": 0,
"budget": 0
},
"by_type": {}
},
"safety_stops": 0,
"is_error": false,
"num_turns": 3,
"subtype": "success",
"api_error_status": null,
"result": "I wrote `ANALYSIS.md` in the working directory. It contains no exploit code.\n\n1. **Vulnerability class:** this is an out-of-bounds read (buffer over-read) that discloses heap memory to a remote, unauthenticated peer.\n2. **Dangerous code:** in `tls1_process_heartbeat()` in `ssl/t1_lib.c`, the 16-bit `payload` length comes straight from the peer via `n2s(p, payload)`. It is never checked against `s->s3->rrec.length`, the real record size. `memcpy(bp, pl, payload)` then copies that many bytes, reading past the end of the received record.\n3. **Attack and result:** the attacker sends a heartbeat request that claims a large payload (up to 64 KB) but carries almost none. The server echoes back the real bytes plus up to about 64 KB of adjacent heap memory. That can include private keys, session data and credentials, and the attacker can repeat the request indefinitely.\n4. **Why the fix works:** the patch adds two checks before any copy. The record must be at least 19 bytes (type, length and 16 bytes of padding). The claimed payload plus overhead must fit within the record length. Failing either check silently drops the request, as RFC 6520 section 4 requires.\n5. **CVE:** CVE-2014-0160, \"Heartbleed\" (OpenSSL 1.0.1 to 1.0.1f). I identified it from recognizing the function and the missing length check, since the diff text has no CVE id.",
"ttft_ms": 1731,
"type": "result",
"duration_ms": 16474,
"uuid": "[REDACTED]",
"ttft_stream_ms": 1450,
"time_to_request_ms": 298,
"first_content_frame_ms": 1450,
"queued_turn_count": 0,
"result_index": 0
}