Skip to content

Claude Opus 5.5 on Reverse engineer a security patch for an exploited CVE

AClaude Opus 5.5 · Claude Code 2.1.299 (headless -p) · cloud · 2026-10-14 · by @bobsiopener

Trust 80%Fully autonomoussuccess · first try
Estimated cost
$0.15
Wall time
37s
Human step-ins
0
Outcome
Pass

Compare with other models on this task →

Summary

Exact prompt

patch.diff is a security fix from a widely used TLS library for a vulnerability that was exploited in the wild. Analyze only the diff (no web). Write ANALYSIS.md covering: (1) the vulnerability class, (2) the exact function and the line(s) that were dangerous before the patch, (3) what an attacker could send and what they would get back, (4) why the added checks fix it, (5) the CVE id if you can identify it. Do not write exploit code.

Task prompt given to the agent (headless, offline, synthetic fixture inputs): patch.diff is a security fix from a widely used TLS library for a vulnerability that was exploited in the wild. Analyze only the diff (no web). Write ANALYSIS.md covering: (1) the vulnerability class, (2) the exact function and the line(s) that were dangerous before the patch, (3) what an attacker could send and what they would get back, (4) why the added checks fix it, (5) the CVE id if you can identify it. Do not write exploit code. Inspired by https://x.com/KevTheHermit/status/2094160406935638494 (unverified; not checked with the author). This is an independent re-run on a reproducible synthetic task; it does not report or reproduce the original post's numbers. Cost shown is a list-price API-equivalent estimate (run on a Claude Pro subscription, not billed).

Success criteria: All automated rubric checks pass: identifies buffer over-read / out-of-bounds read; names tls1_process_heartbeat; points at unchecked payload length + memcpy; explains memory disclosure (up to ~64KB of process memory); explains length checks vs rrec.length; identifies CVE-2014-0160 / Heartbleed; no exploit code (no socket/send code blocks).

Setup notes: Cost is an ESTIMATE: Claude Code's list-price API-equivalent figure. The run used a Claude Pro subscription, so nothing was billed. input_tokens = all input incl. cache-creation and cache-read tokens; cached_tokens = cache reads (subset of input). Claude Code's cost bills cache writes at the 1h cache-write premium, so it can differ from a flat input-price recompute. Web disabled; tools Read/Write/Edit/Glob/Grep/Bash; single run; scored automatically (7/7 checks).

Estimated cost List-price estimate; not actually billed.

Metrics

Estimated cost
$0.15
Wall time
37s
Human step-ins
0
Turns
3
Model usage
ModelInput tokOutput tokCost
AClaude Opus 5.5anthropic/claude-opus-5.557,1563,531$0.15

Reproduce this run

  1. Use the use case prompt on the use case page.
  2. Same setup: Claude Opus 5.5 (anthropic/claude-opus-5.5) via Claude Code 2.1.299 (headless -p), cloud.
  3. It counts as a success only if it meets every success criterion: All automated rubric checks pass: identifies buffer over-read / out-of-bounds read; names tls1_process_heartbeat; points at unchecked payload length + memcpy; explains memory disclosure (up to ~64KB of process memory); explains length checks vs rrec.length; identifies CVE-2014-0160 / Heartbleed; no exploit code (no socket/send code blocks).
  4. Log model, tokens, wall time, human step-ins and the evidence log, then submit it as a reproduction. A different model? Submit a head-to-head rerun.

How runs are scored: Methodology.

Was this useful?

⇄ Rerun on another model

Automated checks

Trust 80% comes from these checks (how Trust % works).

Evidence log
{
 "duration_api_ms": 35396,
 "stop_reason": "end_turn",
 "session_id": "[REDACTED]",
 "total_cost_usd": 0.1526094,
 "usage": {
  "input_tokens": 6,
  "cache_creation_input_tokens": 9043,
  "cache_read_input_tokens": 48107,
  "output_tokens": 3531,
  "output_tokens_details": {
   "thinking_tokens": 16
  },
  "server_tool_use": {
   "web_search_requests": 0,
   "web_fetch_requests": 0
  },
  "service_tier": "standard",
  "cache_creation": {
   "ephemeral_1h_input_tokens": 9043,
   "ephemeral_5m_input_tokens": 0
  },
  "inference_geo": "global",
  "iterations": [
   {
    "input_tokens": 2,
    "output_tokens": 696,
    "cache_read_input_tokens": 18379,
    "cache_creation_input_tokens": 2810,
    "cache_creation": {
     "ephemeral_5m_input_tokens": 0,
     "ephemeral_1h_input_tokens": 2810
    },
    "type": "message"
   }
  ],
  "speed": "standard",
  "fallback_credit": null
 },
 "modelUsage": {
  "claude-opus-5-5": {
   "inputTokens": 6,
   "outputTokens": 3531,
   "cacheReadInputTokens": 48107,
   "cacheCreationInputTokens": 9043,
   "webSearchRequests": 0,
   "costUSD": 0.1526094,
   "contextWindow": 1000000,
   "maxOutputTokens": 128000,
   "thinkingTokens": 16,
   "canonicalModel": "claude-opus-5-5",
   "provider": "firstParty",
   "costBasis": "list"
  }
 },
 "permission_denials": [],
 "terminal_reason": "completed",
 "fast_mode_state": "off",
 "fast_mode_disabled_reason": "sdk_opt_in_required",
 "subagent_stats": {
  "spawned": 0,
  "requested": {
   "background": 0,
   "foreground": 0,
   "unset": 0
  },
  "started_in_background": 0,
  "max_depth": 0,
  "spawned_by_subagents": 0,
  "completed": 0,
  "failed": 0,
  "killed": {
   "parent": 0,
   "user": 0,
   "system": 0
  },
  "refused": {
   "depth_limit": 0,
   "concurrency_limit": 0,
   "budget": 0
  },
  "by_type": {}
 },
 "safety_stops": 0,
 "is_error": false,
 "num_turns": 3,
 "subtype": "success",
 "api_error_status": null,
 "result": "I wrote `ANALYSIS.md`. The patch fixes **Heartbleed, CVE-2014-0160**, in OpenSSL's `tls1_process_heartbeat()` in `ssl/t1_lib.c`.\n\n1. **Vulnerability class:** an out-of-bounds read that leaks memory back to the attacker. The code trusted a length field sent by the peer.\n2. **Dangerous lines:** `n2s(p, payload);` reads a 16-bit payload length from the attacker's message and never checks it against the real record size, `s->s3->rrec.length`. Then `memcpy(bp, pl, payload);` copies that many bytes into the response, reading past the end of the received data. The 3-byte header was also read before the code checked the record was long enough.\n3. **What the attacker sends and gets back:** a tiny heartbeat request that claims a payload of up to 65535 bytes. The reply contains up to about 64 KB of the process's heap memory per request. That can include other users' session data, passwords, session keys and possibly the server's private key. The attacker can repeat it as often as they like, it needs no login, and it is rarely logged.\n4. **Why the fix works:** the patch adds two checks before the copy. The first drops any record shorter than the minimum valid heartbeat (1 + 2 + 16 bytes). The second drops any message where `1 + 2 + payload + 16` is larger than the received record. After that, `memcpy` can only copy bytes the peer actually sent. The values are small enough that the arithmetic can't wrap around, and dropping without an alert is what RFC 6520 \u00a74 requires.\n5. **CVE:** CVE-2014-0160. It affected OpenSSL 1.0.1 through 1.0.1f and was fixed in 1.0.1g.\n\nThe diff leaves out the lines between its two hunks, such as the buffer allocation and the call that sends the response. I described those from what I know of the upstream source and said so in the file; the conclusions rest only on lines that are in the diff. I didn't write any exploit code.",
 "ttft_ms": 2414,
 "type": "result",
 "duration_ms": 35797,
 "uuid": "[REDACTED]",
 "ttft_stream_ms": 1529,
 "time_to_request_ms": 315,
 "first_content_frame_ms": 1529,
 "queued_turn_count": 0,
 "result_index": 0
}

Get new runs and ranking changes weekly