Skip to content

DeepSeek V4.1 Flash (Venice) on Implement 5 Python functions from a task list

DDeepSeek V4.1 Flash (Venice) · venice-mini-agent v1.2 · cloud · 2026-10-15 · by @bobsiopener

Trust 80%Fully autonomoussuccess · first try
Cost
$0.0039
Wall time
24s
Human step-ins
0
Outcome
Pass

Compare with other models on this task →

Summary

Exact prompt

Read tasks/TASKS.md and implement all 5 functions in ./solution.py in the current directory. Python 3 stdlib only.

Task prompt given to the agent (headless, offline, synthetic fixture inputs): Read tasks/TASKS.md and implement all 5 functions in ./solution.py in the current directory. Python 3 stdlib only. Inspired by https://x.com/Vladic_ETH/status/2108591593120756156 (unverified; not checked with the author). This is an independent re-run on a reproducible synthetic task; it does not report or reproduce the original post's numbers. Cost is real Venice spend (paid from DIEM credit), as reported by Venice.

Success criteria: All automated rubric checks pass: hidden test test_duration; hidden test test_lru; hidden test test_merge; hidden test test_roman; hidden test test_topk.

Setup notes: Harness: venice-mini-agent v1.2 via Venice OpenAI-compatible API. Minimal OpenAI-compatible tool loop (/workspace/usecase-runner/venice_agent.py): tools list_files/read_file/write_file/run_shell in the work dir, max 25 turns, max_tokens 16000/call, temperature 0.2, Venice system prompt and web search off, $0.08 per-run cost cap. Identical for all Venice models. Cost source: Venice /api/v1/models model_spec.pricing (USD per 1M tokens), fetched 2026-10-14; cost computed from per-call usage. Paid from Bob's DIEM daily allowance. input_tokens includes cached reads; cached_tokens = cache reads; cache_write_tokens = cache writes (0 if not reported). Single run, no retries; scored automatically (5/5 checks). Runner notes: Single run, no retries. Same rubric/score.py as Claude runs. Web disabled.

Metrics

Cost
$0.0039
Wall time
24s
Human step-ins
0
Turns
4
Model usage
ModelInput tokOutput tokCost
DDeepSeek V4.1 Flash (Venice)venice/deepseek-v4-1-flash6,1122,435$0.0039

Reproduce this run

  1. Use the use case prompt on the use case page.
  2. Same setup: DeepSeek V4.1 Flash (Venice) (venice/deepseek-v4-1-flash) via venice-mini-agent v1.2, cloud.
  3. It counts as a success only if it meets every success criterion: All automated rubric checks pass: hidden test test_duration; hidden test test_lru; hidden test test_merge; hidden test test_roman; hidden test test_topk.
  4. Log model, tokens, wall time, human step-ins and the evidence log, then submit it as a reproduction. A different model? Submit a head-to-head rerun.

How runs are scored: Methodology.

Was this useful?

⇄ Rerun on another model

Automated checks

Trust 80% comes from these checks (how Trust % works).

Evidence log
{"turn": 1, "response": {"id": "chatcmpl-d304df2a537f4d1c92358c5d753ecaea", "object": "chat.completion", "created": 1792023430, "model": "deepseek-v4-1-flash", "choices": [{"index": 0, "message": {"role": "assistant", "content": "I'll start by reading the task file and exploring the current directory.", "tool_calls": [{"index": 0, "id": "chatcmpl-tool-962f6d20c2964dea", "type": "function", "function": {"name": "read_file", "arguments": "{\"path\": \"tasks/TASKS.md\"}"}, "name": null}, {"index": 1, "id": "chatcmpl-tool-a45c322b3fb8be14", "type": "function", "function": {"name": "list_files", "arguments": "{\"path\": \".\"}"}, "name": null}], "tools": null}, "finish_reason": "tool_calls"}], "usage": {"prompt_tokens": 532, "completion_tokens": 83, "total_tokens": 615, "prompt_tokens_details": {"cached_tokens": 0}, "completion_tokens_details": {"reasoning_tokens": 0}}, "cost": {"usd": 0, "diem": 0.0002592}, "venice_parameters": {"strip_thinking_response": false, "disable_thinking": false, "enable_e2ee": true, "enable_web_search": "off", "enable_web_scraping": false, "enable_web_citations": false, "include_search_results_in_stream": false, "include_venice_system_prompt": false, "enable_x_search": false, "web_search_citations": [], "return_search_results_as_documents": false}}}
{"turn": 1, "tool": "read_file", "args": {"path": "tasks/TASKS.md"}, "result": "Implement these 5 functions in solution.py (Python 3 stdlib only):\n1. roman_to_int(s: str) -> int  \u2014 convert a valid Roman numeral (I..MMMCMXCIX) to int.\n2. merge_intervals(iv: list[list[int]]) -> list[list[int]] \u2014 merge overlapping/touching closed intervals, return sorted.\n3. lru_cache_sim(capacity: int, ops: list[tuple]) -> list \u2014 simulate an LRU cache. ops are (\"put\", k, v) or (\"get\", k); return the results of each get in order (-1 if missing).\n4. top_k_words(text: str, k: int) -> list[str] \u2014 k most frequent words, case-insensitive, words are [a-z']+ runs; ties broken alphabetically.\n5. parse_duration(s: str) -> int \u2014 parse strings like \"1h30m\", \"45s\", \"2h\", \"1h2m3s\", \"90m\" into total seconds; raise ValueError for invalid input such as \"\", \"h\", \"1x\", \"1h1h\".\nWrite your own quick tests if you like, then finish.\n"}
{"turn": 1, "tool": "list_files", "args": {"path": "."}, "result": "./tasks/TASKS.md"}
{"turn": 2, "response": {"id": "chatcmpl-f11849f58c404fd3b75cc2df135eb8a9", "object": "chat.completion", "created": 1792023436, "model": "deepseek-v4-1-flash", "choices": [{"index": 0, "message": {"role": "assistant", "content": "", "reasoning_content": "Let me check if solution.py exists.\n\nIt doesn't exist. Let me create it.\n\nLet me think about each function carefully.\n\n1. roman_to_int: standard.\n\n2. merge_intervals: merge overlapping/touching closed intervals. Touching means [1,2] and [2,3] merge into [1,3]. Sort by start.\n\n3. lru_cache_sim: capacity, ops. Standard LRU. Return results of each get in order.\n\nEdge: capacity could be 0? Probably handle gracefully. Let's handle capacity <= 0: put does nothing, get returns -1.\n\n4. top_k_words: words are [a-z']+ runs, case-insensitive. So lowercase the text, find runs of [a-z']+. Count frequencies. Sort by (-count, word). Return top k words. Note: should we strip apostrophes? The spec says words are [a-z']+ runs. So \"don't\" is one word. Keep as is.\n\n5. parse_duration: parse \"1h30m\", \"45s\", \"2h\", \"1h2m3s\", \"90m\". Raise ValueError for \"\", \"h\", \"1x\", \"1h1h\". So format: sequence of number+unit pairs, units h, m, s, each unit at most once, must be in order h, m, s? \"1h1h\" invalid because repeated. \"1x\" invalid unit. \"h\" invalid no number. \"90m\" valid = 5400.\n\nShould we require order? \"1m1h\" \u2014 probably invalid. Let's enforce strictly descending order h > m > s. Also numbers must be non-negative integers? Let's allow digits only (no leading + or -). Probably require at least one digit. Let's use regex: ^(\\d+h)?(\\d+m)?(\\d+s)?$ and require non-empty match and at least one component.\n\nLet me write it.\n\nFor parse_duration, should \"0s\" be valid? Yes probably. Should \"1h0m\" valid? Yes.\n\nLet me write the solution and tests.\n\nFor merge_intervals: input list of [start, end]. Sort by start. Merge if next.start <= current.end (touching merges). Return list of lists.\n\nLet me write.\n\nFor top_k_words, k could be larger than number of unique words \u2014 return all. k <= 0 returns [].\n\nLet me write the file.", "tool_calls": [{"index": 0, "id": "chatcmpl-tool-a4d70cf5e48510c8", "type": "function", "function": {"name": "write_file", "arguments": "{\"path\": \"solution.py\", \"content\": \"\\\"\\\"\\\"Five small utility functions (Python 3 stdlib only).\\\"\\\"\\\"\\n\\nimport re\\nfrom collections import Counter, OrderedDict\\n\\n_ROMAN_VALUES = {\\n    \\\"I\\\": 1,\\n    \\\"V\\\": 5,\\n    \\\"X\\\": 10,\\n    \\\"L\\\": 50,\\n    \\\"C\\\": 100,\\n    \\\"D\\\": 500,\\n    \\\"M\\\": 1000,\\n}\\n\\n\\ndef roman_to_int(s: str) -> int:\\n    \\\"\\\"\\\"Convert a valid Roman numeral (I..MMMCMXCIX) to an int.\\\"\\\"\\\"\\n    if not isinstance(s, str) or not s:\\n        raise ValueError(\\\"invalid roman numeral\\\")\\n    total = 0\\n    prev = 0\\n    for ch in reversed(s.upper()):\\n        if ch not in _ROMAN_VALUES:\\n            raise ValueError(\\\"invalid roman numeral: %r\\\" % (s,))\\n        val = _ROMAN_VALUES[ch]\\n        if val < prev:\\n            total -= val\\n        else:\\n            total += val\\n            prev = val\\n    return total\\n\\n\\ndef merge_intervals(iv):\\n    \\\"\\\"\\\"Merge overlapping/touching closed intervals; return sorted list.\\\"\\\"\\\"\\n    if not iv:\\n        return []\\n    intervals = sorted((list(pair) for pair in iv), key=lambda p: (p[0], p[1]))\\n    merged = [intervals[0][:]]\\n    for start, end in intervals[1:]:\\n        last = merged[-1]\\n        if start <= last[1]:  # overlapping or touching\\n            if end > last[1]:\\n                last[1] = end\\n        else:\\n            merged.append([start, end])\\n    return merged\\n\\n\\ndef lru_cache_sim(capacity, ops):\\n    \\\"\\\"\\\"Simulate an LRU cache; return the results of each get in order.\\\"\\\"\\\"\\n    cache = OrderedDict()\\n    results = []\\n    for op in ops:\\n        kind = op[0]\\n        if kind == \\\"put\\\":\\n            _, key, value = op\\n            if capacity <= 0:\\n                continue\\n            if key in cache:\\n                cache.move_to_end(key)\\n            cache[key] = value\\n            if len(cache) > capacity:\\n                cache.popitem(last=False)\\n        elif kind == \\\"get\\\":\\n            _, key = op\\n            if key in cache:\\n                cache.move_to_end(key)\\n                results.append(cache[key])\\n            else:\\n                results.append(-1)\\n        else:\\n            raise ValueError(\\\"unknown op: %r\\\" % (kind,))\\n    return results\\n\\n\\n_WORD_RE = re.compile(r\\\"[a-z']+\\\")\\n\\n\\ndef top_k_words(text, k):\\n    \\\"\\\"\\\"Return the k most frequent words (case-insensitive), ties alphabetical.\\\"\\\"\\\"\\n    if k <= 0:\\n        return []\\n    counts = Counter(_WORD_RE.findall(text.lower()))\\n    ordered = sorted(counts.items(), key=lambda kv: (-kv[1], kv[0]))\\n    return [word for word, _ in ordered[:k]]\\n\\n\\n_DURATION_RE = re.compile(r\\\"^(?:(\\\\d+)h)?(?:(\\\\d+)m)?(?:(\\\\d+)s)?$\\\")\\n\\n\\ndef parse_duration(s):\\n    \\\"\\\"\\\"Parse strings like '1h30m', '45s', '2h', '1h2m3s', '90m' to seconds.\\\"\\\"\\\"\\n    if not isinstance(s, str):\\n        raise ValueError(\\\"invalid duration: %r\\\" % (s,))\\n    match = _DURATION_RE.match(s)\\n    if not match or not any(g is not None for g in match.groups()):\\n        raise ValueError(\\\"invalid duration: %r\\\" % (s,))\\n    hours, minutes, seconds = (int(g) if g is not None else 0 for g in match.groups())\\n    return hours * 3600 + minutes * 60 + seconds\\n\\n\\nif __name__ == \\\"__main__\\\":\\n    # Quick self-tests.\\n    assert roman_to_int(\\\"I\\\") == 1\\n    assert roman_to_int(\\\"IV\\\") == 4\\n    assert roman_to_int(\\\"MCMXCIX\\\") == 1999\\n    assert roman_to_int(\\\"MMMCMXCIX\\\") == 3999\\n    assert roman_to_int(\\\"LVIII\\\") == 58\\n\\n    assert merge_intervals([[1, 3], [2, 6], [8, 10], [15, 18]]) == [[1, 6], [8, 10], [15, 18]]\\n    assert merge_intervals([[1, 2], [2, 3]]) == [[1, 3]]\\n    assert merge_intervals([]) == []\\n    assert merge_intervals([[5, 6], [1, 2]]) == [[1, 2], [5, 6]]\\n\\n    assert lru_cache_sim(2, [(\\\"put\\\", 1, 1), (\\\"put\\\", 2, 2), (\\\"get\\\", 1), (\\\"put\\\", 3, 3), (\\\"get\\\", 2)]) == [1, -1]\\n    assert lru_cache_sim(1, [(\\\"put\\\", 1, 1), (\\\"put\\\", 1, 2), (\\\"get\\\", 1)]) == [2]\\n    assert lru_cache_sim(0, [(\\\"put\\\", 1, 1), (\\\"get\\\", 1)]) == [-1]\\n\\n    assert top_k_words(\\\"the quick brown fox the lazy dog the\\\", 2) == [\\\"the\\\", \\\"brown\\\"]\\n    assert top_k_words(\\\"a A b B c\\\", 2) == [\\\"a\\\", \\\"b\\\"]\\n    assert top_k_words(\\\"\\\", 3) == []\\n\\n    assert parse_duration(\\\"1h30m\\\") == 5400\\n    assert parse_duration(\\\"45s\\\") == 45\\n    assert parse_duration(\\\"2h\\\") == 7200\\n    assert parse_duration(\\\"1h2m3s\\\") == 3723\\n    assert parse_duration(\\\"90m\\\") == 5400\\n    for bad in (\\\"\\\", \\\"h\\\", \\\"1x\\\", \\\"1h1h\\\", \\\"1m1h\\\", \\\"1\\\", \\\"1h30\\\"):\\n        try:\\n            parse_duration(bad)\\n        except ValueError:\\n            pass\\n        else:\\n            raise AssertionError(\\\"expected ValueError for %r\\\" % (bad,))\\n\\n    print(\\\"all self-tests passed\\\")\\n\"}"}, "name": null}], "tools": null}, "finish_reason": "tool_calls"}], "usage": {"prompt_tokens": 891, "completion_tokens": 1918, "total_tokens": 2809, "prompt_tokens_details": {"cached_tokens": 402}, "cache_read_input_tokens": 402, "completion_tokens_details": {"reasoning_tokens": 518}}, "cost": {"usd": 0, "diem": 0.002451315}, "venice_parameters": {"strip_thinking_response": false, "disable_thinking": false, "enable_e2ee": true, "enable_web_search": "off", "enable_web_scraping": false, "enable_web_citations": false, "include_search_results_in_stream": false, "include_venice_system_prompt": false, "enable_x_search": false, "web_search_citations": [], "return_search_results_as_documents": false}}}
{"turn": 2, "tool": "write_file", "args": {"path": "solution.py", "content": "\"\"\"Five small utility functions (Python 3 stdlib only).\"\"\"\n\nimport re\nfrom collections import Counter, OrderedDict\n\n_ROMAN_VALUES = {\n    \"I\": 1,\n    \"V\": 5,\n    \"X\": 10,\n    \"L\": 50,\n    \"C\": 100,\n    \"D\": 500,\n    \"M\": 1000,\n}\n\n\ndef roman_to_int(s: str) -> int:\n    \"\"\"Convert a valid Roman numeral (I..MMMCMXCIX) to an int.\"\"\"\n    if not isinstance(s, str) or not s:\n        raise ValueError(\"invalid roman numeral\")\n    total = 0\n    prev = 0\n    for ch in reversed(s.upper()):\n        if ch not in _ROMAN_VALUES:\n            raise ValueError(\"invalid roman numeral: %r\" % (s,))\n        val = _ROMAN_VALUES[ch]\n        if val < prev:\n            total -= val\n        else:\n            total += val\n            prev = val\n    return total\n\n\ndef merge_intervals(iv):\n    \"\"\"Merge overlapping/touching closed intervals; return sorted list.\"\"\"\n    if not iv:\n        return []\n    intervals = sorted((list(pair) for pair in iv), key=lambda p: (p[0], p[1]))\n    merged = [intervals[0][:]]\n    for start, end in intervals[1:]:\n        last = merged[-1]\n        if start <= last[1]:  # overlapping or touching\n            if end > last[1]:\n                last[1] = end\n        else:\n            merged.append([start, end])\n    return merged\n\n\ndef lru_cache_sim(capacity, ops):\n    \"\"\"Simulate an LRU cache; return the results of each get in order.\"\"\"\n    cache = OrderedDict()\n    results = []\n    for op in ops:\n        kind = op[0]\n        if kind == \"put\":\n            _, key, value = op\n            if capacity <= 0:\n                continue\n            if key in cache:\n                cache.move_to_end(key)\n            cache[key] = value\n            if len(cache) > capacity:\n                cache.popitem(last=False)\n        elif kind == \"get\":\n            _, key = op\n            if key in cache:\n                cache.move_to_end(key)\n                results.append(cache[key])\n            else:\n                results.append(-1)\n        else:\n            raise ValueError(\"unknown op: %r\" % (kind,))\n    return results\n\n\n_WORD_RE = re.compile(r\"[a-z']+\")\n\n\ndef top_k_words(text, k):\n    \"\"\"Return the k most frequent words (case-insensitive), ties alphabetical.\"\"\"\n    if k <= 0:\n        return []\n    counts = Counter(_WORD_RE.findall(text.lower()))\n    ordered = sorted(counts.items(), key=lambda kv: (-kv[1], kv[0]))\n    return [word for word, _ in ordered[:k]]\n\n\n_DURATION_RE = re.compile(r\"^(?:(\\d+)h)?(?:(\\d+)m)?(?:(\\d+)s)?$\")\n\n\ndef parse_duration(s):\n    \"\"\"Parse strings like '1h30m', '45s', '2h', '1h2m3s', '90m' to seconds.\"\"\"\n    if not isinstance(s, str):\n        raise ValueError(\"invalid duration: %r\" % (s,))\n    match = _DURATION_RE.match(s)\n    if not match or not any(g is not None for g in match.groups()):\n        raise ValueError(\"invalid duration: %r\" % (s,))\n    hours, minutes, seconds = (int(g) if g is not None else 0 for g in match.groups())\n    return hours * 3600 + minutes * 60 + seconds\n\n\nif __name__ == \"__main__\":\n    # Quick self-tests.\n    assert roman_to_int(\"I\") == 1\n    assert roman_to_int(\"IV\") == 4\n    assert roman_to_int(\"MCMXCIX\") == 1999\n    assert roman_to_int(\"MMMCMXCIX\") == 3999\n    assert roman_to_int(\"LVIII\") == 58\n\n    assert merge_intervals([[1, 3], [2, 6], [8, 10], [15, 18]]) == [[1, 6], [8, 10], [15, 18]]\n    assert merge_intervals([[1, 2], [2, 3]]) == [[1, 3]]\n    assert merge_intervals([]) == []\n    assert merge_intervals([[5, 6], [1, 2]]) == [[1, 2], [5, 6]]\n\n    assert lru_cache_sim(2, [(\"put\", 1, 1), (\"put\", 2, 2), (\"get\", 1), (\"put\", 3, 3), (\"get\", 2)]) == [1, -1]\n    assert lru_cache_sim(1, [(\"put\", 1, 1), (\"put\", 1, 2), (\"get\", 1)]) == [2]\n    assert lru_cache_sim(0, [(\"put\", 1, 1), (\"get\", 1)]) == [-1]\n\n    assert top_k_words(\"the quick brown fox the lazy dog the\", 2) == [\"the\", \"brown\"]\n    assert top_k_words(\"a A b B c\", 2) == [\"a\", \"b\"]\n    assert top_k_words(\"\", 3) == []\n\n    assert parse_duration(\"1h30m\") == 5400\n    assert parse_duration(\"45s\") == 45\n    assert parse_duration(\"2h\") == 7200\n    assert parse_duration(\"1h2m3s\") == 3723\n    assert parse_duration(\"90m\") == 5400\n    for bad in (\"\", \"h\", \"1x\", \"1h1h\", \"1m1h\", \"1\", \"1h30\"):\n        try:\n            parse_duration(bad)\n        except ValueError:\n            pass\n        else:\n            raise AssertionError(\"expected ValueError for %r\" % (bad,))\n\n    print(\"all self-tests passed\")\n"}, "result": "wrote 4343 chars"}
{"turn": 3, "response": {"id": "chatcmpl-fd41bd8e0dff4062bf5aa23fc213341c", "object": "chat.completion", "created": 1792023448, "model": "deepseek-v4-1-flash", "choices": [{"index": 0, "message": {"role": "assistant", "content": "", "reasoning_content": "Wait, \"1m1h\" \u2014 my regex requires order h, m, s, so \"1m1h\" fails. Good. But is that desired? Task says raise ValueError for invalid input such as those. Ordering h,m,s is standard. Fine.\n\nRun tests.", "tool_calls": [{"index": 0, "id": "chatcmpl-tool-8629d4fc9a4a3cef", "type": "function", "function": {"name": "run_shell", "arguments": "{\"command\": \"cd . && python3 solution.py\"}"}, "name": null}], "tools": null}, "finish_reason": "tool_calls"}], "usage": {"prompt_tokens": 2308, "completion_tokens": 105, "total_tokens": 2413, "prompt_tokens_details": {"cached_tokens": 410}, "cache_read_input_tokens": 410, "completion_tokens_details": {"reasoning_tokens": 59}}, "cost": {"usd": 0, "diem": 0.0006984749999999999}, "venice_parameters": {"strip_thinking_response": false, "disable_thinking": false, "enable_e2ee": true, "enable_web_search": "off", "enable_web_scraping": false, "enable_web_citations": false, "include_search_results_in_stream": false, "include_venice_system_prompt": false, "enable_x_search": false, "web_search_citations": [], "return_search_results_as_documents": false}}}
{"turn": 3, "tool": "run_shell", "args": {"command": "cd . && python3 solution.py"}, "result": "exit=0\nstdout:\nall self-tests passed\n\nstderr:\n"}
{"turn": 4, "response": {"id": "chatcmpl-c51aa6102ba2432098aba352b6e011a6", "object": "chat.completion", "created": 1792023450, "model": "deepseek-v4-1-flash", "choices": [{"index": 0, "message": {"role": "assistant", "content": "All 5 functions are implemented in `./solution.py` and pass the built-in self-tests (`python3 solution.py` \u2192 \"all self-tests passed\").\n\nSummary of implementations:\n1. **`roman_to_int`** \u2014 right-to-left scan with subtractive-pair handling; validates characters.\n2. **`merge_intervals`** \u2014 sorts by start, merges overlapping/touching (`start <= last_end`) closed intervals.\n3. **`lru_cache_sim`** \u2014 `OrderedDict`-based LRU with `move_to_end`/`popitem(last=False)`; returns each `get` result (`-1` on miss); handles capacity \u2264 0.\n4. **`top_k_words`** \u2014 case-insensitive `[a-z']+` tokenization via `Counter`, sorted by `(-count, word)` for alphabetical tie-breaking.\n5. **`parse_duration`** \u2014 anchored regex for `h`/`m`/`s` components in order; raises `ValueError` for empty, unit-only, unknown-unit, duplicate-unit, or malformed inputs.\n\nOnly Python 3 stdlib (`re`, `collections`) is used.", "reasoning_content": "All tests pass. Let me double-check a few edge cases: top_k_words with apostrophes, parse_duration \"1h1h\" fails (regex anchored, good). Also check \"1h30\" fails \u2014 yes since trailing digits unmatched. Good.\n\nOne consideration: merge_intervals touching intervals \u2014 handled. lru_cache_sim capacity 0 \u2014 handled.\n\nDone.", "tools": null}, "finish_reason": "stop"}], "usage": {"prompt_tokens": 2381, "completion_tokens": 329, "total_tokens": 2710, "prompt_tokens_details": {"cached_tokens": 2178}, "cache_read_input_tokens": 2178, "completion_tokens_details": {"reasoning_tokens": 77}}, "cost": {"usd": 0, "diem": 0.000472035}, "venice_parameters": {"strip_thinking_response": false, "disable_thinking": false, "enable_e2ee": true, "enable_web_search": "off", "enable_web_scraping": false, "enable_web_citations": false, "include_search_results_in_stream": false, "include_venice_system_prompt": false, "enable_x_search": false, "web_search_citations": [], "return_search_results_as_documents": false}}}

Get new runs and ranking changes weekly