AK-Benchbenchmark report

Task

cadence-takeover-001@1.0.0

runs
12
success rate
100.0%
mean time
3 m 60 s
input tokens
2,182,492
generated tokens
53,408
tokens
2,235,900
tool calls
169

What the task asks

A worker with a stale epoch can take a live lease back

LeaseBook.takeover accepts a request whose epoch no longer matches the resource's current lease, so a worker that was cut off and comes back can steal the resource from the worker that holds it now. Restore the documented rule that only the holder of the current epoch may take a resource back.

full description (markdown source)
# A worker with a stale epoch can take a live lease back

`LeaseBook.takeover` in `src/cadence/lease.py` is supposed to answer one
question: does the caller still hold the epoch this resource was granted with?
A worker that was cut off and comes back still believes it holds the resource,
and asks for it back by the epoch it was given.

The rule, as documented in the module:

- the epoch is the whole question — a request for an epoch the book no longer
  carries is **stale**, because somebody else was granted the resource while the
  caller was silent;
- granting a stale request would put two workers on the same work, so a stale
  epoch is refused with `LeaseHeld`, and the current holder keeps its lease and
  its validity;
- a refused takeover must change nothing: not the holder, not the epoch, not
  the deadline.

The repository's own suite is green on the pinned commit, because it only ever
asks a holder back by the epoch it still carries. The defect only shows up when
a request arrives for an epoch the book no longer holds — which is exactly the
request that a worker that lost the lease makes.

Constraints

Recorded identity

fieldvalue
uidcadence-takeover-001@1.0.0
categorybug_fix
difficultyexpert
evaluatorpytest_evaluator
base commitHEAD
repositoryrepo-457ed29341306d49
snapshot digestsha256:20071a11d3adde959c832627ec86d344b2e7697d7bca56d700aca9ddabd1e24a
sourcedatasets/_sources/cadence
refHEAD
mutationnone
timeout900 s
hidden testshidden_tests
public testspublic_tests
definition digestsha256:20071a11d3adde959c832627ec86d344b2e7697d7bca56d700aca9ddabd1e24a
definition path/mnt/data2/dev/develop/ak-bench/datasets/bug_fix/cadence-takeover-001

Runs of this task

in the order they started; the parameters column is the configuration each run was executed under

runparametersstartedagentrepetitionstatusverdictscoretimeinput tokensgenerated tokenstotal tokenstool callsfailure classerror
run-20261102-102224-77a554a3knowledge_enabled=False, model=qwen/Qwen3.8-27B2026-11-02 10:22:24codex@codex-cli 0.166.01completedpassed1.0002 m 25 s87,4412,06289,5039
run-20261102-102450-4c4dafd8knowledge_enabled=False, model=qwen/Qwen3.8-27B2026-11-02 10:24:50codex@codex-cli 0.166.02completedpassed1.0003 m 56 s184,0373,928187,96513
run-20261102-102846-f9da52d1knowledge_enabled=False, model=qwen/Qwen3.8-27B2026-11-02 10:28:46codex@codex-cli 0.166.03completedpassed1.0002 m 53 s166,1052,393168,49814
run-20261102-103139-c174189bknowledge_enabled=False, model=qwen/Qwen3.8-Flash-Next2026-11-02 10:31:39codex@codex-cli 0.166.01completedpassed1.0003 m 20 s142,0814,157146,23813
run-20261102-103500-42595b9bknowledge_enabled=False, model=qwen/Qwen3.8-Flash-Next2026-11-02 10:35:00codex@codex-cli 0.166.02completedpassed1.0003 m 6 s136,1904,248140,43812
run-20261102-103806-ca54befbknowledge_enabled=False, model=qwen/Qwen3.8-Flash-Next2026-11-02 10:38:06codex@codex-cli 0.166.03completedpassed1.0003 m 43 s176,8744,786181,66014
run-20261102-104150-ee470964knowledge_enabled=False, model=auto_codex2026-11-02 10:41:50codex@codex-cli 0.166.01completedpassed1.0003 m 36 s184,2632,829187,09215
run-20261102-104527-df556abcknowledge_enabled=False, model=auto_codex2026-11-02 10:45:27codex@codex-cli 0.166.02completedpassed1.0005 m 33 s231,2506,615237,86516
run-20261102-105101-7f0ea089knowledge_enabled=False, model=auto_codex2026-11-02 10:51:01codex@codex-cli 0.166.03completedpassed1.0002 m 31 s81,7561,51383,2699
run-20261102-105332-509d420cknowledge_enabled=True, model=auto_codex2026-11-02 10:53:32codex@codex-cli 0.166.01completedpassed1.0005 m 51 s181,89210,799192,69115
run-20261102-105924-5844ec5cknowledge_enabled=True, model=auto_codex2026-11-02 10:59:24codex@codex-cli 0.166.02completedpassed1.0007 m 45 s486,0237,446493,46926
run-20261102-110711-12e9e01eknowledge_enabled=True, model=auto_codex2026-11-02 11:07:11codex@codex-cli 0.166.03completedpassed1.0003 m 18 s124,5802,632127,21213

Not verified

Configurations that are not verified. They stay in the report, because a result produced under them is only as trustworthy as they are.
kindsubjectstatusrunnote
pluginagentic_knowledgeunsupportedrun-20261102-102224-77a554a3LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, …
pluginagentic_routing_codexunsupportedrun-20261102-102224-77a554a3descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp …
pluginagentic_knowledgeunsupportedrun-20261102-102450-4c4dafd8LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, …
pluginagentic_routing_codexunsupportedrun-20261102-102450-4c4dafd8descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp …
pluginagentic_knowledgeunsupportedrun-20261102-102846-f9da52d1LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, …
pluginagentic_routing_codexunsupportedrun-20261102-102846-f9da52d1descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp …
pluginagentic_knowledgeunsupportedrun-20261102-103139-c174189bLLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, …
pluginagentic_routing_codexunsupportedrun-20261102-103139-c174189bdescriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp …
pluginagentic_knowledgeunsupportedrun-20261102-103500-42595b9bLLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, …
pluginagentic_routing_codexunsupportedrun-20261102-103500-42595b9bdescriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp …
pluginagentic_knowledgeunsupportedrun-20261102-103806-ca54befbLLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, …
pluginagentic_routing_codexunsupportedrun-20261102-103806-ca54befbdescriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp …
pluginagentic_knowledgeunsupportedrun-20261102-104150-ee470964LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, …
pluginagentic_routing_codexunsupportedrun-20261102-104150-ee470964descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp …
pluginagentic_knowledgeunsupportedrun-20261102-104527-df556abcLLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, …
pluginagentic_routing_codexunsupportedrun-20261102-104527-df556abcdescriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp …
pluginagentic_knowledgeunsupportedrun-20261102-105101-7f0ea089LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, …
pluginagentic_routing_codexunsupportedrun-20261102-105101-7f0ea089descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp …
pluginagentic_knowledgeunsupportedrun-20261102-105332-509d420cLLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, …
pluginagentic_routing_codexunsupportedrun-20261102-105332-509d420cdescriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp …
pluginagentic_knowledgeunsupportedrun-20261102-105924-5844ec5cLLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, …
pluginagentic_routing_codexunsupportedrun-20261102-105924-5844ec5cdescriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp …
pluginagentic_knowledgeunsupportedrun-20261102-110711-12e9e01eLLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, …
pluginagentic_routing_codexunsupportedrun-20261102-110711-12e9e01edescriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp …
modelqwen/Qwen3.8-27Bunverifiedrun-20261102-102224-77a554a3
modelqwen/Qwen3.8-27Bunverifiedrun-20261102-102450-4c4dafd8
modelqwen/Qwen3.8-27Bunverifiedrun-20261102-102846-f9da52d1
modelqwen/Qwen3.8-Flash-Nextunverifiedrun-20261102-103139-c174189b
modelqwen/Qwen3.8-Flash-Nextunverifiedrun-20261102-103500-42595b9b
modelqwen/Qwen3.8-Flash-Nextunverifiedrun-20261102-103806-ca54befb
modelauto_codexunverifiedrun-20261102-104150-ee470964
modelauto_codexunverifiedrun-20261102-104527-df556abc
modelauto_codexunverifiedrun-20261102-105101-7f0ea089
modelauto_codexunverifiedrun-20261102-105332-509d420c
modelauto_codexunverifiedrun-20261102-105924-5844ec5c
modelauto_codexunverifiedrun-20261102-110711-12e9e01e