AK-Benchbenchmark report

Task

stockpile-writeoff-001@1.0.0

runs
12
success rate
100.0%
mean time
3 m 40 s
input tokens
2,345,811
generated tokens
47,327
tokens
2,393,138
tool calls
156

What the task asks

A write-off destroys the reserved part of an expired lot

Stock.write_off_expired removes the reserved quantity of an expired lot along with its free quantity, so an order that is already holding stock on the lot is left with nothing to commit. Restore the documented rule that a write-off removes only the free quantity and leaves the lot alive while a claim still rides on it.

full description (markdown source)
# A write-off destroys the reserved part of an expired lot

`Stock.write_off_expired` in `src/stockpile/stock.py` answers one question:
which of the expired lots' quantity can go away? The rule, as documented on
the method:

- a write-off removes only what is **free** — quantity nobody is holding;
- committed quantity is already off the shelf, so it is never touched;
- reserved quantity is somebody's claim, so it is never touched: the lot
  stays alive while a claim still rides on it, because the warehouse has to
  be able to commit the order it already promised.

The defect zeroes the reserved quantity along with the free one. An order that
is holding stock on a lot that has since expired is left with nothing to
commit — no error at the write-off, just a claim that no longer corresponds to
anything.

The repository's own suite is green on the pinned commit, because it only ever
writes off an expired lot that carries no reservation, for which keeping the
reserved part and destroying it are the same thing. The defect only shows up
when a reservation is still riding on a lot that has expired.

Constraints

Recorded identity

fieldvalue
uidstockpile-writeoff-001@1.0.0
categorybug_fix
difficultyexpert
evaluatorpytest_evaluator
base commitHEAD
repositoryrepo-747f8117bee5bab7
snapshot digestsha256:f18166b57d49f1ac8badc5c2047738fcfd85d6fa3f8b673f831e05f82eae456b
sourcedatasets/_sources/stockpile
refHEAD
mutationnone
timeout900 s
hidden testshidden_tests
public testspublic_tests
definition digestsha256:f18166b57d49f1ac8badc5c2047738fcfd85d6fa3f8b673f831e05f82eae456b
definition path/mnt/data2/dev/develop/ak-bench/datasets/bug_fix/stockpile-writeoff-001

Runs of this task

in the order they started; the parameters column is the configuration each run was executed under

runparametersstartedagentrepetitionstatusverdictscoretimeinput tokensgenerated tokenstotal tokenstool callsfailure classerror
run-20261102-115850-e5f2348fknowledge_enabled=False, model=qwen/Qwen3.8-27B2026-11-02 11:58:50codex@codex-cli 0.166.01completedpassed1.0003 m 30 s174,8333,032177,86512
run-20261102-120221-58de15afknowledge_enabled=False, model=qwen/Qwen3.8-27B2026-11-02 12:02:21codex@codex-cli 0.166.02completedpassed1.0002 m 13 s131,9341,632133,56610
run-20261102-120435-e4b44447knowledge_enabled=False, model=qwen/Qwen3.8-27B2026-11-02 12:04:35codex@codex-cli 0.166.03completedpassed1.0001 m 40 s86,40794887,3557
run-20261102-120616-d577d4b1knowledge_enabled=False, model=qwen/Qwen3.8-Flash-Next2026-11-02 12:06:16codex@codex-cli 0.166.01completedpassed1.0005 m 15 s256,4468,216264,66216
run-20261102-121131-13e4a8b5knowledge_enabled=False, model=qwen/Qwen3.8-Flash-Next2026-11-02 12:11:31codex@codex-cli 0.166.02completedpassed1.0003 m 55 s216,2535,255221,50812
run-20261102-121528-03d9f220knowledge_enabled=False, model=qwen/Qwen3.8-Flash-Next2026-11-02 12:15:28codex@codex-cli 0.166.03completedpassed1.0005 m 26 s293,3468,697302,04317
run-20261102-122054-f8985461knowledge_enabled=False, model=auto_codex2026-11-02 12:20:54codex@codex-cli 0.166.01completedpassed1.0003 m 3 s130,7163,022133,73810
run-20261102-122358-fec2acfeknowledge_enabled=False, model=auto_codex2026-11-02 12:23:58codex@codex-cli 0.166.02completedpassed1.0004 m 57 s284,3844,090288,47416
run-20261102-122856-43bd96e6knowledge_enabled=False, model=auto_codex2026-11-02 12:28:56codex@codex-cli 0.166.03completedpassed1.0004 m 56 s290,1165,054295,17018
run-20261102-123353-08bae5f2knowledge_enabled=True, model=auto_codex2026-11-02 12:33:53codex@codex-cli 0.166.01completedpassed1.0003 m 28 s174,0553,204177,25912
run-20261102-123722-abf5b6a0knowledge_enabled=True, model=auto_codex2026-11-02 12:37:22codex@codex-cli 0.166.02completedpassed1.0002 m 50 s159,4521,673161,12512
run-20261102-124013-4e335fc0knowledge_enabled=True, model=auto_codex2026-11-02 12:40:13codex@codex-cli 0.166.03completedpassed1.0002 m 48 s147,8692,504150,37314

Not verified

Configurations that are not verified. They stay in the report, because a result produced under them is only as trustworthy as they are.
kindsubjectstatusrunnote
pluginagentic_knowledgeunsupportedrun-20261102-115850-e5f2348fLLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, …
pluginagentic_routing_codexunsupportedrun-20261102-115850-e5f2348fdescriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp …
pluginagentic_knowledgeunsupportedrun-20261102-120221-58de15afLLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, …
pluginagentic_routing_codexunsupportedrun-20261102-120221-58de15afdescriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp …
pluginagentic_knowledgeunsupportedrun-20261102-120435-e4b44447LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, …
pluginagentic_routing_codexunsupportedrun-20261102-120435-e4b44447descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp …
pluginagentic_knowledgeunsupportedrun-20261102-120616-d577d4b1LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, …
pluginagentic_routing_codexunsupportedrun-20261102-120616-d577d4b1descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp …
pluginagentic_knowledgeunsupportedrun-20261102-121131-13e4a8b5LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, …
pluginagentic_routing_codexunsupportedrun-20261102-121131-13e4a8b5descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp …
pluginagentic_knowledgeunsupportedrun-20261102-121528-03d9f220LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, …
pluginagentic_routing_codexunsupportedrun-20261102-121528-03d9f220descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp …
pluginagentic_knowledgeunsupportedrun-20261102-122054-f8985461LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, …
pluginagentic_routing_codexunsupportedrun-20261102-122054-f8985461descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp …
pluginagentic_knowledgeunsupportedrun-20261102-122358-fec2acfeLLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, …
pluginagentic_routing_codexunsupportedrun-20261102-122358-fec2acfedescriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp …
pluginagentic_knowledgeunsupportedrun-20261102-122856-43bd96e6LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, …
pluginagentic_routing_codexunsupportedrun-20261102-122856-43bd96e6descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp …
pluginagentic_knowledgeunsupportedrun-20261102-123353-08bae5f2LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, …
pluginagentic_routing_codexunsupportedrun-20261102-123353-08bae5f2descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp …
pluginagentic_knowledgeunsupportedrun-20261102-123722-abf5b6a0LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, …
pluginagentic_routing_codexunsupportedrun-20261102-123722-abf5b6a0descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp …
pluginagentic_knowledgeunsupportedrun-20261102-124013-4e335fc0LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, …
pluginagentic_routing_codexunsupportedrun-20261102-124013-4e335fc0descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp …
modelqwen/Qwen3.8-27Bunverifiedrun-20261102-115850-e5f2348f
modelqwen/Qwen3.8-27Bunverifiedrun-20261102-120221-58de15af
modelqwen/Qwen3.8-27Bunverifiedrun-20261102-120435-e4b44447
modelqwen/Qwen3.8-Flash-Nextunverifiedrun-20261102-120616-d577d4b1
modelqwen/Qwen3.8-Flash-Nextunverifiedrun-20261102-121131-13e4a8b5
modelqwen/Qwen3.8-Flash-Nextunverifiedrun-20261102-121528-03d9f220
modelauto_codexunverifiedrun-20261102-122054-f8985461
modelauto_codexunverifiedrun-20261102-122358-fec2acfe
modelauto_codexunverifiedrun-20261102-122856-43bd96e6
modelauto_codexunverifiedrun-20261102-123353-08bae5f2
modelauto_codexunverifiedrun-20261102-123722-abf5b6a0
modelauto_codexunverifiedrun-20261102-124013-4e335fc0