AK-Benchbenchmark report

Task

bookshop-tiers-001@1.0.0

runs
12
success rate
100.0%
mean time
4 m 37 s
input tokens
1,224,693
generated tokens
32,561
tokens
1,257,254
tool calls
111

What the task asks

The five percent tier is lost at exactly one hundred dollars

A basket whose subtotal is exactly 10 000 cents (or 50 000, or 200 000) is charged the wrong discount. Restore the documented tier behaviour.

full description (markdown source)
# The five percent tier is lost at exactly one hundred dollars

The catalogue is a Flask service in `src/bookshop`. Prices are computed by
`price_basket` in `src/bookshop/pricing.py`, and the rules it follows are
documented in that module's docstring. Read them before touching anything.

The defect is in the tier discount. A basket whose subtotal is **exactly**
`10 000` cents should earn the 5 % tier, and the same holds for `50 000`
(10 %) and `200 000` (15 %). Today, a basket at exactly those amounts is
charged as if it were one cent below the threshold.

## What to check

- `discount_tier` in `src/bookshop/pricing.py` returns the tier for a subtotal.
- A subtotal of exactly `10 000` must return the 5 % tier, not `None`.
- The same applies at `50 000` and `200 000`.
- The rest of the pricing pipeline (voucher, tax, total) must keep producing
  the same numbers it produced before.

## Boundaries

- Do not change the tier thresholds, the tax rate, or the rounding rule.
- Do not modify the repository test suite under `tests/`.
- The public API of the `bookshop` package stays as it is.
- The repository's own test suite has to stay green after your change.

Constraints

Recorded identity

fieldvalue
uidbookshop-tiers-001@1.0.0
categorybug_fix
difficultyhard
evaluatorpytest_evaluator
base commitHEAD
repositoryrepo-8b7a77081db02132
snapshot digestsha256:37d9e6edaac1b1ac25ed9eb08906ed20649dff0f3c7c368b675fe460d40e3660
sourcedatasets/_sources/bookshop
refHEAD
mutationnone
timeout600 s
hidden testshidden_tests
public testspublic_tests
definition digestsha256:37d9e6edaac1b1ac25ed9eb08906ed20649dff0f3c7c368b675fe460d40e3660
definition path/mnt/data2/dev/develop/ak-bench/datasets/bug_fix/bookshop-tiers-001

Runs of this task

in the order they started; the parameters column is the configuration each run was executed under

runparametersstartedagentrepetitionstatusverdictscoretimeinput tokensgenerated tokenstotal tokenstool callsfailure classerror
run-20261102-024213-cd12e80fknowledge_enabled=False, model=qwen/Qwen3.8-27B2026-11-02 02:42:13codex@codex-cli 0.166.01completedpassed1.0001 m 60 s51,44782652,2735
run-20261102-024414-b4a4c0f1knowledge_enabled=False, model=qwen/Qwen3.8-27B2026-11-02 02:44:14codex@codex-cli 0.166.02completedpassed1.0002 m 22 s73,2531,40774,6607
run-20261102-024637-153cbef0knowledge_enabled=False, model=qwen/Qwen3.8-27B2026-11-02 02:46:37codex@codex-cli 0.166.03completedpassed1.0002 m 22 s84,7901,31086,1008
run-20261102-024900-b4e495b9knowledge_enabled=False, model=qwen/Qwen3.8-Flash-Next2026-11-02 02:49:00codex@codex-cli 0.166.01completedpassed1.0004 m 48 s204,4546,473210,92718
run-20261102-025349-a41ee544knowledge_enabled=False, model=qwen/Qwen3.8-Flash-Next2026-11-02 02:53:49codex@codex-cli 0.166.02completedpassed1.0004 m 48 s190,9166,504197,42016
run-20261102-025838-8e5d4349knowledge_enabled=False, model=qwen/Qwen3.8-Flash-Next2026-11-02 02:58:38codex@codex-cli 0.166.03completedpassed1.0004 m 52 s185,1266,992192,11817
run-20261102-030330-6e680c03knowledge_enabled=False, model=auto_codex2026-11-02 03:03:30codex@codex-cli 0.166.01completedpassed1.0002 m 49 s81,2491,66482,9137
run-20261102-030620-307d982cknowledge_enabled=False, model=auto_codex2026-11-02 03:06:20codex@codex-cli 0.166.02completedpassed1.0003 m 43 s62,5881,46264,0506
run-20261102-031005-8ee792e8knowledge_enabled=False, model=auto_codex2026-11-02 03:10:05codex@codex-cli 0.166.03completedpassed1.0002 m 39 s52,1581,25653,4145
run-20261102-031245-d48061aaknowledge_enabled=True, model=auto_codex2026-11-02 03:12:45codex@codex-cli 0.166.01completedpassed1.0007 m 31 s65,0701,52766,5976
run-20261102-032017-d7e7635dknowledge_enabled=True, model=auto_codex2026-11-02 03:20:17codex@codex-cli 0.166.02completedpassed1.0006 m 36 s84,4421,52985,9718
run-20261102-032654-eef72cc9knowledge_enabled=True, model=auto_codex2026-11-02 03:26:54codex@codex-cli 0.166.03completedpassed1.00010 m 50 s89,2001,61190,8118

Not verified

Configurations that are not verified. They stay in the report, because a result produced under them is only as trustworthy as they are.
kindsubjectstatusrunnote
pluginagentic_knowledgeunsupportedrun-20261102-024213-cd12e80fLLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, …
pluginagentic_routing_codexunsupportedrun-20261102-024213-cd12e80fdescriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp …
pluginagentic_knowledgeunsupportedrun-20261102-024414-b4a4c0f1LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, …
pluginagentic_routing_codexunsupportedrun-20261102-024414-b4a4c0f1descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp …
pluginagentic_knowledgeunsupportedrun-20261102-024637-153cbef0LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, …
pluginagentic_routing_codexunsupportedrun-20261102-024637-153cbef0descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp …
pluginagentic_knowledgeunsupportedrun-20261102-024900-b4e495b9LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, …
pluginagentic_routing_codexunsupportedrun-20261102-024900-b4e495b9descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp …
pluginagentic_knowledgeunsupportedrun-20261102-025349-a41ee544LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, …
pluginagentic_routing_codexunsupportedrun-20261102-025349-a41ee544descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp …
pluginagentic_knowledgeunsupportedrun-20261102-025838-8e5d4349LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, …
pluginagentic_routing_codexunsupportedrun-20261102-025838-8e5d4349descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp …
pluginagentic_knowledgeunsupportedrun-20261102-030330-6e680c03LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, …
pluginagentic_routing_codexunsupportedrun-20261102-030330-6e680c03descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp …
pluginagentic_knowledgeunsupportedrun-20261102-030620-307d982cLLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, …
pluginagentic_routing_codexunsupportedrun-20261102-030620-307d982cdescriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp …
pluginagentic_knowledgeunsupportedrun-20261102-031005-8ee792e8LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, …
pluginagentic_routing_codexunsupportedrun-20261102-031005-8ee792e8descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp …
pluginagentic_knowledgeunsupportedrun-20261102-031245-d48061aaLLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, …
pluginagentic_routing_codexunsupportedrun-20261102-031245-d48061aadescriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp …
pluginagentic_knowledgeunsupportedrun-20261102-032017-d7e7635dLLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, …
pluginagentic_routing_codexunsupportedrun-20261102-032017-d7e7635ddescriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp …
pluginagentic_knowledgeunsupportedrun-20261102-032654-eef72cc9LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, …
pluginagentic_routing_codexunsupportedrun-20261102-032654-eef72cc9descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp …
modelqwen/Qwen3.8-27Bunverifiedrun-20261102-024213-cd12e80f
modelqwen/Qwen3.8-27Bunverifiedrun-20261102-024414-b4a4c0f1
modelqwen/Qwen3.8-27Bunverifiedrun-20261102-024637-153cbef0
modelqwen/Qwen3.8-Flash-Nextunverifiedrun-20261102-024900-b4e495b9
modelqwen/Qwen3.8-Flash-Nextunverifiedrun-20261102-025349-a41ee544
modelqwen/Qwen3.8-Flash-Nextunverifiedrun-20261102-025838-8e5d4349
modelauto_codexunverifiedrun-20261102-030330-6e680c03
modelauto_codexunverifiedrun-20261102-030620-307d982c
modelauto_codexunverifiedrun-20261102-031005-8ee792e8
modelauto_codexunverifiedrun-20261102-031245-d48061aa
modelauto_codexunverifiedrun-20261102-032017-d7e7635d
modelauto_codexunverifiedrun-20261102-032654-eef72cc9