Variant
v-e816befad22e of router-config-comparison@1.0.0
runs
15
evaluated
13
success rate
73.3%
mean time
8 m 2 s
input tokens
6,520,869
generated tokens
230,871
tokens
6,751,740
tool calls
295
What this recording was
| field | value |
|---|---|
| variant | v-e816befad22e |
| experiment | router-config-comparison@1.0.0 |
| configuration | knowledge_enabled=False, model=qwen/Qwen3.8-Flash-Next |
| config hash | sha256:0aee5d2eab7cab432cd594b511295ef358b950541b6b1f85a17ee1a0a4e96949 |
| baseline | no |
In the configuration ranking
the ranking states one row per configuration, over all of its recordings; these are the numbers of that row, not of this recording alone
| place | configuration | recordings | runs | evaluated | success rate | mean time | mean input tokens | mean generated tokens | mean total tokens | mean tool calls | against the baseline |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 4 | knowledge_enabled=False, model=qwen/Qwen3.8-Flash-Next | 2 | 33 | 27 | 75.8% | 8 m 0 s | 278,068 | 10,337 | 288,405 | 16 | success -3%, time 1.46x, tokens 0.99x |
The other recordings of this configuration
the same values, recorded under another variant uid; the ranking counts this page's runs and these together
| variant | baseline | runs | evaluated | passed | success rate | mean time | input tokens | generated tokens | total tokens | tool calls |
|---|---|---|---|---|---|---|---|---|---|---|
| v-1bd895ea9679 | no | 18 | 14 | 14 | 77.8% | 7 m 59 s | 2,655,384 | 110,250 | 2,765,634 | 235 |
Runs of this recording
in the order they started; the parameters column is the configuration each run was executed under
| run | parameters | started | task | agent | repetition | status | verdict | score | time | input tokens | generated tokens | total tokens | tool calls | failure class | error |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| run-20261102-103139-c174189b | knowledge_enabled=False, model=qwen/Qwen3.8-Flash-Next | 2026-11-02 10:31:39 | cadence-takeover-001@1.0.0 | codex@codex-cli 0.166.0 | 1 | completed | passed | 1.000 | 3 m 20 s | 142,081 | 4,157 | 146,238 | 13 | ||
| run-20261102-103500-42595b9b | knowledge_enabled=False, model=qwen/Qwen3.8-Flash-Next | 2026-11-02 10:35:00 | cadence-takeover-001@1.0.0 | codex@codex-cli 0.166.0 | 2 | completed | passed | 1.000 | 3 m 6 s | 136,190 | 4,248 | 140,438 | 12 | ||
| run-20261102-103806-ca54befb | knowledge_enabled=False, model=qwen/Qwen3.8-Flash-Next | 2026-11-02 10:38:06 | cadence-takeover-001@1.0.0 | codex@codex-cli 0.166.0 | 3 | completed | passed | 1.000 | 3 m 43 s | 176,874 | 4,786 | 181,660 | 14 | ||
| run-20261102-111839-c5f2e4c1 | knowledge_enabled=False, model=qwen/Qwen3.8-Flash-Next | 2026-11-02 11:18:39 | ledgerlite-rates-001@1.0.0 | codex@codex-cli 0.166.0 | 1 | completed | passed | 1.000 | 4 m 26 s | 197,743 | 6,744 | 204,487 | 15 | ||
| run-20261102-112306-6bc27784 | knowledge_enabled=False, model=qwen/Qwen3.8-Flash-Next | 2026-11-02 11:23:06 | ledgerlite-rates-001@1.0.0 | codex@codex-cli 0.166.0 | 2 | completed | passed | 1.000 | 6 m 46 s | 359,678 | 11,459 | 371,137 | 20 | ||
| run-20261102-112953-68b1af75 | knowledge_enabled=False, model=qwen/Qwen3.8-Flash-Next | 2026-11-02 11:29:53 | ledgerlite-rates-001@1.0.0 | codex@codex-cli 0.166.0 | 3 | completed | passed | 1.000 | 5 m 12 s | 266,657 | 7,835 | 274,492 | 19 | ||
| run-20261102-120616-d577d4b1 | knowledge_enabled=False, model=qwen/Qwen3.8-Flash-Next | 2026-11-02 12:06:16 | stockpile-writeoff-001@1.0.0 | codex@codex-cli 0.166.0 | 1 | completed | passed | 1.000 | 5 m 15 s | 256,446 | 8,216 | 264,662 | 16 | ||
| run-20261102-121131-13e4a8b5 | knowledge_enabled=False, model=qwen/Qwen3.8-Flash-Next | 2026-11-02 12:11:31 | stockpile-writeoff-001@1.0.0 | codex@codex-cli 0.166.0 | 2 | completed | passed | 1.000 | 3 m 55 s | 216,253 | 5,255 | 221,508 | 12 | ||
| run-20261102-121528-03d9f220 | knowledge_enabled=False, model=qwen/Qwen3.8-Flash-Next | 2026-11-02 12:15:28 | stockpile-writeoff-001@1.0.0 | codex@codex-cli 0.166.0 | 3 | completed | passed | 1.000 | 5 m 26 s | 293,346 | 8,697 | 302,043 | 17 | ||
| run-20261102-132819-55ee19e1 | knowledge_enabled=False, model=qwen/Qwen3.8-Flash-Next | 2026-11-02 13:28:19 | cadence-catchup-001@1.0.0 | codex@codex-cli 0.166.0 | 1 | timeout | not measured | 15 m 5 s | 949,884 | 33,200 | 983,084 | 30 | agent | The `covers` helper was wrong — history-settled occurrences belong to neither list by spec. Fixing the helper. | |
| run-20261102-134326-1f193cec | knowledge_enabled=False, model=qwen/Qwen3.8-Flash-Next | 2026-11-02 13:43:26 | cadence-catchup-001@1.0.0 | codex@codex-cli 0.166.0 | 2 | completed | partial | 0.960 | 13 m 32 s | 946,332 | 28,707 | 975,039 | 30 | ||
| run-20261102-135659-dfdbb7ee | knowledge_enabled=False, model=qwen/Qwen3.8-Flash-Next | 2026-11-02 13:56:59 | cadence-catchup-001@1.0.0 | codex@codex-cli 0.166.0 | 3 | timeout | not measured | 15 m 5 s | 452,414 | 32,517 | 484,931 | 19 | agent | Several more of my expectations were wrong (an always-planned upstream settles its dependents, so real waits need staggered schedules). Rewriting with `(day, hour)` tuples and genuinely blocked upstreams. | |
| run-20261102-161153-16c04a4e | knowledge_enabled=False, model=qwen/Qwen3.8-Flash-Next | 2026-11-02 16:11:53 | freight-shop-001@1.0.0 | codex@codex-cli 0.166.0 | 1 | completed | partial | 0.941 | 11 m 36 s | 709,380 | 23,886 | 733,266 | 25 | ||
| run-20261102-162330-279c2d0e | knowledge_enabled=False, model=qwen/Qwen3.8-Flash-Next | 2026-11-02 16:23:30 | freight-shop-001@1.0.0 | codex@codex-cli 0.166.0 | 2 | completed | passed | 1.000 | 11 m 50 s | 597,870 | 25,858 | 623,728 | 24 | ||
| run-20261102-163521-b51708f9 | knowledge_enabled=False, model=qwen/Qwen3.8-Flash-Next | 2026-11-02 16:35:21 | freight-shop-001@1.0.0 | codex@codex-cli 0.166.0 | 3 | completed | passed | 1.000 | 12 m 16 s | 819,721 | 25,306 | 845,027 | 29 |
Not verified
Configurations that are not verified. They stay in the report, because a result produced under them is only as trustworthy as they are.
| kind | subject | status | run | note |
|---|---|---|---|---|
| plugin | agentic_knowledge | unsupported | run-20261102-103139-c174189b | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-103139-c174189b | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-103500-42595b9b | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-103500-42595b9b | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-103806-ca54befb | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-103806-ca54befb | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-111839-c5f2e4c1 | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-111839-c5f2e4c1 | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-112306-6bc27784 | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-112306-6bc27784 | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-112953-68b1af75 | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-112953-68b1af75 | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-120616-d577d4b1 | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-120616-d577d4b1 | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-121131-13e4a8b5 | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-121131-13e4a8b5 | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-121528-03d9f220 | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-121528-03d9f220 | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-132819-55ee19e1 | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-132819-55ee19e1 | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-134326-1f193cec | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-134326-1f193cec | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-135659-dfdbb7ee | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-135659-dfdbb7ee | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-161153-16c04a4e | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-161153-16c04a4e | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-162330-279c2d0e | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-162330-279c2d0e | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-163521-b51708f9 | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-163521-b51708f9 | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| model | qwen/Qwen3.8-Flash-Next | unverified | run-20261102-103139-c174189b | |
| model | qwen/Qwen3.8-Flash-Next | unverified | run-20261102-103500-42595b9b | |
| model | qwen/Qwen3.8-Flash-Next | unverified | run-20261102-103806-ca54befb | |
| model | qwen/Qwen3.8-Flash-Next | unverified | run-20261102-111839-c5f2e4c1 | |
| model | qwen/Qwen3.8-Flash-Next | unverified | run-20261102-112306-6bc27784 | |
| model | qwen/Qwen3.8-Flash-Next | unverified | run-20261102-112953-68b1af75 | |
| model | qwen/Qwen3.8-Flash-Next | unverified | run-20261102-120616-d577d4b1 | |
| model | qwen/Qwen3.8-Flash-Next | unverified | run-20261102-121131-13e4a8b5 | |
| model | qwen/Qwen3.8-Flash-Next | unverified | run-20261102-121528-03d9f220 | |
| model | qwen/Qwen3.8-Flash-Next | unverified | run-20261102-132819-55ee19e1 | |
| model | qwen/Qwen3.8-Flash-Next | unverified | run-20261102-134326-1f193cec | |
| model | qwen/Qwen3.8-Flash-Next | unverified | run-20261102-135659-dfdbb7ee | |
| model | qwen/Qwen3.8-Flash-Next | unverified | run-20261102-161153-16c04a4e | |
| model | qwen/Qwen3.8-Flash-Next | unverified | run-20261102-162330-279c2d0e | |
| model | qwen/Qwen3.8-Flash-Next | unverified | run-20261102-163521-b51708f9 |