Variant
v-5f6f603add50 of router-config-comparison@1.0.0
runs
15
evaluated
13
success rate
73.3%
mean time
7 m 0 s
input tokens
4,103,911
generated tokens
158,523
tokens
4,262,434
tool calls
227
What this recording was
| field | value |
|---|---|
| variant | v-5f6f603add50 |
| experiment | router-config-comparison@1.0.0 |
| configuration | knowledge_enabled=False, model=auto_codex |
| config hash | sha256:9157803bd654cf57d23f1d0537a8173a182e549942bb454be6dae74b44ee7386 |
| baseline | no |
In the configuration ranking
the ranking states one row per configuration, over all of its recordings; these are the numbers of that row, not of this recording alone
| place | configuration | recordings | runs | evaluated | success rate | mean time | mean input tokens | mean generated tokens | mean total tokens | mean tool calls | against the baseline |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 2 | knowledge_enabled=False, model=auto_codex | 2 | 33 | 29 | 81.8% | 6 m 41 s | 196,230 | 6,933 | 203,163 | 13 | success +3%, time 1.22x, tokens 0.70x |
The other recordings of this configuration
the same values, recorded under another variant uid; the ranking counts this page's runs and these together
| variant | baseline | runs | evaluated | passed | success rate | mean time | input tokens | generated tokens | total tokens | tool calls |
|---|---|---|---|---|---|---|---|---|---|---|
| v-17e7e69bf087 | no | 18 | 16 | 16 | 88.9% | 6 m 25 s | 2,371,680 | 70,279 | 2,441,959 | 199 |
Runs of this recording
in the order they started; the parameters column is the configuration each run was executed under
| run | parameters | started | task | agent | repetition | status | verdict | score | time | input tokens | generated tokens | total tokens | tool calls | failure class | error |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| run-20261102-104150-ee470964 | knowledge_enabled=False, model=auto_codex | 2026-11-02 10:41:50 | cadence-takeover-001@1.0.0 | codex@codex-cli 0.166.0 | 1 | completed | passed | 1.000 | 3 m 36 s | 184,263 | 2,829 | 187,092 | 15 | ||
| run-20261102-104527-df556abc | knowledge_enabled=False, model=auto_codex | 2026-11-02 10:45:27 | cadence-takeover-001@1.0.0 | codex@codex-cli 0.166.0 | 2 | completed | passed | 1.000 | 5 m 33 s | 231,250 | 6,615 | 237,865 | 16 | ||
| run-20261102-105101-7f0ea089 | knowledge_enabled=False, model=auto_codex | 2026-11-02 10:51:01 | cadence-takeover-001@1.0.0 | codex@codex-cli 0.166.0 | 3 | completed | passed | 1.000 | 2 m 31 s | 81,756 | 1,513 | 83,269 | 9 | ||
| run-20261102-113507-5543ffe1 | knowledge_enabled=False, model=auto_codex | 2026-11-02 11:35:07 | ledgerlite-rates-001@1.0.0 | codex@codex-cli 0.166.0 | 1 | completed | passed | 1.000 | 6 m 49 s | 273,433 | 8,650 | 282,083 | 17 | ||
| run-20261102-114156-1ff0ba59 | knowledge_enabled=False, model=auto_codex | 2026-11-02 11:41:56 | ledgerlite-rates-001@1.0.0 | codex@codex-cli 0.166.0 | 2 | completed | passed | 1.000 | 2 m 49 s | 97,473 | 2,294 | 99,767 | 9 | ||
| run-20261102-114446-b3513377 | knowledge_enabled=False, model=auto_codex | 2026-11-02 11:44:46 | ledgerlite-rates-001@1.0.0 | codex@codex-cli 0.166.0 | 3 | completed | passed | 1.000 | 3 m 5 s | 137,674 | 3,039 | 140,713 | 10 | ||
| run-20261102-122054-f8985461 | knowledge_enabled=False, model=auto_codex | 2026-11-02 12:20:54 | stockpile-writeoff-001@1.0.0 | codex@codex-cli 0.166.0 | 1 | completed | passed | 1.000 | 3 m 3 s | 130,716 | 3,022 | 133,738 | 10 | ||
| run-20261102-122358-fec2acfe | knowledge_enabled=False, model=auto_codex | 2026-11-02 12:23:58 | stockpile-writeoff-001@1.0.0 | codex@codex-cli 0.166.0 | 2 | completed | passed | 1.000 | 4 m 57 s | 284,384 | 4,090 | 288,474 | 16 | ||
| run-20261102-122856-43bd96e6 | knowledge_enabled=False, model=auto_codex | 2026-11-02 12:28:56 | stockpile-writeoff-001@1.0.0 | codex@codex-cli 0.166.0 | 3 | completed | passed | 1.000 | 4 m 56 s | 290,116 | 5,054 | 295,170 | 18 | ||
| run-20261102-141204-1573e7b2 | knowledge_enabled=False, model=auto_codex | 2026-11-02 14:12:04 | cadence-catchup-001@1.0.0 | codex@codex-cli 0.166.0 | 1 | timeout | not measured | 15 m 4 s | 469,387 | 26,193 | 495,580 | 19 | agent | Both were again my own expectation errors — the plan output is correct in each case. Now the critical rules: `depends_on_past`, the cap, ordering, and purity. | |
| run-20261102-142710-0136dc94 | knowledge_enabled=False, model=auto_codex | 2026-11-02 14:27:10 | cadence-catchup-001@1.0.0 | codex@codex-cli 0.166.0 | 2 | completed | passed | 1.000 | 11 m 32 s | 409,647 | 23,502 | 433,149 | 16 | ||
| run-20261102-143842-25bc3410 | knowledge_enabled=False, model=auto_codex | 2026-11-02 14:38:42 | cadence-catchup-001@1.0.0 | codex@codex-cli 0.166.0 | 3 | timeout | not measured | 15 m 5 s | 507,660 | 28,184 | 535,844 | 21 | agent | My assertion there forgot that `a`'s missing days are also planned and `b@1` is satisfied by history. Let me make the harness self-reporting so mismatches are diagnosable. | |
| run-20261102-164738-43ff8923 | knowledge_enabled=False, model=auto_codex | 2026-11-02 16:47:38 | freight-shop-001@1.0.0 | codex@codex-cli 0.166.0 | 1 | completed | partial | 0.941 | 6 m 29 s | 254,425 | 9,577 | 264,002 | 14 | ||
| run-20261102-165407-d907f577 | knowledge_enabled=False, model=auto_codex | 2026-11-02 16:54:07 | freight-shop-001@1.0.0 | codex@codex-cli 0.166.0 | 2 | completed | partial | 0.941 | 11 m 28 s | 361,986 | 19,008 | 380,994 | 17 | ||
| run-20261102-170536-da2abfe7 | knowledge_enabled=False, model=auto_codex | 2026-11-02 17:05:36 | freight-shop-001@1.0.0 | codex@codex-cli 0.166.0 | 3 | completed | passed | 1.000 | 8 m 7 s | 389,741 | 14,953 | 404,694 | 20 |
Not verified
Configurations that are not verified. They stay in the report, because a result produced under them is only as trustworthy as they are.
| kind | subject | status | run | note |
|---|---|---|---|---|
| plugin | agentic_knowledge | unsupported | run-20261102-104150-ee470964 | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-104150-ee470964 | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-104527-df556abc | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-104527-df556abc | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-105101-7f0ea089 | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-105101-7f0ea089 | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-113507-5543ffe1 | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-113507-5543ffe1 | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-114156-1ff0ba59 | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-114156-1ff0ba59 | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-114446-b3513377 | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-114446-b3513377 | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-122054-f8985461 | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-122054-f8985461 | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-122358-fec2acfe | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-122358-fec2acfe | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-122856-43bd96e6 | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-122856-43bd96e6 | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-141204-1573e7b2 | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-141204-1573e7b2 | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-142710-0136dc94 | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-142710-0136dc94 | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-143842-25bc3410 | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-143842-25bc3410 | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-164738-43ff8923 | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-164738-43ff8923 | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-165407-d907f577 | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-165407-d907f577 | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-170536-da2abfe7 | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-170536-da2abfe7 | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| model | auto_codex | unverified | run-20261102-104150-ee470964 | |
| model | auto_codex | unverified | run-20261102-104527-df556abc | |
| model | auto_codex | unverified | run-20261102-105101-7f0ea089 | |
| model | auto_codex | unverified | run-20261102-113507-5543ffe1 | |
| model | auto_codex | unverified | run-20261102-114156-1ff0ba59 | |
| model | auto_codex | unverified | run-20261102-114446-b3513377 | |
| model | auto_codex | unverified | run-20261102-122054-f8985461 | |
| model | auto_codex | unverified | run-20261102-122358-fec2acfe | |
| model | auto_codex | unverified | run-20261102-122856-43bd96e6 | |
| model | auto_codex | unverified | run-20261102-141204-1573e7b2 | |
| model | auto_codex | unverified | run-20261102-142710-0136dc94 | |
| model | auto_codex | unverified | run-20261102-143842-25bc3410 | |
| model | auto_codex | unverified | run-20261102-164738-43ff8923 | |
| model | auto_codex | unverified | run-20261102-165407-d907f577 | |
| model | auto_codex | unverified | run-20261102-170536-da2abfe7 |