Variant
v-dccd72463be1 of router-config-comparison@1.0.0
runs
15
evaluated
12
success rate
66.7%
mean time
7 m 47 s
input tokens
5,258,048
generated tokens
182,534
tokens
5,440,582
tool calls
276
What this recording was
| field | value |
|---|---|
| variant | v-dccd72463be1 |
| experiment | router-config-comparison@1.0.0 |
| configuration | knowledge_enabled=True, model=auto_codex |
| config hash | sha256:82ce525d9ca9c06d2d26e107882e87de84783989996d9507f54e846d37a2f90d |
| baseline | no |
In the configuration ranking
the ranking states one row per configuration, over all of its recordings; these are the numbers of that row, not of this recording alone
| place | configuration | recordings | runs | evaluated | success rate | mean time | mean input tokens | mean generated tokens | mean total tokens | mean tool calls | against the baseline |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | knowledge_enabled=True, model=auto_codex | 2 | 33 | 30 | 84.8% | 6 m 25 s | 218,490 | 7,243 | 225,734 | 14 | success +6%, time 1.17x, tokens 0.78x |
The other recordings of this configuration
the same values, recorded under another variant uid; the ranking counts this page's runs and these together
| variant | baseline | runs | evaluated | passed | success rate | mean time | input tokens | generated tokens | total tokens | tool calls |
|---|---|---|---|---|---|---|---|---|---|---|
| v-a0ed565bcc26 | no | 18 | 18 | 18 | 100.0% | 5 m 16 s | 1,952,135 | 56,493 | 2,008,628 | 178 |
Runs of this recording
in the order they started; the parameters column is the configuration each run was executed under
| run | parameters | started | task | agent | repetition | status | verdict | score | time | input tokens | generated tokens | total tokens | tool calls | failure class | error |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| run-20261102-105332-509d420c | knowledge_enabled=True, model=auto_codex | 2026-11-02 10:53:32 | cadence-takeover-001@1.0.0 | codex@codex-cli 0.166.0 | 1 | completed | passed | 1.000 | 5 m 51 s | 181,892 | 10,799 | 192,691 | 15 | ||
| run-20261102-105924-5844ec5c | knowledge_enabled=True, model=auto_codex | 2026-11-02 10:59:24 | cadence-takeover-001@1.0.0 | codex@codex-cli 0.166.0 | 2 | completed | passed | 1.000 | 7 m 45 s | 486,023 | 7,446 | 493,469 | 26 | ||
| run-20261102-110711-12e9e01e | knowledge_enabled=True, model=auto_codex | 2026-11-02 11:07:11 | cadence-takeover-001@1.0.0 | codex@codex-cli 0.166.0 | 3 | completed | passed | 1.000 | 3 m 18 s | 124,580 | 2,632 | 127,212 | 13 | ||
| run-20261102-114752-9f3f4f92 | knowledge_enabled=True, model=auto_codex | 2026-11-02 11:47:52 | ledgerlite-rates-001@1.0.0 | codex@codex-cli 0.166.0 | 1 | completed | passed | 1.000 | 2 m 59 s | 109,708 | 3,384 | 113,092 | 10 | ||
| run-20261102-115051-39be5247 | knowledge_enabled=True, model=auto_codex | 2026-11-02 11:50:51 | ledgerlite-rates-001@1.0.0 | codex@codex-cli 0.166.0 | 2 | completed | passed | 1.000 | 5 m 19 s | 233,226 | 5,848 | 239,074 | 16 | ||
| run-20261102-115611-ee478152 | knowledge_enabled=True, model=auto_codex | 2026-11-02 11:56:11 | ledgerlite-rates-001@1.0.0 | codex@codex-cli 0.166.0 | 3 | completed | passed | 1.000 | 2 m 39 s | 82,773 | 2,114 | 84,887 | 10 | ||
| run-20261102-123353-08bae5f2 | knowledge_enabled=True, model=auto_codex | 2026-11-02 12:33:53 | stockpile-writeoff-001@1.0.0 | codex@codex-cli 0.166.0 | 1 | completed | passed | 1.000 | 3 m 28 s | 174,055 | 3,204 | 177,259 | 12 | ||
| run-20261102-123722-abf5b6a0 | knowledge_enabled=True, model=auto_codex | 2026-11-02 12:37:22 | stockpile-writeoff-001@1.0.0 | codex@codex-cli 0.166.0 | 2 | completed | passed | 1.000 | 2 m 50 s | 159,452 | 1,673 | 161,125 | 12 | ||
| run-20261102-124013-4e335fc0 | knowledge_enabled=True, model=auto_codex | 2026-11-02 12:40:13 | stockpile-writeoff-001@1.0.0 | codex@codex-cli 0.166.0 | 3 | completed | passed | 1.000 | 2 m 48 s | 147,869 | 2,504 | 150,373 | 14 | ||
| run-20261102-145348-58a7a3f3 | knowledge_enabled=True, model=auto_codex | 2026-11-02 14:53:48 | cadence-catchup-001@1.0.0 | codex@codex-cli 0.166.0 | 1 | completed | partial | 0.920 | 12 m 29 s | 836,667 | 23,155 | 859,822 | 30 | ||
| run-20261102-150618-18c32d85 | knowledge_enabled=True, model=auto_codex | 2026-11-02 15:06:18 | cadence-catchup-001@1.0.0 | codex@codex-cli 0.166.0 | 2 | timeout | not measured | 15 m 4 s | 543,457 | 28,631 | 572,088 | 23 | agent | The scratch script itself is buggy. Let me print actual values to debug quickly. | |
| run-20261102-152123-401b3de4 | knowledge_enabled=True, model=auto_codex | 2026-11-02 15:21:23 | cadence-catchup-001@1.0.0 | codex@codex-cli 0.166.0 | 3 | timeout | not measured | 15 m 5 s | 405,548 | 24,862 | 430,410 | 22 | agent | Most failures are test-harness artifacts, not implementation bugs — let me fix the script (tz-aware string formatting, wrong window for the staggered case, empty `reasons()` on a fully-planned chain) and re-run. | |
| run-20261102-171345-07e43bdb | knowledge_enabled=True, model=auto_codex | 2026-11-02 17:13:45 | freight-shop-001@1.0.0 | codex@codex-cli 0.166.0 | 1 | timeout | not measured | 15 m 3 s | 581,220 | 26,757 | 607,977 | 24 | agent | All 19 spec checks pass. Final verification — full suite plus a review of the finished body: | |
| run-20261102-172849-b26fa228 | knowledge_enabled=True, model=auto_codex | 2026-11-02 17:28:49 | freight-shop-001@1.0.0 | codex@codex-cli 0.166.0 | 2 | completed | partial | 0.941 | 10 m 30 s | 515,522 | 17,044 | 532,566 | 23 | ||
| run-20261102-173920-82f7a48c | knowledge_enabled=True, model=auto_codex | 2026-11-02 17:39:20 | freight-shop-001@1.0.0 | codex@codex-cli 0.166.0 | 3 | completed | passed | 1.000 | 11 m 43 s | 676,056 | 22,481 | 698,537 | 26 |
Not verified
Configurations that are not verified. They stay in the report, because a result produced under them is only as trustworthy as they are.
| kind | subject | status | run | note |
|---|---|---|---|---|
| plugin | agentic_knowledge | unsupported | run-20261102-105332-509d420c | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-105332-509d420c | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-105924-5844ec5c | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-105924-5844ec5c | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-110711-12e9e01e | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-110711-12e9e01e | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-114752-9f3f4f92 | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-114752-9f3f4f92 | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-115051-39be5247 | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-115051-39be5247 | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-115611-ee478152 | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-115611-ee478152 | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-123353-08bae5f2 | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-123353-08bae5f2 | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-123722-abf5b6a0 | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-123722-abf5b6a0 | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-124013-4e335fc0 | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-124013-4e335fc0 | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-145348-58a7a3f3 | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-145348-58a7a3f3 | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-150618-18c32d85 | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-150618-18c32d85 | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-152123-401b3de4 | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-152123-401b3de4 | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-171345-07e43bdb | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-171345-07e43bdb | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-172849-b26fa228 | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-172849-b26fa228 | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-173920-82f7a48c | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-173920-82f7a48c | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| model | auto_codex | unverified | run-20261102-105332-509d420c | |
| model | auto_codex | unverified | run-20261102-105924-5844ec5c | |
| model | auto_codex | unverified | run-20261102-110711-12e9e01e | |
| model | auto_codex | unverified | run-20261102-114752-9f3f4f92 | |
| model | auto_codex | unverified | run-20261102-115051-39be5247 | |
| model | auto_codex | unverified | run-20261102-115611-ee478152 | |
| model | auto_codex | unverified | run-20261102-123353-08bae5f2 | |
| model | auto_codex | unverified | run-20261102-123722-abf5b6a0 | |
| model | auto_codex | unverified | run-20261102-124013-4e335fc0 | |
| model | auto_codex | unverified | run-20261102-145348-58a7a3f3 | |
| model | auto_codex | unverified | run-20261102-150618-18c32d85 | |
| model | auto_codex | unverified | run-20261102-152123-401b3de4 | |
| model | auto_codex | unverified | run-20261102-171345-07e43bdb | |
| model | auto_codex | unverified | run-20261102-172849-b26fa228 | |
| model | auto_codex | unverified | run-20261102-173920-82f7a48c |