Variant
v-4d3337664165 of router-config-comparison@1.0.0
runs
15
evaluated
11
success rate
60.0%
mean time
7 m 1 s
input tokens
6,072,271
generated tokens
134,079
tokens
6,206,350
tool calls
281
What this recording was
| field | value |
|---|---|
| variant | v-4d3337664165 |
| experiment | router-config-comparison@1.0.0 |
| configuration | knowledge_enabled=False, model=qwen/Qwen3.8-27B |
| config hash | sha256:8da91d2e41ca62fb67cac3bde630ea18000e4d68c0e62430ffda49d61f483fa9 |
| baseline | yes |
In the configuration ranking
the ranking states one row per configuration, over all of its recordings; these are the numbers of that row, not of this recording alone
| place | configuration | recordings | runs | evaluated | success rate | mean time | mean input tokens | mean generated tokens | mean total tokens | mean tool calls | against the baseline |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 3 | knowledge_enabled=False, model=qwen/Qwen3.8-27B | 2 | 33 | 28 | 78.8% | 5 m 28 s | 284,222 | 6,408 | 290,630 | 16 |
The other recordings of this configuration
the same values, recorded under another variant uid; the ranking counts this page's runs and these together
| variant | baseline | runs | evaluated | passed | success rate | mean time | input tokens | generated tokens | total tokens | tool calls |
|---|---|---|---|---|---|---|---|---|---|---|
| v-5dfe3345dc0f | yes | 18 | 17 | 17 | 94.4% | 4 m 10 s | 3,307,047 | 77,381 | 3,384,428 | 244 |
Runs of this recording
in the order they started; the parameters column is the configuration each run was executed under
| run | parameters | started | task | agent | repetition | status | verdict | score | time | input tokens | generated tokens | total tokens | tool calls | failure class | error |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| run-20261102-102224-77a554a3 | knowledge_enabled=False, model=qwen/Qwen3.8-27B | 2026-11-02 10:22:24 | cadence-takeover-001@1.0.0 | codex@codex-cli 0.166.0 | 1 | completed | passed | 1.000 | 2 m 25 s | 87,441 | 2,062 | 89,503 | 9 | ||
| run-20261102-102450-4c4dafd8 | knowledge_enabled=False, model=qwen/Qwen3.8-27B | 2026-11-02 10:24:50 | cadence-takeover-001@1.0.0 | codex@codex-cli 0.166.0 | 2 | completed | passed | 1.000 | 3 m 56 s | 184,037 | 3,928 | 187,965 | 13 | ||
| run-20261102-102846-f9da52d1 | knowledge_enabled=False, model=qwen/Qwen3.8-27B | 2026-11-02 10:28:46 | cadence-takeover-001@1.0.0 | codex@codex-cli 0.166.0 | 3 | completed | passed | 1.000 | 2 m 53 s | 166,105 | 2,393 | 168,498 | 14 | ||
| run-20261102-111029-a9c04726 | knowledge_enabled=False, model=qwen/Qwen3.8-27B | 2026-11-02 11:10:29 | ledgerlite-rates-001@1.0.0 | codex@codex-cli 0.166.0 | 1 | completed | passed | 1.000 | 2 m 57 s | 118,837 | 2,616 | 121,453 | 12 | ||
| run-20261102-111326-3c584bd3 | knowledge_enabled=False, model=qwen/Qwen3.8-27B | 2026-11-02 11:13:26 | ledgerlite-rates-001@1.0.0 | codex@codex-cli 0.166.0 | 2 | completed | passed | 1.000 | 2 m 10 s | 81,280 | 1,903 | 83,183 | 7 | ||
| run-20261102-111537-b9ab2fef | knowledge_enabled=False, model=qwen/Qwen3.8-27B | 2026-11-02 11:15:37 | ledgerlite-rates-001@1.0.0 | codex@codex-cli 0.166.0 | 3 | completed | passed | 1.000 | 3 m 1 s | 130,324 | 2,843 | 133,167 | 12 | ||
| run-20261102-115850-e5f2348f | knowledge_enabled=False, model=qwen/Qwen3.8-27B | 2026-11-02 11:58:50 | stockpile-writeoff-001@1.0.0 | codex@codex-cli 0.166.0 | 1 | completed | passed | 1.000 | 3 m 30 s | 174,833 | 3,032 | 177,865 | 12 | ||
| run-20261102-120221-58de15af | knowledge_enabled=False, model=qwen/Qwen3.8-27B | 2026-11-02 12:02:21 | stockpile-writeoff-001@1.0.0 | codex@codex-cli 0.166.0 | 2 | completed | passed | 1.000 | 2 m 13 s | 131,934 | 1,632 | 133,566 | 10 | ||
| run-20261102-120435-e4b44447 | knowledge_enabled=False, model=qwen/Qwen3.8-27B | 2026-11-02 12:04:35 | stockpile-writeoff-001@1.0.0 | codex@codex-cli 0.166.0 | 3 | completed | passed | 1.000 | 1 m 40 s | 86,407 | 948 | 87,355 | 7 | ||
| run-20261102-124303-45bdfb12 | knowledge_enabled=False, model=qwen/Qwen3.8-27B | 2026-11-02 12:43:03 | cadence-catchup-001@1.0.0 | codex@codex-cli 0.166.0 | 1 | timeout | not measured | 15 m 4 s | 1,030,274 | 21,301 | 1,051,575 | 39 | agent | Suite is green. Let me re-verify the full spec behavior with the final implementation, including a few extra edge cases I hadn't covered (staggered schedules, attempts, DST). | |
| run-20261102-125808-280ac90d | knowledge_enabled=False, model=qwen/Qwen3.8-27B | 2026-11-02 12:58:08 | cadence-catchup-001@1.0.0 | codex@codex-cli 0.166.0 | 2 | timeout | not measured | 15 m 5 s | 823,347 | 18,432 | 841,779 | 33 | agent | The implementation is complete and syntactically valid. Let me run the full test suite and my spec-based checks to confirm everything is green. | |
| run-20261102-131314-47751b92 | knowledge_enabled=False, model=qwen/Qwen3.8-27B | 2026-11-02 13:13:14 | cadence-catchup-001@1.0.0 | codex@codex-cli 0.166.0 | 3 | timeout | not measured | 15 m 5 s | 1,793,968 | 20,171 | 1,814,139 | 53 | agent | Now let me fix my test's history times (they must match the 06:00 occurrence instants, per "at that exact logical time") and re-run. | |
| run-20261102-153629-de5ac66d | knowledge_enabled=False, model=qwen/Qwen3.8-27B | 2026-11-02 15:36:29 | freight-shop-001@1.0.0 | codex@codex-cli 0.166.0 | 1 | completed | partial | 0.941 | 9 m 40 s | 514,930 | 12,666 | 527,596 | 25 | ||
| run-20261102-154610-09481f31 | knowledge_enabled=False, model=qwen/Qwen3.8-27B | 2026-11-02 15:46:10 | freight-shop-001@1.0.0 | codex@codex-cli 0.166.0 | 2 | timeout | not measured | 15 m 3 s | 345,908 | 24,250 | 370,158 | 18 | agent | The code works correctly — my test had the wrong assumption. I forgot the default divisor is 5000, so 130×30×20 gives volumetric 15.6 kg (billed 16.0 kg), not 3.0 kg. The linehaul is 4200, surcharge 500, total 4700 — and the group logic worked perfectly: `oversize` won, `heavy` was suppressed, winner named. | |
| run-20261102-160115-cb48ab31 | knowledge_enabled=False, model=qwen/Qwen3.8-27B | 2026-11-02 16:01:15 | freight-shop-001@1.0.0 | codex@codex-cli 0.166.0 | 3 | completed | partial | 0.941 | 10 m 38 s | 402,646 | 15,902 | 418,548 | 17 |
Not verified
Configurations that are not verified. They stay in the report, because a result produced under them is only as trustworthy as they are.
| kind | subject | status | run | note |
|---|---|---|---|---|
| plugin | agentic_knowledge | unsupported | run-20261102-102224-77a554a3 | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-102224-77a554a3 | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-102450-4c4dafd8 | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-102450-4c4dafd8 | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-102846-f9da52d1 | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-102846-f9da52d1 | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-111029-a9c04726 | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-111029-a9c04726 | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-111326-3c584bd3 | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-111326-3c584bd3 | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-111537-b9ab2fef | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-111537-b9ab2fef | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-115850-e5f2348f | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-115850-e5f2348f | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-120221-58de15af | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-120221-58de15af | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-120435-e4b44447 | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-120435-e4b44447 | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-124303-45bdfb12 | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-124303-45bdfb12 | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-125808-280ac90d | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-125808-280ac90d | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-131314-47751b92 | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-131314-47751b92 | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-153629-de5ac66d | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-153629-de5ac66d | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-154610-09481f31 | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-154610-09481f31 | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-160115-cb48ab31 | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-160115-cb48ab31 | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| model | qwen/Qwen3.8-27B | unverified | run-20261102-102224-77a554a3 | |
| model | qwen/Qwen3.8-27B | unverified | run-20261102-102450-4c4dafd8 | |
| model | qwen/Qwen3.8-27B | unverified | run-20261102-102846-f9da52d1 | |
| model | qwen/Qwen3.8-27B | unverified | run-20261102-111029-a9c04726 | |
| model | qwen/Qwen3.8-27B | unverified | run-20261102-111326-3c584bd3 | |
| model | qwen/Qwen3.8-27B | unverified | run-20261102-111537-b9ab2fef | |
| model | qwen/Qwen3.8-27B | unverified | run-20261102-115850-e5f2348f | |
| model | qwen/Qwen3.8-27B | unverified | run-20261102-120221-58de15af | |
| model | qwen/Qwen3.8-27B | unverified | run-20261102-120435-e4b44447 | |
| model | qwen/Qwen3.8-27B | unverified | run-20261102-124303-45bdfb12 | |
| model | qwen/Qwen3.8-27B | unverified | run-20261102-125808-280ac90d | |
| model | qwen/Qwen3.8-27B | unverified | run-20261102-131314-47751b92 | |
| model | qwen/Qwen3.8-27B | unverified | run-20261102-153629-de5ac66d | |
| model | qwen/Qwen3.8-27B | unverified | run-20261102-154610-09481f31 | |
| model | qwen/Qwen3.8-27B | unverified | run-20261102-160115-cb48ab31 |