Task
slugify-001@1.0.0
runs
12
success rate
91.7%
mean time
5 m 5 s
input tokens
2,058,223
generated tokens
51,938
tokens
2,110,161
tool calls
156
What the task asks
Implement slugify from the specification
textkit.slugify is declared and specified but raises NotImplementedError. The agent implements it from the written contract, keeping casefold_word intact.
full description (markdown source)
# Implement `slugify` from the specification
`textkit.slugify` in `src/textkit/slugify.py` is declared, documented and
tested, but its body raises `NotImplementedError`. Write it.
## Contract
`slugify(text: str, separator: str = "-") -> str` builds a URL slug:
1. The result is lowercase.
2. ASCII letters (`a-z`, `A-Z`) and digits (`0-9`) are kept.
3. Every other character — including spaces, punctuation and non-ASCII letters
such as `ż` or `é` — acts as a separator. There is no transliteration.
4. A run of one or more separator characters collapses to a single `separator`.
5. The result never starts or ends with `separator`.
6. An input with no letters or digits yields an empty string.
7. `separator` is a single character and is inserted verbatim.
## Examples
| call | result |
|---|---|
| `slugify("Hello World")` | `"hello-world"` |
| `slugify("Hello, World!")` | `"hello-world"` |
| `slugify("one -- two ___ three")` | `"one-two-three"` |
| `slugify(" --Trimmed-- ")` | `"trimmed"` |
| `slugify("AK-Bench 2026")` | `"ak-bench-2026"` |
| `slugify("Zażółć")` | `"za"` |
| `slugify("Zażółć gęślami")` | `"za-g-lami"` |
| `slugify("!!!")` | `""` |
| `slugify("")` | `""` |
| `slugify("Hello World", separator="_")` | `"hello_world"` |
Applying `slugify` to its own result must not change it again.
## Boundaries
- Implement it in `src/textkit/slugify.py`; the public API of the `textkit`
package stays as it is.
- Do not modify the tests and do not add third-party dependencies — the standard
library is enough.
- The repository's own test suite (`pytest`) has to stay green, so
`textkit.casefold_word` must keep working.
Constraints
- do not change the public API of the textkit package
- do not modify the tests
- do not add third-party dependencies
Notes
The base state fails the hidden tests by design: the feature is missing, which is the opposite of the bug-fix convention where hidden tests pass on the base commit.
Recorded identity
| field | value |
|---|---|
| uid | slugify-001@1.0.0 |
| category | new_feature |
| difficulty | easy |
| evaluator | pytest_evaluator |
| base commit | HEAD |
| repository | repo-004eb785a5e88d8a |
| snapshot digest | sha256:2c39bf1195b0a52e05d98d63da379bdb54eb4cf185e30c2eb24e81eb42230177 |
| source | datasets/_sources/textkit |
| ref | HEAD |
| mutation | none |
| timeout | 300 s |
| hidden tests | hidden_tests |
| public tests | public_tests |
| definition digest | sha256:2c39bf1195b0a52e05d98d63da379bdb54eb4cf185e30c2eb24e81eb42230177 |
| definition path | /mnt/data2/dev/develop/ak-bench/datasets/new_feature/slugify-001 |
Runs of this task
in the order they started; the parameters column is the configuration each run was executed under
| run | parameters | started | agent | repetition | status | verdict | score | time | input tokens | generated tokens | total tokens | tool calls | failure class | error |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| run-20261102-080204-89ac58c6 | knowledge_enabled=False, model=qwen/Qwen3.8-27B | 2026-11-02 08:02:04 | codex@codex-cli 0.166.0 | 1 | completed | passed | 1.000 | 2 m 5 s | 88,760 | 1,406 | 90,166 | 9 | ||
| run-20261102-080410-d2c89897 | knowledge_enabled=False, model=qwen/Qwen3.8-27B | 2026-11-02 08:04:10 | codex@codex-cli 0.166.0 | 2 | timeout | not measured | 15 m 2 s | 989,571 | 18,787 | 1,008,358 | 51 | agent | I'm looking at the test results more carefully now—most of them are passing, but there are two specific cases where the hyphen handling is producing unexpected triple hyphens instead of the expected output. The issue seems to be that when I'm building the test strings with the hyphen character, something about how the input is being processed is causing the hyphens to multiply in the output. | |
| run-20261102-081913-0789e06a | knowledge_enabled=False, model=qwen/Qwen3.8-27B | 2026-11-02 08:19:13 | codex@codex-cli 0.166.0 | 3 | completed | passed | 1.000 | 2 m 35 s | 113,233 | 2,116 | 115,349 | 11 | ||
| run-20261102-082149-abdd844b | knowledge_enabled=False, model=qwen/Qwen3.8-Flash-Next | 2026-11-02 08:21:49 | codex@codex-cli 0.166.0 | 1 | completed | passed | 1.000 | 11 m 17 s | 147,994 | 7,191 | 155,185 | 12 | ||
| run-20261102-083306-81799937 | knowledge_enabled=False, model=qwen/Qwen3.8-Flash-Next | 2026-11-02 08:33:06 | codex@codex-cli 0.166.0 | 2 | completed | passed | 1.000 | 2 m 6 s | 77,350 | 2,527 | 79,877 | 8 | ||
| run-20261102-083513-c8a810d4 | knowledge_enabled=False, model=qwen/Qwen3.8-Flash-Next | 2026-11-02 08:35:13 | codex@codex-cli 0.166.0 | 3 | completed | passed | 1.000 | 9 m 13 s | 139,450 | 7,093 | 146,543 | 15 | ||
| run-20261102-084427-5c4e6c25 | knowledge_enabled=False, model=auto_codex | 2026-11-02 08:44:27 | codex@codex-cli 0.166.0 | 1 | completed | passed | 1.000 | 2 m 54 s | 92,917 | 2,289 | 95,206 | 9 | ||
| run-20261102-084722-12cb8519 | knowledge_enabled=False, model=auto_codex | 2026-11-02 08:47:22 | codex@codex-cli 0.166.0 | 2 | completed | passed | 1.000 | 2 m 3 s | 68,437 | 1,564 | 70,001 | 7 | ||
| run-20261102-084925-db233c9d | knowledge_enabled=False, model=auto_codex | 2026-11-02 08:49:25 | codex@codex-cli 0.166.0 | 3 | completed | passed | 1.000 | 1 m 60 s | 67,029 | 1,571 | 68,600 | 7 | ||
| run-20261102-085126-7081e5a7 | knowledge_enabled=True, model=auto_codex | 2026-11-02 08:51:26 | codex@codex-cli 0.166.0 | 1 | completed | passed | 1.000 | 5 m 50 s | 82,112 | 2,404 | 84,516 | 8 | ||
| run-20261102-085717-2d40efc9 | knowledge_enabled=True, model=auto_codex | 2026-11-02 08:57:17 | codex@codex-cli 0.166.0 | 2 | completed | passed | 1.000 | 3 m 35 s | 112,642 | 2,849 | 115,491 | 11 | ||
| run-20261102-090053-801966e8 | knowledge_enabled=True, model=auto_codex | 2026-11-02 09:00:53 | codex@codex-cli 0.166.0 | 3 | completed | passed | 1.000 | 2 m 15 s | 78,728 | 2,141 | 80,869 | 8 |
Not verified
Configurations that are not verified. They stay in the report, because a result produced under them is only as trustworthy as they are.
| kind | subject | status | run | note |
|---|---|---|---|---|
| plugin | agentic_knowledge | unsupported | run-20261102-080204-89ac58c6 | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-080204-89ac58c6 | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-080410-d2c89897 | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-080410-d2c89897 | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-081913-0789e06a | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-081913-0789e06a | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-082149-abdd844b | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-082149-abdd844b | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-083306-81799937 | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-083306-81799937 | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-083513-c8a810d4 | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-083513-c8a810d4 | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-084427-5c4e6c25 | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-084427-5c4e6c25 | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-084722-12cb8519 | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-084722-12cb8519 | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-084925-db233c9d | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-084925-db233c9d | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-085126-7081e5a7 | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-085126-7081e5a7 | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-085717-2d40efc9 | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-085717-2d40efc9 | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| plugin | agentic_knowledge | unsupported | run-20261102-090053-801966e8 | LLM_ROUTER_KNOWLEDGE_AGENTIC_ENABLED is not set, so the plugin keeps its built-in default; verification methods effective_router_config, plugin_runtime_events, … |
| plugin | agentic_routing_codex | unsupported | run-20261102-090053-801966e8 | descriptor declares no plugin-level toggle (benchmark.enabled_when), so activation is not observable and enabled is reported as false; verification methods resp … |
| model | qwen/Qwen3.8-27B | unverified | run-20261102-080204-89ac58c6 | |
| model | qwen/Qwen3.8-27B | unverified | run-20261102-080410-d2c89897 | |
| model | qwen/Qwen3.8-27B | unverified | run-20261102-081913-0789e06a | |
| model | qwen/Qwen3.8-Flash-Next | unverified | run-20261102-082149-abdd844b | |
| model | qwen/Qwen3.8-Flash-Next | unverified | run-20261102-083306-81799937 | |
| model | qwen/Qwen3.8-Flash-Next | unverified | run-20261102-083513-c8a810d4 | |
| model | auto_codex | unverified | run-20261102-084427-5c4e6c25 | |
| model | auto_codex | unverified | run-20261102-084722-12cb8519 | |
| model | auto_codex | unverified | run-20261102-084925-db233c9d | |
| model | auto_codex | unverified | run-20261102-085126-7081e5a7 | |
| model | auto_codex | unverified | run-20261102-085717-2d40efc9 | |
| model | auto_codex | unverified | run-20261102-090053-801966e8 |