AK-Benchbenchmark report

Tasks

every recorded task

taskcategorydifficultyevaluatorrunsevaluatedpassedsuccess ratemean timeinput tokensgenerated tokenstotal tokenstool calls
bookshop-stock-002@1.0.0bug_fixexpertpytest_evaluator121212100.0%4 m 5 s1,966,13952,1142,018,253160
bookshop-tiers-001@1.0.0bug_fixhardpytest_evaluator121212100.0%4 m 37 s1,224,69332,5611,257,254111
cadence-catchup-001@1.0.0new_featureexpertpytest_evaluator12318.3%14 m 26 s9,168,585308,8559,477,440335
cadence-takeover-001@1.0.0bug_fixexpertpytest_evaluator121212100.0%3 m 60 s2,182,49253,4082,235,900169
freight-shop-001@1.0.0new_featureexpertpytest_evaluator1210433.3%11 m 12 s6,169,405237,6886,407,093262
ledgerlite-rates-001@1.0.0bug_fixexpertpytest_evaluator121212100.0%4 m 1 s2,088,80658,7292,147,535157
ledgerlite-reconcile-001@1.0.0new_featureexpertpytest_evaluator128433.3%11 m 29 s6,164,860235,3776,400,237259
notes-search-001@1.0.0new_featurehardpytest_evaluator128866.7%8 m 52 s2,184,30583,3612,267,666175
notes-sync-001@1.0.0new_featureexpertpytest_evaluator121212100.0%5 m 50 s1,971,92174,1782,046,099152
slugify-001@1.0.0new_featureeasypytest_evaluator12111191.7%5 m 5 s2,058,22351,9382,110,161156
stockpile-fefo-001@1.0.0new_featureexpertpytest_evaluator121212100.0%6 m 12 s3,412,912103,4283,516,340203
stockpile-writeoff-001@1.0.0bug_fixexpertpytest_evaluator121212100.0%3 m 40 s2,345,81147,3272,393,138156
wordcount-001@1.0.0bug_fixeasypytest_evaluator12101083.3%7 m 17 s880,96520,251901,216102