- Published
- Sep 16, 2026, 06:28 AM UTC
- Question pack
- coding-fast-v4.12
- Tested
- 2 measured configurations
- Batch
- evaluation-497e806be8b8470345169c1aad6752d74a0c1ce9e195a9eb7fcc02b628701c77
Results from this batch
| Measure | Qwen 3.8 / Max | Kimi K3 / High |
|---|---|---|
| Rank | #28 | #21 |
| Score | 60/100 | 65/100 |
| Elapsed | 47m 48s | 37m 56s |
| Reference cost | $1.07 | $0.89 |
| Route | Custom endpoint | Custom endpoint |
| Black-box Regression Audit | 12/20 | 16/20 |
| Retry Planner Counterexamples | 17/20 | 15/20 |
| CI Adversarial Audit | 8/20 | 14/20 |
| Transaction Regression Design | 14/20 | 12/20 |
| Cache Propagation Certificate | 9/20 | 8/20 |
How to choose from these results
Does the measured quality difference remain useful after you account for route reliability, elapsed time, and your own task profile?
Start with the score and question profile. Use elapsed time only after both configurations clear your quality floor, and leave cost undecided when either side is not covered.