- Published
- Oct 27, 2026, 12:22 AM UTC
- Question pack
- coding-fast-v4.12
- Tested
- 2 measured configurations
- Batch
- pricing-558102e552269389e09ab11477f8c63d3aedee5bab43565c55e4168c61b36cf4
This is a publication snapshot of retained results. Evaluations may have completed on different dates; these scores describe the backend axis, not the three-axis overall score.
Published backend results
| Measure | Qwen 3.8 / default | Kimi K3 / high |
|---|---|---|
| Backend configuration rank | #64 | #46 |
| Backend score | 64/100 | 70/100 |
| Backend tested | Sep 9, 2026, 09:53 PM UTC | Sep 9, 2026, 10:01 PM UTC |
| Elapsed | 47m 48s | 37m 56s |
| Reference cost | $1.07 | $0.89 |
| Route | Custom endpoint | Custom endpoint |
| Black-box Regression Audit | 12/20 | 16/20 |
| Retry Planner Counterexamples | 17/20 | 15/20 |
| CI Adversarial Audit | 8/20 | 14/20 |
| Transaction Regression Design | 14/20 | 12/20 |
| Cache Propagation Certificate | 13/20 | 13/20 |
How to choose from these results
Does the measured quality difference remain useful after you account for route reliability, elapsed time, and your own task profile?
Start with the backend score and question profile. Use elapsed time only after both configurations clear your quality floor, and leave cost undecided when either side is not covered.