- Published
- Sep 16, 2026, 06:28 AM UTC
- Question pack
- coding-fast-v4.12
- Tested
- 15 measured configurations
- Batch
- evaluation-497e806be8b8470345169c1aad6752d74a0c1ce9e195a9eb7fcc02b628701c77
Measured configurations
| Configuration | Rank | Score | Elapsed | Reference cost | Route |
|---|---|---|---|---|---|
| GPT-5.6 Sol / Max | #6 | 84/100 | 34m 23s | $1.57 | Custom endpoint |
| GPT-5.6 Sol / XHigh | #9 | 75/100 | 35m 6s | $1.22 | Custom endpoint |
| GPT-5.6 Sol / High | #11 | 73/100 | 13m 36s | $0.66 | Custom endpoint |
| GPT-5.6 Terra / Max | #12 | 73/100 | 36m 23s | $1.34 | Custom endpoint |
| GPT-5.6 Sol / Medium | #18 | 66/100 | 28m 4s | $0.74 | Custom endpoint |
| GPT-5.6 Sol / Low | #19 | 65/100 | 10m 43s | $0.53 | Custom endpoint |
| GPT-5.6 Luna / Max | #24 | 63/100 | 41m 48s | $0.16 | Custom endpoint |
| GPT-5.6 Terra / High | #35 | 55/100 | 27m 10s | $0.42 | Custom endpoint |
| GPT-5.6 Terra / Medium | #38 | 53/100 | 9m 16s | $0.25 | Custom endpoint |
| GPT-5.6 Luna / XHigh | #40 | 53/100 | 31m 15s | $0.12 | Custom endpoint |
| GPT-5.6 Terra / XHigh | #42 | 51/100 | 15m 5s | $0.50 | Custom endpoint |
| GPT-5.6 Terra / Low | #46 | 44/100 | 23m 23s | $0.26 | Custom endpoint |
| GPT-5.6 Luna / Medium | #49 | 39/100 | 10m 29s | $0.0334 | Custom endpoint |
| GPT-5.6 Luna / High | #50 | 39/100 | 17m 35s | $0.0623 | Custom endpoint |
| GPT-5.6 Luna / Low | #51 | 33/100 | 7m 40s | $0.0237 | Custom endpoint |
Five-question profile
| Question | GPT-5.6 Sol / Max | GPT-5.6 Sol / XHigh | GPT-5.6 Sol / High | GPT-5.6 Terra / Max | GPT-5.6 Sol / Medium | GPT-5.6 Sol / Low | GPT-5.6 Luna / Max | GPT-5.6 Terra / High | GPT-5.6 Terra / Medium | GPT-5.6 Luna / XHigh | GPT-5.6 Terra / XHigh | GPT-5.6 Terra / Low | GPT-5.6 Luna / Medium | GPT-5.6 Luna / High | GPT-5.6 Luna / Low |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Black-box Regression Audit | 14/20 | 14/20 | 16/20 | 15/20 | 12/20 | 16/20 | 7/20 | 12/20 | 8/20 | 10/20 | 9/20 | 7/20 | 6/20 | 7/20 | 10/20 |
| Retry Planner Counterexamples | 18/20 | 19/20 | 17/20 | 15/20 | 16/20 | 15/20 | 19/20 | 15/20 | 15/20 | 18/20 | 15/20 | 14/20 | 10/20 | 13/20 | 12/20 |
| CI Adversarial Audit | 15/20 | 12/20 | 9/20 | 11/20 | 13/20 | 12/20 | 11/20 | 11/20 | 8/20 | 9/20 | 12/20 | 5/20 | 11/20 | 10/20 | 1/20 |
| Transaction Regression Design | 17/20 | 17/20 | 19/20 | 17/20 | 13/20 | 13/20 | 17/20 | 12/20 | 13/20 | 16/20 | 12/20 | 9/20 | 12/20 | 0/20 | 10/20 |
| Cache Propagation Certificate | 20/20 | 13/20 | 12/20 | 15/20 | 12/20 | 9/20 | 9/20 | 5/20 | 9/20 | 0/20 | 3/20 | 9/20 | 0/20 | 9/20 | 0/20 |
What the same-batch spread shows
The table keeps each available variant and effort level as a separate configuration. This makes it possible to compare Sol, Terra, and Luna like for like without assigning one family-wide score.
Compare the same effort first, then check whether a higher-scoring configuration still meets your elapsed-time requirement. Reference cost remains undecided when usage pricing coverage is absent.