OpenAI has published a post titled “How enabling two settings tripled our scores on the ARC-AGI-3 benchmark”, and the-decoder.com reports that the company is using those results to claim a GPT-5.6 Sol ARC-AGI-3 lead over the rival Opus 5 model when the benchmark is run through OpenAI’s latest API with two additional settings switched on. The claim is notable less for the leaderboard position than for what it says about how much benchmark results depend on the exact configuration used to produce them — a detail that rarely survives the journey from a vendor blog post to a chart on a slide.
Key takeaways
- OpenAI has published a post stating that enabling two settings tripled its scores on the ARC-AGI-3 benchmark, according to the company’s own headline.
- the-decoder.com reports that OpenAI claims GPT-5.6 Sol beats Opus 5 on ARC-AGI-3 using its latest API plus those two additional settings.
- The reported threefold improvement comes from configuration changes rather than a new model release, based on the framing in OpenAI’s own post.
- Neither source snippet gives specific scores, the identity of the two settings, or the evaluation dates, so those figures should not be assumed.
- For developers, the practical lesson is that API-level configuration can move benchmark outcomes substantially — results are only comparable when the harness is identical.
What OpenAI has actually claimed about GPT-5.6 Sol on ARC-AGI-3
The primary source here is OpenAI itself. The company’s post is titled “How enabling two settings tripled our scores on the ARC-AGI-3 benchmark”, which establishes three things: the benchmark in question is ARC-AGI-3, the improvement is attributed to enabling two settings, and the magnitude of that improvement is described as tripling the score.
The second source, the-decoder.com, adds the competitive framing. Its report states that OpenAI claims GPT-5.6 Sol beats Opus 5 on ARC-AGI-3 with its latest API and two additional settings. That is the extent of what is on the record from the material available at the time of writing. The underlying score values, the ordering of models before the settings were enabled, the number of tasks run, and the compute or token budget per attempt are not specified in the source snippets, and no responsible reading should fill those gaps by inference.
It is worth stating the shape of the claim precisely, because it is easy to misread. OpenAI is not reported to be announcing a new model. It is reported to be announcing a new result for an existing model, obtained by changing how that model is invoked.
Why “two settings” is the most important phrase in the story
A tripling of a benchmark score from configuration changes alone is a large swing, and it points to a broader issue in how the industry compares models. Frontier evaluations are not simply a matter of sending a prompt and recording an answer. The result depends on the scaffolding around the model: how many attempts it is allowed, how long it can reason before responding, whether it can call tools, how its outputs are parsed, and how failures are retried. Change any of those and the number changes with it.
This is general context rather than a reported detail — the sources do not identify which two settings OpenAI enabled. But the direction of the finding is consistent with a well-known pattern: on benchmarks designed to test reasoning and adaptation rather than recall, the harness often accounts for as much variance as the underlying weights do. That makes a headline like “model A beats model B” incomplete without the configuration that produced it.
For anyone maintaining a shortlist of candidate models, this is a reason to treat vendor-published comparisons as a starting point rather than a verdict. Our AI models database tracks published specifications across providers, but no static table can capture the runtime settings that a benchmark run used.
GPT-5.6 Sol vs Opus 5: what the sources establish, and what they do not
Given the volume of secondary coverage that a claim like this tends to generate, it is useful to separate the reported facts from the details that are simply absent. The table below reflects only what appears in the two available sources.
| Detail | Status in the available sources |
|---|---|
| Benchmark | ARC-AGI-3, named in both OpenAI’s post title and the-decoder.com’s report |
| Models compared | GPT-5.6 Sol and Opus 5, per the-decoder.com |
| Claimed outcome | OpenAI claims GPT-5.6 Sol beats Opus 5, per the-decoder.com |
| Reported improvement | Scores “tripled” after enabling two settings, per OpenAI’s post title |
| Conditions | OpenAI’s latest API plus two additional settings, per the-decoder.com |
| Specific scores | Not present in the source snippets |
| Identity of the two settings | Not present in the source snippets |
| Independent verification | Not present in the source snippets |
The last row deserves emphasis. As reported, this is a vendor claim about a vendor’s own model, published by the vendor, and relayed by the-decoder.com as a claim rather than a confirmed third-party result. That is not a criticism of the work — publishing configuration details is more transparent than publishing a bare number — but it does set the standard of evidence.
What this means for developers building on the OpenAI API
The practical takeaway for developers is not “switch models”. It is that the defaults you inherit when you call an API may not be the settings that produced the numbers you read about. If a published result depends on two non-default settings, then a team reproducing that workload with default parameters should expect a materially different outcome.
Three consequences follow. First, benchmark-driven model selection should be paired with an internal evaluation on your own tasks, run through the exact configuration you intend to ship. Second, any configuration that raises scores by expanding reasoning effort or retries will usually raise cost as well, so the comparison that matters is quality per unit of spend rather than quality alone — something our AI API cost calculator is designed to make concrete. Third, when comparing across providers, the settings must be matched on both sides or the comparison is not a comparison.
None of the sources state the cost implications of OpenAI’s two settings, so that connection is offered here as general analysis rather than reported fact. But it is the question most engineering teams will ask next, and it is a reasonable one to put to any vendor publishing a configuration-dependent result.
ARC-AGI-3 and the shift towards interactive evaluation
The ARC-AGI family of benchmarks has become a reference point in discussions of general reasoning precisely because it resists the memorisation strategies that inflate scores on older test sets. Successive versions have pushed towards tasks that require a model to work out rules it has not seen before, rather than retrieve a pattern it has.
That design has a side effect relevant to this story: benchmarks that reward exploration and multi-step problem solving are unusually sensitive to how much room the harness gives the model to explore. A configuration change that allows more deliberation or more structured interaction can therefore produce a larger jump than the same change would on a single-turn question-answering test. Again, this is background context on how such benchmarks behave, not a claim about OpenAI’s specific settings, which the sources do not describe.
Teams evaluating models for autonomous, multi-step work will recognise the pattern from their own testing — the same sensitivity to scaffolding shows up across the tooling covered in our roundup of AI coding agents, where harness design often separates a usable agent from an unusable one running the same underlying model.
How to read vendor benchmark claims without overcorrecting
There is a temptation to dismiss configuration-dependent results as marketing. That would be the wrong lesson. A vendor that publishes which settings it enabled has given the community more to work with than one that publishes a number alone, and the resulting claim is at least testable in principle.
The useful posture sits between credulity and dismissal. Treat the claim as reported: OpenAI says enabling two settings tripled its ARC-AGI-3 scores, and OpenAI says the resulting figures put GPT-5.6 Sol ahead of Opus 5. Wait for independent runs before treating the ordering as settled. And when the specific settings and scores are published in full, check whether the same configuration is available and affordable in the tier you actually use. Where cost is part of the decision, our AI price-performance index tracks how far a given quality level stretches across providers.
Frequently asked questions
What exactly did OpenAI claim about GPT-5.6 Sol on ARC-AGI-3? According to the-decoder.com, OpenAI claims GPT-5.6 Sol beats Opus 5 on ARC-AGI-3 when run with its latest API and two additional settings enabled. OpenAI’s own post is titled “How enabling two settings tripled our scores on the ARC-AGI-3 benchmark”.
Which two settings were enabled? The available source snippets do not name them. Until OpenAI’s full post details are confirmed, any specific identification would be speculation.
Does this mean GPT-5.6 Sol is a new model? Nothing in the sources indicates a new model release. The reported change concerns how an existing model was configured and invoked through OpenAI’s latest API.
Has the result been independently verified? Not according to the material available. the-decoder.com frames it as an OpenAI claim, and no third-party confirmation appears in the sources.
Should this change which model I use in production? Not on its own. A configuration-dependent benchmark result is weak evidence for a production decision; an evaluation on your own workload, using your own settings and budget, is stronger.
The bottom line
The reported GPT-5.6 Sol ARC-AGI-3 claim is a useful data point wrapped in an important caveat. OpenAI says two settings tripled its scores, and the-decoder.com reports the company now places GPT-5.6 Sol ahead of Opus 5 on that benchmark using its latest API. What the sources do not provide — the scores, the settings, the cost, or independent replication — is exactly what would be needed to treat the ordering as durable rather than provisional. Until those details are on the record, the safest reading is that configuration has been shown to matter a great deal on ARC-AGI-3, and that any leaderboard comparison drawn without matching configurations is not measuring what it appears to measure.
Sources: news.google.com. Reported July 30, 2026.
