BLOG
GPT-6 Claims the 'AGI Era' — I Tore Apart the Benchmarks: It's Genuinely Powerful, But AGI Is Just Marketing Talk
Opening: The Moment GPT-6 Launched, Everyone Was Shouting “The AGI Era Is Here”
On September 3, OpenAI released its new flagship model, GPT-6 Astra. President Greg Brockman said it “may ultimately be seen as the arrival of AGI,” and the media translated that directly into headlines like “Welcome to the AGI Era.”
Then came the familiar scene: screenshots of benchmarks flooded social feeds, news of a 98.6% score on ARC-AGI-3 spread everywhere, and comment sections were split half-and-half between “AGI is here” and “just another hype cycle.”
Shiwen: So this GPT-6 launch — is it a real breakthrough, or just launch event rhetoric? Yongliang: Put simply: it’s genuinely powerful, but “AGI” is just marketing talk. Shiwen: Alright, let’s break this down into what’s actually impressive and what’s overblown.
Q1: What Makes GPT-6 Actually Powerful? And Is That Power Real?
Yongliang: The power is real, and it’s concrete — not just vague claims.
Let’s start with a few hard advancements, every one of which you can verify on the official page and in third-party evaluations:
First, computer use — it can actually “use your computer.” This is GPT-6’s flagship capability: give it a goal, and it can operate a mouse and keyboard on its own, open software, fill out forms, and complete multi-step workflows end-to-end. This was an area every company hyped but delivered poorly on before. This time, it’s a genuine step forward.
Second, agentic coding — the “agent” ability to write code. On DeepSWE v1.1, a benchmark of 113 agentic programming tasks, Astra scored 74.1%, beating the previous generation GPT-5.6 Sol’s 72.7%. Don’t dismiss that two-point gap — at this high score range, every single point is hard-won.
Third, extremely long context. A 1.1M token context window means it can ingest an entire book of hundreds of thousands of words in one go and work with it, no chunking required.
Fourth, training scale. For the first time, pre-training used over 100,000 GPUs — the largest in OpenAI’s history. That’s real money poured into real compute.
In OpenAI’s official overall comparison chart, Astra sits at 59.3%, Claude Opus 5 at 55.5%, and the previous Sol at 53.6%. It has genuinely gotten better — this isn’t just a rebrand. On this point, I’ll give OpenAI credit: there’s real substance here.
Q2: It Scored 98.6% on ARC-AGI-3 — Is AGI Actually Here?
Yongliang: No. And that 98.6% number itself has some caveats.
Let’s unpack that score first. For the ARC-AGI-3 benchmark, ARC Prize ran tests under two setups: one where the model had access to its full conversation history to answer questions, and another where it only had access to its own notes. For the exact same GPT-6, one setup yielded 99.9% and the other only 62.7% — the same benchmark, a 37-point gap just from changing the setup.
What does that tell us? That “98.6%” isn’t an objective measure of capability — it’s a number under a very specific configuration. Change the harness, and the score drops dramatically. The media always grabs the highest number, but the real boundary of its ability swings somewhere between 62.7% and 99.9%.
Now let’s unpack AGI itself. What does ARC-AGI-3 actually measure? Per ARC Prize’s own description, it measures “agentic intelligence” — the ability of a model to explore, infer goals, and build internal models in novel, abstract environments. It tests whether something can “think its way through a new environment,” not whether it’s “general intelligence.” Using it as the final verdict on AGI is a bait-and-switch.
Then there’s OpenAI’s own definition of AGI — they used to say it’s “a system that can perform all economically valuable work as well as or better than a human.” Has GPT-6 reached that level? It can’t even guarantee the stability of its own service (it had a widespread outage the day before launch) — it’s still a million miles away from “all work.”
One last point that’s easy to miss: back in the ARC-AGI-2 era, GPT-5.4 scored only 0.26%, while humans scored 100%. This generation suddenly jumps to 98.6% — a leap that big naturally raises questions about whether it’s targeted benchmark gaming. The score is real, but the “weight of capability” behind that score deserves a big question mark.
So my conclusion is straightforward: it’s genuinely powerful, but AGI is just marketing talk. Framing “powerful” as “AGI” is launch event language, not technical language.
Q3: 100,000 GPUs, Recurrent Depth, Restricted Cybersecurity Capabilities, 2.5x the Price — What Is OpenAI Betting On, and What Is It Afraid Of?
Yongliang: Put these signals together, and it’s clear OpenAI is betting big while also being scared.
What’s it betting on? Compute scale. 100,000 GPUs for training, 2.5x the price, 1.1M context — this is a bet that the “throw more resources at capability” path still works, and that if it works, you can charge a premium for it. GPT-6 is the most expensive public API model in OpenAI’s history: $10 per million input tokens, $50 per million output tokens. It’s not betting on “good enough” — it’s betting that the people who want the absolute best will pay for the absolute best.
What’s it afraid of? Capability getting out of control. Three details:
First, recurrent depth “hides” the reasoning process. GPT-6 uses a new reasoning technique that obscures parts of the model’s chain of thought — you can’t see how it arrived at an answer. The Information directly called this a safety risk — the more powerful the model, the less you can see how it thinks. That alone should give you pause.
Second, cybersecurity capabilities are locked down. GPT-6 scored 100% on security benchmarks like ExploitBench, and its arbitrary code execution rate is significantly higher than the previous generation. Sounds like a good thing? But a model that can write exploit code that quickly, if made widely available, becomes a ready-made attack tool. So OpenAI is restricting its strongest cybersecurity capabilities to a small group of vetted testers for now. It hit the brakes on itself first.
Third, this launch was already delayed. Back in July, OpenAI had a security incident on Hugging Face, after which it specifically delayed the launch to add safety measures. GPT-6 is only seeing the light of day today because it’s been driven with the brakes on the whole way.
Put all this together, and you understand OpenAI’s real position: it wants you to believe AGI is here (for valuation and competition), but it knows full well that the more capable a model is, the more carefully it has to be deployed. Both mindsets exist in the same company at the same time.
Q4: Models Are Calling Themselves “AGI” — What Should Professionals Actually Be Worried About?
Yongliang: Most people are anxious about the wrong thing.
Every time a model launches, the most common comment is “That’s it, AGI is here, am I going to be replaced?” But have you really thought this through? GPT-6 being powerful doesn’t mean people who use GPT-6 are powerful.
The same knife, in the hands of a chef, makes a full Manchu Han Imperial Feast. In the hands of an ordinary person, it just cuts fingers. AI is the same way: the model’s capability going up just raises the ceiling. Whether you can reach that ceiling depends on your own ability.
So what professionals should actually be worried about isn’t “the model is too strong” — it’s these three things:
First, you only know how to “use” it, not how to judge it. Ask AI to write a proposal or a piece of code — can you confidently say “this part works, this part doesn’t”? The stronger AI gets, the more people are needed to oversee it — people who can make judgments and take responsibility for outcomes become more valuable, not less.
Second, you can’t “prove” your ability. An interviewer asks “do you know how to use AI?” and you say “yes” — then what? If you can’t explain a coherent approach, you’ll be exposed. “I sort of know AI” is the most dangerous position to be in.
Third, you’re still consuming fragmented content. One tutorial here, one video there, hundreds of items in your bookmarks, but nothing systematic in your head. Fragmented learning is the least efficient way to learn in the AI era — because the model itself can give you fragments. What you’re missing isn’t fragments, it’s a system.
Put simply: the stronger models get, the more the gap between professionals shifts from “who knows how to use the tool” to “who can prove they know how to use it, and who can use it to solve real problems.”
Q5: Models Keep Getting Stronger — How Should Professionals Learn, and How Can They Prove They Know How to Use AI?
Yongliang: Three things — just do them.
First, don’t chase the latest model — chase “provable capability.” GPT-6 is out, and next month there might be GPT-6.5. You’ll never catch up chasing versions, and you don’t need to. What you should chase is “can I use AI to do my entire job, end to end?” — if you can, that’s capability. If you can’t, the newest model in the world won’t help.
Second, turn fragments into a system. Instead of scrolling through 100 random tutorials, follow one complete learning path: from “what can AI do” to “how do I apply it to specific scenarios in my job,” until you can answer one question — “how do I actually use this stuff in my work?” What you’ve organized and can explain is what’s truly yours.
Third, make your skills verifiable and deliverable. I recommend learning systematically and getting a certification. Note that the value of a certification isn’t the piece of paper — it’s that it forces you to turn scattered experience into a system — when you’re done, you can clearly say “what I learned, what I can do, and in what scenarios it’s been validated,” instead of “I’ve watched a lot of tutorials.” Writing “proficient with AI” on your resume is worth less than one verifiable certification. For employers, “systematic and verifiable” is what gets you in the door and makes your resume stand out.
Don’t mistake getting certified as “get certified and you’ll get a raise” — that’s a lie. Think of it as “a hard constraint that forces me to organize my capabilities into a system” — that’s its real value.
Closing
Shiwen: One sentence to wrap up this episode? Yongliang: Models are still far from AGI, but your distance from “knowing how to use AI” is closer than you think — the key isn’t chasing the newest model, it’s turning what you know into something you can prove. Shiwen: That’s a line for everyone. See you next time.
[Technical Deep Dive] Why You Can’t Take Benchmark Scores at Face Value
We’ve been talking about “unpacking scores” a lot this episode, so let’s get into the technical details — why can the same model have such wildly different scores?
First, harness configuration. For the same ARC-AGI-3 benchmark, whether the model has full conversation history, and how much “scratch paper” it’s allowed, both change the result. GPT-6’s 62.7% and 99.9% are products of exactly this kind of configuration difference. A “score” is never an objective constant — it’s an output under a specific experimental setup. The media only reports the highest score, which is like only showing you your most flattering angle.
Second, overfit (benchmark gaming). The problem with public benchmarks is that model teams know what’s on the test, so they can train for it specifically. How hard was ARC-AGI-2 back in the day? GPT-5.4 only got 0.26%. Less than a year later, ARC-AGI-3 jumps straight to 98.6%. Did capability really improve that much? Some of it is real, but a big part of it is “the model learned the patterns of this specific exam.” That’s why the community is already calling for new, closed benchmarks.
Third, benchmarks “saturate,” so they keep getting replaced. ARC-AGI-2 was gamed to near-perfect scores, so ARC-AGI-3 came out with a new test format. That 98.6% you see today will probably be “reset back to earth” by a new benchmark next year. Scores chase benchmarks, not “intelligence.”
For regular people, just remember one thing: to judge if a model is good, don’t look at the headline highest score — look at how it performs on your own real tasks. Have GPT-6 write your weekly report, make a spreadsheet, look up information — whether it feels natural and useful to you is more accurate than any benchmark.
Fact-Check Table
| Claim in Article | Source |
|---|---|
| GPT-6 Astra limited preview launched 9/3, public access 9/5 | Wikipedia “GPT-6 Astra” entry |
| First training with over 100,000 GPUs (Stargate, Texas) | Wikipedia quoting OpenAI Research VP Aidan Clark |
| 1.1M token context window | llm-stats.com / ComputingForGeeks, etc. |
| Pricing $10/M input + $50/M output, ~2.5x Sol | OpenAI official + overchat.ai (Sol is $4/M) |
| Overall comparison: 59.3% (Astra) vs 55.5% (Opus 5) vs 53.6% (Sol) | OpenAI official launch page |
| DeepSWE v1.1: Astra 74.1% vs Sol 72.7% | Vellum (citing OpenAI official Coding table) |
| ARC-AGI-3 same benchmark yields 62.7% and 99.9% (harness configuration difference) | ofox.ai / ARC Prize official |
| ARC-AGI-3 measures “agentic intelligence” (exploring new environments, inferring goals) | arcprize.org official description |
| ARC-AGI-2 era GPT-5.4 only 0.26% (humans 100%) | happycapyguide citing ARC data |
| Brockman said it “may ultimately be seen as the arrival of AGI” | Wikipedia entry |
| ”Welcome to the AGI era” | Axios headline (not OpenAI official quote) |
| Recurrent depth obscures part of reasoning chain, raising monitoring concerns | Wikipedia + The Information |
| Cybersecurity capabilities restricted (Daybreak Blue testers) | Wikipedia + CNBC |
| ExploitBench 100%, arbitrary code execution rate significantly higher than previous generation | gptunnel / OpenAI official launch page |
| Launch delayed after July Hugging Face incident | Wikipedia entry |
| Certification “resume boost, foot in the door, systematic learning”, not “guaranteed pass/raise” | Guest’s personal opinion, compliant with regulatory guidelines |
Zhihu Version Differences
- Alternative title: “Does GPT-6’s 98.6% Have Caveats? Debunking the Benchmark Trap of the ‘AGI Era’” (long-tail search, keyword-rich)
- Move technical deep dive earlier: Zhihu readers prefer underlying reasoning — move the [Technical Deep Dive] section on “harness configuration / overfit / benchmark saturation” to right after Q2, and add a methodological discussion of “why closed benchmarks still struggle to prevent gaming”
- Add critical perspective: Add a line in Q2 about “the essence of launch language vs technical language” — why AI companies universally love terms like AGI and milestones: because attention equals funding, which explains why every launch comes with exaggeration
- Add “The above are personal opinions, for reference only” at the end
WeChat Official Account Layout Notes
- The opening golden line “it’s genuinely powerful, but AGI is just marketing talk” should be bolded and prominent above the fold to ensure 3-second dwell time
- Keep golden lines bolded: “the more powerful the model, the less you can see how it thinks”, “people who can make judgments and take responsibility become more valuable, not less”, “the value of a certification isn’t the piece of paper — it’s that it forces you to organize into a system”
- The four “strengths” in Q1 can be made into a “GPT-6 Real Capabilities List” card graphic, and Q2’s “62.7% vs 99.9%” can be made into a comparison graphic for easy mobile scanning