What were Glasswing's actual performance in seven months?

According to VulnCheck's third-party tracking data: 442 CVEs, averaging about 63 per month, accounting for approximately 0.5% of this year's total CVEs. Only 2 were exploited in the wild; Anthropic's logs report 37,966 reports, 217 fixes, and 7,349 public disclosures, with more reports being withdrawn or duplicate than fixes. Both figures fall short of the narrative of "thousands of high-risk and critical vulnerabilities" at the time of launch.

Can this report be completely trusted?

No. VulnCheck is a vulnerability intelligence provider that competes with Anthropic. HN’s comments indicate that its conclusions are consistent with their own business model; moreover, 80.6% of the findings in their report are not disclosed, and their quality is unknown to the outside world, so it’s a case of survivor analysis. A reasonable interpretation is that the two sets of numbers have separate reporting frameworks, and judgments are made only at the intersection of them, without making one-sided conclusions.

Hot Tracking 041 | 37,966 Vulnerability Reports, Only 217 Fixed

Q: Why is the verification result of curl important?

This is a rare example of maintainers verifying AI vulnerability reports one by one and publishing their conclusions: 34 out of 45 reports have been verified, with 19 identified as not vulnerabilities, 6 confirmed as low-risk, and 9 pending evaluation. The false-positive rate is 55.9%; whereas Anthropic’s self-assessment has a true-positive rate of 94.2%. These two sets of figures cannot be the same, highlighting the problem of the verification burden being shifted to maintainers.

What Happened

Background. On April 7, 2026, Anthropic launched Project Glasswing, officially stating, “Thousands of additional high-risk and critical vulnerabilities have been identified and are being responsibly disclosed to open-source maintainers.” The launch received significant attention: post HN 1541▲/836💬 was posted, with the first update on May 22 at 561▲ and an expansion on June 2 at 180▲. In May, Anthropic launched its own disclosure ledger, with updates only periodically in batches. On October 8, they released the OSS Scanner service, claiming to have discovered over 29,000 candidate vulnerabilities but only manually reviewed about 6,000 of them, stating, “The bottleneck is our manual verification capability.” They also sent nearly 5,000 unverified reports to maintainers who requested full access. “Complete model generation, no manual review, or three-part approach” are the official statements from Anthropic.

Audit. VulnCheck started tracking on April 17, earlier than Anthropic’s own ledger. The method involved maintaining open-source repositories labeled as Anthropic Credited CVEs, comparing CVE records, GHSA, Linux kernel commits, and Anthropic’s own ledger, and manually auditing the announcements. This was not the first time VulnCheck released tracking results: since April 15’s first observation, it has continuously published ledger observations (on June 9), utilization reports (on July 28), and last phase reconciliation (on September 8). This is the fifth update in seven months. The results over seven months (all according to VulnCheck’s tracking): a total of 442 CVEs attributable to Anthropic, averaging about 63 per month, accounting for approximately 0.5% of this year’s 93,450 CVEs. Of the 442, only 2 were exploited in the wild (0.45%, CVE-2026-61500 and CVE-2026-26980, compared to VulnCheck KEV). Another comparison: the 442 tracked by VulnCheck exceeded twice the number of fixes marked by Anthropic’s own ledger (217, according to Anthropic’s self-report).

Verified Maintainers. Around November 6, three days before the blog post, curl maintainer Daniel Stenberg announced that he had received 45 vulnerability reports from Anthropic. Anthropic assessed only 1 as moderately risky, with the rest being low-risk. Daniel deliberately slowed down the transfer to GitHub to “reduce the impact on the team” and planned to make all reports public this week. Of the 34 already published, the verification results showed: 19 were marked as “not vulnerabilities,” 6 confirmed as low-risk, and 9 pending evaluation. The false-positive rate was 55.9%, while the confirmation rate was 17.6% and all were low-risk. This was not the first time: in May this year, Daniel wrote that five Mythos findings ultimately confirmed only one vulnerability. As of publication, no positive response from Anthropic has been seen ( neither HN nor the blog has any response records ).

Ledger Audit: Surge in Discoveries, Stagnant Fixing

When arranging the numbers reported by Anthropic chronologically, the issues will speak for themselves. At launch in May: 23,019 findings, with a claimed true positive rate of 90.8%. The latest update on November 2nd: 37,966 findings, with only 217 marked as fixed, and 7,349 (19.4%) made public.

Looking at the change over nearly two months from September 8th to November 2nd (all reported by Anthropic): total findings increased from 26,153 to 37,966 (+11,813); publicly disclosed findings increased from 2,736 to 7,349; fixed findings increased from 202 to 217 (only +15); retracted findings increased from 245 to 419 (+174); duplicate findings decreased from 2 to 1,670 (+1,668). In total, there were 1,654 duplicate findings (4.4%), about 7.5 times the number of fixed findings. The surge in duplicates is not accidental: VulnCheck warned in its September update that the longer reports are stored in the ledger, the higher the probability of others discovering and fixing the same vulnerabilities first. Two months later, this prediction came true: the discovery side could not keep up with the volume, leading to blockages in the fixing side, with duplicates and retractions accumulating in between.

The number of findings increased by nearly half over two months, while the number of fixed findings only increased by 15; both the increase in retracted and duplicate findings was greater than that of fixed findings. The issue VulnCheck identified is this: more retractions and duplicates than fixed findings, while the claimed true positive rate rose from 90.8% to 94.2% (91.4% at September 8th, +2.8 points). The ledger may lack a feedback loop for “three out of three failures”; reports deemed “not vulnerabilities” by maintainers are not counted. Additionally, VulnCheck found a bug in the ledger that inflated disclosed numbers (the “disclosed” filter can be reproduced).

The severity also does not match. In VulnCheck’s September findings, Claude rated 91.5% of the findings in the ledger as severe/high-risk, while maintainers only recognized 51.3%. For the same set of reports, there was a difference of 40 percentage points between machine self-assessment and maintainer judgment.

Transfer of Verification Burden: From OSS Scanner to curl

The second main criticism is not about numbers but processes. Anthropic itself acknowledged that manually classifying each finding as clearly unextensible, so on October 8th, they switched to a self-service mode: maintainers applied for scanning and classified the results themselves. After the OSS Scanner was launched, only about 6,000 of the 29,000 candidate vulnerabilities were manually reviewed, and nearly 5,000 unverified reports were directly sent to maintainers who requested full access. According to Anthropic’s own description, these reports were entirely generated by the model, without manual review or classification. If the discovery side could not keep up, who would fill the gap in verification? The case of curl provided the answer: maintainers filled it themselves. Here lies a layer of Anthropic’s own logic: the justification for introducing self-service scanning was that “if the true positive rate were as high as claimed, maintainingers classifying the results themselves should not be a problem.” However, the 55.9% false positive rate in curl precisely shows that this premise does not hold, and the cost of the self-service mode is actually borne by maintainer hours.

Daniel’s actions after receiving 45 reports deserve careful scrutiny: he slowed down the process of uploading them to GitHub to “reduce the impact on the team,” verified each report individually, and made all low- to medium-risk issues public. VulnCheck’s comment on this was that maintainers need to “control the pace” to protect their teams, which itself indicates a burden; while curl decided to make everything public is a demonstration of transparency.

It’s important to clarify the boundaries of criticism: neither this article nor VulnCheck denies that “AI can find vulnerabilities effectively.” VulnCheck repeatedly stated that AI is an effective tool, and the 442 CVEs are indeed genuine contributions. What’s criticized are two issues: the disconnect between the number of findings reported and the reality of fixes, and the systematic transfer of verification costs to open-source maintainers.

Audit of Words and Actions: The Promise Not Yet Met Seven Months Later

The first aspect is the promise regarding the scale of findings. At the start, it was claimed that “thousands of additional high-risk and critical vulnerabilities” were found; seven months later, VulnCheck reported 442 CVEs, accounting for about 0.5% of the total annual count, with only 2 being exploited in the wild. Between thousands and 442, even if all 7,349 publicly disclosed findings were counted as CVEs, there’s still a conversion factor from “finding” to CVE—while curl’s public verification rate is 17.6%, and all are low-risk.

The second aspect is the quality of disclosures. The other side of “responsible disclosure” is the quality of the disclosed information: VulnCheck’s self-assessed true positive rate was 94.2% (reported by Anthropic), compared to curl’s maintained maintainer’s false positive rate of 55.9%; Claude’s self-assessed severe/high-risk rate was 91.5%, while the maintainers only recognized 51.3%. These two sets of numbers cannot be true.

The third aspect is the way in which responses were given. The report was updated in batches, lacking a feedback loop; after the audit was released, there was no positive response until publication. Between the initial hype of 1541 ▲ and the quietness of the audit report of 35 ▲, the official side didn’t make any effort to bridge the gap. The pace itself deserves note: three official releases increased the attention, but in face of seven months of third-party verification, there was silence until publication.

VulnCheck’s original statement was: “The evidence still doesn’t match the launch narrative”—the evidence still doesn’t align with the initial claims. This article follows this principle: it doesn’t accuse of “failure” or “fabrication,” only that “there’s no match.” It must also limit both sides: VulnCheck has competing interests, and its conclusions may be suspected of sponsored content (as HN user jrflo pointed out); meanwhile, the quality of 80.6% of undisclosed findings is unknown to the outside world, and we cannot assume that 19.4% of publicly disclosed findings represent everything. This is survivor analysis; HN user dfee’s reminder holds true, and this article doesn’t make one-sided accusations.

What the Comments Say

There were 35 HN posts, with only 6 comments, a small number but with direction worth writing about, summarized into four points.

First, the “wolf came” scenario. User fny said that the most obvious sign was that no catastrophic exploits occurred in the past year: if thousands of high-risk vulnerabilities were indeed found, the number of wild exploits shouldn’t be this low.

II. The survivor bias path. User dfee reminds: Only 19.4% of findings are publicly available. This article analyzes the survivors, while the remaining 80% may be inaccurate or unverified. Conclusions should be cautious. User hologramofham questions similarly: The article draws large conclusions based on 19.4% of the data, but this ratio itself could indicate something else—the undisclosed parts might contain sensitive findings that are not suitable for public release. It’s impossible to distinguish between these two possibilities solely based on the disclosed data.

III. The conflict of interest path. User jrflo points out that VulnCheck’s conclusions are highly consistent with its business model: Companies selling vulnerability intelligence criticize others for poor vulnerability reporting quality, which could be seen as advertising.

IV. As of verification, Anthropic has not responded on HN or its blog.

Three things ordinary users can do

When receiving AI-generated vulnerability reports, follow your own three-step process. If you are a maintainer or security officer: Do not directly accept the severity labels in the reports—91.5% self-assessment is considered severe/high-risk, while maintainers only recognize 51.3%. There are empirical cases of label inflation. You can follow curl’s approach: receive reports in batches, control the pace, verify each one individually, and make conclusions public.
Before patching, investigate real-world exploitation situations and don’t be led by “high-risk” labels. Out of 442 CVEs, only 2 have been exploited in the wild (0.45%). When prioritizing fixes, refer to the KEV directory and GHSA records, and spend limited repair time on actual vulnerabilities.
Watch two key points next. First, the complete false positive rate after all 45 reports from curl are publicly available this week. Second, whether Anthropic responds and whether it addresses the three-step feedback loop. If any of these nodes change, this article will be updated to v2.

Conclusion

Statement of propriety: This article does not mention “Glasswing failure” or “data fraud.” All numbers are marked in two different ways. 442, 2, 0.5%, 55.9%, 17.6%, 91.5% vs 51.3% are from VulnCheck’s tracking or maintainer verification; 23,019, 37,966, 217, 7,349, 94.2% are from Anthropic’s self-reporting. The target of criticism is not “AI finding vulnerabilities,” but the disconnect between discovered incidents and actual fixes, as well as the shifting of verification burden to open-source maintainers. Both sides’ uncertainties are included in the text; no unilateral judgment is made.

Verifying itself is more valuable than making a qualitative assessment: For the account to convince everyone, what’s lacking is not a higher true positive rate for self-assessment, but a mechanism that allows maintainer verification results to return to the denominator. Within 48 hours (from the blog post on November 9th to HN listing on November 10th), if Anthropic responds or curl’s full verification results are available, they will be updated in this article.

Reference sources

VulnCheck blog “After 7-months, Glasswing Overwhelms Maintainers with Underwhelming Findings” (Published by Patrick Garrity on November 9, 2026; full text directly accessed for 200 verifications; primary source, but note its competing vendor stance).
Anthropic official statement: Project Glasswing launch announcement (April 7, 2026), ledger disclosure (launched in May, latest update on November 2), OSS Scanner documentation (October 8, 2026; “complete model generation, no manual review or triage” are the official statements).
Curly maintainer Daniel Stenberg public statement (beginning around November 6, 2026; 45 reports, 34 verification results are the primary source for maintainers).
Hacker News story 50447074 (listed on November 10, 2026; 35▲/6💬 when writing, Algolia snapshot; all comments are quoted, username retained).
HN historical popularity data (start post on April 7 with 1541▲/836💬, updated on May 22 with 561▲, expanded on June 2 with 180▲, all historical snapshot data).

Author: Yongliang