MagSolutionsAIField report: 1496 real pull requests scanned, 0 hallucinated packages confirmed
Correction — 21 September 2026. This article was published automatically this morning with a headline number that was wrong, and a sentence next to it that was false. Both are corrected below, and the original claim is left visible rather than quietly edited away.
- It said 7 hallucinated packages confirmed. The correct number is 0.
- It said “each one reviewed by hand before it entered the count.” Nobody had reviewed them. The weekly sweep wrote them straight into the index, and this article read the total back out.
Six of the seven were unpublished siblings inside a monorepo.
@memberjunction/*has twenty published siblings on npm;prismarine-xbox-serviceshas a hundred and fifty-three. The seventh,@akp/application, we cannot call either way, so it stays uncounted.The mistake underneath was ours and it fits in a line: a 404 means a name is not published. It does not mean anyone invented it. An unpublished workspace package returns exactly the same 404 as a name a model made up, and no HTTP request can tell them apart.
The index now counts only names a person has confirmed, and publishes the pending and discarded counts beside the total so the discard rate is visible. Sorry — this is precisely the kind of number we ask people to hold us to.
1496 real public pull requests scanned, 1728 dependencies resolved against the live registries, 0 hallucinated packages confirmed. Measured, not estimated, and re-checked weekly.
We run a security scanner on public pull requests and publish what it finds — including what it gets wrong. This is the weekly number.
What the field looks like this week
| Pull requests analysed | 1496 |
| Dependencies resolved against live PyPI and npm | 1728 |
| Distinct package names seen | 1566 |
| Hallucinated packages confirmed | 0 |
| Candidates discarded on review | 6 |
| Candidates still pending review | 1 |
| Sweeps, since 2026-09-04 | 6 |
Zero confirmed this week, and six discarded. That discard rate is the interesting number, and it is why it is published next to the count: a 404 from a registry is a candidate, not a finding, and most candidates do not survive being looked at.
Check any of this yourself
- The raw record: https://api.magsolutionsai.com/quality
- The live field figures: https://api.magsolutionsai.com/measurement
- The corpus behind both, with every cited pull request: https://github.com/MagSolutionsAI/MagSolutionsAI.github.io/tree/main/evidence
- Written up in full: https://magsolutionsai.com/quality.html
Generated on 2026-09-21 from measurements taken on public repositories. Every figure above is resolved against the endpoints linked here before publishing, and re-checked weekly afterwards — if one stops being true, this article gets a dated correction at the top.
This report was generated from measurements, not written by hand: every figure above is resolved against our public endpoints before publishing and re-checked once a week afterwards. If one stops being true, a dated correction appears at the top of this page. How often we are wrong →