Back to leaderboard 
robots.txt AI Crawler Audit - GPTBot, ClaudeBot, Google-Extended
ReadyStack.robots-txt-ai-crawler-audit
Checks robots.txt line by line and says what each AI crawler token actually controls - training, AI-search citation, or user fetch - and which lines silently do nothing.
ai-crawlerccbotgptbotllms-txtperplexitybotrobots-txtseoLinters
Open VSX
Total downloads
6,374
Since yesterday
+60
Current version
v0.1.17
Days tracked
34
Download history
Version history
v0.1.17 from v0.1.16
Detected 2026-10-20 02:00 UTC
v0.1.16 from v0.1.15
Detected 2026-10-12 02:00 UTC
v0.1.15 from v0.1.14
Detected 2026-10-11 02:00 UTC
v0.1.14 from v0.1.13
Detected 2026-10-09 02:00 UTC
v0.1.13 from v0.1.12
Detected 2026-10-03 02:00 UTC
v0.1.12 from v0.1.11
Detected 2026-10-02 02:00 UTC
v0.1.11 from v0.1.10
Detected 2026-09-30 02:00 UTC
v0.1.10 from v0.1.8
Detected 2026-09-29 02:00 UTC
v0.1.8 from v0.1.7
Detected 2026-09-28 02:00 UTC
Snapshot log
| Scraped at (UTC) | Open VSX DLs | Version |
|---|---|---|
| 2026-10-30 02:00:00 | 6,374 | v0.1.17 |
| 2026-10-29 02:00:00 | 6,314 | v0.1.17 |
| 2026-10-28 02:00:00 | 6,268 | v0.1.17 |
| 2026-10-27 02:00:00 | 6,236 | v0.1.17 |
| 2026-10-26 02:00:00 | 6,218 | v0.1.17 |
| 2026-10-25 02:00:00 | 6,194 | v0.1.17 |
| 2026-10-24 02:00:00 | 6,176 | v0.1.17 |
| 2026-10-23 02:00:00 | 6,138 | v0.1.17 |
| 2026-10-22 02:00:00 | 6,098 | v0.1.17 |
| 2026-10-21 02:00:00 | 6,070 | v0.1.17 |
| 2026-10-20 02:00:00 | 6,024 | v0.1.17 |
| 2026-10-19 02:00:00 | 5,818 | v0.1.16 |
| 2026-10-18 02:00:00 | 5,810 | v0.1.16 |
| 2026-10-17 02:00:00 | 5,794 | v0.1.16 |
| 2026-10-16 02:00:00 | 5,786 | v0.1.16 |
| 2026-10-15 02:00:00 | 5,652 | v0.1.16 |
| 2026-10-14 02:00:00 | 5,550 | v0.1.16 |
| 2026-10-13 02:00:00 | 5,404 | v0.1.16 |
| 2026-10-12 02:00:00 | 5,282 | v0.1.16 |
| 2026-10-11 02:00:00 | 4,948 | v0.1.15 |
| 2026-10-10 02:00:00 | 4,668 | v0.1.14 |
| 2026-10-09 02:00:00 | 4,508 | v0.1.14 |
| 2026-10-08 02:00:00 | 4,178 | v0.1.13 |
| 2026-10-07 02:00:00 | 4,042 | v0.1.13 |
| 2026-10-06 02:00:00 | 3,932 | v0.1.13 |
| 2026-10-05 02:00:00 | 3,824 | v0.1.13 |
| 2026-10-04 02:00:00 | 3,674 | v0.1.13 |
| 2026-10-03 02:00:00 | 3,460 | v0.1.13 |
| 2026-10-02 02:00:00 | 3,252 | v0.1.12 |
| 2026-10-01 02:00:00 | 2,924 | v0.1.11 |
| 2026-09-30 02:00:00 | 2,722 | v0.1.11 |
| 2026-09-29 02:00:00 | 2,348 | v0.1.10 |
| 2026-09-28 02:00:00 | 2,030 | v0.1.8 |
| 2026-09-27 02:00:00 | 1,686 | v0.1.7 |