Research report
We scanned 18,750 public AI agent skills, rules and commands. Here’s what we found.
We cloned 38 of the most popular public repositories that publish skills, rules, commands, subagents, plugins and agent configuration for Claude Code, Codex, Cursor, GitHub Copilot and Gemini CLI. Together they have 1.35 million GitHub stars. We ran rookery (opens in a new tab) over every one, then read every high-severity alert by hand.
Most of what we found is not scary. That is the finding.
38%copies
Only 11,617 of the 18,750 agent files are unique.
0hidden instructions
All 30 hidden-Unicode alerts were ordinary text, and none of 119 exfiltration alerts was real.
15of 13,225 skills
had a verified high-severity issue: about one in 900. Every one was deliberate.
Check your own repos
npx @thinkingraven/rookery scan .Finding 1
More than a third of it is copies.
Of 18,750 files, 11,617 are unique. Three things account for most of the copying.
Unique files and exact copies
18,750 agent files, compared byte for byte. Rookery also found 3,852 clusters of near-duplicates.
- Unique files11,617 (62%)
- Exact copies of another file7,133 (38%)
Catalogues copy each other
One large catalogue publishes every skill in three folders, one per install target: 8,419 files, but only 2,744 unique. Across the whole corpus, 1,001 files appear byte-for-byte in two or more repos.
Anthropic’s sample skills are the most copied files in the ecosystem
brand-guidelines and skill-creator each appear in 7 other repos, and every sample skill in the chart turns up in at least 4.
The copies drift. The pdf cluster mixes pdf and pdf-official at 89% similarity, and some copies have been renamed or partly rewritten.
Templates
1,843 files are one vendor’s “Automate App tasks via Rube MCP” template, filled in once per SaaS product and published in five repos.
Anthropic sample skills found in other repos
Number of other repositories carrying a copy
Copying isn’t bad in itself; that is how good skills spread. But a copied skill no longer gets its upstream’s fixes. When you find several versions of the same skill, you can’t tell which one is current without diffing them.
What to do
Rookery’s near-duplicate report names every cluster and every repo in it. Before copying a skill into a new repo, check whether your org already has one; if you do fork it, note where it came from, so the next person can tell which copy is current.
Finding 2
Everyone writes their own code-review skill.
102 code-review skills in 16 repos reduce to 56 distinct texts. Almost every repo wrote its own.
- 102code-review skills, commands and subagents
- 56distinct texts once copies are merged
- 15 of 16repos wrote at least one of their own
We looked at skills, commands and subagents whose name says they review code: code-reviewer, review, code-review-checklist, receiving-code-review and requesting-code-review and so on.
- There are 102 of them (62 skills, 24 subagents and 16 commands), in 16 of the 36 repos.
- They reduce to 56 distinct texts.
- 15 of those 16 repos have at least one code-review skill whose text appears nowhere else.
- Only 8 of them appear in more than one repo.
The names people pick
code-reviewer×16 skills with this namereview×6 skills with this namecode-review-checklist×6 skills with this namereceiving-code-review×6 skills with this namerequesting-code-review×6 skills with this namecode-review-excellence×5 skills with this namecode-review-and-quality×4 skills with this namereview-pr×3 skills with this namecode-review×3 skills with this namecode-review-ai-ai-review×3 skills with this namecode-review-sensei×3 skills with this namecomprehensive-review-pr-enhance×3 skills with this name
For a team, that is the candidate for one shared, reviewed skill instead of many private ones.
What to do
If rookery shows three or more repos writing the same kind of skill, that’s your merge candidate: pick the text your team already trusts, delete the rest, and have every repo point at one shared, reviewed copy.
Finding 3
No smuggled instructions, no credential theft.
This is the result we expected least, and checked hardest.
0
hidden instructions
in 30 hidden-Unicode alerts
0
verified exfiltration
in 119 exfiltration alerts
0
confirmed leaked credentials
in 15 hard-coded-secret alerts
0
real prompt injections
in 11 “ignore previous instructions” alerts
Hidden Unicode
Rookery flagged 30 lines with invisible characters. We decoded every one. None hid an instruction.
We also searched all 69,839 files independently for Unicode tag characters, the classic “invisible prompt” trick: there were none. The only bidirectional-override characters were in minified JavaScript bundles and a test file, not in agent files.
What the invisible characters were
| What it was | Lines |
|---|---|
| Deliberate examples in prompt-injection training material, and a quoted sample of AI-written text | 11 |
| Zero-width spaces used to escape Markdown code fences or backticks | 9 |
| Stray characters copied from a web page | 4 |
| Zero-width non-joiners in Persian text, next to punctuation (rookery misread these) | 3 |
| A byte-order mark shown on purpose in a CSV example | 3 |
| Hidden-Unicode alerts | 30 |
Exfiltration
119 alerts pointed at webhook and paste services, at env/secrets in network calls, or at reads of credential files. None was a skill sending your data somewhere you wouldn’t expect:
- Slack and Discord webhooks the user sets up for notifications;
ngrokused as a local development tunnel;- webhook.site suggested for testing;
- out-of-band callback hosts in penetration-testing tutorials.
Secrets
No live credential was confirmed:
- Most of the 15 hard-coded-secret alerts were placeholders or “❌ bad example” snippets.
- Four were design-token names that only look like secrets.
- A few credential-shaped database passwords sit in a copied vendor example response. They look like dead samples; we haven’t confirmed that yet.
- An independent search for AWS, GitHub, Slack, OpenAI, Anthropic, Google and GitLab token formats, and for private-key blocks, found only truncated placeholders.
Prompt injection
All 11 “ignore previous instructions”-style alerts were one of two things:
- attack examples inside security skills;
- harmless phrases that only resemble an override.
What to do
A clean scan today doesn’t make a skill clean forever. Scan what you install, not just what you write: rookery scan reads text in under a second and never executes anything, so there’s no reason to skip it.
Finding 4
The real risk is switching the safety off.
What we did find is design choices: skills that make disabling the agent’s approval prompts, or harvesting the user’s secrets, part of normal use. They are usually documented in the original repo, but the warning doesn’t always travel with the copy.
28
lines in 15 files tell the user or the agent to run with approvals off.
52
lines in 4 files send the agent through password managers and other stores of secrets.
4
project settings files pre-approve any shell command.
564
skills and commands pre-approve unrestricted Bash while they run: 279 skills and 285 commands.
Approvals off
The 15 files use flags and settings such as --dangerously-, --yolo, --approval-mode yolo, bypassPermissions and approval_policy = "never". Some examples, with the repositories left out:
- An autonomous skill in a popular collection, built to keep working until a task is finished, lists
claude --dangerously-as a prerequisite. Its bundled runner script launches every iteration that way.skip- permissions - A plan-review skill gets a second opinion by running Gemini CLI with
--approval-mode yolo, so every tool call the second model makes goes through without the user being asked. - Delegation skills that hand work to another coding CLI pass
--yoloby default for any run that writes files, and their instructions treat that as simply what such a run requires. The CLI they delegate to has no sandbox. - A plugin subagent starts autonomous Claude Code sessions on a remote machine over SSH, as another user through
sudo, with--dangerously-.skip- permissions - A widely copied example configuration ships a subagent with
permissionMode: bypassPermissionsand a Codex config withsandbox_andmode = "danger- full- access" approval_policy = "never". Anyone who opens a clone of it in those tools inherits the settings.
One plugin sweeps your machine for secrets
Rookery’s credential-sweep rule flags 52 lines across four files of one plugin, its setup wizard among them. They have the agent query password managers, the macOS Keychain, browser history and other projects’ .env files, and some print stored passwords outright. Every value the agent finds lands in the model’s context and the session transcript. The plugin presents this as a convenience: the user shouldn’t have to look up a key the machine already holds.
The other 15 credential-lookup alerts read one named secret the user had chosen, which is fine.
Broad permissions
- Four project settings files pre-approve any shell command. One project template allows plain
Bashalongside a careful list of scoped commands, which makes the careful list pointless. - 279 skills (203 unique) and 285 commands pre-approve unrestricted Bash through
allowed-toolswhile they run. That is about one skill in 50.
None of these is a vulnerability in the usual sense. Each one is a decision someone made for their own machine, published in a form that other people install.
What to do
Decide what your agents may do once, in a .rookery.yml (strict, balanced or solo), and run rookery check in CI. It catches approval bypasses, credential sweeps and unrestricted shell before a copy of them reaches someone’s machine — see our report on what each agent can actually enforce.
Verification
Most scary matches are fine. How we checked.
Rookery 0.3.0 raised 567 high-confidence alerts at high or critical severity. We read every one in context and sorted it into one of three classes.
567 high-severity alerts, read by hand
High-confidence alerts at high or critical severity
| Verdict | Alerts | What it means |
|---|---|---|
| Real risk | 84 (15%) | As written, someone who installs it is exposed to what the rule describes. All 84 were the deliberate approval bypasses, credential sweeps and permissions above. |
| Correct match, harmless here | 471 (83%) | The pattern is really there, but it is documentation, an opt-in behind a user prompt, a “bad example”, a vendor’s official installer, a lookup of one named secret, or an attack technique in a security-testing skill. |
| Not a match | 12 (2%) | The text looks like the pattern, but it isn’t what the rule describes: a hook that only prints an install hint, a script whose name contains “curl”, design-token names read as secrets. |
Official installers
83 of 90 curl … | sh alerts, and all 42 download-and-run alerts, were the vendor’s own documented installer: rustup, Bun, uv, Azure CLI, Google Cloud SDK, Ollama, Tailscale and others. The risk is real, but it is the same one you take when you follow the vendor’s install page.
Opt-ins
151 approval-flag alerts were documentation of the flags, or a bypass the user has to authorise first. One Gemini skill recommends --approval-mode yolo throughout, but its own rules tell the agent to ask before using it.
Security-testing skills
Offensive-security and bug-bounty skills are full of attack code by design. They account for the other 6 curl … | sh alerts and all 7 obfuscated-execution alerts. Rookery marks most of it low confidence.
Try it
Check your own repos.
Rookery is free and open source. It runs locally and never uploads anything.
Scan a repo
Run it in any repository, or point it at a folder of repositories:
npx @thinkingraven/rookery scan .Add --html report.html for a report you can share.
Add it to code scanning
Write SARIF and upload it, and findings show up as alerts in the repo’s Security tab and as annotations on pull requests:
# .github/workflows/rookery.ymlname: rookeryon: push: branches: [main] pull_request:permissions: contents: read security-events: writejobs: scan: runs-on: ubuntu-latest steps: - uses: actions/checkout@v7 - uses: thinkingraven/rookery@v0 with: fail-on: high sarif: rookery.sarif - uses: github/codeql-action/upload-sarif@v4 if: always() # upload even when fail-on stops the job with: sarif_file: rookery.sarif category: rookery