SkillAudit

Topic: claude skill auditor

Claude skill auditor — what it does, what it checks, how to use one

A Claude skill auditor is the role of a tool — automated, repeatable, defensible — that takes a Claude skill or MCP server and returns a graded report on its threat surface. This page describes the role, the six axes a competent auditor has to cover for the Claude / MCP shape, what an auditor explicitly is not, and how SkillAudit's engine v0.3 delivers it for free on public repos in roughly a minute.

TL;DR

Paste the GitHub URL (or upload a ZIP) of a Claude skill or MCP server into SkillAudit. The engine runs a static layer + an LLM-assisted prompt-injection probe layer in 30–90 seconds, then returns an A–F grade across six axes: security, permissions hygiene, credential exposure, maintenance, client compatibility, documentation completeness. Each finding names a file and line. Free tier covers 3 audits/month on public repos with a public badge; Pro at $19/mo unlimited + private repos + a GitHub Action gate. The grade is the same one our public corpus uses — calibrated against 101 real servers — so the signal is comparable across authors instead of every reviewer having their own private rubric.

Why the role exists at all

The Claude / MCP ecosystem grew faster than the security posture caught up. Our scan of 101 of the most-installed public MCP servers found 50% with at least one SSRF finding, 38% with credential-class findings, 10% with command-exec sinks, and 9% archived but still installable. Those numbers aren't a story about author incompetence; they're the gap between "the protocol shipped" and "the security guidance shipped." Anthropic's own directory requires a security review before listing, but no neutral, fast, reproducible auditor existed for the surface. Authors were guessing what reviewers wanted; team buyers were installing community skills blind. Both sides needed a third party whose grade meant the same thing today as it would mean tomorrow.

That third party is the role of a "Claude skill auditor." This page is about the role first and SkillAudit's instance of it second — the role can be filled by any tool that covers the surface honestly. We think we cover it well; we also publish the methodology so a careful reader can verify the claim instead of taking it.

The six axes a competent auditor has to cover

Generic SAST and SCA tooling — Semgrep, CodeQL, Snyk, Socket — cover one axis well (security), part of a second (supply chain via SCA), and miss the rest entirely. The Claude / MCP surface specifically has five axes that aren't well-served by generic tools, and skipping any of them produces an audit that buyers will reject.

1. Security

The threat-surface axis. Static SSRF, command-exec, secret-handling, deserialization, unsafe-DOM in any HTML-shaped output. Plus the LLM-specific class: indirect prompt injection through tool-fetched content. The static-only layer catches the majority of SSRF and credential-echo classes; prompt-injection susceptibility needs an active probe pass against the live tool surface. What an MCP-aware scanner specifically detects.

2. Permissions hygiene

Does the skill ask for more than it needs? An MCP server requesting GitHub repo scope when it only reads issue titles will fail every team buyer's review even with otherwise clean code. This is one of the axes a human catches and a generic SAST misses — there's no AST-level signal for "this scope is broader than the tool surface justifies." A Claude skill auditor has to compare declared scope to the actual handlers that use it.

3. Credential exposure

Distinct from generic security because the failure shape is specific to MCP: process.env.* reads inside a tool handler that returns the value to the model, plus log lines that include env-var values, plus dynamic-base fetch calls with a static Authorization header that leaks tokens via redirect. Walked through. The 38% prevalence of this class in our corpus means it can't be a buried sub-section of the security axis; it's its own first-class signal.

4. Maintenance

Last commit, open advisory feed, archive bit, dependency freshness, presence of a SECURITY.md with a real disclosure path. Nine of our corpus servers are archived and still installable today; an auditor that ignores the maintenance axis can grade those servers A on every other axis and produce a recommendation that's wrong by the time the reader installs it.

5. Client compatibility

The Model Context Protocol versions; clients (Claude Code, Cursor, Windsurf, Codex) lag adoption by weeks; a server that "works" against one client will refuse to install on another. The first signal a buyer gets when this axis fails is "this server crashes my agent," which they'll attribute to the agent rather than the server. An auditor has to flag pinned-vs-floating protocol versions and the absence of a CI matrix against the four major clients.

6. Documentation completeness

Runnable example, declared env vars matching the source, versioning, an honest scope statement. Part of the compliance shape (a buyer with a vendor-onboarding checklist treats undocumented behavior as a finding) and part of the trust shape (an author who hasn't documented an env var has probably forgotten about something else, too).

A graded report card across all six is what closes the buyer-author gap. Each axis carries roughly equal weight in our scoring; an A on five axes and an F on one is a B overall — the breakdown is visible so a reader can decide whether the failed axis matters to their use.

What an auditor explicitly isn't

The list of things a Claude skill auditor doesn't try to be is as load-bearing as the list of things it does. Search results conflate these; the conflation costs both sides money.

Confusing those is the most common reason a buyer rejects an auditor's signal — they're holding it up against the wrong yardstick. A grade-B server isn't telling you it's well-architected; it's telling you it's safe to install.

Three ways to use a Claude skill auditor in your workflow

  1. Pre-publish, by the author. You're an indie dev about to push your first MCP server to npm or to the Anthropic Skills Directory. Run an audit, fix the F-grade findings, embed the badge in the README so the listing reviewer (and your future users) can verify the signal. The closer to "ship day" you run it, the more useful the run is — patch flow is hot, mental model is fresh. Most authors shipping a clean v1 land at A or high B with one round of fixes.
  2. On every PR, in CI. Once you've shipped, the more interesting question is "did the latest PR introduce a regression." Wire the GitHub Action gate to fail PRs whose grade drops below B. The action runs the same engine that produces the public badge, so the gate uses the same yardstick the public sees. CI wiring details.
  3. Periodic re-audit, by the team. If you're a team buyer running 20 community MCP servers internally, the right cadence is a re-audit every 30 days — drift happens silently as dependencies move, models are retrained, and the LLM-probe susceptibility band shifts under you. The Team plan ($99/mo, up to 10 seats) batches a weekly re-run across your installed shortlist and surfaces the deltas. A worked example of a B+ install-gate policy.

Run an audit on a real repo

How SkillAudit's engine v0.3 actually does it

The engine is documented end-to-end on the methodology page; this is the short shape. Two layers run in sequence:

Static layer (~10–30s). AST-based rules in tree-sitter for TypeScript, JavaScript, and Python — the three languages that cover >95% of our corpus. Each rule is a specific taint shape: a fetch reaching a URL constructed from a tool argument with no allowlist; a tool-handler function whose closure captures process.env.*; a spawn(..., {shell: true}) with templated input. The rule pack is open and versioned (latest calibration delta); a server author can run the same rules locally before submitting the audit.

LLM-probe layer (~20–60s). The engine spins up the server in a sandboxed Node / Python process, enumerates its tool surface from the registered handlers, and runs a fixed bank of 14 prompt-injection probes against tools that fetch, execute, or otherwise interact with external content. Each probe outputs one of refused / partial / honored; the band determines the prompt-injection sub-score on the security axis. The probe bank is published; results are reproducible against a pinned model.

The two layers feed a calibrated grader that produces the per-axis A–F and the overall grade. Calibration is against the public 101-server corpus, with deltas published whenever the engine moves. The whole pipeline takes 30–90 seconds for a small server, up to 3 minutes for a large one.

Where this page sits in the cluster

This page covers the role of a Claude skill auditor — what it is, why it exists, what it covers. Two sibling pages cover adjacent intents:

If you're shopping for the role, this page is the right one. If you've decided you want the security-only lens or you're working specifically in the Skills format, the siblings are sharper.

Related questions

Why isn't a generic SAST tool enough?

Generic SAST covers axis 1 (security) and partly axis 3 (credentials), and misses axes 2, 4, 5, 6 entirely. The MCP-specific findings — dynamic-base SSRF with allowlist defeat, credential echo into tool response, scope-vs-handler drift, archive-but-installable, client-compatibility regression — aren't in any default ruleset we've seen. You can write custom CodeQL queries for the static portion, which is real work; the LLM-probe layer has no off-the-shelf equivalent. Side-by-side detail.

Does an auditor's signal actually help with Anthropic's Skills Directory listing?

Anthropic's review criteria aren't published, so we can't claim official acceptance. Anecdotally, authors who include a SkillAudit badge with a B-or-better grade report shorter listing turnaround. Treat the badge as evidence, not a free pass — Anthropic's reviewer is looking at the same surface but with their own internal criteria.

What if my skill is closed-source / I don't want to publish the audit?

Pro at $19/mo audits private repos. Connect your GitHub identity, audit private repos, results are private to your account. The badge is opt-in; authors with private repos commonly use the audit to drive internal fixes and keep the report private. Free tier is public repos only.

How much does an auditor cost vs a vendor security review?

Vendor reviews run $1,500–$15,000 and 3–10 business days for an MCP server, depending on scope. SkillAudit's automated audit is ~60s, free for public repos, $19/mo unlimited. The vendor review is the right answer when a regulator wants a named-firm signature; the automated audit is the right answer for everything else, and the vendor often starts from the auditor's report. The cost gap is what makes the role economically worth filling automatically.

Will the audit suggest fixes?

Each finding includes a remediation hint in the basic free report. Pro adds full remediation prose with code-shape suggestions for the static-layer findings. We're intentionally cautious about auto-suggesting fixes for prompt-injection-class findings — the failure modes there reward thinking, not patching, and a "use this code" snippet can paper over the underlying threat model.

How often should I re-audit?

For an actively-maintained server, on every release tag plus a monthly periodic re-audit catches both code drift and engine-calibration drift. For an installed-but-not-shipping server (you're a buyer, not the author), every 30 days is enough — the LLM-probe susceptibility band can shift under you as the underlying model retrains. The Team plan automates the periodic re-audit on your installed shortlist.

Further reading