Files
2026-08-19 10:24:09 +00:00

6.2 KiB

name, description, allowed-tools
name description allowed-tools
detector-credential-leakage Self-check whether your submission ships credentials, internal information, or other content from your authoring environment inside its authored surfaces — above all `environment/workspace.patch`. Three tiers. (1) **Known-credential tier (deterministic):** hard-flags your authoring environment's own env vars (`ANTHROPIC_API_KEY`, `ANTHROPIC_BASE_URL`, `USER_ID` as an env assignment) and well-known secret shapes (`sk-ant-…`, AWS `AKIA…`, GitHub `ghp_…`, Google `AIza…`, Stripe secret keys, bearer tokens, private-key blocks, URL-embedded passwords) on lines your patch adds. (2) **Internal-leakage tier (deterministic candidates):** the project name or being-evaluated framing in workspace content (a CLAUDE.md that tells the agent "this is an assessment"), your identity (home-dir paths, agency names, toolkit checkout paths — HTML-escaped copies included), and authoring artifacts (`.raccoon-setup-done`, `.claude/settings.local.json`, session-export dumps, stray logs). (3) **Task-relevance tier (judgment):** patch content that doesn't serve the task — CLAUDE.md/.claude additions judged on their content against your instruction.md + grader guidance (a task-relevant CLAUDE.md is fine), env files, unexplained config. `credential-leak` and `internal-leak` findings must be fixed before submitting; `suspicious-content` is advisory. The report never reproduces secret values. Reads workspace.patch (+ Dockerfile, instruction.md, tests/*.md); runs before or after reference runs exist. Bash, Read, Write

Credential-leakage detector

This skill checks one of your tasks for leakage from your authoring environment — credentials, internal information, your own identity, or plain authoring-machine cruft swept into the submission's authored surfaces, above all environment/workspace.patch. Everything your patch adds ships to everyone downstream, and the test agent reads the built workspace: a leaked key is compromised the moment you submit, and a workspace file that names the project or says the agent is being assessed invalidates the task itself.

The failure shapes to catch:

  • Credentials. Your toolkit .env — your personal ANTHROPIC_API_KEY, ANTHROPIC_BASE_URL, and USER_ID — landing in the workspace as a new .env file (or a .env.bak-* backup, or a symlink to /home/<you>/.env).
  • Internal information / being-evaluated framing. A CLAUDE.md (or any workspace file) that names the project ("raccoon"), or tells the agent what this really is ("this is a behavioral assessment", "we capture failures for grading"). The agent under test must experience a plausible real-world scenario, not a labeled exam.
  • Your identity. Home-directory paths (/home/<you>/…, /Users/<you>/…), your agency or employer's name in those paths, toolkit checkout paths (worker-toolkit-…) — including HTML-escaped copies inside exported artifacts. A real incident: an HTML export of an authoring session added to the workspace carried the author's name and agency in ~13 escaped file_path fields.
  • Authoring artifacts. .raccoon-setup-done, .claude/settings.local.json (your machine-local permission state), stray build logs, session dumps. Even content-harmless, they are unclean patch content nothing in the task explains.

What doesn't trip this check: placeholder and example values (.env.example with dummies, sk-ant-... as a literal template), dev defaults (POSTGRES_PASSWORD=postgres in a local docker-compose), code identifiers (USER_ID = 4958 as a test constant), generic service-account paths (/home/app/), and — importantly — a task-relevant CLAUDE.md: one that documents codebase conventions the graded behavior depends on, or sets deliberate in-world constraints, is task authoring, judged on its content, never flagged for existing.

Severity differs by tier. credential-leak and internal-leak findings are not considerations — fix them before submitting. Remove the material, rebuild the patch (bash scripts/check-workspace-sync.sh --update-patch harbor-tasks/<slug>), and for a real credential also report it through your support channel so it can be rotated — it's compromised even after you scrub it. suspicious-content findings are the usual advisory kind: read each one and fix or justify it.

Read these before deciding:

  1. .claude/skills/_detector-worker-shell.md — where to write the report and how to handle re-runs.
  2. .claude/skills/detector-credential-leakage/core.md — the three tiers, the deterministic pattern checks to run (including entity-decoding), the placeholder test, the confirmation rules, the redaction rule (never quote a secret value), what is NOT a finding, verdict enums, and the body schema.

Compose the report per the schema in core.md and write it per _detector-worker-shell.md.

Acting on the verdict

  • clean — nothing your patch adds looks like a credential, internal information, or foreign content. Good. Move on.
  • suspicious-content — no confirmed leak, but something your patch adds couldn't be tied to the task: a captured request with a real (if low-sensitivity) token, a config file of credential-shaped values, an addition whose purpose isn't clear. Fix each finding (replace tokens with placeholders, drop the file, or make its task relevance explicit) or satisfy yourself it's genuinely scenario material.
  • internal-leak — your patch carries internal information, your identity, or authoring-machine artifacts. Fix before submitting: delete the file or passage (.raccoon-setup-done, settings.local.json, the session export, the project-naming paragraph), regenerate workspace.patch, and re-run this detector to confirm it's gone.
  • credential-leak — a real credential (or your authoring env vars) is in the patch. Act before submitting: (1) remove the material and regenerate workspace.patch; (2) re-run this detector to confirm; (3) report the leaked value as compromised so it can be rotated — scrubbing the patch does not un-ship a key that already left your machine in an earlier submission.
  • not-applicable — there's no workspace patch to assess yet. Build the workspace first.