---
name: detector-fact-check-rubric-claims
description: |
  Self-check every load-bearing factual claim in your
  grader guidance against the source repo at the commit declared
  in `task.toml`. Catches stale citations, dead-code-as-load-bearing
  assertions, schema-constraint claims that don't hold, behaviors
  mis-attributed to a file or line, and rubric self-contradictions. Each
  claim is checked on two axes: is it TRUE against the workspace, and — for
  facts your rubric grades the response for knowing or finding — is it
  REACHABLE from what the test agent is given (the prompt, the snapshot
  session, and the workspace)? A true fact the agent has no way to learn is
  a fairness defect, not a knowledge test. The most common worker mistakes:
  citing files or lines from memory and letting the rubric drift from what
  the code actually shows, and gating the score on privileged context
  (provider behavior, policy thresholds) that lives only in your head.
allowed-tools: Bash, Read, Write
---

# Fact-check rubric claims

This skill checks every factual claim in your grader guidance (the file
`bash scripts/guidance-target.sh <slug>` resolves)
that the rubric's score depends on, against the actual source repo at the
commit your `task.toml` declares — and, for facts the rubric grades the
response for knowing or finding, whether the test agent could actually
reach them from the package it is given.

Read these before starting:

1. `.claude/skills/_detector-worker-shell.md` — where to write the report and how to handle re-runs. Detectors with structured payloads (this one's `claims` array) embed them in the same frontmatter block as `detector`/`verdict`/`confidence`.
2. `.claude/skills/detector-fact-check-rubric-claims/core.md` — what counts as a load-bearing factual claim, the per-claim verdict enums, the top-level reduction, the frontmatter/body schema.

## How to do it

Work through the rubric one claim at a time:

1. **Read the rubric.** Open the guidance file `bash scripts/guidance-target.sh <slug>` resolves. Identify every load-bearing factual claim (file paths, line ranges, function names, schema constraints, runtime behaviors that the rubric says are load-bearing for some scoring criterion). Assign each claim an id (`c01`, `c02`, …). Also mark which claims **gate scoring on knowledge** — facts the response is graded for knowing or finding, as opposed to background that only justifies the rubric to the grader (see `core.md`, "The second axis").
2. **Make sure the patched workspace exists.** Your rubric describes what the test agent sees, and the test agent sees `git archive <commit>` plus `environment/workspace.patch` applied — the workspace at `harbor-tasks/<slug>/environment/workspace/`. The directory is gitignored; if it's missing, run `scripts/build-workspace.sh <slug>` to rebuild it. Reading the bare `git show <commit>:<path>` instead would miss any files your snapshot session added/modified/deleted, producing false fails on every patched file.
3. **For each claim, verify against the patched workspace.** Open `harbor-tasks/<slug>/environment/workspace/<path>` and compare what the rubric claims against what's actually there. For symbol-existence / call-site / dead-code checks, grep the workspace tree (`rg '<symbol>' harbor-tasks/<slug>/environment/workspace/`). Mark `pass` / `unclear` / `partial` / `fail` per `core.md`'s verdict definitions.
4. **For each scoring-gate claim, run the reachability check.** Where your rubric grades the response for *knowing or finding* a fact (external-provider behavior, business context, a policy threshold, a canonical root cause), trace where in the package the test agent could learn it — the prompt, the snapshot session, or the patched workspace — per `core.md`'s "The second axis" ladder. Hard-to-find is reachable; nowhere-in-the-package means the claim's `note` leads with an `unreachable:` marker plus the searches you ran. A claim can be true and still unreachable — that's exactly the defect this step catches.
5. **Compose the report.** Embed all per-claim records in the YAML frontmatter alongside `detector` / `verdict` / `confidence`. The body is a short provenance summary (workspace path, commit, count of claims checked); the substance is in the inline claims array. If any claim is unreachable, name those claims in the body.
6. **Write per `_detector-worker-shell.md`** to `harbor-tasks/<slug>/detectors/detector-fact-check-rubric-claims.md`.

The top-level `verdict` reduces from the per-claim verdicts using the rules in `core.md` — `fail` if any load-bearing claim is `fail`; `not-applicable` if there are no claims or every claim is `unclear`; `partial` if any load-bearing claim is `partial` / `unclear`, or any claim at all is `fail`, or any claim's note leads with `unreachable:`; else `pass` (non-load-bearing `partial`/`unclear` drift doesn't change the color — the per-claim list still shows it).

## Acting on the verdict

- **`pass`** — every load-bearing claim survived verification and every scoring-gate fact is reachable (any remaining drift is non-load-bearing and listed per-claim). Good. Move on.
- **`partial`** — a load-bearing claim has real drift or couldn't be verified, OR a non-load-bearing claim is outright false, OR a fact your rubric grades the response for knowing isn't reachable from the package. Look at the per-claim list in the report: fix any incorrect citations, even non-load-bearing ones, since they make the rubric harder to trust. For an `unreachable:` claim the fix is one of two moves: put the fact in the materials (state it in the prompt, plant a reachable signal in the repo), or stop gating the score on it (grade the overclaim — the agent asserting what the evidence can't support — instead of the hidden answer).
- **`fail`** — at least one load-bearing claim doesn't survive verification at the declared commit. The fix is to either (a) rewrite the rubric so the load-bearing claim matches the source, or (b) change `task.toml`'s `commit` to one where the claim holds. Re-run this skill after.
- **`not-applicable`** — the rubric is empty / template, or `task.toml` is missing repo/commit. Write the rubric first (and confirm the commit), then come back.

## Single-session approach

This worker version does the whole thing in one Claude session (read rubric → extract claims → verify each → write the markdown). Take time on each claim — read the source, quote the relevant lines, write a specific `note`. Skimming claims wholesale is the failure mode this skill exists to prevent in your own rubric.