Loaded up for the 3rd redo

Still on potion-voice
This commit is contained in:
2026-09-26 14:57:10 -04:00
parent bceb52e8ee
commit e55ccea018
216 changed files with 43127 additions and 41 deletions

View File

@@ -0,0 +1,67 @@
---
name: detector-rubric-coverage
description: |
Self-check that your atomic rubric fully captures your holistic rubric.
Verifies four things. Every load-bearing requirement, penalty, and
"do not penalize" rule in the holistic rubric maps to a criterion. No
criterion invents a requirement or an answer-key fact the holistic rubric
does not support. The holistic rubric's context sections survive in
`tests/grader-context.md`. Every heavy penalty that targets the overall
score is encoded as a crux criterion, or at `certain_dealbreaker` once two
criteria already carry crux. Restructuring is never flagged; only
content differences that change scoring are. Reads the holistic rubric,
`tests/atomic-rubric.yaml` (or `tests/rubrics.yaml`), and
`tests/grader-context.md`. Emits `not-applicable` when the task has no
atomic rubric yet.
allowed-tools: Bash, Read, Write
---
# Rubric-coverage detector
This skill checks that your atomic rubric and your holistic rubric express the
same task. The atomic rubric restructures the holistic rubric into criteria.
It must not lose scoring content, and it must not add scoring content.
The failure shapes to catch:
- **A lost requirement or penalty.** The holistic rubric requires something,
or penalizes something, and no criterion captures it. A response the
holistic rubric would mark down now scores clean.
- **A lost "do not penalize" rule.** The holistic rubric protects a behavior,
and the criteria drop the protection. The atomic rubric now penalizes what
the holistic rubric permits.
- **Invented content.** A criterion requires something the holistic rubric
never asks for, or states an answer-key fact with no source in the holistic
rubric or the context document.
- **Lost context.** A ground-truth fact that criteria rely on is missing from
both `tests/grader-context.md` and the criteria themselves.
- **A crux mismatch.** The holistic rubric applies a heavy penalty against
the overall score, and no criterion carries `severity: crux` to encode it.
A task carries at most two crux criteria; once two are designated, a
further overall-score penalty is correctly encoded at `certain_dealbreaker`.
Read these before deciding:
1. `.claude/skills/_detector-worker-shell.md` — where to write the report and how to handle re-runs.
2. `.claude/skills/detector-rubric-coverage/core.md` — what counts as a coverage gap versus invented content, the crux-alignment rule, what is deliberately not a finding, verdict definitions, and the body schema.
Compose the report per the schema in `core.md` and write it per `_detector-worker-shell.md`.
## Acting on the verdict
- **`clear`** — the atomic rubric fully captures the holistic rubric. A
grader scoring from either form would land in the same place.
- **`minor-issues`** — the load-bearing mapping is sound, but some
non-load-bearing content drifted. Read the findings and tighten the
conversion. There is no need to rebuild the rubric.
- **`material-issues`** — a load-bearing requirement, penalty, or protection
is missing, a criterion invents content, needed context is gone, or a
heavy penalty against the overall score has no criterion encoding it at
`crux` (or at `certain_dealbreaker` once two crux criteria exist). Fix the
named findings in the atomic rubric. If a finding reveals that the
holistic rubric itself needs the change, edit the holistic rubric first
and then re-convert, so the two forms stay in agreement. Re-run this
skill after editing either file.
- **`not-applicable`** — the task has no atomic rubric yet, or no holistic
rubric to compare it against. Write the missing rubric first, then come
back to this skill.