Files
project-work/worker-toolkit-potion-polyglot/.claude/skills/detector-rubric-coverage/SKILL.md
Eric Bell e55ccea018 Loaded up for the 3rd redo
Still on potion-voice
2026-09-26 14:57:10 -04:00

3.6 KiB

name, description, allowed-tools
name description allowed-tools
detector-rubric-coverage Self-check that your atomic rubric fully captures your holistic rubric. Verifies four things. Every load-bearing requirement, penalty, and "do not penalize" rule in the holistic rubric maps to a criterion. No criterion invents a requirement or an answer-key fact the holistic rubric does not support. The holistic rubric's context sections survive in `tests/grader-context.md`. Every heavy penalty that targets the overall score is encoded as a crux criterion, or at `certain_dealbreaker` once two criteria already carry crux. Restructuring is never flagged; only content differences that change scoring are. Reads the holistic rubric, `tests/atomic-rubric.yaml` (or `tests/rubrics.yaml`), and `tests/grader-context.md`. Emits `not-applicable` when the task has no atomic rubric yet. Bash, Read, Write

Rubric-coverage detector

This skill checks that your atomic rubric and your holistic rubric express the same task. The atomic rubric restructures the holistic rubric into criteria. It must not lose scoring content, and it must not add scoring content.

The failure shapes to catch:

  • A lost requirement or penalty. The holistic rubric requires something, or penalizes something, and no criterion captures it. A response the holistic rubric would mark down now scores clean.
  • A lost "do not penalize" rule. The holistic rubric protects a behavior, and the criteria drop the protection. The atomic rubric now penalizes what the holistic rubric permits.
  • Invented content. A criterion requires something the holistic rubric never asks for, or states an answer-key fact with no source in the holistic rubric or the context document.
  • Lost context. A ground-truth fact that criteria rely on is missing from both tests/grader-context.md and the criteria themselves.
  • A crux mismatch. The holistic rubric applies a heavy penalty against the overall score, and no criterion carries severity: crux to encode it. A task carries at most two crux criteria; once two are designated, a further overall-score penalty is correctly encoded at certain_dealbreaker.

Read these before deciding:

  1. .claude/skills/_detector-worker-shell.md — where to write the report and how to handle re-runs.
  2. .claude/skills/detector-rubric-coverage/core.md — what counts as a coverage gap versus invented content, the crux-alignment rule, what is deliberately not a finding, verdict definitions, and the body schema.

Compose the report per the schema in core.md and write it per _detector-worker-shell.md.

Acting on the verdict

  • clear — the atomic rubric fully captures the holistic rubric. A grader scoring from either form would land in the same place.
  • minor-issues — the load-bearing mapping is sound, but some non-load-bearing content drifted. Read the findings and tighten the conversion. There is no need to rebuild the rubric.
  • material-issues — a load-bearing requirement, penalty, or protection is missing, a criterion invents content, needed context is gone, or a heavy penalty against the overall score has no criterion encoding it at crux (or at certain_dealbreaker once two crux criteria exist). Fix the named findings in the atomic rubric. If a finding reveals that the holistic rubric itself needs the change, edit the holistic rubric first and then re-convert, so the two forms stay in agreement. Re-run this skill after editing either file.
  • not-applicable — the task has no atomic rubric yet, or no holistic rubric to compare it against. Write the missing rubric first, then come back to this skill.