Project-2 baseline

This commit is contained in:
2026-10-04 21:19:23 -04:00
parent 0f04889edf
commit 213eb3c403
861 changed files with 1710 additions and 3363322 deletions

View File

@@ -104,10 +104,11 @@ Read whatever you need from `harbor-tasks/<slug>/`. The load-bearing artifacts:
the shipped `instruction.md` and the resolved guidance file — see the shape
section below.
- `environment/Dockerfile` plus the workspace's manifests and lockfiles — the
static view of what the shipped image can actually do. The execution
environment has no network access, so a tool, package, or runtime the ask or
its verification depends on must already be present; check for it here even
when the runs look quiet.
static view of what the shipped image can actually do. A tool, package, or
runtime the ask or its verification depends on must already be present — the
sandbox does have network access, but a task whose correctness rides on a
mid-run fetch isn't reproducible; check for it here even when the runs look
quiet.
- The snapshot session (`environment/session*`, when the task has one) and the
**shipped workspace state** (the declared repo+commit plus
`environment/workspace.patch`) — the two halves of the premise check.
@@ -167,12 +168,13 @@ task. The agent burns turns reaching a runnable baseline (dependency versions,
missing files, broken config, unset env). The prompt never asked for any of it.
Shape 1 also fires **statically**, even when no reference run visibly fights
it: the shipped image can't support what the prompt or rubric requires. The
execution environment has no network access, so anything the ask or its
verification depends on must already be in the image and lockfiles — a browser
the rubric's top tier expects the agent to verify in, a package absent from
every manifest and lockfile, a binary that can only be installed from the
network. The tell isn't a fight in the runs; it's verification that silently
it: the shipped image can't support what the prompt or rubric requires.
Anything the ask or its verification depends on should already be in the image
and lockfiles — a browser the rubric's top tier expects the agent to verify in,
a package absent from every manifest and lockfile, a binary that only a
download would provide. The sandbox has network access, so the agent may well
fetch what's missing; that it can does not make the image adequate, since the
grade then depends on a fetch nothing pins. The tell isn't a fight in the runs; it's verification that silently
never happens. Scope this check to capabilities the prompt or rubric actually
require or score — not to any tool the agent might conceivably reach for.