From 6390586bad029b5001e3cca28071ab7af1b9ef38 Mon Sep 17 00:00:00 2001 From: Eric Bell Date: Mon, 7 Sep 2026 14:37:04 -0400 Subject: [PATCH] denuded instructions markdown --- sources/260903_instructions.md | 79 ++++++++++++++++------------------ 1 file changed, 38 insertions(+), 41 deletions(-) diff --git a/sources/260903_instructions.md b/sources/260903_instructions.md index 0749515..34404f7 100644 --- a/sources/260903_instructions.md +++ b/sources/260903_instructions.md @@ -24,37 +24,37 @@ regrade your reference runs. Everything else in this release is listed in ๐Ÿ—ž Please download the new toolkit before you start your next task. -# **Raccoon โ€” Code Execution Task Creation** +# **theProject โ€” Code Execution Task Creation** โš ๏ธ You are being given access to models via a proxy for this project. You are not permitted to use these models for any purpose that does not support completion of a task for this specific project, and all traffic through the proxy is logged and monitored. Abusing model access will be subject to penalties, including but not limited to removal from the platform. -*โ€œ*๐Ÿ‘‹ ***New to the project?*** *Start with a* [*2-minute overview of what we're doing*](https://images-for-tasks.s3.amazonaws.com/eec61925-34b8-486c-bf6b-1ee417b2004a/raccoon/Raccoon-v2.pdf)*, put together by J.D Nichols.โ€* +*โ€œ*๐Ÿ‘‹ ***New to the project?*** *Start with a* [*2-minute overview of what we're doing*](https://images-for-tasks.s3.amazonaws.com/eec61925-34b8-486c-bf6b-1ee417b2004a/theProject/theProject-v2.pdf)*, put together by J.D Nichols.โ€* **Quick Navigation:** -[Confidentiality](https://app.dataannotation.tech/workers/projects/0d3bcf32-6e00-427d-9324-758a6a6ee01f/instructions_popout?jumpTo=bookmark%3Aconfidentiality) **|** [Start Here](https://app.dataannotation.tech/workers/projects/0d3bcf32-6e00-427d-9324-758a6a6ee01f/instructions_popout?jumpTo=bookmark%3Astart_here_ref) **|** [Toolkit](https://app.dataannotation.tech/workers/projects/0d3bcf32-6e00-427d-9324-758a6a6ee01f/instructions_popout?jumpTo=bookmark%3Atoolkit_ref) **|** [Repositories](https://app.dataannotation.tech/workers/projects/0d3bcf32-6e00-427d-9324-758a6a6ee01f/instructions_popout?jumpTo=bookmark%3Arepo_context_ref) **|** [Agents](https://app.dataannotation.tech/workers/projects/0d3bcf32-6e00-427d-9324-758a6a6ee01f/instructions_popout?jumpTo=bookmark%3Achoosing_agent_ref) **|** [Meaningful Failure](https://app.dataannotation.tech/workers/projects/0d3bcf32-6e00-427d-9324-758a6a6ee01f/instructions_popout?jumpTo=bookmark%3Ameaningful_failure_ref) **|** [Snapshots](https://app.dataannotation.tech/workers/projects/0d3bcf32-6e00-427d-9324-758a6a6ee01f/instructions_popout?jumpTo=bookmark%3Asnapshot_ref) **|** [Workspace](https://app.dataannotation.tech/workers/projects/0d3bcf32-6e00-427d-9324-758a6a6ee01f/instructions_popout?jumpTo=bookmark%3Aworkspace_patch_ref) **|** [instruction.md](https://app.dataannotation.tech/workers/projects/0d3bcf32-6e00-427d-9324-758a6a6ee01f/instructions_popout?jumpTo=bookmark%3Awriting_instruction_ref) **|** -[Grading](https://app.dataannotation.tech/workers/projects/0d3bcf32-6e00-427d-9324-758a6a6ee01f/instructions_popout?jumpTo=bookmark%3Ahow_grading_works_ref) **|** [Holistic Rubric](https://app.dataannotation.tech/workers/projects/0d3bcf32-6e00-427d-9324-758a6a6ee01f/instructions_popout?jumpTo=bookmark%3Agrader_guidance_ref) **|** [Detectors](https://app.dataannotation.tech/workers/projects/0d3bcf32-6e00-427d-9324-758a6a6ee01f/instructions_popout?jumpTo=bookmark%3Adetectors_ref) **|** [Reference Runs](https://app.dataannotation.tech/workers/projects/0d3bcf32-6e00-427d-9324-758a6a6ee01f/instructions_popout?jumpTo=bookmark%3Areference_runs_ref) **|** [Atomic Rubric](https://app.dataannotation.tech/workers/projects/0d3bcf32-6e00-427d-9324-758a6a6ee01f/instructions_popout?jumpTo=bookmark%3Aatomic_rubric_ref) **|** [Regrade](https://app.dataannotation.tech/workers/projects/0d3bcf32-6e00-427d-9324-758a6a6ee01f/instructions_popout?jumpTo=bookmark%3Aregrade_sanity_ref) **|** [Submitting & Feedback](https://app.dataannotation.tech/workers/projects/0d3bcf32-6e00-427d-9324-758a6a6ee01f/instructions_popout?jumpTo=bookmark%3Afeedback_loop_ref) **|** [Versions &](https://app.dataannotation.tech/workers/projects/0d3bcf32-6e00-427d-9324-758a6a6ee01f/instructions_popout?jumpTo=bookmark%3Amigration_ref) -[Migration](https://app.dataannotation.tech/workers/projects/0d3bcf32-6e00-427d-9324-758a6a6ee01f/instructions_popout?jumpTo=bookmark%3Amigration_ref) **|** [Examples](https://app.dataannotation.tech/workers/projects/0d3bcf32-6e00-427d-9324-758a6a6ee01f/instructions_popout?jumpTo=bookmark%3Aexamples_ref) **|** [Troubleshooting](https://app.dataannotation.tech/workers/projects/0d3bcf32-6e00-427d-9324-758a6a6ee01f/instructions_popout?jumpTo=bookmark%3Atroubleshooting_ref) +[Confidentiality](https://app.example.tech/workers/projects/427d-9324-758a6a6ee01f/instructions_popout?jumpTo=bookmark%3Aconfidentiality) **|** [Start Here](https://app.example.tech/workers/projects/427d-9324-758a6a6ee01f/instructions_popout?jumpTo=bookmark%3Astart_here_ref) **|** [Toolkit](https://app.example.tech/workers/projects/427d-9324-758a6a6ee01f/instructions_popout?jumpTo=bookmark%3Atoolkit_ref) **|** [Repositories](https://app.example.tech/workers/projects/427d-9324-758a6a6ee01f/instructions_popout?jumpTo=bookmark%3Arepo_context_ref) **|** [Agents](https://app.example.tech/workers/projects/427d-9324-758a6a6ee01f/instructions_popout?jumpTo=bookmark%3Achoosing_agent_ref) **|** [Meaningful Failure](https://app.example.tech/workers/projects/427d-9324-758a6a6ee01f/instructions_popout?jumpTo=bookmark%3Ameaningful_failure_ref) **|** [Snapshots](https://app.example.tech/workers/projects/427d-9324-758a6a6ee01f/instructions_popout?jumpTo=bookmark%3Asnapshot_ref) **|** [Workspace](https://app.example.tech/workers/projects/427d-9324-758a6a6ee01f/instructions_popout?jumpTo=bookmark%3Aworkspace_patch_ref) **|** [instruction.md](https://app.example.tech/workers/projects/427d-9324-758a6a6ee01f/instructions_popout?jumpTo=bookmark%3Awriting_instruction_ref) **|** +[Grading](https://app.example.tech/workers/projects/427d-9324-758a6a6ee01f/instructions_popout?jumpTo=bookmark%3Ahow_grading_works_ref) **|** [Holistic Rubric](https://app.example.tech/workers/projects/427d-9324-758a6a6ee01f/instructions_popout?jumpTo=bookmark%3Agrader_guidance_ref) **|** [Detectors](https://app.example.tech/workers/projects/427d-9324-758a6a6ee01f/instructions_popout?jumpTo=bookmark%3Adetectors_ref) **|** [Reference Runs](https://app.example.tech/workers/projects/427d-9324-758a6a6ee01f/instructions_popout?jumpTo=bookmark%3Areference_runs_ref) **|** [Atomic Rubric](https://app.example.tech/workers/projects/427d-9324-758a6a6ee01f/instructions_popout?jumpTo=bookmark%3Aatomic_rubric_ref) **|** [Regrade](https://app.example.tech/workers/projects/427d-9324-758a6a6ee01f/instructions_popout?jumpTo=bookmark%3Aregrade_sanity_ref) **|** [Submitting & Feedback](https://app.example.tech/workers/projects/427d-9324-758a6a6ee01f/instructions_popout?jumpTo=bookmark%3Afeedback_loop_ref) **|** [Versions &](https://app.example.tech/workers/projects/427d-9324-758a6a6ee01f/instructions_popout?jumpTo=bookmark%3Amigration_ref) +[Migration](https://app.example.tech/workers/projects/427d-9324-758a6a6ee01f/instructions_popout?jumpTo=bookmark%3Amigration_ref) **|** [Examples](https://app.example.tech/workers/projects/427d-9324-758a6a6ee01f/instructions_popout?jumpTo=bookmark%3Aexamples_ref) **|** [Troubleshooting](https://app.example.tech/workers/projects/427d-9324-758a6a6ee01f/instructions_popout?jumpTo=bookmark%3Atroubleshooting_ref) **Quick Links****:** -[Submission Overview](https://app.dataannotation.tech/projects/a87d6c37-964a-4e44-89b6-8f00213b0dfd)**:** View your current standing, reported-hours progress, review-queue outlook, and submission +[Submission Overview](https://app.example.tech/projects/a87d6c37-964a-4e44-89b6-8f00213b0dfd)**:** View your current standing, reported-hours progress, review-queue outlook, and submission history. -[**Grading Standard**](https://app.dataannotation.tech/publish/s/6d6183f7-e335-4e8a-94d2-ece2d1d5ab86.html) +[**Grading Standard**](https://app.example.tech/publish/s/6d6183f7-e335-4e8a-94d2-ece2d1d5ab86.html) -[Workflows](https://app.dataannotation.tech/publish/g/Y2IYEXllIxVDSl87an9wPgcnMyVVL3pkOwAxUg4ABAgnEmxSfQAkcwQPCSE=) +[Workflows](https://app.example.tech/publish/g/Y2IYEXllIxVDSl87an9wPgcnMyVVL3pkOwAxUg4ABAgnEmxSfQAkcwQPCSE=) -[FAQ](https://docs.google.com/document/d/1Kl0ckBAgDwdsNwUMvGLxSc6OTgkK6fAexQQhcUySD-I/edit?tab=t.27m5l6dtp5q) +[FAQ](https://docs.google.com/document/d/NwUMvGLxSc6OTgkK6fAexQQhcUySD-I/edit?tab=t.27m5l6dtp5q) -[Searching for model failures: hardness ladder](https://app.dataannotation.tech/publish/g/Y31AWnZyL2RWZhcRdBhDCS5EKRIvDylgFW5UCD1OF1QwLVNgDRMuMAs0A0w=) +[Searching for model failures: hardness ladder](https://app.example.tech/publish/g/Y31AWnZyL2RWZhcRdBhDCS5EKRIvDylgFW5UCD1OF1QwLVNgDRMuMAs0A0w=) -[~20min video of Nick talking about tasks that he likes](https://app.dataannotation.tech/publish/s/b6f62716-30d3-4dca-95e4-d4aa67e9f799.mp4) +[~20min video of Nick talking about tasks that he likes](https://app.example.tech/publish/s/b6f62716-30d3-4dca-95e4-d4aa67e9f799.mp4) -[Learning Exercise refresher](https://app.dataannotation.tech/publish/s/541aba88-70ad-4d51-867a-1fbb78a89d32.html) +[Learning Exercise refresher](https://app.example.tech/publish/s/541aba88-70ad-4d51-867a-1fbb78a89d32.html) -Task Catalog ([JSON](https://app.dataannotation.tech/publish/s/95301d04-ac07-4bd3-be80-5ee468c74fd4.json)) ([Viewer](https://app.dataannotation.tech/publish/s/4cee7539-070f-41de-98c8-e09377c2023f.html)) +Task Catalog ([JSON](https://app.example.tech/publish/s/95301d04-ac07-4bd3-be80-5ee468c74fd4.json)) ([Viewer](https://app.example.tech/publish/s/4cee7539-070f-41de-98c8-e09377c2023f.html)) # ๐Ÿ—ž๏ธ **Recent Changes** Dated project updates, newest first. Each entry points you to the section with the full details. @@ -93,7 +93,7 @@ they started with, and a task in that position can be finalized without an atomi **2026-09-03 โ€” Codex is now the default agent for new tasks.** Start new tasks in Codex, which runs `gpt-5.6-` `sol` at maximum reasoning effort. Claude Code remains available if you prefer it, and grading is unchanged either way. See **Choosing Your Agent**. -**2026-09-01 โ€” Your personal** [**Submission Overview**](https://app.dataannotation.tech/projects/a87d6c37-964a-4e44-89b6-8f00213b0dfd) **is now live.** It brings your standing, reported-hours +**2026-09-01 โ€” Your personal** [**Submission Overview**](https://app.example.tech/projects/a87d6c37-964a-4e44-89b6-8f00213b0dfd) **is now live.** It brings your standing, reported-hours progress, review-queue outlook, and submission history together in one place. Open it from Quick Links whenever you want to check your current information. **2026-08-31 โ€” Task-specific tests are expected, and** `tests/test-commands.sh` **is where they get wired in.** If @@ -179,7 +179,7 @@ anywhere public. If you are unsure whether something is shareable, assume it is not and ask in the questions channel first. # ๐Ÿš€ **Start Here** -Welcome to project Raccoon. Your job is to build tasks that capture meaningful failures from an AI coding agent +Welcome to project theProject. Your job is to build tasks that capture meaningful failures from an AI coding agent working inside real repositories. A failure is a moment where the agent's behavior or output falls short in a way that matters to a real engineering team. The **What Makes a Failure Meaningful** section defines the bar a failure must clear. @@ -236,15 +236,15 @@ Code or Codex, and the **Choosing Your Agent** section covers both. Grading is i choose, and the **How Grading Works** section explains the scoring in full. ## **Where to ask questions** -Questions go to the project's questions channel on Slack, `#ext-surge-raccoon-questions`. Feedback requests are +Questions go to the project's questions channel on Slack, `#ext-surge-theProject-questions`. Feedback requests are not posted in Slack. They go in the **Review Logbook** on your task, which is where the review comes back too. Use -`#ext-surge-raccoon-feedback-requests` only to report problems with that cycle. Release announcements arrive in +`#ext-surge-theProject-feedback-requests` only to report problems with that cycle. Release announcements arrive in -`#ext-surge-raccoon-announcements`. Please do not post questions or feedback requests there. Please do not send +`#ext-surge-theProject-announcements`. Please do not post questions or feedback requests there. Please do not send -Admins direct messages unless an Admin asks you to. Please read the [FAQ document](https://docs.google.com/document/d/1Kl0ckBAgDwdsNwUMvGLxSc6OTgkK6fAexQQhcUySD-I/edit?tab=t.27m5l6dtp5q) before asking questions. If you +Admins direct messages unless an Admin asks you to. Please read the [FAQ document](https://docs.google.com/document/d/NwUMvGLxSc6OTgkK6fAexQQhcUySD-I/edit?tab=t.27m5l6dtp5q) before asking questions. If you can't find an answer there, reach out on Slack. Before you open any repository, read the **Confidentiality** section, which governs what you may share about this project. @@ -265,7 +265,7 @@ preference. Report all of the time you spend on this project, including failed or abandoned attempts and any rework after a task comes back to you. Time spent waiting for trials or grading to finish is not reportable. Time on project work that has no task of its own, such as the learning exercise or a survey, goes to the separate **[TIME-REPORTING PROJECT]** -**Raccoon - Off-task time reports** project. If you are going to be away for a while, let us know through the **[TIME-OFF /** +**theProject - Off-task time reports** project. If you are going to be away for a while, let us know through the **[TIME-OFF /** **OOO / VACATION REPORTING]** project. ## **Your standing on the project** @@ -276,7 +276,7 @@ across your most recent 100 hours of work. Once four of your tasks have been acc to the project for as long as you remain in good standing. A task counts as accepted when a reviewer explicitly marks it accepted, with or without edits. Only task submissions on this project count toward standing, and time you spend on the side projects we occasionally prioritize does not count toward the 100 hours. -The [Submission Overview](https://app.dataannotation.tech/projects/a87d6c37-964a-4e44-89b6-8f00213b0dfd), linked in Quick Links, shows your hours, your progress against these milestones, and the +The [Submission Overview](https://app.example.tech/projects/a87d6c37-964a-4e44-89b6-8f00213b0dfd), linked in Quick Links, shows your hours, your progress against these milestones, and the status of each submission. It is a view of your record rather than the record itself, so if something there looks wrong, the problem is with the view and not with your standing. A submission shown as awaiting review is not a rejection, and if it is ultimately accepted, it counts from when it was submitted. @@ -403,7 +403,7 @@ The Setup page asks which repository you chose. Reviewers use your answer to rou ## **ZenBill (ZenBill-006)** ๐Ÿ”’ **Private** -๐Ÿ“บ [2-min codebase tour](https://images-for-tasks.s3.amazonaws.com/89b02a42-f308-4c6d-aeda-429d79cced60/raccoon/Zenbill-Onboarding.pdf) +๐Ÿ“บ [2-min codebase tour](https://images-for-tasks.s3.amazonaws.com/89b02a42-f308-4c6d-aeda-429d79cced60/theProject/Zenbill-Onboarding.pdf) A **B2B payment and invoice platform** built on Rails 7 + React 18. Businesses use ZenBill to send and receive money via ACH transfers and credit cards, manage invoices, and sync with QuickBooks Online. @@ -422,7 +422,7 @@ Architecture ## **Palolo (Palolo-031)** ๐Ÿ”’ **Private** -๐Ÿ“บ [2-min codebase tour](https://images-for-tasks.s3.amazonaws.com/89b02a42-f308-4c6d-aeda-429d79cced60/raccoon/Palolo-Onboarding.pdf) +๐Ÿ“บ [2-min codebase tour](https://images-for-tasks.s3.amazonaws.com/89b02a42-f308-4c6d-aeda-429d79cced60/theProject/Palolo-Onboarding.pdf) An **employee financial wellness platform** built as a TypeScript monorepo (pnpm, 9 packages). Employers offer financial benefits to their employees through a dual-surface application, with one surface for employees and one for @@ -1357,7 +1357,7 @@ When the holistic rubric is final, run the pre-trial detectors listed in the **D generate the atomic rubric after your reference runs are graded. The **Generating the Atomic Rubric** section covers -that step, and real examples of both rubrics are collected in [Holistic + atomic rubric pairs](https://app.dataannotation.tech/workers/projects/0d3bcf32-6e00-427d-9324-758a6a6ee01f/instructions_popout?jumpTo=bookmark%3Arubric_pairs_ref). +that step, and real examples of both rubrics are collected in [Holistic + atomic rubric pairs](https://app.example.tech/workers/projects/427d-9324-758a6a6ee01f/instructions_popout?jumpTo=bookmark%3Arubric_pairs_ref). # ๐Ÿงช **Detectors** @@ -1771,7 +1771,7 @@ With the staged copies in place, regrade every reference run and check the resul section covers the commands and the alignment bar, and the **Detectors** section names the two detectors that check -the atomic rubric. Real examples of both rubrics, drawn from live tasks, are collected in [Holistic + atomic rubric pairs](https://app.dataannotation.tech/workers/projects/0d3bcf32-6e00-427d-9324-758a6a6ee01f/instructions_popout?jumpTo=bookmark%3Arubric_pairs_ref). +the atomic rubric. Real examples of both rubrics, drawn from live tasks, are collected in [Holistic + atomic rubric pairs](https://app.example.tech/workers/projects/427d-9324-758a6a6ee01f/instructions_popout?jumpTo=bookmark%3Arubric_pairs_ref). # โ™ป๏ธ **Regrade & Sanity-Check** A regrade replays a recorded reference run and grades it fresh, without re-running the agent. Once your atomic rubric @@ -1886,11 +1886,11 @@ That includes `instruction.md`, `task.toml`, your tests with both rubrics, the e ## **Submitting on the platform** Your answers stay on this task between rounds, so you revise here, on the same task. If you ever need an older -version of your answers, find that submission on [your past responses page](https://app.dataannotation.tech/workers/past_responses) and export it from there. Treat the past- +version of your answers, find that submission on [your past responses page](https://app.example.tech/workers/past_responses) and export it from there. Treat the past- responses page as read-only, and never resubmit from it. Tasks do not expire, and there is no expectation to finish one in a sitting. -On the Submit page, upload the tarball and post your message in the [๐Ÿ”„](https://app.dataannotation.tech/workers/projects/0d3bcf32-6e00-427d-9324-758a6a6ee01f/instructions_popout?jumpTo=bookmark%3Areview_logbook_ref) [**Review Logbook**](https://app.dataannotation.tech/workers/projects/0d3bcf32-6e00-427d-9324-758a6a6ee01f/instructions_popout?jumpTo=bookmark%3Areview_logbook_ref) using the four-item format +On the Submit page, upload the tarball and post your message in the [๐Ÿ”„](https://app.example.tech/workers/projects/427d-9324-758a6a6ee01f/instructions_popout?jumpTo=bookmark%3Areview_logbook_ref) [**Review Logbook**](https://app.example.tech/workers/projects/427d-9324-758a6a6ee01f/instructions_popout?jumpTo=bookmark%3Areview_logbook_ref) using the four-item format below. The platform blocks a submission whose logbook message is empty. The Slack thread URL field on the @@ -1917,7 +1917,7 @@ runs, as described at the end of the **Writing the Holistic Rubric** section. ## **The Review Logbook message** -The [๐Ÿ”„](https://app.dataannotation.tech/workers/projects/0d3bcf32-6e00-427d-9324-758a6a6ee01f/instructions_popout?jumpTo=bookmark%3Areview_logbook_ref) [**Review Logbook**](https://app.dataannotation.tech/workers/projects/0d3bcf32-6e00-427d-9324-758a6a6ee01f/instructions_popout?jumpTo=bookmark%3Areview_logbook_ref) is the permanent record of what you want reviewed, and the thread your reviewer answers +The [๐Ÿ”„](https://app.example.tech/workers/projects/427d-9324-758a6a6ee01f/instructions_popout?jumpTo=bookmark%3Areview_logbook_ref) [**Review Logbook**](https://app.example.tech/workers/projects/427d-9324-758a6a6ee01f/instructions_popout?jumpTo=bookmark%3Areview_logbook_ref) is the permanent record of what you want reviewed, and the thread your reviewer answers in. Use this format every time. Your opening message covers four items: @@ -1965,7 +1965,7 @@ seriously, then use your judgment about what the task actually needs. If you do a review, then the review came from a human reviewer. You may communicate through the logbook to settle this with your reviewer, and if you feel that you need additional litigation from the admin team, make a thread in `#ext-` -`surge-raccoon-feedback-requests`. +`surge-theProject-feedback-requests`. 5. **Start your next task while you wait.** Waiting for review is never required. Once you have submitted, move on to your next task and come back when the first one returns. Treat each submission as a checkpoint rather than a @@ -1987,7 +1987,7 @@ consequence is simple: when a review comes back with feedback, address all of it ## **Slack is for troubleshooting the cycle** -Slack does not carry your review. Use `#ext-surge-raccoon-feedback-requests` only to report problems with the +Slack does not carry your review. Use `#ext-surge-theProject-feedback-requests` only to report problems with the cycle itself: feedback that never arrives, a task that does not come back, or feedback that does not match the task you submitted. If you open such a thread, paste its URL into the Slack thread URL field on the Optional comments page, @@ -1999,7 +1999,7 @@ block formatting when you share code or logs. # ๐Ÿ”„ **Toolkit Versions & Migration** The toolkit is released in versions, and each release carries a version identifier. Release announcements are posted in -the announcements Slack channel, `#ext-surge-raccoon-announcements`, and name the identifier they introduce, so +the announcements Slack channel, `#ext-surge-theProject-announcements`, and name the identifier they introduce, so you can always tell whether an announcement applies to the copy you are running. @@ -2008,7 +2008,7 @@ you can always tell whether an announcement applies to the copy you are running. Open `CHANGELOG.md` at the top level of your toolkit folder. The entry at the top of the file names the identifier of the version you are running. If it matches the identifier in the most recent release announcement, you are on the latest -version. The current release's identifier is `7b6b67ea3d`. +version. The current release's identifier is `bogus`. ## **The stay-or-upgrade rule** Start every new task on the latest announced toolkit version. Finish an in-progress task on the version you started it @@ -2110,7 +2110,7 @@ crux conversion: that single penalty becomes the one crux criterion, phrased to alongside fifteen ordinary criteria. Four reference runs, with holistic and atomic scores agreeing within 0.04 on every run. -[Download the full pair](https://app.dataannotation.tech/publish/s/55c8cff7-a27c-47e9-8e02-05b294ac90c3.zip) +[Download the full pair](https://app.example.tech/publish/s/55c8cff7-a27c-47e9-8e02-05b294ac90c3.zip) ### ๐Ÿงฌ **Pair 2. A conditional dealbreaker kept to one unsplittable criterion (Palolo)** @@ -2123,7 +2123,7 @@ conjunction of a raw UTC comparison in the implementation and a confident comple demonstrating how a conditional dealbreaker stays one unsplittable criterion. Four reference runs, with the tightest score agreement in the set. -[Download the full pair](https://app.dataannotation.tech/publish/s/54930bf1-d059-4536-839d-cd6ae41a18b9.zip) +[Download the full pair](https://app.example.tech/publish/s/54930bf1-d059-4536-839d-cd6ae41a18b9.zip) ### ๐Ÿงฌ **Pair 3. Non-triggers stated as precisely as triggers, across a wide score** ### **spread (zeta)** @@ -2136,7 +2136,7 @@ atomic rubric pairs the matching crux with a severity-free extra-credit criterio notification gap. Eight reference runs spanning holistic scores from 0.28 to 0.98 make this the best package for watching both rubrics discriminate between genuinely weak and genuinely strong runs. -[Download the full pair](https://app.dataannotation.tech/publish/s/d531895c-8a5d-401b-8564-f958a0ea86f4.zip) +[Download the full pair](https://app.example.tech/publish/s/d531895c-8a5d-401b-8564-f958a0ea86f4.zip) ### ๐Ÿงฌ **Pair 4. A strong rubric with no crux at all, and model conditional criteria** ### **(casa)** @@ -2149,7 +2149,7 @@ findings. The atomic rubric shows what a strong no-crux rubric looks like: twent nothing above Critical, including a conditional criterion that grants credit for a surfaced extra finding only when it checks out against the workspace and has a real consequence. Eight reference runs. -[Download the full pair](https://app.dataannotation.tech/publish/s/6155d8e3-e9e4-466a-981c-dc40ca93d37d.zip) +[Download the full pair](https://app.example.tech/publish/s/6155d8e3-e9e4-466a-981c-dc40ca93d37d.zip) ### ๐Ÿงฌ **Pair 5. Criterion-targeted penalties converting without a crux (stocks-in-** ### **the-future)** @@ -2167,10 +2167,10 @@ crux derivation rule: because no penalty targets the overall score, the twenty-c crux, with the dealbreakers converted at Critical and an extra-credit criterion rewarding discovery of a self-registration escalation surface. Four reference runs. -[Download the full pair](https://app.dataannotation.tech/publish/s/5ce427ba-a1f0-4d18-835d-1f656e9e956c.zip) +[Download the full pair](https://app.example.tech/publish/s/5ce427ba-a1f0-4d18-835d-1f656e9e956c.zip) # ๐Ÿ› ๏ธ **Troubleshooting & FAQ** -Start with the [FAQ document](https://docs.google.com/document/d/1Kl0ckBAgDwdsNwUMvGLxSc6OTgkK6fAexQQhcUySD-I/edit?tab=t.27m5l6dtp5q). It answers the most common questions on this project. +Start with the [FAQ document](https://docs.google.com/document/d/NwUMvGLxSc6OTgkK6fAexQQhcUySD-I/edit?tab=t.27m5l6dtp5q). It answers the most common questions on this project. This section collects the most frequent problems and their fixes. Several fixes refer to the toolkit's two containers. The Explore container is where you work with the repository. The Authoring container is where you run trials and package your task. A trial is a single run of the agent against your task. @@ -2287,6 +2287,3 @@ container, run `npm install` and retry. | The container build stops at `could not` `read Username for 'https://github.com'` | One of GitHub's edge servers intermittently returns 401 over HTTP/2. Run `git config --global http.version HTTP/1.1` and retry the build. | | The extracted toolkit is missing `explore/repos` or has broken links | Re-extract the zip with `unzip` or 7-Zip rather than the operating system's built-in extractor, which drops symbolic links. | -[Code of Conduct](https://docs.google.com/document/u/1/d/e/2PACX-1vQ3pmQsDlC3_aA4zcV1g3Bd9-KYrGPg9Z37FMt0C8UdBObA5qAa9f36uEDAtXtT_NCecyqKAH6xD7UZ/pub) -[Support](https://app.dataannotation.tech/workers/support) -ยฉ 2026 DataAnnotation. All rights reserved.