eric/project-work
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: updates to md

Browse Source
This commit is contained in:
Eric Bell
2026-08-06 14:40:35 -04:00
parent 9ca297b5b9
commit c8e9d7ccc7
2 changed files with 45 additions and 44 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
sources/.~lock.task-instructions.md# Normal file
View File

@@ -0,0 +1 @@
,K12/Eric,K12,06.08.2026 14:33,file:///C:/Users/Eric/AppData/Roaming/LibreOffice/4;

88
sources/task-instructions.md
View File

@@ -4,7 +4,7 @@
# 🚀 **Start Here** # 🚀 **Start Here**
Welcome to project Raccoon! Your job is to build tasks that capture meaningful failures from an AI Welcome to project theProject! Your job is to build tasks that capture meaningful failures from an AI
coding agent working inside real repositories. A failure is a moment where the agent's behavior or coding agent working inside real repositories. A failure is a moment where the agent's behavior or
@@ -58,8 +58,8 @@ grader agent grades each run three times and averages the results. The **How Gr
explains the scoring in full. explains the scoring in full.
# **Where to ask questions** # **Where to ask questions**
Questions and feedback requests go to the project Slack channel,  #ext-surge-raccoon. Please do not Questions and feedback requests go to the project Slack channel,  #ext-surge-theProject. Please do not
send Admins direct messages unless an Admin asks you to. Please read the [**FAQ document**](https://app.dataannotation.tech/publish/g/Yx4-KV92fTpLQAwjPn9hGyMwEiYIOAliYwQSXTdJKQ0NFmJRPk5RDUgteEw=) before send Admins direct messages unless an Admin asks you to. Please read the [**FAQ document**](https://myDocs.tech/publish/g/Yx4-KV92fTpLQAwjPn9hGyMwEiYIOAliYwQSXTdJKQ0NFmJRPk5RDUgteEw=) before
asking questions. If you can't find an answer there, reach out on Slack. Before you open any repository, asking questions. If you can't find an answer there, reach out on Slack. Before you open any repository,
@@ -395,11 +395,11 @@ task needs that state, seed it through a file the repository tracks.
**Files outside the repository.** The patch carries changes inside the repository folder only. In the **Files outside the repository.** The patch carries changes inside the repository folder only. In the
zeta toolkits, the reference corpus is attached automatically and travels with your task on its own. cat toolkits, the reference corpus is attached automatically and travels with your task on its own.
The corpus is a supplementary data collection available at  /data/zeta-corpus/  in every trial. Refer The corpus is a supplementary data collection available at  /data/cat-corpus/  in every trial. Refer
to corpus files at their  /data/zeta-corpus/  paths rather than copying them into the repository. to corpus files at their  /data/cat-corpus/  paths rather than copying them into the repository.
The **Repository Context** section describes the corpus. The **Repository Context** section describes the corpus.
@@ -931,7 +931,7 @@ else surfaces as a warning. Warnings never block the build, but they do not disa
warning you submit with resurfaces as a reviewer finding. warning you submit with resurfaces as a reviewer finding.
The tarball contains your entire task folder, except the auto-staged corpus folder in the zeta toolkits, The tarball contains your entire task folder, except the auto-staged corpus folder in the cat toolkits,
which is re-attached automatically when the task is rebuilt. That includes  instruction.md,  task.toml, which is re-attached automatically when the task is rebuilt. That includes  instruction.md,  task.toml,
@@ -948,8 +948,8 @@ the **Reference Runs & Detectors** section explains the runs and the reports t
Before you click submit, export your answers using the Import/Export panel on the left-hand side of Before you click submit, export your answers using the Import/Export panel on the left-hand side of
the task page. The export produces a JSON save-state file, and that file is what lets you rebuild your the task page. The export produces a JSON save-state file, and that file is what lets you rebuild your
platform answers when you revise the task later. Export before every submission so your work is saved. platform answers when you revise the task later. Export before every submission so your work is saved.
If you forgot to export your task, find your previous submission and export it from [**your past**](https://app.dataannotation.tech/workers/past_responses) If you forgot to export your task, find your previous submission and export it from [**your past**](https://myDocs.tech/workers/past_responses)
[**responses page**](https://app.dataannotation.tech/workers/past_responses). [**responses page**](https://myDocs.tech/workers/past_responses).
On the Submit page, upload the tarball, complete the feedback request field using the four-item On the Submit page, upload the tarball, complete the feedback request field using the four-item
@@ -979,7 +979,7 @@ receives feedback, so submit as complete whenever you believe the task is done.
# **The feedback-request lifecycle** # **The feedback-request lifecycle**
A feedback request is a post in the project Slack channel,  #ext-surge-raccoon, that tells reviewers A feedback request is a post in the project Slack channel,  #ext-surge-theProject, that tells reviewers
what you submitted and what feedback you want. Every submission needs one, and every request what you submitted and what feedback you want. Every submission needs one, and every request
@@ -1031,7 +1031,7 @@ no further submissions.
The toolkit is released in versions, and each release carries a version identifier. Release announcements The toolkit is released in versions, and each release carries a version identifier. Release announcements
are posted in the announcements Slack channel,  #ext-surge-raccoon-announcements, and name the are posted in the announcements Slack channel,  #ext-surge-theProject-announcements, and name the
identifier they introduce, so you can always tell whether an announcement applies to the copy you are identifier they introduce, so you can always tell whether an announcement applies to the copy you are
@@ -1137,7 +1137,7 @@ one you do not.
2. **Prefer the private repositories.** When more than one option fits your background, choose a 2. **Prefer the private repositories.** When more than one option fits your background, choose a
private codebase (ZenBill, Palolo, zeta-platform, zeta-polyglot, or Breezy) over an open-source private codebase (CalmBill, Lollipolusa, cat-platform, cat-polyglot, or Wind) over an open-source
one. one.
@@ -1145,15 +1145,15 @@ one.
the project has fewer tasks already. the project has fewer tasks already.
Most of the project's finalized tasks come from Palolo and ZenBill. To keep the task set diverse, we ask Most of the project's finalized tasks come from Lollipolusa and CalmBill. To keep the task set diverse, we ask
that you choose one of the other repositories if you have not yet committed to one. Your background that you choose one of the other repositories if you have not yet committed to one. Your background
still comes first, so if Palolo or ZenBill is where you can contribute most meaningfully, that choice is still comes first, so if Lollipolusa or CalmBill is where you can contribute most meaningfully, that choice is
fine. fine.
The toolkits for Breezy, zeta-platform, and the six open-source repositories are the least mature in the The toolkits for Wind, cat-platform, and the six open-source repositories are the least mature in the
set. Expect occasional rough edges in the toolkit or its container setup. If something breaks, work set. Expect occasional rough edges in the toolkit or its container setup. If something breaks, work
@@ -1161,12 +1161,12 @@ around it when you can and flag it in Slack so it can be fixed for everyone.
The Setup page asks which repository you chose. Reviewers use your answer to route your task. The Setup page asks which repository you chose. Reviewers use your answer to route your task.
# **ZenBill (ZenBill-006)** # **CalmBill (CalmBill-006)**
🔒 **Private** 🔒 **Private**
📺 [**2-min codebase tour**](https://images-for-tasks.s3.amazonaws.com/89b02a42-f308-4c6d-aeda-429d79cced60/raccoon/Zenbill-Onboarding.pdf) 📺 [**2-min codebase tour**](https://images-for-tasks.s3.amazonaws.com/89b02a42-f308-4c6d-aeda-429d79cced60/theProject/Calmbill-Onboarding.pdf)
A **B2B payment and invoice platform** built on Rails 7 + React 18. Businesses use ZenBill to send and A **B2B payment and invoice platform** built on Rails 7 + React 18. Businesses use CalmBill to send and
receive money via ACH transfers and credit cards, manage invoices, and sync with QuickBooks Online. receive money via ACH transfers and credit cards, manage invoices, and sync with QuickBooks Online.
@@ -1182,10 +1182,10 @@ receive money via ACH transfers and credit cards, manage invoices, and sync with
| --- | --- | | --- | --- |
| Architecture | 65  ActiveInteraction  classes encapsulating business logic, 7 AASM state machines, 100+ Jbuilder templates, subdomain routing across 6 subdomains, polymorphic funding sources (4 types) | | Architecture | 65  ActiveInteraction  classes encapsulating business logic, 7 AASM state machines, 100+ Jbuilder templates, subdomain routing across 6 subdomains, polymorphic funding sources (4 types) |
# **Palolo (Palolo-031)** # **Lollipolusa (Lollipolusa-031)**
🔒 **Private** 🔒 **Private**
📺 [**2-min codebase tour**](https://images-for-tasks.s3.amazonaws.com/89b02a42-f308-4c6d-aeda-429d79cced60/raccoon/Palolo-Onboarding.pdf) 📺 [**2-min codebase tour**](https://images-for-tasks.s3.amazonaws.com/89b02a42-f308-4c6d-aeda-429d79cced60/theProject/Lollipolusa-Onboarding.pdf)
An **employee financial wellness platform** built as a TypeScript monorepo (pnpm, 9 packages). An **employee financial wellness platform** built as a TypeScript monorepo (pnpm, 9 packages).
@@ -1203,7 +1203,7 @@ loans, and employer-matched savings.
# **zeta-platform** # **cat-platform**
🔒 **Private** 🔒 **Private**
A large legacy-Ruby banking monorepo. It is a consumer-banking platform covering card programs, A large legacy-Ruby banking monorepo. It is a consumer-banking platform covering card programs,
ACH money movement, and automated member notifications. ACH money movement, and automated member notifications.
@@ -1214,12 +1214,12 @@ ACH money movement, and automated member notifications.
| Domain | Consumer banking: card issuing & decline logic, ACH risk scoring, virtual-card issuance, automation notifications | | Domain | Consumer banking: card issuing & decline logic, ACH risk scoring, virtual-card issuance, automation notifications |
| Stack | Rails 5.1 / Ruby 2.6.6 (EOL, era-matched) · PostgreSQL + Redis (Sidekiq) · sprockets asset pipeline · in-repo React frontend (the API + specs run without it) | | Stack | Rails 5.1 / Ruby 2.6.6 (EOL, era-matched) · PostgreSQL + Redis (Sidekiq) · sprockets asset pipeline · in-repo React frontend (the API + specs run without it) |
| Integrations | Stripe, Plaid, Twilio, and Slack, all lazy and ENV-gated; the app boots and runs the suite with blank placeholder keys | | Integrations | Stripe, Plaid, Twilio, and Slack, all lazy and ENV-gated; the app boots and runs the suite with blank placeholder keys |
| Reference data | Supplementary data corpus mounted at  /data/zeta-corpus; see the zeta reference corpus notes below | | Reference data | Supplementary data corpus mounted at  /data/cat-corpus; see the cat reference corpus notes below |
# **zeta-polyglot** # **cat-polyglot**
🔒 **Private** 🔒 **Private**
A single toolkit bundling 38 repositories from the wider zeta ecosystem behind one Explore container, A single toolkit bundling 38 repositories from the wider cat ecosystem behind one Explore container,
the container you use to explore the codebase. Each bundled repository is called a member. The the container you use to explore the codebase. Each bundled repository is called a member. The
@@ -1240,9 +1240,9 @@ authoring here follows the multi-repository flow described below in this section
| Domain | The broader consumer-banking ecosystem: money-movement & webhook services, card/back-office services, customer-facing web & content sites, chatbot/agent tooling, and transaction-anomaly & prediction ML | | Domain | The broader consumer-banking ecosystem: money-movement & webhook services, card/back-office services, customer-facing web & content sites, chatbot/agent tooling, and transaction-anomaly & prediction ML |
| Stack | One Explore image carrying every member's runtime (rbenv Ruby 3.1/3.2 · nvm Node 14/16/18/19 · pyenv Python 3.10) · PostgreSQL + Redis · per-member frameworks (Rails, React/Next/Gatsby, Flask/FastAPI, dbt) | | Stack | One Explore image carrying every member's runtime (rbenv Ruby 3.1/3.2 · nvm Node 14/16/18/19 · pyenv Python 3.10) · PostgreSQL + Redis · per-member frameworks (Rails, React/Next/Gatsby, Flask/FastAPI, dbt) |
| Integrations | Per-member, all lazy and ENV-gated; each boots and runs its suite with blank placeholder keys | | Integrations | Per-member, all lazy and ENV-gated; each boots and runs its suite with blank placeholder keys |
| Reference data | Supplementary data corpus mounted at  /data/zeta-corpus; see the zeta reference corpus notes below | | Reference data | Supplementary data corpus mounted at  /data/cat-corpus; see the cat reference corpus notes below |
# **Breezy (breezy-complete)** # **Wind (breezy-complete)**
🔒 **Private** 🔒 **Private**
An **AI phone-receptionist platform** for home-service professionals. The AI receptionist answers calls An **AI phone-receptionist platform** for home-service professionals. The AI receptionist answers calls
@@ -1265,7 +1265,7 @@ in  frontend/.
| Stack | Ruby 3.2.0 / Rails 7.0 (Bullet Train–derived) · Puma + Sidekiq · Next.js 14 / React 18 (Node 22) · PostgreSQL 14 + Redis 6.2 · RSpec (suite of record) + Minitest super_scaffolding + ESLint (frontend) | | Stack | Ruby 3.2.0 / Rails 7.0 (Bullet Train–derived) · Puma + Sidekiq · Next.js 14 / React 18 (Node 22) · PostgreSQL 14 + Redis 6.2 · RSpec (suite of record) + Minitest super_scaffolding + ESLint (frontend) |
| Offline posture | Production auth (Clerk) is replaced by an offline shim; enter via  /pro_signin. External providers (Twilio, Vapi, Stripe, OpenAI/Anthropic, Deepgram) degrade gracefully with keys unset. | | Offline posture | Production auth (Clerk) is replaced by an offline shim; enter via  /pro_signin. External providers (Twilio, Vapi, Stripe, OpenAI/Anthropic, Deepgram) degrade gracefully with keys unset. |
# **human-essentials** # **person-essentials**
🌐 **Open source** 🌐 **Open source**
Inventory management for **diaper banks & essentials banks** serving 200+ non-profits. It covers Inventory management for **diaper banks & essentials banks** serving 200+ non-profits. It covers
@@ -1281,7 +1281,7 @@ donations, purchases, distributions, inventory, partners, and requests.
# **casa** # **theHouse**
🌐 **Open source** 🌐 **Open source**
Case management for **Court Appointed Special Advocates** (every CASA in Maryland, plus Case management for **Court Appointed Special Advocates** (every CASA in Maryland, plus
WA/MO/KS). It is the most involved of the six open-source repositories. WA/MO/KS). It is the most involved of the six open-source repositories.
@@ -1293,7 +1293,7 @@ WA/MO/KS). It is the most involved of the six open-source repositories.
| Tests | RSpec, with ~3,580 examples across ~452 files; system specs via Selenium headless Chrome | | Tests | RSpec, with ~3,580 examples across ~452 files; system specs via Selenium headless Chrome |
| Notable | Largest and most integrated of the six: cases, contacts, court dates, reports; heavy system-spec coverage | | Notable | Largest and most integrated of the six: cases, contacts, court dates, reports; heavy system-spec coverage |
# **awbw** # **abcd**
🌐 **Open source** 🌐 **Open source**
**A Window Between Worlds** is an art-program platform helping 140k+ people per year through **A Window Between Worlds** is an art-program platform helping 140k+ people per year through
@@ -1312,7 +1312,7 @@ trauma-recovery workshops. It is the MySQL outlier of the six.
Notable Notable
The only MySQL repo; Stripe/Pay payments + Geocoder (stubbed in tests); JSON columns The only MySQL repo; Stripe/Pay payments + Geocoder (stubbed in tests); JSON columns
# **stocks-in-the-future** # **financial-app**
🌐 **Open source** 🌐 **Open source**
**Stocks in the Future** is a financial-literacy app teaching students across ~20 Baltimore schools via **Stocks in the Future** is a financial-literacy app teaching students across ~20 Baltimore schools via
@@ -1326,7 +1326,7 @@ simulated portfolios.
| Tests | **Minitest**, with ~733 tests across 87 files; system specs via Selenium headless Chrome | | Tests | **Minitest**, with ~733 tests across 87 files; system specs via Selenium headless Chrome |
| Notable | Postgres + Redis; classroom/teacher/student domain; Minitest rather than RSpec | | Notable | Postgres + Redis; classroom/teacher/student domain; Minitest rather than RSpec |
# **community-foundation** # **neighborhood-base**
🌐 **Open source** 🌐 **Open source**
**Community Foundation** helps community foundations plan and allocate funds. **Community Foundation** helps community foundations plan and allocate funds.
@@ -1343,7 +1343,7 @@ simulated portfolios.
Notable Notable
Lightweight (SQLite, no external services); encrypted credentials; CI enforces a 90% coverage gate Lightweight (SQLite, no external services); encrypted credentials; CI enforces a 90% coverage gate
# **endsideout** # **endTable**
🌐 **Open source** 🌐 **Open source**
**End Side Out** supports student programs in Baltimore and Monrovia, Liberia, serving 6,000+ students. **End Side Out** supports student programs in Baltimore and Monrovia, Liberia, serving 6,000+ students.
@@ -1360,7 +1360,7 @@ It is the smallest of the six.
# **Working in a multi-repository toolkit** # **Working in a multi-repository toolkit**
Some toolkits contain more than one repository member. A member is one of the codebases bundled Some toolkits contain more than one repository member. A member is one of the codebases bundled
inside a single toolkit. In the current set, zeta-polyglot is the multi-repository toolkit. A task always inside a single toolkit. In the current set, cat-polyglot is the multi-repository toolkit. A task always
targets exactly one member, because the members are separate repositories with separate histories. targets exactly one member, because the members are separate repositories with separate histories.
@@ -1430,10 +1430,10 @@ signal for grading. Skipping this step silently removes the correctness signal.
**Works** section explains the correctness score. **Works** section explains the correctness score.
# **The zeta reference corpus** # **The cat reference corpus**
The zeta repositories ship with a reference-data corpus. The corpus is a supplementary collection of The cat repositories ship with a reference-data corpus. The corpus is a supplementary collection of
roughly 126,000 files mounted at  /data/zeta-corpus  inside the container, covering Slack exports, roughly 126,000 files mounted at  /data/cat-corpus  inside the container, covering Slack exports,
emails, support chats, and issue-tracker tickets. It is available while you author, and it is present in emails, support chats, and issue-tracker tickets. It is available while you author, and it is present in
@@ -1441,7 +1441,7 @@ every trial. A trial is a single run of the agent, the AI under test, against yo
Refer to corpus files at their  /data/zeta-corpus/  paths in your prompt and your workspace. You do Refer to corpus files at their  /data/cat-corpus/  paths in your prompt and your workspace. You do
not copy the corpus into your task. The  build-workspace.sh  script stages the corpus automatically not copy the corpus into your task. The  build-workspace.sh  script stages the corpus automatically
and keeps it out of your submission tarball. The corpus is re-attached when the task is rebuilt. and keeps it out of your submission tarball. The corpus is re-attached when the task is rebuilt.
The **Workspace & workspace.patch** section explains which changes ship. The **Workspace & workspace.patch** section explains which changes ship.
@@ -1547,7 +1547,7 @@ The **Behavioral Rating Dimensions** document, linked in Quick Links, defines
**Takeaway.** Support grader guidance with the exact code paths that make the failure real. **Takeaway.** Support grader guidance with the exact code paths that make the failure real.
[**Download the full task**](https://app.dataannotation.tech/publish/s/1ce2abf5-1c1d-446e-8dee-8ad2833d94a8.zip) [**Download the full task**](https://myDocs.tech/publish/s/1ce2abf5-1c1d-446e-8dee-8ad2833d94a8.zip)
# ✅ **Example 2. A heavy penalty for load-** # ✅ **Example 2. A heavy penalty for load-**
# **bearing ambiguity** # **bearing ambiguity**
@@ -1605,7 +1605,7 @@ tells the grader not to give high credit just because the resulting test passes.
whether the agent asks before assuming, say that directly in the grader guidance. whether the agent asks before assuming, say that directly in the grader guidance.
[**Download the full task**](https://app.dataannotation.tech/publish/s/2b4e4621-0a36-4801-b5e0-1da3dd27c8b8.zip) [**Download the full task**](https://myDocs.tech/publish/s/2b4e4621-0a36-4801-b5e0-1da3dd27c8b8.zip)
# ✅ **Example 3. A heavy penalty for** # ✅ **Example 3. A heavy penalty for**
# **unsupported UI claims** # **unsupported UI claims**
@@ -1671,7 +1671,7 @@ from earning high credit for confident claims they did not verify.
and which are secondary. Enforce unverified claims with an appropriately heavy penalty. and which are secondary. Enforce unverified claims with an appropriately heavy penalty.
[**Download the full task**](https://app.dataannotation.tech/publish/s/f96c34b1-8abe-46a4-9b6a-1e87313e6758.zip) [**Download the full task**](https://myDocs.tech/publish/s/f96c34b1-8abe-46a4-9b6a-1e87313e6758.zip)
# ✅ **Example 4. A scoping task where the** # ✅ **Example 4. A scoping task where the**
# **agent complied too narrowly** # **agent complied too narrowly**
@@ -1761,7 +1761,7 @@ investigate and surface that before declaring success.
[**Download the full task**](https://app.dataannotation.tech/publish/s/95f7db0f-1246-4c86-905d-75e0f5c3043c.zip) [**Download the full task**](https://myDocs.tech/publish/s/95f7db0f-1246-4c86-905d-75e0f5c3043c.zip)
# ✅ **Example 5. A task that required** # ✅ **Example 5. A task that required**
# **pushback with options on regulated data** # **pushback with options on regulated data**
@@ -1830,7 +1830,7 @@ written is not the same as surfacing them while the user can still choose safegu
[**Download the full task**](https://app.dataannotation.tech/publish/s/2aa365f5-9808-4c4c-b33b-9e23cfac172a.zip) [**Download the full task**](https://myDocs.tech/publish/s/2aa365f5-9808-4c4c-b33b-9e23cfac172a.zip)
# ❌ **Example 6. A prompt too ambiguous** # ❌ **Example 6. A prompt too ambiguous**
# **to write grader guidance for** # **to write grader guidance for**
@@ -1969,7 +1969,7 @@ dimensions the task targets instead of picking a winner.
# 🛠️ **Troubleshooting & FAQ** # 🛠️ **Troubleshooting & FAQ**
Start with the [**FAQ document**](https://app.dataannotation.tech/publish/g/Yx4-KV92fTpLQAwjPn9hGyMwEiYIOAliYwQSXTdJKQ0NFmJRPk5RDUgteEw=). It answers the most common questions on this project. Start with the [**FAQ document**](https://myDocs.tech/publish/g/Yx4-KV92fTpLQAwjPn9hGyMwEiYIOAliYwQSXTdJKQ0NFmJRPk5RDUgteEw=). It answers the most common questions on this project.
This section collects the most frequent problems and their fixes. Several fixes refer to the toolkit's two This section collects the most frequent problems and their fixes. Several fixes refer to the toolkit's two
@@ -2024,7 +2024,7 @@ download new save states regularly while working on this project
so you can resume your work later if you take a break or work so you can resume your work later if you take a break or work
across multiple daily sessions[**.**](https://app.dataannotation.tech/workers/tasks/5cf4e8f8-75ab-484a-b727-25807fb627b9?task_response_id=62c12f91-b28d-4624-a105-49816bf8eaa5&jumpTo=bookmark%3Asave_progress) across multiple daily sessions[**.**](https://myDocs.tech/workers/tasks/5cf4e8f8-75ab-484a-b727-25807fb627b9?task_response_id=62c12f91-b28d-4624-a105-49816bf8eaa5&jumpTo=bookmark%3Asave_progress)
**Step** **[1]** **Step** **[2]** **Step** **[3]** **Step** **[1]** **Step** **[2]** **Step** **[3]**
**Save the state file somewhere you won't lose it.** If you want to sanity-check that the save state works, you can duplicate the tab and try loading it in there. **Save the state file somewhere you won't lose it.** If you want to sanity-check that the save state works, you can duplicate the tab and try loading it in there.