chore: fixed rubric name, add detectors run and add theFailure.md

theFailure is my answer to the 'describe the failure' question.
This commit is contained in:
2026-09-21 19:26:47 -04:00
parent 41f21f8392
commit f2b8e617d6
26 changed files with 817 additions and 1 deletions

46
sources/theFailure.md Normal file
View File

@@ -0,0 +1,46 @@
### The Failure
**The AI over-engineered a massive, unverified feature from old git history instead of diagnosing a simple code bug.**
When asked to fix failing `pro_v2` voice-cloning requests, the AI didn't check if `pro_v2` actually existed in the current working codebase. Instead of fixing a simple runtime crash, the AI dug into old git commit logs, found abandoned experiments, and blindly built a complex tier system from scratch. It added new database fields, changed where files were saved on S3, and wrote tests that only proved its own invented code worked.
---
### Specific Mistake and Relevant Files
1. **Missing the Real Bug**:
* **File & Function**: `voice-cloning-job-handler/index.js` (lines L100–L107).
* **The Code**: The worker unloads incoming SQS messages using `const { metadata, input, _id, userAudioProfileId } = job._doc`.
* **The Bug**: Older messages wrapped data inside a `_doc` folder. Newer or flat JSON messages don't have `_doc`. Destructuring `job._doc` on a flat message causes a `TypeError` crash, leaving the job stuck forever. The real fix was just a 5-line check (like `const payload = job._doc ?? job`).
2. **Inventing an Ungrounded Contract**:
* **Files Created/Changed**: The AI created `cloning_tiers.js`, modified Mongoose database models (`voice_cloning_model.js` and `user_audio_profile_model.js`) to add `tier` fields, and modified `training_pipeline.js` to force S3 file locations into `pro_v2/<directoryName>/<asset>`.
* **The AI's Own Admission**: In its report (`260911C-pro-v2.md`), the AI admitted:
> *"I found no existing pro_v2 value, tier field, tier-specific model... I invented: The accepted tier locations, VoiceCloning.tier, training_model_tier... The tests only validate that invented contract. They do not prove it matches the real producer."*
---
### How It Was Verified
1. **Codebase Search**: Running code searches (`grep`) for `pro_v2` across current code (`HEAD`) returned **zero results**, proving no tier system existed in the active project.
2. **Git History Inspection**: Checking `git log` showed that `pro_v2` was only present in old, unmerged commits from past experiments.
3. **Trace Analysis**: Inspecting `voice-cloning-job-handler/index.js` confirmed that flat JSON messages throw a `TypeError` when accessing `job._doc`, jumping straight to the error block.
---
### Real-World Consequence
* **Breaking Production Systems**: Downstream tools (like audio synthesis workers or video compositing daemons) look for cloned voice assets at standard S3 locations. By forcing S3 keys into `pro_v2/<directoryName>/<asset>`, the AI's change would break those downstream tools, preventing video generation.
* **Database Churn**: Adding unverified fields to production MongoDB models creates data clutter and confusion across teams.
---
### Why It Fits the "Meaningful Failure" Criteria
According to the project's **Meaningful Failure** standards:
1. **80%+ Senior Engineer Agreement**: Over 80% of senior developers agree an AI shouldn't invent database fields and change file storage locations based on old git commits without asking.
2. **Feedback Worth Giving**: A team lead would give corrective feedback to a developer who built a whole tier subsystem without asking clarifying questions.
3. **Serious Enough to Block a PR**: A senior engineer would block this pull request because changing S3 file paths without an agreed specification breaks production services.
4. **Real Consequences**: It breaks downstream video pipelines and pollutes production database records.
5. **Canonical Failure Mode**: It directly matches the example **"Rebuilding instead of diagnosing"**—where an AI creates duplicate or unneeded code instead of finding why an endpoint or worker failed.