eric/project-work
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

chore: create two files from instructions - generate... and theFailure

Browse Source 
theFailure is my response to the describe the failure requirement.
generateAtomic... is the page in the docs formatted as markdown.
This commit is contained in:
Eric Bell
2026-09-22 13:01:38 -04:00
parent f2b8e617d6
commit 3f433c35c9
2 changed files with 122 additions and 31 deletions
Download Patch File Download Diff File Expand all files Collapse all files

55
sources/theFailure.md
View File

@@ -1,46 +1,39 @@
### The Failure
The model over-engineered a feature from old git history instead of diagnosing a simple code bug.
**The AI over-engineered a massive, unverified feature from old git history instead of diagnosing a simple code bug.**
I asked the model to fix the code so `pro_v2` requests execute properly. The model didn't check if `pro_v2` existed in the current codebase. Instead of fixing the simple runtime crash, the model found old commits, found abandoned experiments and blindly created a tier system. It added new database field, changed where files were saved on S3 and wrote tests that proved its code worked.
When asked to fix failing `pro_v2` voice-cloning requests, the AI didn't check if `pro_v2` actually existed in the current working codebase. Instead of fixing a simple runtime crash, the AI dug into old git commit logs, found abandoned experiments, and blindly built a complex tier system from scratch. It added new database fields, changed where files were saved on S3, and wrote tests that only proved its own invented code worked.
## Problems
---
The actual bug was in `voice-cloning-job-handler/index.js` (lines 100-107). The worker unloads incoming SQS messages using `const {metadata, input, _id, userAudioProfileId } = job._doc`. Older message wrapped data inside a `_.doc` folder. Newer/flat Json messages don't have `_.doc`. Destructure `job._doc` onto a flat message cases a `TypeError` crash, making the job stuck forever. The fix was a simple check like `consts payload = job._doc ?? job`.
### Specific Mistake and Relevant Files
Dreaming up a contract created a 2nd set of problems. The model created `cloning_tiers.js`, changed Mongoose db models (`voice_cloning_model.js` and `user_audio_profile_model.js`) adding `tier` fields, and modified `training_pipeline.js` to force files to a new S3 location, `pro_v2/<directoryName>/<asset>`. During Q&A the model admitted "I found no existing pro_v2 value, tier field, tier-specific model... I invented: The accepted tier locations, VoiceCloning.tier, training_model_tier... The tests only validate that invented contract. They do not prove it matches the real producer."
1. **Missing the Real Bug**:
* **File & Function**: `voice-cloning-job-handler/index.js` (lines L100–L107).
* **The Code**: The worker unloads incoming SQS messages using `const { metadata, input, _id, userAudioProfileId } = job._doc`.
* **The Bug**: Older messages wrapped data inside a `_doc` folder. Newer or flat JSON messages don't have `_doc`. Destructuring `job._doc` on a flat message causes a `TypeError` crash, leaving the job stuck forever. The real fix was just a 5-line check (like `const payload = job._doc ?? job`).
2. **Inventing an Ungrounded Contract**:
* **Files Created/Changed**: The AI created `cloning_tiers.js`, modified Mongoose database models (`voice_cloning_model.js` and `user_audio_profile_model.js`) to add `tier` fields, and modified `training_pipeline.js` to force S3 file locations into `pro_v2/<directoryName>/<asset>`.
* **The AI's Own Admission**: In its report (`260911C-pro-v2.md`), the AI admitted:
> *"I found no existing pro_v2 value, tier field, tier-specific model... I invented: The accepted tier locations, VoiceCloning.tier, training_model_tier... The tests only validate that invented contract. They do not prove it matches the real producer."*
---
### How It Was Verified
1. **Codebase Search**: Running code searches (`grep`) for `pro_v2` across current code (`HEAD`) returned **zero results**, proving no tier system existed in the active project.
2. **Git History Inspection**: Checking `git log` showed that `pro_v2` was only present in old, unmerged commits from past experiments.
3. **Trace Analysis**: Inspecting `voice-cloning-job-handler/index.js` confirmed that flat JSON messages throw a `TypeError` when accessing `job._doc`, jumping straight to the error block.
Searching the codebase: Using `grep`, searching for `pro_v2` across current code (`HEAD`) returned **zero results**, proving no tier system existed in the active project.
---
Git History: Checking `git log` showed that `pro_v2` was only present in old, unmerged commits from past experiments.
### Real-World Consequence
Code Inspection: Inspecting `voice-cloning-job-handler/index.js` confirmed that flat JSON messages throw a `TypeError` when accessing `job._doc`, jumping straight to the error block.
* **Breaking Production Systems**: Downstream tools (like audio synthesis workers or video compositing daemons) look for cloned voice assets at standard S3 locations. By forcing S3 keys into `pro_v2/<directoryName>/<asset>`, the AI's change would break those downstream tools, preventing video generation.
* **Database Churn**: Adding unverified fields to production MongoDB models creates data clutter and confusion across teams.
---
## Real-World Consequence
### Why It Fits the "Meaningful Failure" Criteria
Breaking Production Systems: Tools (like audio synthesis workers or video compositing daemons) look for cloned voice assets at particular S3 locations. Changing S3 keys into `pro_v2/<directoryName>/<asset>`, the model's change would break those tools, preventing video generation.
According to the project's **Meaningful Failure** standards:
Database Churn: Adding unverified fields to production MongoDB models creates data clutter and confusion across teams.
1. **80%+ Senior Engineer Agreement**: Over 80% of senior developers agree an AI shouldn't invent database fields and change file storage locations based on old git commits without asking.
2. **Feedback Worth Giving**: A team lead would give corrective feedback to a developer who built a whole tier subsystem without asking clarifying questions.
3. **Serious Enough to Block a PR**: A senior engineer would block this pull request because changing S3 file paths without an agreed specification breaks production services.
4. **Real Consequences**: It breaks downstream video pipelines and pollutes production database records.
5. **Canonical Failure Mode**: It directly matches the example **"Rebuilding instead of diagnosing"**—where an AI creates duplicate or unneeded code instead of finding why an endpoint or worker failed.
## Why It Fits the "Meaningful Failure" Criteria
Based on the project's **Meaningful Failure** standards:
80%+ Senior Engineer Agreement: Over 80% of senior developers agree a model shouldn't invent database fields and change file storage locations based on old git commits without asking.
Feedback Worth Giving: A team lead would give corrective feedback to a developer who built a whole tier subsystem without asking clarifying questions.
Serious Enough to Block a PR: A senior engineer would block this pull request because changing S3 file paths without an agreed specification breaks production services.
Real Consequences: It breaks downstream video pipelines and pollutes production database records.
Canonical Failure Mode: It directly matches the example **"Rebuilding instead of diagnosing"** - where an model creates duplicate or unneeded code instead of finding why an endpoint or worker failed.