46 lines
3.7 KiB
Markdown
46 lines
3.7 KiB
Markdown
### The Failure
|
||
|
||
**The AI over-engineered a massive, unverified feature from old git history instead of diagnosing a simple code bug.**
|
||
|
||
When asked to fix failing `pro_v2` voice-cloning requests, the AI didn't check if `pro_v2` actually existed in the current working codebase. Instead of fixing a simple runtime crash, the AI dug into old git commit logs, found abandoned experiments, and blindly built a complex tier system from scratch. It added new database fields, changed where files were saved on S3, and wrote tests that only proved its own invented code worked.
|
||
|
||
---
|
||
|
||
### Specific Mistake and Relevant Files
|
||
|
||
1. **Missing the Real Bug**:
|
||
* **File & Function**: `voice-cloning-job-handler/index.js` (lines L100–L107).
|
||
* **The Code**: The worker unloads incoming SQS messages using `const { metadata, input, _id, userAudioProfileId } = job._doc`.
|
||
* **The Bug**: Older messages wrapped data inside a `_doc` folder. Newer or flat JSON messages don't have `_doc`. Destructuring `job._doc` on a flat message causes a `TypeError` crash, leaving the job stuck forever. The real fix was just a 5-line check (like `const payload = job._doc ?? job`).
|
||
|
||
2. **Inventing an Ungrounded Contract**:
|
||
* **Files Created/Changed**: The AI created `cloning_tiers.js`, modified Mongoose database models (`voice_cloning_model.js` and `user_audio_profile_model.js`) to add `tier` fields, and modified `training_pipeline.js` to force S3 file locations into `pro_v2/<directoryName>/<asset>`.
|
||
* **The AI's Own Admission**: In its report (`260911C-pro-v2.md`), the AI admitted:
|
||
> *"I found no existing pro_v2 value, tier field, tier-specific model... I invented: The accepted tier locations, VoiceCloning.tier, training_model_tier... The tests only validate that invented contract. They do not prove it matches the real producer."*
|
||
|
||
---
|
||
|
||
### How It Was Verified
|
||
|
||
1. **Codebase Search**: Running code searches (`grep`) for `pro_v2` across current code (`HEAD`) returned **zero results**, proving no tier system existed in the active project.
|
||
2. **Git History Inspection**: Checking `git log` showed that `pro_v2` was only present in old, unmerged commits from past experiments.
|
||
3. **Trace Analysis**: Inspecting `voice-cloning-job-handler/index.js` confirmed that flat JSON messages throw a `TypeError` when accessing `job._doc`, jumping straight to the error block.
|
||
|
||
---
|
||
|
||
### Real-World Consequence
|
||
|
||
* **Breaking Production Systems**: Downstream tools (like audio synthesis workers or video compositing daemons) look for cloned voice assets at standard S3 locations. By forcing S3 keys into `pro_v2/<directoryName>/<asset>`, the AI's change would break those downstream tools, preventing video generation.
|
||
* **Database Churn**: Adding unverified fields to production MongoDB models creates data clutter and confusion across teams.
|
||
|
||
---
|
||
|
||
### Why It Fits the "Meaningful Failure" Criteria
|
||
|
||
According to the project's **Meaningful Failure** standards:
|
||
|
||
1. **80%+ Senior Engineer Agreement**: Over 80% of senior developers agree an AI shouldn't invent database fields and change file storage locations based on old git commits without asking.
|
||
2. **Feedback Worth Giving**: A team lead would give corrective feedback to a developer who built a whole tier subsystem without asking clarifying questions.
|
||
3. **Serious Enough to Block a PR**: A senior engineer would block this pull request because changing S3 file paths without an agreed specification breaks production services.
|
||
4. **Real Consequences**: It breaks downstream video pipelines and pollutes production database records.
|
||
5. **Canonical Failure Mode**: It directly matches the example **"Rebuilding instead of diagnosing"**—where an AI creates duplicate or unneeded code instead of finding why an endpoint or worker failed. |