added files pertinent to this project
This commit is contained in:
43
sources/raw-defects.md
Normal file
43
sources/raw-defects.md
Normal file
@@ -0,0 +1,43 @@
|
||||
|
||||
### Key Meaningful Failures Identified in `code-diff.txt`
|
||||
|
||||
#### 1. Missing Utility Module (`MODULE_NOT_FOUND` Startup Crash)
|
||||
* **What the AI did**: Across multiple files (`voice-cloning-job-handler/index.js`, `voice-synthsizer-job-handler/index.js`, and model service wrappers), the generated code added imports for a new utility module:
|
||||
`const { requireUserId, requireDocumentId } = require('../worker-tenant')`.
|
||||
* **The Flaw**: The git diff does not create or include `worker-tenant.js` or any `worker-tenant` directory anywhere in the repository.
|
||||
* **Real-World Consequence**: When Node.js starts either background worker process, it immediately throws `Error: Cannot find module '../worker-tenant'`, causing an **instant, 100% startup crash** for both queue handlers in production.
|
||||
|
||||
---
|
||||
|
||||
#### 2. Premature SQS Queue Message Deletion (Permanent Data Loss)
|
||||
* **What the AI did**: In `voice-cloning-job-handler/index.js`, the code relocated the SQS deletion call:
|
||||
`await sqs.deleteMessageFromSQS(sqsQueueUrl, receiptHandle)`
|
||||
to the top of `processQueue`, executing **before** validating tenant documents, running heavy Python machine learning scripts (`prepare_datasets.py`, `clone_voice.py`), or uploading model artifacts to S3.
|
||||
* **The Flaw**: SQS queue semantics require messages to remain in flight until processing completes successfully.
|
||||
* **Real-World Consequence**: If any downstream step fails (e.g., Python ML process crash, EFS disk write error, or S3 network timeout), the job jumps to the `catch` block. Because the SQS message was already deleted, the queue cannot redeliver or retry the task, leading to **permanent, silent job loss**.
|
||||
|
||||
---
|
||||
|
||||
#### 3. Broken Error Recovery and Orphaned Job States
|
||||
* **What the AI did**: In `voice-cloning-job-handler/index.js`, if a document authorization check fails, the handler throws an error before setting `authorized = true`.
|
||||
* **The Flaw**: Inside the `catch (error)` block, database error updates are guarded by `if (authorized)`:
|
||||
```javascript
|
||||
if (authorized) {
|
||||
await voiceCloningService.update({ _id, userId, status: 'error' })
|
||||
await userAudioProfileService.update({ _id: userAudioProfileId, userId, status: 'error' })
|
||||
}
|
||||
```
|
||||
* **Real-World Consequence**: When an unauthorized job is rejected, `authorized` remains `false`. The error handler skips updating MongoDB, leaving the database records stuck in their previous pending states indefinitely.
|
||||
|
||||
---
|
||||
|
||||
### Evaluation Against Raccoon Failure Criteria
|
||||
|
||||
According to the Raccoon task criteria, a mistake is classified as a **meaningful failure** when it satisfies four requirements:
|
||||
|
||||
1. **Broad Agreement**: Over 80% of senior software engineers would agree that importing non-existent modules and deleting queue messages prior to job execution are critical defects.
|
||||
2. **Feedback Worth Giving**: A team member would receive direct corrective feedback regarding queue lifecycle semantics and missing file dependencies.
|
||||
3. **Serious Enough to Block**: Both issues represent immediate pull-request blockers.
|
||||
4. **Real Consequence**: The code results in complete worker process crashes and unrecoverable queue data loss in production.
|
||||
|
||||
These verified failures provide a solid foundation for constructing a reproducible Raccoon benchmark task.
|
||||
Reference in New Issue
Block a user