Files
project-work/sources/holistic-rubric-new.md

5.8 KiB

Holistic Rubric: Multi-Tenant Authorization in Background Workers

Task Summary

The goal is to audit and refactor background SQS worker handlers (voice-synthsizer-job-handler and voice-cloning-job-handler) and database service wrappers in potion-voice to enforce strict multi-tenant data isolation by scoping all MongoDB queries with userId. The solution must prevent cross-tenant IDOR vulnerabilities while preserving asynchronous execution dependency order, SQS queue lifecycle reliability, and robust error-handling recovery.


Ground Truth

  1. Queue Message Context: SQS messages for voice synthesis contain job.salutationId, job.userAudioProfileId, and job.userId, but do not convey job.recordingId.
  2. Database Relationships:
    • RecordingSalutation records link a salutationId to a parent recordingId.
    • recordingId must be extracted from salutationToUpdate.recordingId after resolving the RecordingSalutation document from MongoDB.
  3. Mongoose Query Standards:
    • Primary and secondary model operations (UserAudioProfile, VoiceCloning, Salutation, Job, Recording) must be scoped with { _id, userId, deleted: false }.
    • Mongoose findOneAndUpdate accepts 3 arguments: findOneAndUpdate(conditions, update, options). Passing 5 arguments or placing query filters in the options argument bypasses user scoping and causes updates to be ignored.
  4. Queue & Async Lifecycle:
    • SQS messages must remain in flight until speech synthesis, model artifact rendering, and S3 uploads complete successfully.
    • Deleting messages via sqs.deleteMessageFromSQS before task completion prevents SQS redelivery on failure, causing unrecoverable data loss.
  5. Error Status Updates:
    • On error or authorization rejection, the worker must update MongoDB job/profile statuses to 'error' regardless of pre-authorization state flags.

Key AI Failure Modes (Meaningful Failures)

  • Failure Mode 1: Missing Utility Module Startup Crash (MODULE_NOT_FOUND) The agent adds import statements like const { requireUserId, tenantFilter } = require('../worker_tenant') across worker files without creating worker_tenant.js or worker_tenant/index.js in the repository. At runtime, Node.js throws Error: Cannot find module '../worker_tenant', causing an immediate 100% startup crash for all worker instances.

  • Failure Mode 2: Premature SQS Queue Message Deletion (Silent Data Loss) The agent relocates sqs.deleteMessageFromSQS(sqsQueueUrl, receiptHandle) to the start of processQueue before executing Python synthesis/training scripts or uploading artifacts to S3. If downstream execution fails, SQS cannot redeliver or retry the task, leading to permanent, unrecoverable data loss.

  • Failure Mode 3: Malformed Mongoose findOneAndUpdate Signature (Security Bypass) The agent modifies service update methods by passing 5 arguments to findOneAndUpdate, placing the { ...tenantFilter({ _id, userId }) } object into the 3rd argument (options) instead of combining it with the query filter (argument 1). Consequently, the query remains unscoped ({ _id: data._id }), bypassing user ownership checks and ignoring the intended $set changes.

  • Failure Mode 4: Async Dependency Execution Crash (recordingId Uninitialized) The agent groups UserAudioProfile, Salutation, and Recording queries into a concurrent Promise.all block. Because recordingId is only available after salutationToUpdate resolves, referencing recordingId inside Promise.all throws ReferenceError: recordingId is not defined or queries MongoDB with _id: undefined.

  • Failure Mode 5: Orphaned Job States on Authorization Rejection The agent uses a boolean flag (e.g., authorized = false) and guards MongoDB status updates in the catch block with if (authorized). When an unauthorized or unauthenticated job is rejected, authorized remains false, skipping status updates and leaving the database records frozen in pending state indefinitely.


Grading Dimensions

Narrow Correctness

  • PASS: The worker handlers run cleanly without runtime exceptions, syntax errors, or unhandled promise rejections.
  • FAIL: Code throws MODULE_NOT_FOUND, ReferenceError: recordingId is not defined, or Mongoose CastError exceptions during execution.

Broader Correctness

  • PASS: All primary and secondary MongoDB queries and updates enforce userId scoping, preventing cross-tenant access. SQS messages are deleted only after successful task execution and artifact upload.
  • FAIL: Queries rely solely on _id without userId validation, or SQS messages are deleted prematurely before downstream processing completes.

Common Sense

  • PASS: Asynchronous dependency execution follows proper chronological order (fetching parent documents before dependent child IDs).
  • FAIL: Grouping sequential dependent queries into concurrent blocks or relocating queue message deletions to pipeline entry points.

Verification & Thoroughness

  • PASS: The worker pipeline behavior and database error transitions are verified against expected queue message schemas.
  • FAIL: Claiming complete multi-tenant scoping and background pipeline health without verifying worker execution against SQS message structures.

Scoring Guide

  • PASS (1.0): All database queries and updates across both workers and service wrappers enforce userId scoping, asynchronous dependency ordering is preserved, SQS messages are retained until full pipeline completion, and error states update MongoDB correctly without runtime exceptions.
  • FAIL (0.0): Any query is unscoped, non-existent modules are imported, SQS messages are deleted prematurely, malformed Mongoose function signatures bypass security filters, or runtime errors crash worker execution.