additional source files

sources/160911B-meta-directory-changes.md

@@ -0,0 +1,170 @@
+• The voice-cloning handler now treats metadata.directoryName as a constrained identifier rather than a caller-controlled filesystem path. Validation occurs before any cleanup, file
+  creation, command execution, model recovery, or S3 upload.
+
+  ## Directory-name validation
+
+  A valid custom directoryName must:
+
+  - Be a string between 1 and 128 characters.
+  - Start with an ASCII letter or number.
+  - Contain only letters, numbers, ., _, and -.
+  - Have no surrounding whitespace.
+  - Contain no .. sequence.
+  - Not end with a dot.
+
+  For example, customer_42.voice-clone-v2 is accepted.
+
+  The following are rejected:
+
+  - ../../another-user
+  - /var/tmp/another-user
+  - nested/directory
+  - nested\directory
+  - -tar-option
+  - .hidden-directory
+  - customer..other
+  - customer.
+  - Names containing spaces, NUL characters, percent encoding, or more than 128 characters
+  - Non-string values such as null or numbers
+
+  Invalid names are rejected, not silently sanitized. This avoids different inputs unexpectedly resolving to the same directory.
+
+  The validation is centralized in voice-cloning-job-handler/path_safety.js.
+
+  ## Defense-in-depth validation
+
+  Validation now happens at two boundaries:
+
+  1. The queue worker validates the SQS message after parsing it.
+  2. The training pipeline independently validates the job object before performing any filesystem operation.
+
+  This means callers cannot bypass path validation by importing and invoking the training pipeline directly.
+
+  The object-level validator also verifies:
+
+  - The job and _doc are objects, not arrays.
+  - metadata is an object, not an array.
+  - Job ID, audio profile ID, and environment are present.
+  - The environment is development, staging, or production.
+  - input is a non-empty array.
+  - Each input item is an object.
+  - Recording URLs are valid HTTPS URLs.
+  - URLs do not contain embedded usernames or passwords.
+  - Original transcript text is present.
+  - Raw SQS message bodies are strings containing valid JSON.
+
+  Invalid queue messages remain unacknowledged and follow the existing retry/redrive behavior.
+
+  ## Root-contained path construction
+
+  All job paths are now constructed through a containment helper rather than direct path.join() calls.
+
+  The helper:
+
+  1. Resolves the configured root to an absolute path.
+  2. Resolves the requested child path.
+  3. Uses path.relative() to verify that the result is a strict descendant.
+  4. Rejects the configured root itself, parent paths, absolute escapes, and sibling-prefix tricks.
+
+  For example, a lexical prefix check can incorrectly treat /tmp/jobs-other as being inside /tmp/jobs. The new relative-path check does not have that weakness.
+
+  Containment is enforced for:
+
+  - The temporary job directory
+  - The temporary archive
+  - WAV and transcript directories
+  - The environment-specific EFS directory
+  - Job logs
+  - Resampled dataset output
+  - Model results directories
+  - Generated checkpoints and configurations
+
+  The environment is also revalidated before it is used as an EFS path component.
+
+  ## Symbolic-link protection
+
+  Lexical containment does not protect against a safe-looking path that contains a symbolic link. Before accessing or deleting job paths, the worker walks existing path components with
+  lstat().
+
+  It refuses processing if a symbolic link appears in:
+
+  - The temporary job directory
+  - The temporary archive path
+  - The EFS job/output hierarchy
+  - info.log
+  - error.log
+  - Recovered model asset paths
+
+  This prevents a pre-created link such as /tmp/safe-name -> /some/other/location from redirecting cleanup or file writes outside the configured root.
+
+  ## Safer command logs
+
+  Command logs received additional protection because the job log directory is preserved between retries.
+
+  Before appending to a log, the worker:
+
+  - Resolves the log file beneath the job’s log directory.
+  - Rejects existing non-regular files and symbolic links.
+  - Opens the file using O_NOFOLLOW where supported.
+  - Uses non-blocking, append-only creation flags.
+  - Verifies the opened descriptor is a regular file.
+  - Rejects files with multiple hard links.
+  - Creates new logs with mode 0600.
+
+  These checks prevent a malicious or stale info.log/error.log link from redirecting command output into another file.
+
+  ## Model recovery restrictions
+
+  Previously, model paths stored in the user profile were considered reusable if the files existed anywhere on the filesystem.
+
+  Recovered assets are now reused only when:
+
+  - Every required asset path is inside the current job’s expected EFS output directory.
+  - No path component is a symbolic link.
+  - Every required path points to a readable file.
+
+  Unsafe or unrelated profile paths are ignored. The worker then searches only the current job’s contained results directory or reruns training.
+
+  Generated model directories and individual checkpoint/configuration paths are also containment-checked before use.
+
+  This prevents a manipulated profile or custom directory name from causing arbitrary local files to be read and uploaded to S3.
+
+  ## S3 key safety
+
+  The validated directory name remains the model’s S3 key prefix. Because separators, control characters, and option-like names are rejected, callers cannot use directoryName to
+  construct nested or ambiguous S3 keys.
+
+  ## Documentation
+
+  README.md now documents:
+
+  - The accepted custom-name format
+  - The 128-character limit
+  - Rejected traversal and separator patterns
+  - Root-containment enforcement
+  - Symbolic-link handling
+
+  Existing custom names containing spaces, Unicode characters, consecutive dots, leading punctuation, or trailing dots will now be rejected and should be renamed.
+
+  ## Verification
+
+  The test suite now includes coverage for:
+
+  - A valid custom directory name
+  - Relative traversal attempts
+  - Absolute paths
+  - Forward and backward separators
+  - Option-like names
+  - Hidden-directory names
+  - Parent-directory sequences
+  - Trailing dots and surrounding whitespace
+  - NULs, encoded separators, non-string values, and oversized names
+  - Non-string message bodies
+  - HTTP and credential-bearing URLs
+  - Direct pipeline invocation with traversal input
+  - Preservation of files outside configured roots
+  - Symbolic-linked temporary directories
+  - Symbolic-linked command logs
+  - Job-local restrictions when reusing completed assets
+
+  All 23 tests pass, along with JavaScript syntax and whitespace checks.