9.7 KiB
9.7 KiB
Potion — Repository Descriptions
Short descriptions of every git repo under repos/. Potion is a personalized-video
sales platform: users record a template video once, and AI (lip-sync, voice cloning,
background replacement, screen recording) generates a personalized variant per
recipient. The repos below split roughly into product apps, the job pipeline, AI
services, and infrastructure.
Product applications
| Folder | Runtime | Description |
|---|---|---|
potion-app |
node:16 |
The main Potion product — a Nuxt 2 / Vue web app with a custom Express server. Handles recording, the video editor, campaigns, billing (Stripe), auth, and integrations. Largest repo in the set. |
potion-web |
node:18 |
Nuxt 3 rewrite of the Potion front end. Same product surface as potion-app (pages, components, editor) on the newer framework and TypeScript config. |
potion-api |
node:20 |
Express backend API for the Potion app — serves the app's REST endpoints, talks to MongoDB, S3/GCS, Pub/Sub, SendGrid, and ffmpeg-based media helpers. |
potion-custom-domain-app |
none |
Nuxt app plus a small DNS/certificate API that lets customers serve Potion landing pages from their own domain (validates the A record, then issues a certificate). |
potion-website |
node:16 |
The public marketing website — a static Gulp + Webpack build with GSAP/Swiper animations. |
potion-wp-site |
none |
A WordPress installation (theme, assets, and SQL dumps) used for an earlier or secondary marketing site. |
browser-extensions |
node:18 |
Chrome extension source for the Potion screen/webcam recorder, with per-environment configs, manifests, and build scripts. |
potion-analytics |
node:20 |
TypeScript/Express service exposing analytics endpoints over the Potion MongoDB data, deployed via Cloud Build. |
Job pipeline (queueing and scheduling)
| Folder | Runtime | Description |
|---|---|---|
potion-job-producer |
node:18 |
Lambda / Cloud Function that builds the job payload and pushes AI jobs onto the queue (SQS on AWS, Pub/Sub on GCP). Has variants for GPU, CPU-only, and voice AI. |
potion-job-consumer |
node:18 |
The other half of the pair — consumes queued payloads and dispatches them to the AI workers. Same AWS/GCP dual deployment. |
potion-watcher |
node:18 |
Lambda that polls the AI queue depth and triggers the producer when work is waiting. |
potion-multi-dsr-watcher |
node:18 |
Cron-driven Cloud Function that watches for stalled or pending dynamic-screen-recording jobs in MongoDB and re-triggers them. |
lambda-potion-schedular |
node:14 |
AWS SAM umbrella project (template.yml) that packages the job producer, consumer, and watcher lambdas together as one scheduler stack. |
lambda-potion-transcription-scheduler |
node:14 |
Small Lambda that schedules audio/video transcription jobs. |
lambda-potion-engagement |
node:14 |
Lambda that queries MongoDB for product-engagement metrics and emails/exports CSV reports via SendGrid. |
elasticmq-container |
none |
Dockerfile and config for an ElasticMQ server — a local, SQS-compatible queue used for development. |
Video and media processing
| Folder | Runtime | Description |
|---|---|---|
potion-video-processing |
node:14 |
The original video-processing microservice: ffmpeg-based transcoding/assembly worker reading jobs from SQS and writing to S3. |
lambda-video-processing |
node:18 |
Later iteration of the same worker, packaged for both AWS Lambda and GCP Cloud Functions with Docker-based local dev. |
potion-video-processing-devops |
none |
Terraform for the video-processing service — IAM user/roles, ECR repository, and the Lambda that runs the container. |
microservice-dynamic-screen-recording |
node:18 |
Dynamic Screen Recording (DSR) worker — drives Puppeteer to load a prospect's website, records the browsing session, and produces the clip embedded in personalized videos. |
potion-dynamic-screen-recording-lambda |
node:14 |
Lambda-packaged DSR worker (chrome-aws-lambda, puppeteer-core), with urlbox as an alternative capture backend and a template-matching script. |
potion-stitch |
python:3.10 |
Python/Flask + ffmpeg service that stitches generated segments into the final personalized output and normalizes audio volume. |
potion-video-background-change |
python:3.10 |
Node worker that swaps the video background using MODNet matting (bundled ONNX/TorchScript models). |
potion-website-recording-handler |
node:18 |
Webhook handler receiving urlbox website-screenshot/recording callbacks and routing results back into the Potion app. |
urlbox-experiments |
python:3.10 |
Throwaway Python scripts evaluating urlbox.io as a replacement for Puppeteer screen capture (ad blocking, cookie banners, SSL behavior). |
AI models and inference services
| Folder | Runtime | Description |
|---|---|---|
potion-ai |
python:3.10 |
Vendored Wav2Lip — the upstream lip-sync research code that the personalization pipeline was originally built on. |
wav2lip-fa |
python:3.10 |
Potion's internal fork of Wav2Lip ("face alignment"): multiprocess preprocessing, distributed discriminator/generator training, perceptual loss at 384px, 3DDFA_v2 landmarks, MLflow logging. |
potion-ai-gpu |
python:3.10 |
Packaging of the current potion-ai inference stack for GKE — Dockerfiles, Cloud Build configs, k8s deployments, and KEDA autoscaling for GPU pods. |
potion-ai-cpu |
python:3.10 |
The same inference stack targeted at Cloud Run CPU instances, split into full-length-generation and greeting/edit images. |
video-synth-api |
python:3.10 |
The modularized GCP rearchitecture of potion-ai: each subfolder (3D reconstruction, face-landmark extraction, chunking, lip-sync, final render) is its own Cloud Run service, chained by two Cloud Workflows (template and editing). |
potion-tryon |
python:3.10 |
AI backend for Potion's virtual try-on feature — CatVTON diffusion pipeline with DensePose/Detectron2 and MODNet masking, deployed to GKE. |
yeahsure-tryon |
python:3.10 |
A second virtual try-on backend built on a hacked Stable Diffusion XL inpainting pipeline with IP-Adapter garment conditioning. |
MODNet-with-training |
python:3.10 |
MODNet portrait-matting fork with training code adapted for custom datasets (VideoMatte240k composited over BG-20K). |
potion-ai-pretrained-models-infra |
none |
Terraform + Lambda that pulls pre-trained model weights from an external source into a designated S3 bucket, per environment. |
Voice / speech
| Folder | Runtime | Description |
|---|---|---|
potion-voice |
node:14 |
Potion's text-to-speech service: multi-speaker baseline model training, voice cloning, and speech synthesis, plus the job handlers for each. |
microservice-potion-voice |
node:14 |
The Node worker that fronts the voice service — pulls voice jobs off the queue, runs ffmpeg audio work, and reports back to MongoDB. |
potion-voice-dataset |
python:3.10 |
Scripts for assembling the voice training corpus — converting Mozilla Common Voice to VCTK layout, pulling Potion recordings, trimming silence, generating filelists. |
potion-voice-utils |
python:3.10 |
Shared Python package of helpers used across the voice repos. |
lambda-text-to-speech |
node:18 |
Lambda wrapping the ElevenLabs TTS API, with S3/SQS plumbing and a Docker local-invoke setup. |
sentence-split-service |
python:3.10 |
Whisper (whisper-timestamped) transcription service that transcribes audio in parallel, splits it into sentences with NLTK, and exports matching text and audio slices. |
Datasets and data cleaning
| Folder | Runtime | Description |
|---|---|---|
avds-cleaner |
python:3.10 |
Audio/video dataset cleaning routines derived from SyncNet — detects and drops clips where audio and lip motion are out of sync, plus frame-rate post-processing. |
avspeech |
python:3.10 |
Processing pipeline and notes for the AVSpeech dataset: metadata filtering, AWS Transcribe language detection, Mechanical Turk review, and train/val/test splitting for wav2lip-fa training. |
Infrastructure and DevOps
| Folder | Runtime | Description |
|---|---|---|
potion-app-infra |
none |
Terraform for the main application estate, organized per component (network, web app, lambdas, analytics, potion-ai-cpu, DSR reporting, custom-domain NLBs), driven by workspaces and per-env tfvars. |
potion-devops |
none |
Jenkins pipelines and build/deploy Dockerfiles for each environment (development, qa, staging, production). |
potion-bastion |
none |
Terraform for the SSH bastion host per environment, including the public-key drop mechanism for granting access to private instances. |
gcp-infrastructure |
none |
Reusable Terraform module for GCP networking — subnetworks, firewall rules, flow logs, secondary IP ranges. |
gcp-cloud-infrastructure |
none |
GCP Deployment Manager template defining the dev VPC with public and private subnets. |
gcp-application |
node:18 |
Minimal "Hi Potion!" Express app with a Dockerfile and Cloud Build config — a smoke test / template for GCP deployments. |
lambda-cloudwatch-logs-to-loggly |
node:14 |
Lambda that forwards CloudWatch Logs to Loggly, deployed with Claudia.js. |
lambda-datadog-forwarder |
python:3.10 |
Vendored Datadog AWS log/metric forwarder Lambda bundle (dependencies checked in). |
Testing / QA
| Folder | Runtime | Description |
|---|---|---|
potion-qa |
node:18 |
Selenium WebDriver + Jest end-to-end suite covering auth, dashboard, recorder, editor, subtitles, settings, and pricing flows. |
potion-snapshot-testing |
node:18 |
Playwright visual-regression suite that compares screenshots across the app for basic and professional user accounts. |