# AI Custom Templates — Progress Tracker
## Master Plan
Replace the fixed `UniversalScene.tsx` renderer (9 layouts, 10 element types) with AI-generated unique React components per brand. Each template becomes a code string stored in DB, compiled at runtime for preview, written to temp files for rendering.
**No fallback to old system.** `UniversalScene.tsx`, `styleEngine.ts`, and the old custom template types will be **deleted** as a final cleanup step after all phases are implemented, tested, and stable. No dual-path code long-term — the new system must stand on its own so bugs are visible, not hidden behind a fallback.
**Full plan:** `/Users/faisalnazir/Desktop/Blog2Video - AI Generated Custom Templates Plan.md`
## Phases Overview
| Phase | Description | Status |
|-------|-------------|--------|
| 1 | DB Models + Enhanced Scraping | DONE |
| 2 | AI Code Generation + Preview | DONE |
| 3 | Chat-Based Editing | DEFERRED — removed from codebase, will revisit in a future phase |
| 4 | Render Pipeline Update | DONE |
| 5 | Polish & Preview System | DONE |
| 6 | Cleanup — Remove old UniversalScene/styleEngine/fallback code | DONE |
| H1 | Rate Limiting on Claude API Endpoints | DONE |
| H2 | Code Validator Hardening | DONE |
| H3 | Template Cache TTL | DONE |
| H4 | Render Workspace Locking | DONE |
| H5 | Frontend Code Sandbox (CSP + Error Boundary) | DONE |
| H6 | Self-healing current_version_id | DONE |
| F1 | BrandKit Population During Theme Extraction | DONE |
| F2 | Brand Asset Uploads (Logo) | DONE |
| F5 | Version Snapshots on Chat Edits | DEFERRED — depends on Phase 3 |
| F6 | custom_prompt_builder.py Cleanup | DONE |
| F3 | Enhanced Scraping with Firecrawl Branding Format | DONE |
| F4 | Preview Thumbnails for Template Gallery | DONE |
| F7 | Multiple Unique Content Scene Variants | DONE |
| OPT | Performance & Quality Optimization (5 fixes) | DONE |
| V2 | Content-Aware Architecture (feat/customv2) | DONE |
---
## Phase 1: DB Models + Enhanced Scraping — DONE
### What Was Done
**New file created:**
- `backend/app/models/brand_kit.py` — BrandKit model (brand_kits table)
- Fields: id, user_id, source_url, brand_name, colors (JSON), fonts (JSON), design_language (JSON), logos (JSON), images (JSON), timestamps
- Relationships: user, custom_templates
**Files modified:**
- `backend/app/models/custom_template.py` — Added:
- `component_code` (Text, nullable) — will hold AI-generated React code
- `intro_code` (Text, nullable) — intro scene variant
- `outro_code` (Text, nullable) — outro scene variant
- `brand_kit_id` (Integer FK → brand_kits.id, nullable, indexed)
- `brand_kit` relationship
- `backend/app/models/scene.py` — Added:
- `scene_type` (String(20), nullable) — "intro"/"content"/"outro"; NULL = "content"
- `backend/app/models/user.py` — Added:
- `brand_kits` relationship (cascade delete)
- `backend/app/models/__init__.py` — Added BrandKit import and export
- `backend/app/database.py` — Updated:
- `_migrate_sqlite()`: added component_code, intro_code, outro_code, brand_kit_id to ct_migrations; added scene_type to scene_migrations
- `init_db()`: added BrandKit to model imports
- `backend/alembic/env.py` — Added BrandKit import
- `backend/alembic/versions/phase1_brand_kit_and_schema_fields.py` — New Alembic migration:
- Creates brand_kits table
- Adds 4 columns to custom_templates
- Adds scene_type to scenes
- Down revision: 7ed594338d45
- `backend/app/services/theme_scraper.py` — Enhanced:
- `ScrapedThemeData` now has `logo_urls`, `og_image`, `screenshot_url` fields
- New `_extract_logo_urls()` helper extracts logos from HTML (favicon links + img tags with "logo" in attributes)
- Tries Firecrawl screenshot format with graceful fallback
- Extracts OG image from metadata
### Backward Compatibility
- All new columns nullable — existing data unaffected
- No API endpoint changes — responses identical
- No frontend changes
- ScrapedThemeData new fields are additive — ThemeExtractor ignores them
### Verification Steps
1. Start backend: `cd backend && uvicorn app.main:app --reload --port 8000`
2. Confirm no startup errors
3. Check brand_kits table exists in SQLite
4. Check new columns on custom_templates and scenes
5. Test existing custom template flow (paste URL → extract → save)
---
## Phase 2: AI Code Generation + Preview — DONE
### What Was Done
**New files created:**
- `backend/app/services/code_validator.py` — Validates generated code (blocklist for dangerous APIs, SceneComponent declaration check)
- `backend/app/services/code_generator.py` — Claude API-based code generation (claude-sonnet-4-5-20250929). Generates 3 variants (intro/content/outro) in one call using XML delimiters. Retry logic on validation failure.
- `frontend/src/utils/compileComponent.ts` — @babel/standalone JIT compiler. Lazy-loads Babel (~3MB), transpiles JSX, creates component via Function() factory with injected Remotion APIs.
- `frontend/src/components/RemotionPreviewPlayer.tsx` — Remotion Player wrapper. Compiles code string, renders live preview with sample props. Error state with retry button.
**Files modified:**
- `backend/app/routers/custom_templates.py` — Added:
- `POST /{template_id}/generate-code` endpoint (calls code_generator, validates, stores in DB)
- `GET /{template_id}/code` endpoint (lightweight code-only response)
- `_serialize_template()` now includes `component_code`, `intro_code`, `outro_code`
- `CustomTemplateOut` schema updated with code fields
- `frontend/src/components/templatePreviews/CustomPreview.tsx` — Added:
- Optional `componentCode` prop
- When `componentCode` exists, renders `RemotionPreviewPlayer` (lazy-loaded) instead of carousel
- Existing carousel unchanged as fallback for templates without generated code
- `frontend/src/api/client.ts` — Added:
- `component_code`, `intro_code`, `outro_code` fields to `CustomTemplateItem`
- `generateTemplateCode(templateId)` API method
- `getTemplateCode(templateId)` API method
- `frontend/src/components/CustomTemplateCreator.tsx` — Added:
- Step 3: "Generating Template" — shows spinner during code generation, then live Remotion preview
- After saving template (Step 2), automatically triggers code generation
- Error handling with retry button
- "Done" button to close modal
- `frontend/src/pages/CustomTemplates.tsx` — Updated:
- Template grid passes `componentCode` to CustomPreview for live Remotion previews
- `frontend/package.json` — Added:
- `@babel/standalone` dependency
- `@types/babel__standalone` dev dependency
### Architecture Decisions
- **Claude API for code gen** (not DSPy) — raw code output doesn't fit DSPy's structured fields
- **Separate endpoint** — code gen is slow (10-30s), doesn't block template creation
- **One LLM call for 3 variants** — intro/content/outro in one call for visual coherence
- **Lazy-loaded Babel** — ~3MB chunk only loaded when viewing templates with generated code
- **No fallback** — compilation errors surface inline with retry option
### Verification Steps
1. Start backend: `cd backend && uvicorn app.main:app --reload --port 8000`
2. Create a custom template via existing flow (paste URL → extract → save)
3. Step 3 should auto-trigger code generation with spinner
4. After generation, live Remotion preview should play in the modal
5. On Custom Templates page, templates with generated code show live previews
6. `npm run build` passes ✓
---
## Phase 3: Chat-Based Editing — DONE
### What Was Done
**New files created:**
- `backend/app/models/code_edit_history.py` — CodeEditHistory model (code_edit_history table)
- Fields: id, template_id (FK, CASCADE), code_type ("component"/"intro"/"outro"), previous_code, instruction, summary, created_at
- Composite index on (template_id, code_type)
- Relationship: template (back to CustomTemplate)
- `backend/alembic/versions/phase3_code_edit_history.py` — Alembic migration
- Creates code_edit_history table
- Down revision: phase1_brand_kit
- `frontend/src/components/TemplateCodeEditor.tsx` — Full-screen code editor overlay
- Two-panel layout: Remotion preview (left) + chat panel (right)
- Variant tabs (Intro/Content/Outro) to switch which code is previewed/edited
- Chat messages reconstructed from edit history on mount
- Typing indicator dots during Claude processing
- Undo button per variant (reverts to previous code version)
- Suggestion chips for common edit requests
- Error display for failed edits
**Files modified:**
- `backend/app/services/code_generator.py` — Added:
- `EDIT_SYSTEM_PROMPT` — system prompt for code editing (modify existing code, keep everything else intact)
- `_parse_edit_response()` — parses `<CODE>` and `<SUMMARY>` XML tags from response
- `edit_component_code(existing_code, instruction)` — calls Claude to edit code, validates, retries on failure. Returns `{ code, summary }`
- `backend/app/routers/custom_templates.py` — Added:
- `EditCodeRequest` / `UndoEditRequest` Pydantic schemas
- `VARIANT_FIELD_MAP` helper dict
- `POST /{template_id}/edit-code` — sends existing code + instruction to Claude, saves edit history, updates template code
- `POST /{template_id}/undo-edit` — reverts to previous code version from history, deletes history row
- `GET /{template_id}/edit-history` — returns list of edits (instruction + summary, no code) for chat reconstruction
- `backend/app/models/custom_template.py` — Added:
- `edit_history` relationship (cascade delete-orphan)
- `backend/app/models/__init__.py` — Added CodeEditHistory import and export
- `backend/app/database.py` — Added CodeEditHistory to init_db imports (SQLite auto-creates via create_all)
- `backend/alembic/env.py` — Added CodeEditHistory import
- `frontend/src/api/client.ts` — Added:
- `CodeVariant` type ("component" | "intro" | "outro")
- `CodeEditHistoryItem` interface
- `editTemplateCode(templateId, instruction, variant)` API method
- `undoTemplateEdit(templateId, variant)` API method
- `getTemplateEditHistory(templateId)` API method
- `frontend/src/pages/CustomTemplates.tsx` — Added:
- `codeEditTarget` state for full-screen code editor
- "Customize" button on template cards (only shown when component_code exists, purple filled)
- `TemplateCodeEditor` rendering when codeEditTarget is set
- `preloadBabel()` call on page mount
- `handleCodeEditorClose()` to update template in grid after editing
### Architecture Decisions
- **Separate edit function** — `edit_component_code()` uses a different system prompt than generation (focused on modifying existing code, not creating from scratch)
- **Per-variant editing** — user selects which variant (Intro/Content/Outro) to edit; edits apply to one variant at a time (saves cost, more predictable)
- **Edit history table** — follows SceneEditHistory pattern; stores previous_code for multi-level undo
- **Chat reconstruction** — edit history reconstructed into chat messages on mount (no separate chat persistence)
- **No new npm dependencies** — reuses existing @babel/standalone, @remotion/player, RemotionPreviewPlayer
- **Separate from CustomTemplateEditor** — existing editor kept for name/style changes; new TemplateCodeEditor is full-screen for code editing
### Verification Steps
1. Start backend: `cd backend && uvicorn app.main:app --reload --port 8000` — no errors, code_edit_history table created
2. Create a template with generated code (Phase 2 flow)
3. On Custom Templates page, "Customize" button appears on templates with code
4. Click "Customize" → full-screen editor opens with preview + chat
5. Type "make the title bigger" → Claude edits code → preview updates live
6. Click "Undo" → code reverts to previous version → preview updates
7. Close editor → template card on grid shows updated preview
8. `npm run build` passes ✓
---
## Phase 4: Render Pipeline Update — DONE
### What Was Done
**New files created:**
- `remotion-video/src/templates/generated/types.ts` — TypeScript types for generated video data (GeneratedSceneProps, GeneratedVideoData, GeneratedSceneData)
- `remotion-video/src/templates/generated/SceneIntro.tsx` — Placeholder intro scene component (overwritten with AI code at render time)
- `remotion-video/src/templates/generated/SceneContent.tsx` — Placeholder content scene component (overwritten with AI code at render time)
- `remotion-video/src/templates/generated/SceneOutro.tsx` — Placeholder outro scene component (overwritten with AI code at render time)
- `remotion-video/src/templates/generated/GeneratedVideo.tsx` — New Remotion composition:
- Reads data.json (including brandColors and sceneType per scene)
- Statically imports SceneIntro, SceneContent, SceneOutro components
- Maps each scene to the right component based on sceneType (explicit from DB, or inferred: first=intro, last=outro, middle=content)
- Handles font loading via delayRender/continueRender
- Sequences scenes with Audio and LogoOverlay (same pattern as CustomVideo)
- calculateGeneratedMetadata for dynamic duration/dimensions
**Files modified:**
- `remotion-video/src/Root.tsx` — Added:
- Import of GeneratedVideo + calculateGeneratedMetadata
- New `<Composition id="GeneratedVideo" .../>` registration
- `backend/app/services/template_service.py` — Updated:
- `_load_custom_template_data()` now includes `has_generated_code`, `component_code`, `intro_code`, `outro_code` in cached result
- `_get_custom_meta()` passes `has_generated_code` to `build_custom_meta`
- `backend/app/services/custom_prompt_builder.py` — Updated:
- `build_custom_meta()` accepts `has_generated_code` parameter
- Returns `composition_id: "GeneratedVideo"` when template has AI code, `"CustomVideo"` otherwise
- `backend/app/services/remotion.py` — Updated:
- `provision_workspace()` calls `_write_generated_scene_files()` for custom templates with AI code
- `_write_generated_scene_files()` (NEW) — loads component code from DB, wraps in proper .tsx module with imports, writes to workspace `src/templates/generated/` directory
- `_wrap_generated_code()` (NEW) — wraps raw AI-generated code (`const SceneComponent = ...`) in a proper ESM module with Remotion imports + default export
- `write_remotion_data()` — adds `brandColors` and `sceneType` per scene to data.json when template has generated code
### Architecture Decisions
- **Static file overwrite** — generated code is written as actual .tsx files in the workspace, overwriting placeholders. Vite bundles them normally — no runtime compilation, no @babel/standalone needed server-side.
- **Placeholder pattern** — the repo contains stub scene components so Root.tsx always compiles. At render time, these stubs are replaced with the real AI-generated code.
- **composition_id routing** — `GeneratedVideo` vs `CustomVideo` is selected via `build_custom_meta` based on whether the template has `component_code`. Old templates without generated code continue to use UniversalScene.
- **brandColors in data.json** — derived from theme colors, passed to GeneratedSceneProps. Generated components use `props.brandColors` directly.
- **sceneType tagging** — each scene gets `sceneType` ("intro"/"content"/"outro") from the DB `scene_type` field, falling back to positional inference (first=intro, last=outro).
- **Code wrapping** — raw AI code (`const SceneComponent = (props) => { ... }`) is wrapped with Remotion imports and `export default SceneComponent`. This makes the generated files proper ES modules that Vite can bundle.
### Verification Steps
1. Start backend: `cd backend && uvicorn app.main:app --reload --port 8000` — no errors
2. Create a template with generated code (Phase 2 flow)
3. Trigger a video render for a project using the generated template
4. Check logs: should see `[REMOTION] GeneratedVideo: brandColors and sceneTypes set for N scenes`
5. Check workspace: `src/templates/generated/SceneIntro.tsx` etc. should contain the AI-generated code wrapped in proper imports
6. Render should produce an MP4 using the AI-generated components
7. Templates without generated code should still render via CustomVideo + UniversalScene
8. TypeScript check passes: `cd remotion-video && npx tsc --noEmit` ✓
---
## Phase 5: Polish & Preview System — DONE
### What Was Done
**New file created:**
- `backend/app/models/template_version.py` — TemplateVersion model (template_versions table)
- Fields: id, template_id (FK, CASCADE), component_code, intro_code, outro_code, label, created_at
- Relationship: template (back to CustomTemplate)
- `backend/alembic/versions/phase5_template_versions.py` — Alembic migration
- Creates template_versions table
- Adds current_version_id column to custom_templates
- Down revision: phase3_code_edit_history
**Files modified:**
- `backend/app/models/custom_template.py` — Added:
- `current_version_id` (Integer, nullable) — pointer to active version
- `versions` relationship (cascade delete-orphan, ordered by created_at desc)
- `backend/app/models/__init__.py` — Added TemplateVersion import and export
- `backend/app/database.py` — Added:
- TemplateVersion to init_db imports
- `current_version_id` to SQLite migrations for custom_templates
- `backend/alembic/env.py` — Added TemplateVersion import
- `backend/app/routers/custom_templates.py` — Added:
- `_save_version()` helper — snapshots code fields into a new TemplateVersion row
- Updated `generate_code()` — saves "Initial generation" version after first code gen
- `POST /{template_id}/regenerate-code` — regenerates all code variants from scratch; snapshots current code before overwriting, saves new "Regenerated" version
- `GET /{template_id}/versions` — lists all saved versions (newest first) with current_version_id
- `POST /{template_id}/versions/{version_id}/rollback` — restores code from a previous version; snapshots current state as "Before rollback" first
- `CustomTemplateOut` and `_serialize_template()` now include `current_version_id`
- `frontend/src/api/client.ts` — Added:
- `current_version_id` to `CustomTemplateItem` interface
- `TemplateVersionItem` and `TemplateVersionsResponse` interfaces
- `regenerateTemplateCode(templateId)` API method
- `getTemplateVersions(templateId)` API method
- `rollbackTemplateVersion(templateId, versionId)` API method
- `frontend/src/pages/CustomTemplates.tsx` — Added:
- `regeneratingId` state for tracking which template is regenerating
- `handleRegenerate()` function — calls regenerate API, updates template in grid
- Regenerate icon button on each template card (next to Customize), with spinning SVG animation during regeneration
- `frontend/src/components/TemplateCodeEditor.tsx` — Added:
- Version history state: `versions`, `currentVersionId`, `showVersions`, `isRegenerating`, `isRollingBack`
- `loadVersions()` — fetches version list from API on mount
- `handleRegenerate()` — regenerates code from within editor, clears chat, reloads versions
- `handleRollback(versionId)` — restores a previous version, reloads version list
- Regenerate button in preview panel header (purple outline, with spinner)
- "Versions (N)" toggle button in chat header — opens collapsible version history panel
- Version history panel: lists all versions with labels, timestamps, "(current)" badge, and "Restore" button per version
### Architecture Decisions
- **Version snapshots** — each version stores a full copy of all 3 code variants (intro/component/outro). Not incremental diffs — simpler, no reconstruction needed.
- **Automatic versioning** — versions are created automatically on generate, regenerate, and rollback. No manual "save version" step needed.
- **Pre-rollback snapshots** — before a rollback overwrites code, the current state is saved as a "Before rollback" version, so users can always undo a rollback.
- **current_version_id** — pointer on the template to its active version. Used by frontend to highlight "(current)" in the version list. Not a FK constraint (flexible).
- **No preview thumbnails yet** — this phase focused on regeneration + versioning. Server-side mini-renders for gallery thumbnails can be added later as an optimization.
### Verification Steps
1. Start backend: `cd backend && uvicorn app.main:app --reload --port 8000` — no errors, template_versions table created
2. Create a template with generated code (Phase 2 flow) — a version is saved automatically
3. On Custom Templates page, hover a template card → Regenerate button (refresh icon) appears next to Customize
4. Click Regenerate → spinner shows → new code generated → preview updates → old version preserved
5. Click Customize → full-screen editor opens → "Versions (N)" button in chat header
6. Click "Versions (N)" → version history panel opens with all versions listed
7. Click "Restore" on an older version → code rolls back → preview updates
8. TypeScript check passes (no new errors) ✓
---
## Phase 6: Cleanup — Remove Old UniversalScene/StyleEngine/Fallback Code — DONE
### What Was Done
**Files deleted:**
- `remotion-video/src/templates/custom/` — entire directory removed:
- `UniversalScene.tsx` (36KB) — old fixed layout engine (9 arrangements, 10 element types)
- `CustomVideo.tsx` (10KB) — old Remotion composition using UniversalScene
- `types.ts` (3KB) — old CustomTheme, SceneLayoutConfig types
- `utils/styleEngine.ts` (11KB) — theme style → CSS decoration mapper
- `utils/normalize.ts` (7KB) — layout config normalization helpers
- `frontend/src/components/remotion/custom/` — entire directory removed:
- `UniversalScene.tsx` — frontend copy of server UniversalScene
- `CustomVideoComposition.tsx` — frontend copy of CustomVideo
- `types.ts` — frontend copy of custom types
- `utils/` — frontend copy of utils
**Files modified:**
- `remotion-video/src/Root.tsx` — Removed:
- `CustomVideo` + `calculateCustomMetadata` import
- `<Composition id="CustomVideo" .../>` registration
- Only `GeneratedVideo` composition remains for custom templates
- `backend/app/services/remotion.py` — Updated:
- Removed `has_generated_code` conditional in `write_remotion_data()` — brandColors and sceneType are now always set for custom templates
- Replaced `print()` calls with `logger.info()`/`logger.warning()`
- `backend/app/services/custom_prompt_builder.py` — Updated:
- Removed `has_generated_code` parameter from `build_custom_meta()`
- `composition_id` is now always `"GeneratedVideo"` — no fallback to `"CustomVideo"`
- `backend/app/services/template_service.py` — Updated:
- Removed `has_generated_code` kwarg from `build_custom_meta()` call
- `frontend/src/components/remotion/remotionAdapters.tsx` — Removed:
- `UniversalScene` import from `@remotion-video/templates/custom/UniversalScene`
- `CustomTheme`, `SceneLayoutConfig` type imports from `@remotion-video/templates/custom/types`
- `RemotionCustomSceneInput` interface
- `RemotionCustomVideoCompositionProps` interface
- `RemotionCustomVideoComposition` component (entire ~100 line composition)
- `frontend/src/components/remotion/templateConfig.tsx` — Removed:
- `CustomVideoComposition` import
- `CUSTOM_ARRANGEMENTS` set
- `custom` entry from `TEMPLATE_REGISTRY`
- `custom_` routing in `getTemplateConfig()` (no longer routes to old system)
- `RemotionCustomVideoComposition` from the remotion source override chain
- `frontend/src/types/remotion-video.d.ts` — Removed:
- `@remotion-video/templates/custom/UniversalScene` module declaration
- `@remotion-video/templates/custom/types` module declaration
- `frontend/src/components/templatePreviews/CustomPreview.tsx` — Rewritten:
- Removed 3-slide carousel fallback (SlideHero, SlideContent, SlideTypo, SlideDots, ScaledCanvas)
- Removed all old style helpers (getStyleBg, getCardStyle, getDecorations)
- Now only renders `RemotionPreviewPlayer` when `componentCode` exists
- Shows simple placeholder when `componentCode` is missing
- `frontend/src/components/SceneEditModal.tsx` — Updated:
- Changed comment from "UniversalScene handles sizing" to "Custom template arrangements"
### Architecture Changes
- **Single rendering path** — All custom templates now use `GeneratedVideo` composition exclusively. No fallback to `CustomVideo` + `UniversalScene`.
- **~107KB of dead code removed** — UniversalScene (36KB), styleEngine (11KB), normalize (7KB), CustomVideo (10KB × 2 copies), types (3KB × 2 copies), carousel preview (~400 lines).
- **No dual-path conditionals** — `build_custom_meta()` always returns `composition_id: "GeneratedVideo"`. `write_remotion_data()` always writes brandColors + sceneType for custom templates.
### Verification Steps
1. `cd remotion-video && npx tsc --noEmit` — passes ✓
2. `cd frontend && npx tsc --noEmit` — passes (only pre-existing @babel/standalone type issue from missing npm install) ✓
3. Backend imports: `python -c "from app.services.remotion import provision_workspace; ..."` — passes ✓
4. Custom templates page shows live Remotion previews (no carousel fallback)
5. Video rendering for custom templates uses GeneratedVideo composition exclusively
---
## Phase H1: Rate Limiting on Claude API Endpoints — DONE
### What Was Done
**File modified:**
- `backend/app/routers/custom_templates.py` — Added:
- `_ai_call_counts` in-memory dict: `user_id → (date_string, count)`
- `AI_DAILY_LIMIT = 5` constant
- `_check_ai_rate_limit(user_id)` helper — checks daily count, resets on new day, raises HTTP 429 if exceeded
- Rate limit check added at top of `generate_code()`, `regenerate_code()`, and `edit_code()` endpoints
### Design Decisions
- **In-memory counter** — matches existing `_render_progress` pattern. Single Cloud Run instance, so in-memory is fine.
- **Combined limit** — 5 calls/day total across generate + regenerate + edit (not 5 per endpoint).
- **Daily reset** — counter resets on date change, no cron needed.
- **Future extensibility** — `AI_DAILY_LIMIT` can be made per-plan (Free: 5, Standard: 20, Pro: 50).
---
## Phase H2: Code Validator Hardening — DONE
### What Was Done
**File modified:**
- `backend/app/services/code_validator.py` — Enhanced:
- Added `DANGEROUS_REGEX` list — 18 compiled regex patterns with word-boundary matching for `eval`, `Function`, `fetch`, `document`, `window`, `process`, `globalThis`, `require`, `import`, `__proto__`, `constructor[]`, `Proxy`, `Reflect`, `XMLHttpRequest`, `WebSocket`, `localStorage`, `sessionStorage`, `cookie`
- Added `MAX_CODE_LENGTH = 15,000` — rejects code exceeding this limit
- Added `MAX_NESTING_DEPTH = 20` — rejects excessively nested brace structures
- Added balanced braces check — ensures `{` and `}` counts match
- Two-pass validation: fast substring check first, then thorough regex pass
### Design Decisions
- **Regex word boundaries** — `\beval\b` catches `eval` but not `evaluation`, and is harder to bypass with string concatenation tricks
- **New blocked APIs** — `Proxy`, `Reflect` (metaprogramming), `WebSocket` (network), `localStorage`/`sessionStorage`/`cookie` (storage)
- **Structural checks** — max length prevents abuse via massive code payloads; nesting depth catches obfuscated code; balanced braces catch malformed output
---
## Phase H3: Template Cache TTL — DONE
### What Was Done
**Files modified:**
- `backend/app/services/template_service.py` — Updated:
- Cache type changed from `dict[str, data]` to `dict[str, (data, timestamp)]`
- Added `_CACHE_TTL = 30.0` seconds constant
- `_load_custom_template_data()` now checks `time.monotonic()` age on cache hit — refetches from DB if expired
- Explicit invalidation (`invalidate_custom_template_cache()`) still works for same-process freshness
- `backend/app/services/remotion.py` — Updated:
- `provision_workspace()` calls `invalidate_custom_template_cache(template_id)` before writing scene files, ensuring fresh DB data
### Design Decisions
- **30s TTL** — short enough for multi-worker consistency, long enough to avoid hammering DB during a single request lifecycle
- **`time.monotonic()`** — immune to system clock changes
- **Explicit invalidation preserved** — same-process updates still get instant cache busting
---
## Phase H4: Render Workspace Locking — DONE
### What Was Done
**File modified:**
- `backend/app/services/remotion.py` — Added:
- `_workspace_locks: dict[int, threading.Lock]` — per-project lock registry
- `_get_workspace_lock(project_id)` — returns or creates a lock for a project
- `provision_workspace()` body wrapped in `with _get_workspace_lock(project_id):`
### Design Decisions
- **Per-project locks** — different projects can provision concurrently; only same-project requests serialize
- **Same pattern as `_render_progress`** — uses module-level dict for lock storage
- **`threading.Lock`** — sufficient for single-process multi-thread scenarios (uvicorn workers)
---
## Phase H5: Frontend Code Sandbox — DONE
### What Was Done
**Files modified:**
- `frontend/index.html` — Added:
- CSP meta tag: `script-src 'self' 'unsafe-eval'` (needed for Babel), `connect-src 'self' https://*.blog2video.app` (blocks generated code from fetching external URLs), restricted `style-src`, `img-src`, `media-src`, `font-src`
- `frontend/src/components/RemotionPreviewPlayer.tsx` — Added:
- `PlayerErrorBoundary` class component — catches React render errors from AI-generated components
- Shows friendly error message with "Retry" button instead of crashing the app
- Player JSX wrapped in `<PlayerErrorBoundary>`
### Design Decisions
- **CSP as first line of defense** — even if generated code contains `fetch()`, the browser blocks it from reaching external URLs
- **`unsafe-eval` required** — @babel/standalone needs `eval` for JIT compilation; CSP restricts everything else
- **Error boundary on Player** — catches runtime errors from compiled AI components; shows error UI instead of white screen
---
## Phase H6: Self-healing current_version_id — DONE
### What Was Done
**File modified:**
- `backend/app/routers/custom_templates.py` — Updated:
- `list_versions()` now checks if `tpl.current_version_id` exists in the fetched versions set
- If not found (dangling pointer), auto-sets to the newest version's ID and commits
### Design Decisions
- **Lazy self-healing** — only checks when versions are listed (read path), not on every template access
- **Newest version wins** — if pointer is dangling, the most recent version is the safest default
---
## Phase F1: BrandKit Population During Theme Extraction — DONE
### What Was Done
**Files modified:**
- `backend/app/routers/custom_templates.py` — Updated:
- `ExtractThemeResponse` now includes `logo_urls`, `og_image`, `screenshot_url` from ScrapedThemeData
- `CreateCustomTemplateRequest` now accepts optional `logo_urls`, `og_image`, `screenshot_url`
- `extract_theme()` passes scraped logo/image data through to the response
- `create_custom_template()` now creates a `BrandKit` row from theme data (colors, fonts, design_language, logos, images) and links it via `brand_kit_id`
- `frontend/src/api/client.ts` — Updated:
- `ExtractThemeResponse` interface includes `logo_urls`, `og_image`, `screenshot_url`
- `createCustomTemplate()` accepts optional `logo_urls`, `og_image`, `screenshot_url`
- `frontend/src/components/CustomTemplateCreator.tsx` — Updated:
- Stores scraped `logo_urls`, `og_image`, `screenshot_url` from extract response in state
- Passes them to `createCustomTemplate()` call so BrandKit is populated
### Design Decisions
- **BrandKit auto-created on template creation** — every new template gets a linked BrandKit, populating data that `code_generator.py` already reads (lines 199-207)
- **Logo URLs stored as JSON list** — scraped logo URLs are raw strings; F2 adds structured upload with primary/secondary types
- **OG image stored in BrandKit.images** — serves as the initial brand image until user uploads custom ones
---
## Phase F2: Brand Asset Uploads (Logo) — DONE
### What Was Done
**Files modified:**
- `backend/app/services/r2_storage.py` — Added:
- `brand_asset_key(user_id, brand_kit_id, filename)` helper for R2 key generation
- `backend/app/routers/custom_templates.py` — Added:
- `POST /{template_id}/upload-logo` endpoint — validates file type (PNG/JPEG/WebP/SVG) and size (2MB), auto-creates BrandKit if missing, uploads to R2, updates `bk.logos` JSON with primary logo entry
- FastAPI `UploadFile`, `File` imports
- `backend/app/services/template_service.py` — Updated:
- `_load_custom_template_data()` now includes `brand_kit` dict (colors, fonts, logos, design_language, images) in cached result
- `backend/app/services/remotion.py` — Updated:
- `write_remotion_data()` reads brand_kit logos from custom_data, downloads primary logo to workspace public/ folder, sets `data["brandLogo"]` for GeneratedVideo access via `staticFile()`
- `frontend/src/api/client.ts` — Added:
- `uploadTemplateLogo(templateId, file)` API method
- `frontend/src/components/TemplateCodeEditor.tsx` — Added:
- Hidden file input + "Logo" button in preview panel header
- `handleLogoUpload()` — uploads file, updates template state, shows success/error message in chat
### Design Decisions
- **Auto-create BrandKit on upload** — if template was created before F1, uploading a logo still works by auto-creating a minimal BrandKit
- **Logo format migration** — handles old BrandKit.logos format (list of URL strings) and new format (list of dicts with type/url)
- **Primary logo replacement** — uploading a new logo replaces the previous primary, preserving scraped logos as type "scraped"
- **Brand logo in data.json** — downloaded to workspace `public/brand-logo.png` for `staticFile()` access in GeneratedVideo
---
## Phase F5: Version Snapshots on Chat Edits — DONE
### What Was Done
**File modified:**
- `backend/app/routers/custom_templates.py` — Updated:
- `edit_code()` now counts total edits for the template after each edit
- Every 3rd edit, auto-saves a version snapshot via `_save_version(tpl, f"After {count} edits", db)`
### Design Decisions
- **Every 3rd edit** — balances version granularity vs. list clutter; users get checkpoints at edits 3, 6, 9, etc.
- **Combined with existing versioning** — these auto-versions appear alongside generate/regenerate/rollback versions in the version panel
- **Non-blocking** — version save happens in the same DB transaction as the edit, no extra overhead
---
## Phase F6: custom_prompt_builder.py Cleanup — DONE
### What Was Done
**File modified:**
- `backend/app/services/custom_prompt_builder.py` — Updated:
- `build_custom_meta()` simplified: removed `valid_arrangements`, `preferred_arrangements`, `arrangements_without_image`, `hero_arrangement`, `fallback_arrangement`
- Kept `valid_layouts`, `layouts_without_image` (used by SceneEditModal), `hero_layout`, `fallback_layout` (used by scene generation)
- Added docstring explaining dual purpose: `build_custom_prompt()` is for DSPy AI script generation, `build_custom_meta()` is for pipeline routing
### Design Decisions
- **Prompt preserved** — `build_custom_prompt()` still generates the full 300-line prompt for DSPy scene generation (narration, visual hints) even though GeneratedVideo doesn't use arrangement/element layout data
- **Meta simplified** — removed arrangement-specific fields that GeneratedVideo ignores; kept layout fields that the API pipeline (SceneEditModal dropdown) still uses
---
## Phase F3: Enhanced Scraping with Firecrawl Branding Format — DONE
### What Was Done
**Files modified:**
- `backend/app/services/theme_scraper.py` — Updated:
- `ScrapedThemeData` has new `branding: dict | None` field
- `scrape_for_theme()` tries `formats=["branding", "html", "markdown", "screenshot"]` first
- Falls back to standard formats if branding not available on the Firecrawl plan
- Normalizes branding_data to dict (handles SDK object or raw dict)
- `backend/app/dspy_modules/theme_extractor.py` — Added:
- `_apply_branding_overrides()` static method on `ThemeExtractor`
- After DSPy theme extraction, merges Firecrawl branding data as high-confidence overrides
- Overrides: `colors.primary→accent`, `colors.background→bg`, `colors.text→text`, `colors.secondary→surface`
- Overrides: `typography.headingFont→fonts.heading`, `typography.bodyFont→fonts.body`
- Called in `extract_theme()` only when `scraped.branding` is present and is a dict
### Design Decisions
- **Graceful fallback** — branding format may not be available on all Firecrawl plans; try/except ensures zero regression
- **Override, not replace** — Firecrawl branding values are merged on top of DSPy-extracted theme; DSPy still provides style, animation, patterns, category
- **Only override when present** — each branding field is checked individually; missing fields are left as DSPy extracted them
---
## Phase F4: Preview Thumbnails for Template Gallery — DONE
### What Was Done
**New file created:**
- `backend/app/services/thumbnail_renderer.py` — Renders single-frame PNG thumbnail:
- `render_template_thumbnail(template_id, user_id)` — provisions temporary workspace, writes mock data.json with 1 intro scene, runs `npx remotion still GeneratedVideo` at 480p, uploads PNG to R2
- Uses negative project IDs for temp workspace to avoid collision
- Cleanup: removes temp workspace in finally block
**Files modified:**
- `backend/app/routers/custom_templates.py` — Updated:
- `_render_and_store_thumbnail()` background task helper — calls renderer, stores URL in `tpl.preview_image_url`
- `generate_code()` and `regenerate_code()` now accept `BackgroundTasks` and kick off thumbnail render after code gen
- `_serialize_template()` now includes `preview_image_url`
- `frontend/src/api/client.ts` — Updated:
- `CustomTemplateItem` includes `preview_image_url: string | null`
- `frontend/src/components/templatePreviews/CustomPreview.tsx` — Rewritten:
- If `previewImageUrl` exists and user hasn't hovered, shows static image with play icon overlay (instant load)
- On hover, lazy-loads the full `RemotionPreviewPlayer` (Babel JIT compilation)
- Suspense fallback uses thumbnail image if available
- `frontend/src/pages/CustomTemplates.tsx` — Updated:
- Passes `previewImageUrl={tpl.preview_image_url}` to CustomPreview in template grid
### Design Decisions
- **Background task** — thumbnail rendering is non-critical; failure doesn't affect code generation response
- **`remotion still`** — uses Remotion's built-in still image command (frame 0) instead of full video render; much faster
- **480p resolution** — sufficient for gallery thumbnails, keeps file size small
- **Hover-to-play** — template grid loads instantly with static images; @babel/standalone (~3MB) only loaded when user interacts
- **Thumbnail as Suspense fallback** — when Player is loading, the thumbnail serves as a smooth placeholder instead of a blank rectangle
---
## Phase F7: Multiple Unique Content Scene Variants — DONE
### What Was Done
Previously, Claude generated only 3 variants (1 intro, 1 content, 1 outro). All content scenes reused the same component — different text/image props but identical layout. This was the same fundamental limitation as the old UniversalScene system.
Now, Claude decides how many unique scene variants to create (typically 5-8 total). Each content scene gets a visually distinct layout, and scenes cycle through variants at render time.
**Before:** Intro → Content (same layout × N) → Outro
**After:** Intro → Scene1 → Scene2 → Scene3 → Scene4 → Scene5 → Outro (all unique layouts)
**Files modified:**
- `backend/app/models/custom_template.py` — Added `content_codes` (Text, nullable) — JSON array of content variant code strings
- `backend/app/models/template_version.py` — Added `content_codes` (Text, nullable) — version snapshots include all content variants
- `backend/app/database.py` — Added `content_codes` to SQLite migrations for both custom_templates and template_versions
- `backend/app/services/code_generator.py` — Major rewrite:
- New SYSTEM_PROMPT: asks Claude to generate 5-8 scene variants using `<SCENE_0>`, `<SCENE_1>`, ..., `<SCENE_N>` XML tags
- Claude decides how many unique layouts the brand needs
- Scene 0 = intro, last scene = outro, all middle scenes = unique content variants
- `_parse_scene_variants()` replaces `_parse_variants()` — parses dynamic numbered XML tags
- Returns `content_codes: list[str]` instead of single `component_code`
- `max_tokens` increased from 8000 to 16000 for multiple variants
- `backend/app/routers/custom_templates.py` — Updated:
- `EditCodeRequest.variant` now accepts `content_N` format (e.g., `content_0`, `content_3`)
- `_parse_variant()` helper parses variant strings into (type, index) tuples
- `_get_variant_code()` / `_set_variant_code()` handle reading/writing content codes by index
- `generate_code()` and `regenerate_code()` store `content_codes` as JSON
- `_save_version()` snapshots `content_codes`
- Rollback restores `content_codes`
- `_serialize_template()` includes `content_codes` (parsed from JSON)
- `backend/app/services/template_service.py` — Updated:
- `_load_custom_template_data()` parses and caches `content_codes`
- `has_generated_code` checks both `component_code` and `content_codes`
- `backend/app/services/remotion.py` — Updated:
- `_write_generated_scene_files()` now writes SceneContent0.tsx, SceneContent1.tsx, ..., SceneContentN.tsx
- Generates `contentRegistry.ts` — exports `CONTENT_VARIANTS` array with all content components
- Generates SceneContent.tsx (re-exports Content0 for backward compat)
- `write_remotion_data()` adds `contentVariantIndex` and `contentVariantCount` to data.json
- Content scenes are assigned variant indices cyclically (scene[i] uses variant[i % numVariants])
**New files created:**
- `remotion-video/src/templates/generated/contentRegistry.ts` — Placeholder; at render time, overwritten with imports of all content variant components
- `remotion-video/src/templates/generated/SceneContent0.tsx` — Placeholder; re-exports SceneContent
**Files rewritten:**
- `remotion-video/src/templates/generated/GeneratedVideo.tsx` — Now imports from `contentRegistry.ts` instead of single ContentScene. `getSceneComponent()` picks variant by `contentVariantIndex` from data.json.
- `remotion-video/src/templates/generated/types.ts` — Added `contentVariantIndex` and `contentVariantCount` fields
- `frontend/src/components/TemplateCodeEditor.tsx` — Dynamic variant tabs built from `template.content_codes`. Shows "Intro | Scene 1 | Scene 2 | ... | Scene N | Outro" tabs. Each tab can be independently edited via chat.
- `frontend/src/api/client.ts` — `CodeVariant` type changed from `"component" | "intro" | "outro"` to `"intro" | "outro" | \`content_${number}\``. `CustomTemplateItem` includes `content_codes: string[] | null`.
### Architecture Decisions
- **Claude decides count** — The prompt asks for "at least 5 variants" but lets Claude decide the right number (5-8) based on the brand personality. Complex brands might get more variety.
- **Cyclic assignment** — If there are 5 content variants and 8 content scenes, scenes cycle: 0,1,2,3,4,0,1,2. Simple, predictable, no AI needed at render time.
- **Backward compatibility** — `component_code` is always set to the first content variant. Old code that reads `component_code` still works (preview, gallery, etc).
- **Per-variant editing** — User selects a specific scene tab (e.g., "Scene 3") and edits just that variant. Edits are scoped and precise.
- **contentRegistry.ts pattern** — Instead of dynamic imports (which Vite can't bundle), we generate a registry file at render time that statically imports all content components. Vite bundles them normally.
- **max_tokens increase** — 8000 → 16000 to accommodate 5-8 full React components in one API call.
### Verification Steps
1. Start backend: `cd backend && uvicorn app.main:app --reload --port 8000` — no errors, content_codes column auto-created
2. Create a template and generate code — should see "Generated N scene variants (1 intro + X content + 1 outro)" in backend logs
3. On Custom Templates page, click "Customize" — editor should show tabs: "Intro | Scene 1 | Scene 2 | ... | Outro"
4. Click each tab — different preview for each scene variant
5. Edit a specific variant via chat — only that variant changes
6. Trigger a video render — check workspace: should have SceneContent0.tsx through SceneContentN.tsx
7. Rendered video should have visually different layouts for each content scene
8. TypeScript check passes: `cd remotion-video && npx tsc --noEmit` ✓
---
## Phase OPT: Custom Template Performance & Quality Optimization — DONE
### Overview
5 fixes targeting custom template speed (246s → ~140s), animation quality, and UX. Previously code generation was blocking, scenes were 500-800 lines with repetitive animations, and CSS/logo scraping was broken.
### Fix 1: CSS & Logo Scraping Improvements — DONE
**Files modified:**
- `backend/app/services/theme_scraper.py`
- `_extract_css_content()` now fetches external `<link rel="stylesheet">` CSS files (first 2-3, up to `_MAX_CSS_CHARS`) in addition to inline `<style>` blocks
- Logo fallbacks improved: og:image added to logos list, SVG logo detection via logo-like class names, favicon.ico fallback
- Note: SPA sites (e.g. noon.com) use CSS-in-JS, so CSS=0 is expected for them — the AI reads inline styles from the rendered HTML instead
### Fix 2: Theme Extraction Speed — DONE
**Files modified:**
- `backend/app/services/theme_scraper.py`
- `_MAX_HTML_CHARS` reduced: 40,000 → 15,000 (AI only needs enough HTML for colors/fonts/layout patterns)
- `_MAX_MARKDOWN_CHARS` reduced: 5,000 → 2,000
- `backend/app/dspy_modules/__init__.py`
- `get_theme_lm()` model changed: `anthropic/claude-sonnet-4-6` → `anthropic/claude-haiku-4-5-20251001` (structured JSON extraction doesn't need Sonnet)
**Impact:** Theme extraction ~25s → ~13.8s (tested on noon.com)
### Fix 3: Code Generation Speed — DONE
**Files modified:**
- `backend/app/services/code_generator.py`
- Line target in prompt: "There is NO line limit" → "Target 200-350 lines. Spend tokens on animation logic, not verbose inline styles."
- Brevity reward in `_scene_reward()`: +0.05 for 200-350 lines, -0.10 for >500 lines
- `backend/app/dspy_modules/__init__.py`
- `get_custom_lm()` max_tokens: 8192 → 5120 (~300-400 lines of JSX, matching built-in template complexity)
- All 5 content variants kept (NUM_CONTENT_VARIANTS=5)
**Impact:** Scene sizes reduced from 500-800 lines → 242-354 lines. Fewer truncation retries.
### Fix 4: Background Code Generation — DONE
**Files modified:**
- `backend/app/routers/custom_templates.py`
- `POST /{template_id}/generate-code` now returns 202 immediately, runs generation in background thread via `asyncio.get_event_loop().run_in_executor()`
- In-memory progress dict: `_codegen_progress[template_id]` with status/step/running/error (same pattern as `_pipeline_progress` in pipeline.py)
- New `GET /{template_id}/generation-status` endpoint for polling
- `frontend/src/components/CustomTemplateCreator.tsx`
- After 202 response, polls `generation-status` every 2s
- Shows progress steps, allows closing modal during generation
- `frontend/src/api/client.ts`
- Added `getCodeGenerationStatus(templateId)` API function
**Impact:** Frontend no longer blocks during code generation. User can navigate away.
### Fix 5: Animation Quality & Creative Freedom — DONE
**Files modified:**
- `backend/app/services/code_generator.py`
- **Deleted** `_VARIANT_SPECIALIZATIONS` dict (~150 lines of hardcoded JSX recipes that killed creativity)
- **Added** `_CONTENT_VARIANT_ROLES` — lightweight content-type assignments (name, content_types, role description) giving Claude full creative freedom
- **Rewrote** `_build_creative_direction()` — assigns content specialization but lets Claude design layout/animation freely
- **Enhanced** `GenerateSceneCode` signature:
- Animation technique reference section (word-by-word spring, image glow entrance, decorative orb) — examples not recipes
- Quality rules: ≥3 different spring configs, staggered text reveal, decorative background element, proper Img handling
- Forbidden: defaulting to simple opacity+translateY for everything
- **Strengthened** `_scene_reward()`:
- +0.10 for diverse springs (≥3 unique damping/stiffness configs)
- +0.10 for staggered text animation (.map + spring + stagger timing)
- +0.10 for Img + animation (spring/interpolate near Img usage)
- +0.05 for glow/blur effects
- -0.10 for opacity+translateY only (no other animation types)
- -0.15 for scenes that completely ignore `imageUrl` (images won't render)
- **Quality debug logging**: each scene logs techniques used (staggered-text, blur, radial-glow, scale, rotate, translateX, clip-path, easing) and spring configs
**Impact:** Generated scenes now have diverse animations (staggered-text, blur, radial-glow, scale, rotate, easing), 4 unique spring configs per scene, and scores of 1.00-1.30.
### Design System Optimization — DONE
**Problem:** Design system generation took 43.6s (Sonnet) producing 6323 chars including spring configs. All 7 scenes copied the same springs, producing identical motion feel.
**Files modified:**
- `backend/app/services/code_generator.py`
- `GenerateDesignSystem` signature slimmed: 7 categories → 3 (background treatment, card/container CSS, text treatment)
- Explicitly excludes: "Do NOT include spring configs, animation physics, decorative elements, or entrance patterns. Those are creative choices each scene makes independently."
- Output constrained to ~1500 chars
- Uses `get_theme_lm()` (Haiku) instead of `get_custom_lm()` (Sonnet) — structured CSS extraction doesn't need creative reasoning
- `GenerateSceneCode` prompt updated: "Follow the design_system for visual styling (colors, cards, backgrounds, text treatment). Invent your OWN unique spring physics and animation timing — each scene should have distinctly different motion feel."
**Impact:** Design system ~43.6s → ~8-12s (Haiku + shorter output). Springs now vary per scene instead of being identical across all 7 scenes.
### Content Variant Diversity — DONE
**Problem:** When all blog content is the same type (e.g. noon.com: all bullets), every scene got routed to the same variant (variant 0, lists specialist). All scenes looked identical.
**Files modified:**
- `backend/app/services/remotion.py`
- Variant routing updated: specialist variant used for the **first** occurrence of each content type only
- Subsequent scenes with the same content type cycle through other variants for visual diversity
- Example: 5 bullet scenes → variant 0 (specialist), 1 (cycling), 2, 3, 4 — all different layouts
**Impact:** Even when all content is the same type, every scene gets a different visual treatment.
### Test Results (noon.com)
| Metric | Before | After | Target |
|--------|--------|-------|--------|
| CSS chars | 0 | 0 (expected for SPA) | > 0 for traditional sites |
| Logos | 0 | 1 | > 0 |
| Theme AI time | 25s | 13.8s | < 12s |
| Codegen total | 246s | 177.7s (pending design system fix) | < 140s |
| Scene lines | 500-800 | 242-354 | 200-350 |
| Spring configs | 1 identical | 4 unique per scene | ≥ 3 |
| Animation techniques | opacity+translateY | staggered-text, blur, radial-glow, scale, rotate, translateX, easing | diverse |
| Frontend blocking | yes (60-250s) | 202 + polling | non-blocking |
---
## Phase V2: Content-Aware Architecture (feat/customv2) — DONE
### Problem
Custom templates produced repetitive, content-blind videos. Built-in templates look great because each scene gets a layout matched to its content. Custom templates were generic because:
- 5 hardcoded layout families — every brand got the same structural layouts
- Blind cycling — scenes rotated through variants regardless of content (metrics scene might get kinetic_typography)
- 16 wasted DSPy calls per video generating layoutConfig that the renderer ignored
- Images broken — only one layout family rendered `<Img>`, others silently ignored images
- Preview broken — `contentVariantIndex` written to data.json but not DB
### What Was Done
#### 1. AI-Decided Brand Scene Types (DecideBrandSceneTypes DSPy Signature)
**Removed:** `CONTENT_ARCHETYPES` (10 hardcoded archetypes), `_LAYOUT_FAMILIES`, `NUM_CONTENT_VARIANTS=5`, `_build_creative_direction()`, `_VARIANT_SPECIALIZATIONS`, `_CONTENT_VARIANT_ROLES`
**Added:** `DecideBrandSceneTypes` DSPy signature — one call per brand that outputs 6-8 scene types tailored to the brand.
- Example: Metal (fundraising SaaS) → `brand_hook_intro, feature_spotlight, how_it_works, metrics_proof, vs_traditional, fundraising_timeline, founder_voice, cta_outro`
- Example: Al Jazeera (news) → completely different set of scene types
- Each scene type has `best_for` tags mapping to content types (bullets, metrics, quote, etc.)
**Files:** `backend/app/services/code_generator.py` — complete rewrite of generation flow
#### 2. Content Extraction & Archetype Matching
**New file:** `backend/app/services/content_classifier.py`
- `extract_structured_content_batch()` — ONE cheap Haiku call classifies ALL scene narrations into contentType (metrics/bullets/quote/comparison/timeline/steps/code/plain) and extracts structured data
- `match_scenes_to_archetypes()` — deterministic matching: contentType → best archetype based on `best_for` tags, with anti-repeat logic so consecutive scenes never get the same archetype
**Files modified:**
- `backend/app/routers/pipeline.py` — custom template branch now uses batch extraction instead of 16 per-scene DSPy calls
- `backend/app/services/remotion.py` — content-aware matching replaces blind cycling, persists `contentVariantIndex` to DB
- `frontend/src/components/VideoPreview.tsx` — reads `contentVariantIndex` from DB instead of `CONTENT_TYPE_TO_VARIANT`
#### 3. Alembic Migration for content_archetype_ids
**New file:** `backend/alembic/versions/phase8_content_archetype_ids.py`
- Adds `content_archetype_ids TEXT` column to `custom_templates` table
- Merges 3 divergent alembic heads into one
#### 4. Preview System Enrichment
**File:** `frontend/src/components/templatePreviews/CustomPreview.tsx` — complete rewrite
- Replaced simple `buildSampleData()` with archetype-aware `buildArchetypeSampleData()` returning rich structured data per content type (metrics with values, bullets, quotes, comparison, timeline, steps, code)
- Scene crossfade transitions with dot navigation
- Pre-compiles ALL scene codes on mount (eliminates per-scene "Compiling preview..." flash)
**File:** `frontend/src/components/templatePreviews/CustomPreviewLandscape.tsx`
- Delegates to `CustomPreview` when generated scene code exists
- Falls back to `FallbackSlides` component when no code
**File:** `frontend/src/components/BlogUrlForm.tsx`
- Template picker passes full template data (introCode, outroCode, contentCodes, contentArchetypeIds, logoUrls, ogImage)
- Selected template preview shows real generated scenes instead of hardcoded slides
#### 5. Image & Logo Rendering (MANDATORY in AI-generated scenes)
**Problem:** AI-generated components ignored `props.imageUrl` and `props.logoUrl` — pipeline delivered images correctly but components didn't render them.
**Root cause:** Not the pipeline — `GeneratedVideo.tsx` correctly maps `scene.images[0]` → `imageUrl`, and `VideoPreview.tsx` correctly passes `imageUrl` from `sceneImageMap`. The issue was that AI-generated React code never referenced these props.
**Fix — Prompt (code_generator.py `GenerateSceneCode`):**
- Images & Logo section marked MANDATORY with explicit JSX patterns
- Image techniques from built-in templates: Ken Burns zoom, radial vignette reveal, clipPath slit reveal, gradient overlays (3-layer: vignette + bottom gradient + accent wash)
- Adaptive layout: `const hasImage = !!props.imageUrl` — with image: split layout or full-bleed overlay; without image: text expands, larger fonts, particles/gradient as visual interest
- Both modes must look intentionally designed
**Fix — Text animations:**
- Word-by-word / line-by-line reveals with staggered springs
- Typewriter effect with blinking cursor
- Scale-punch for key words (bouncy overshoot spring)
- Bullet stagger (slide from right, delay=20+i*10)
- Exit animations 20-30 frames before durationInFrames
**Fix — Scene motion:**
- Multiple spring configs with different damping/stiffness/mass
- Metric count-ups, card fly-ins, decorative corner shapes, accent line grows
- Parallax depth, ambient gradient shifts, pulsing accent glows
- Reference configs: fast={damping:22,stiffness:140,mass:1.2}, bouncy={damping:14,stiffness:220,mass:1.1}, smooth={damping:20,stiffness:70}
**Fix — Reward function penalties:**
- `-0.2` if code doesn't reference `logoUrl`
- `-0.2` if code doesn't reference `imageUrl`
- `-0.3` for non-monotonic `interpolate()` inputRange (catches runtime crashes like `[0,60,120,90]`)
- Combined with existing checks (overflow:hidden, hardcoded data, visible contentType/sceneIndex)
#### 6. Code Generator Architecture (current state)
```
generate_component_code(template)
│
├─ _build_brand_context() — raw data only (colors, fonts, patterns, brand kit)
│
├─ _decide_brand_scene_types() — ONE DSPy call → 6-8 brand-specific scene types
│
├─ _generate_design_system() — ONE Haiku call → concise CSS design system (<2000 chars)
│
└─ asyncio.gather(*tasks) — ALL scenes in PARALLEL via ThreadPoolExecutor(max_workers=8)
└─ _generate_single_scene() → dspy.Refine(ChainOfThought(GenerateSceneCode), reward_fn=_scene_reward)
└─ Up to 3 attempts per scene (1 initial + 2 retries if score < 0.75)
```
**Output:** `{ intro_code, outro_code, content_codes: list[str], archetype_ids: list[dict] }`
### Files Changed Summary
| File | Change |
|------|--------|
| `backend/app/services/code_generator.py` | Complete rewrite: DecideBrandSceneTypes, stripped prescriptive code, enriched animation/image prompt, reward penalties for missing image/logo |
| `backend/app/services/content_classifier.py` | NEW: batch content extraction (Haiku) + deterministic archetype matching |
| `backend/app/routers/pipeline.py` | Custom template branch uses batch extraction |
| `backend/app/services/remotion.py` | Content-aware matching, passes structuredContent to scenes |
| `backend/app/routers/custom_templates.py` | content_archetype_ids serialization, regeneration endpoint |
| `backend/app/routers/projects.py` | Scene regeneration for custom templates |
| `backend/app/models/custom_template.py` | content_archetype_ids column |
| `backend/app/database.py` | SQLite migration for content_archetype_ids |
| `backend/app/services/template_service.py` | Loads and caches archetype metadata |
| `backend/alembic/versions/phase8_content_archetype_ids.py` | PostgreSQL migration |
| `frontend/src/components/templatePreviews/CustomPreview.tsx` | Archetype-aware previews with crossfade |
| `frontend/src/components/templatePreviews/CustomPreviewLandscape.tsx` | Delegates to CustomPreview for generated code |
| `frontend/src/components/BlogUrlForm.tsx` | Template picker passes full template data |
| `frontend/src/components/VideoPreview.tsx` | Reads contentVariantIndex from DB |
| `frontend/src/components/CustomTemplateCreator.tsx` | Centered step indicator |
| `frontend/src/pages/CustomTemplates.tsx` | Passes archetype data to previews |
| `frontend/src/pages/ProjectView.tsx` | Layout label from archetype name |
| `frontend/src/api/client.ts` | content_archetype_ids types |
### Test Results
Three brands tested end-to-end (template creation → scene generation → video creation):
| Brand | Scene Types Decided | Content Extraction | Archetype Matching | Generation Time |
|-------|--------------------|--------------------|-------------------|-----------------|
| Metal (fundraising SaaS) | 8 types: brand_hook_intro, feature_spotlight, how_it_works, metrics_proof, vs_traditional, fundraising_timeline, founder_voice, cta_outro | All scenes classified correctly | Anti-repeat working | ~137s |
| Al Jazeera (news) | Unique news-focused types | Classified correctly | Working | ~130s |
| Nestlé Pakistan (FMCG) | Unique consumer brand types | Classified correctly | Working | ~135s |
All regenerated scenes score 1.00 (image/logo handling verified by reward function).
---
## Next Improvements (TODO)
| # | Improvement | Priority | Description |
|---|------------|----------|-------------|
| N2 | brandImages Pass-through in frontend preview | Low | VideoPreview.tsx doesn't pass `brandImages` to scene props (Remotion pipeline does). Add it for preview parity. |
| N7 | Per-Plan AI Limits | Low | AI_DAILY_LIMIT per-plan: Free: 5, Standard: 20, Pro: 50. |
# AI Custom Templates — Progress Tracker
## Master Plan
Replace the fixed `UniversalScene.tsx` renderer (9 layouts, 10 element types) with AI-generated unique React components per brand. Each template becomes a code string stored in DB, compiled at runtime for preview, written to temp files for rendering.
**No fallback to old system.** `UniversalScene.tsx`, `styleEngine.ts`, and the old custom template types will be **deleted** as a final cleanup step after all phases are implemented, tested, and stable. No dual-path code long-term — the new system must stand on its own so bugs are visible, not hidden behind a fallback.
**Full plan:** `/Users/faisalnazir/Desktop/Blog2Video - AI Generated Custom Templates Plan.md`
## Phases Overview
| Phase | Description | Status |
|-------|-------------|--------|
| 1 | DB Models + Enhanced Scraping | DONE |
| 2 | AI Code Generation + Preview | DONE |
| 3 | Chat-Based Editing | DEFERRED — removed from codebase, will revisit in a future phase |
| 4 | Render Pipeline Update | DONE |
| 5 | Polish & Preview System | DONE |
| 6 | Cleanup — Remove old UniversalScene/styleEngine/fallback code | DONE |
| H1 | Rate Limiting on Claude API Endpoints | DONE |
| H2 | Code Validator Hardening | DONE |
| H3 | Template Cache TTL | DONE |
| H4 | Render Workspace Locking | DONE |
| H5 | Frontend Code Sandbox (CSP + Error Boundary) | DONE |
| H6 | Self-healing current_version_id | DONE |
| F1 | BrandKit Population During Theme Extraction | DONE |
| F2 | Brand Asset Uploads (Logo) | DONE |
| F5 | Version Snapshots on Chat Edits | DEFERRED — depends on Phase 3 |
| F6 | custom_prompt_builder.py Cleanup | DONE |
| F3 | Enhanced Scraping with Firecrawl Branding Format | DONE |
| F4 | Preview Thumbnails for Template Gallery | DONE |
| F7 | Multiple Unique Content Scene Variants | DONE |
| OPT | Performance & Quality Optimization (5 fixes) | DONE |
| V2 | Content-Aware Architecture (feat/customv2) | DONE |
---
## Phase 1: DB Models + Enhanced Scraping — DONE
### What Was Done
**New file created:**
- `backend/app/models/brand_kit.py` — BrandKit model (brand_kits table)
- Fields: id, user_id, source_url, brand_name, colors (JSON), fonts (JSON), design_language (JSON), logos (JSON), images (JSON), timestamps
- Relationships: user, custom_templates
**Files modified:**
- `backend/app/models/custom_template.py` — Added:
- `component_code` (Text, nullable) — will hold AI-generated React code
- `intro_code` (Text, nullable) — intro scene variant
- `outro_code` (Text, nullable) — outro scene variant
- `brand_kit_id` (Integer FK → brand_kits.id, nullable, indexed)
- `brand_kit` relationship
- `backend/app/models/scene.py` — Added:
- `scene_type` (String(20), nullable) — "intro"/"content"/"outro"; NULL = "content"
- `backend/app/models/user.py` — Added:
- `brand_kits` relationship (cascade delete)
- `backend/app/models/__init__.py` — Added BrandKit import and export
- `backend/app/database.py` — Updated:
- `_migrate_sqlite()`: added component_code, intro_code, outro_code, brand_kit_id to ct_migrations; added scene_type to scene_migrations
- `init_db()`: added BrandKit to model imports
- `backend/alembic/env.py` — Added BrandKit import
- `backend/alembic/versions/phase1_brand_kit_and_schema_fields.py` — New Alembic migration:
- Creates brand_kits table
- Adds 4 columns to custom_templates
- Adds scene_type to scenes
- Down revision: 7ed594338d45
- `backend/app/services/theme_scraper.py` — Enhanced:
- `ScrapedThemeData` now has `logo_urls`, `og_image`, `screenshot_url` fields
- New `_extract_logo_urls()` helper extracts logos from HTML (favicon links + img tags with "logo" in attributes)
- Tries Firecrawl screenshot format with graceful fallback
- Extracts OG image from metadata
### Backward Compatibility
- All new columns nullable — existing data unaffected
- No API endpoint changes — responses identical
- No frontend changes
- ScrapedThemeData new fields are additive — ThemeExtractor ignores them
### Verification Steps
1. Start backend: `cd backend && uvicorn app.main:app --reload --port 8000`
2. Confirm no startup errors
3. Check brand_kits table exists in SQLite
4. Check new columns on custom_templates and scenes
5. Test existing custom template flow (paste URL → extract → save)
---
## Phase 2: AI Code Generation + Preview — DONE
### What Was Done