| # Ai&R System v1.1 |
|
|
| ## Overview |
|
|
| This is an AI-powered music A&R (Artists & Repertoire) analysis tool called "The Ai&R System." It uses a Dual-Model Orchestration pipeline with a Two-Stage Workflow: |
|
|
| **Stage 1 ("The Breach")**: Gemini Flash extracts Sonic DNA (quantitative audio features, X-Factor keywords, primary reference artist, technical profiles) → Groq generates Suno v4 style tags and 3 suggested song titles. No lyrics generated at this stage. |
|
|
| **Stage 2 ("The Synthesis")**: User selects a title → Groq generates full anti-cringe lyrics (24+ lines) with reference artist vocal branding and X-Factor metatag injection. If Groq fails, Gemini handles everything as a fallback. |
|
|
| The app is a full-stack TypeScript monorepo with a React frontend and Express backend, backed by PostgreSQL for storing analysis history. Apple-style minimalist black UI with action-first layout (Copy Suno Prompt, Copy Lyrics, Go to Suno buttons at top). |
|
|
| ## User Preferences |
|
|
| Preferred communication style: Simple, everyday language. |
|
|
| ## System Architecture |
|
|
| ### Monorepo Structure |
| - `client/` — React SPA (Vite + TypeScript) |
| - `server/` — Express API server (TypeScript, runs via tsx) |
| - `shared/` — Shared types, schemas, and route definitions used by both client and server |
| - `migrations/` — Drizzle-generated database migrations |
| - `attached_assets/` — Reference documents (prompt engineering specs, UI mockups) |
|
|
| ### Frontend Architecture |
| - **Framework**: React 18 with TypeScript, bundled by Vite |
| - **Routing**: Wouter (lightweight client-side router) |
| - **State/Data Fetching**: TanStack React Query for server state management |
| - **UI Components**: shadcn/ui (new-york style) built on Radix UI primitives with Tailwind CSS |
| - **Animations**: Framer Motion for reveal effects and log animations |
| - **Icons**: Lucide React |
| - **Styling**: Tailwind CSS with CSS variables for theming. Dark-only monochromatic theme (pure black background, white text, green accents). Uses three font families: Inter (sans), Newsreader (serif), Geist Mono (monospace). |
| - **Key Pages**: Home page (`/`) is the main analysis interface; Pricing page (`/pricing`); Help page (`/help`); 404 page for unknown routes |
| - **Path Aliases**: `@/` maps to `client/src/`, `@shared/` maps to `shared/` |
|
|
| ### Backend Architecture |
| - **Framework**: Express.js on Node.js with TypeScript |
| - **API Pattern**: Routes defined in `shared/routes.ts` as a typed contract (method, path, input/output schemas via Zod). Both client and server reference the same route definitions. |
| - **AI Integration**: Dual-Model Orchestration with efficient fallback |
| - **Gemini Flash** (Analyst): Extracts raw Music Observation via `@google/genai`, accessed through Replit AI Integrations (`AI_INTEGRATIONS_GEMINI_API_KEY` and `AI_INTEGRATIONS_GEMINI_BASE_URL`) |
| - **Groq Llama-3.3-70b** (Architect & Poet): Stage 1 generates Suno tags and 3 suggested titles. Stage 2 generates full lyrics for a selected title. Uses `groq-sdk` (`GROQ_API_KEY` env var) |
| - **Fallback**: If Groq fails, Gemini Architect Fallback reuses existing Gemini observation data (no re-call). If no Groq API key, Gemini Unified mode handles everything. |
| - **Spotify Integration** (`server/spotify.ts`): Searches Spotify for artist/track, retrieves audio features (if API allows), artist genres, popularity, album art. Real audio features override Gemini estimates for Deep Scan. Connected via Replit Connector. |
| - **Helpers**: `safeParseJson()` centralized JSON parsing, `generateWithGroq()`/`generateWithGemini()` shared AI wrappers, `validateArchitectOutput()` post-parse validation, `trimForResponse()` strips non-UI fields from API response (full data stored in DB), `enrichWithSpotify()` merges Spotify data into sonicDna |
| - **Dev Server**: Vite dev server is used as middleware in development (HMR via `server/vite.ts`); static files served in production from `dist/public` |
| - **Build**: Custom build script (`script/build.ts`) — Vite builds the client, esbuild bundles the server into `dist/index.cjs` |
|
|
| ### API Endpoints |
| - `POST /api/analyze` — Stage 1 endpoint. Accepts `targetArtist` (required). Runs dual-model pipeline: Gemini extracts Sonic DNA → Groq generates Suno tags and 3 suggested titles. Returns structured JSON with sonicDna (tags, suggestedTitles, audioFeatures, highImpactKeywords, primaryReferenceArtist) and pipeline indicator. Response includes `pipeline` field: "dual", "gemini-fallback", or "gemini-only". Note: Suno tags do NOT include artist names. |
| - `POST /api/generate-lyrics` — Stage 2 endpoint. Accepts `selectedTitle` (string) and `sonicDna` (full sonic DNA from Stage 1). Generates full anti-cringe lyrics (24+ lines) with reference artist vocal branding. Returns `{ sunoScript: string }`. |
| - `GET /api/history` — Returns all past analysis results ordered by creation date (newest first) |
|
|
| ### Replit AI Integrations (Pre-built Modules) |
| Located in `server/replit_integrations/`: |
| - **chat/** — Generic chat conversation routes with Gemini (conversations + messages CRUD) |
| - **image/** — Image generation endpoint using `gemini-2.5-flash-image` |
| - **batch/** — Batch processing utilities with rate limiting, retries, and concurrency control |
|
|
| These are supplementary modules. The main analysis feature uses Gemini directly in `server/routes.ts`. |
|
|
| ### Database |
| - **Database**: PostgreSQL (required, via `DATABASE_URL` environment variable) |
| - **ORM**: Drizzle ORM with `drizzle-zod` for schema-to-validation integration |
| - **Schema** (`shared/schema.ts`): |
| - `analysis_results` table: `id` (serial PK), `target_artist` (text), `reference_artist` (text), `sonic_dna` (jsonb — stores tags, BPM, key, instruments, etc.), `narrative` (text), `created_at` (timestamp) |
| - **Additional schemas** (`shared/models/chat.ts`): |
| - `conversations` table: `id`, `title`, `created_at` |
| - `messages` table: `id`, `conversation_id` (FK to conversations), `role`, `content`, `created_at` |
| - **Migrations**: Use `npm run db:push` (drizzle-kit push) to sync schema to database |
| - **Storage Layer**: `server/storage.ts` — `DatabaseStorage` class implements `IStorage` interface with `createAnalysis()` and `getHistory()` methods |
|
|
| ### Key Design Decisions |
| 1. **Shared route contracts** — Route definitions with Zod schemas in `shared/routes.ts` ensure type safety across client-server boundary |
| 2. **JSONB for Sonic DNA** — The analysis output from Gemini is complex nested JSON; storing as JSONB allows flexible querying without rigid column structure |
| 3. **Gemini JSON cleaning** — The backend strips markdown code blocks (` ```json `) from both Gemini and Groq responses before parsing |
| 4. **Dual-Model Orchestration** — Gemini as Analyst (data extraction) + Groq as Architect (creative generation) with automatic fallback |
| 5. **Interscope Deep Scan** — 5 circular audio feature gauges (Energy, Valence, Acousticness, Danceability, Instrumentalness) scored 0.0-1.0. Uses Spotify real data when available (with green "Live" badge), falls back to Gemini estimation |
| 6. **Replit Auth** — Google login via Replit OpenID Connect (passport-based). Users table has credits, freeUsesRemaining, totalAnalyses columns. Freemium model: 3 free analyses, then 10 credits per analysis. Auth middleware on `/api/analyze`. Credits endpoint at `GET /api/credits`. |
|
|
| ## External Dependencies |
|
|
| ### Required Services |
| - **PostgreSQL Database** — Connection string via `DATABASE_URL` environment variable. Required for all data persistence. |
| - **Google Gemini AI** (via Replit AI Integrations) — Two environment variables required: |
| - `AI_INTEGRATIONS_GEMINI_API_KEY` — API key for Gemini |
| - `AI_INTEGRATIONS_GEMINI_BASE_URL` — Base URL for the Gemini-compatible endpoint |
| - Model used: `gemini-2.5-flash` (Analyst role) |
| - **Groq AI** — Environment variable required: |
| - `GROQ_API_KEY` — API key for Groq |
| - Model used: `llama-3.3-70b-versatile` (Architect & Poet role) |
|
|
| ### Key NPM Dependencies |
| - `@google/genai` — Google Gemini AI client |
| - `groq-sdk` — Groq AI client (Llama 3.3 70B) |
| - `drizzle-orm` + `drizzle-kit` + `drizzle-zod` — Database ORM and migration tooling |
| - `express` + `connect-pg-simple` — HTTP server and session store |
| - `@tanstack/react-query` — Client-side data fetching |
| - `framer-motion` — UI animations |
| - `wouter` — Client-side routing |
| - `zod` — Schema validation (shared between client and server) |
| - `shadcn/ui` ecosystem — Radix UI primitives, Tailwind CSS, class-variance-authority, clsx, tailwind-merge |
