Buckets:
| # System Design & AI/ML System Design — Complete Roadmap | |
| > [!info] Target audience: 3+ years of ML/AI engineering experience, weak on production design & system design interviews. | |
| > Goal: Become production-grade engineer + nail system design rounds (incl. ML, LLM, RAG, Agentic, VLM). | |
| --- | |
| ## How to Use This Roadmap | |
| - **Phase 0–1**: Build the *vocabulary* of distributed systems. Without this, ML system design discussions are shallow. | |
| - **Phase 2–3**: Learn classic system design and practice 10–15 problems. | |
| - **Phase 4–5**: ML system design — frameworks, pipelines, serving, MLOps. | |
| - **Phase 6–8**: Modern AI — LLM serving, RAG, Agentic, multimodal, evals, guardrails. | |
| - **Phase 9**: Interview drilling. | |
| **Estimated calendar time**: 10–14 weeks part-time (1.5–2 hrs/day), 5–6 weeks full-time. | |
| **Method per topic**: (1) Read the *concept*, (2) Read a *real-world implementation* (engineering blog), (3) *Draw* the architecture from memory, (4) *Explain it aloud* in <5 min, (5) Identify *trade-offs and failure modes*. | |
| --- | |
| # PART 1 — CLASSIC SYSTEM DESIGN (A–Z) | |
| ## Phase 0 — Foundations of Computing & Networks | |
| ### 0.1 Computer architecture refresh | |
| - CPU vs GPU vs TPU; memory hierarchy (L1/L2/L3 cache, RAM, disk, network). | |
| - Latency numbers every engineer must memorize (Jeff Dean's table): | |
| - L1 cache: ~0.5 ns; L2: ~7 ns; main memory: ~100 ns | |
| - SSD random read: ~150 µs; disk seek: ~10 ms | |
| - Round trip same datacenter: ~500 µs; cross-continent: ~150 ms | |
| - Throughput vs latency vs concurrency. | |
| - Big-O of common operations; back-of-envelope estimation. | |
| ### 0.2 Networking essentials | |
| - OSI model and TCP/IP stack (Layers 3, 4, 7 are interview-relevant). | |
| - TCP vs UDP, three-way handshake, congestion control (TCP slow start), head-of-line blocking. | |
| - HTTP/1.1, HTTP/2 (multiplexing), HTTP/3 (QUIC over UDP), keep-alive. | |
| - DNS: recursive vs iterative resolution, TTL, A/AAAA/CNAME/MX records, GeoDNS. | |
| - TLS handshake, mTLS, certificate chains. | |
| - CDNs: edge caching, push vs pull, origin shielding, Anycast, BGP routing. | |
| - WebSockets, Server-Sent Events (SSE), long polling vs short polling. | |
| - gRPC (HTTP/2 + protobuf, streaming modes: unary, server, client, bidirectional). | |
| - REST vs gRPC vs GraphQL vs WebSocket — when to use each. | |
| ### 0.3 Operating systems & concurrency | |
| - Processes vs threads vs coroutines. | |
| - Context switching cost. | |
| - Blocking vs non-blocking I/O, sync vs async, epoll/kqueue. | |
| - Memory mapping, page cache. | |
| - Locking: mutex, semaphore, read-write lock, optimistic locking, deadlock. | |
| - Concurrency models: thread-per-request, event loop (Node, Nginx), actor (Erlang/Akka), CSP (Go). | |
| ## Phase 1 — Building Blocks of Distributed Systems | |
| ### 1.1 Theoretical foundations | |
| - **CAP theorem** — partition-tolerance is non-negotiable, so it's really CP vs AP. | |
| - **PACELC** — extension to CAP that adds latency-vs-consistency trade-off. | |
| - **ACID** vs **BASE**. | |
| - Consistency models (strongest → weakest): | |
| - Linearizable | |
| - Sequential | |
| - Causal | |
| - Read-your-writes / monotonic reads / monotonic writes / writes-follow-reads | |
| - Eventual | |
| - Isolation levels: read uncommitted, read committed, repeatable read, serializable, snapshot isolation; phenomena: dirty read, non-repeatable read, phantom read, write skew. | |
| ### 1.2 Distributed primitives | |
| - Consensus: Paxos, Raft, ZAB; leader election; quorum (R+W>N). | |
| - Logical clocks: Lamport timestamps, vector clocks; hybrid logical clocks (HLC). | |
| - Two-phase commit (2PC), three-phase commit (3PC); their failure modes. | |
| - Distributed transactions, Saga pattern (orchestration vs choreography). | |
| - Idempotency, exactly-once vs at-least-once vs at-most-once delivery. | |
| - Failure modes: crash, omission, Byzantine; gray failures, partial failures. | |
| - Time: NTP, clock skew, TrueTime (Spanner). | |
| ### 1.3 Hashing & partitioning | |
| - Consistent hashing (with virtual nodes); jump hash; rendezvous hashing. | |
| - Range-based vs hash-based partitioning vs directory-based. | |
| - Hotspot mitigation: salting, key randomization. | |
| - Bloom filters, Count-min sketch, HyperLogLog (when interviewers ask "how to check membership/count of billions cheaply"). | |
| ## Phase 2 — Core Components in Depth | |
| ### 2.1 Databases (SQL) | |
| - Storage engines: B-tree vs LSM-tree (RocksDB). | |
| - Indexing: clustered vs secondary; covering index; multi-column index; partial index; index selectivity. | |
| - Query planning, EXPLAIN, statistics. | |
| - MVCC, undo/redo logs, WAL. | |
| - Read replicas, replication lag, synchronous vs asynchronous replication. | |
| - Failover, split-brain, fencing tokens. | |
| - Connection pooling (PgBouncer); prepared statements. | |
| - Sharding: vertical (by table), horizontal (by row), functional. Shard key selection. | |
| - Postgres-specific tips: TOAST, vacuum, table partitioning, logical replication. | |
| ### 2.2 NoSQL | |
| - Wide-column (Cassandra, ScyllaDB, HBase, BigTable) — data modeling around query patterns. | |
| - Document (MongoDB, DynamoDB) — denormalization, single-table design. | |
| - Key-value (Redis, DynamoDB, Riak). | |
| - Graph (Neo4j, Neptune, JanusGraph) — when to use. | |
| - Search engines (Elasticsearch, OpenSearch, Solr) — inverted index, BM25 scoring, analyzers/tokenizers, replicas/shards. | |
| - Time-series (InfluxDB, TimescaleDB, Prometheus TSDB) — downsampling, retention. | |
| - NewSQL (Spanner, CockroachDB, TiDB, YugabyteDB). | |
| - Vector DBs (covered in Part 3). | |
| ### 2.3 Caching | |
| - Levels: browser, CDN, reverse proxy (Nginx, Varnish), application, database, OS page cache. | |
| - Strategies: | |
| - Cache-aside (lazy loading) | |
| - Read-through | |
| - Write-through | |
| - Write-back / write-behind | |
| - Refresh-ahead | |
| - Eviction policies: LRU, LFU, ARC, FIFO, TTL, random. | |
| - Cache problems: stampede / thundering herd (mitigations: request coalescing, jittered TTL, probabilistic early expiration), penetration, avalanche. | |
| - Distributed cache: Redis (cluster, sentinel), Memcached (consistent hashing client-side). | |
| - Cache consistency: invalidation vs TTL; double-write hazards. | |
| ### 2.4 Message queues, streaming, pub/sub | |
| - Queues vs logs (RabbitMQ vs Kafka). | |
| - Kafka deep-dive: topics, partitions, offsets, consumer groups, ISR, leader/follower, log compaction, exactly-once semantics, KRaft (Zookeeper-less). | |
| - Alternatives: Pulsar, RabbitMQ (AMQP), NATS, ActiveMQ, AWS SQS/SNS/Kinesis, GCP Pub/Sub. | |
| - Stream processing: Flink, Spark Streaming, Kafka Streams, Beam — windowing (tumbling, sliding, session), watermarks, exactly-once. | |
| - Backpressure, dead-letter queues, retry with exponential backoff + jitter. | |
| ### 2.5 Load balancing & traffic management | |
| - L4 (TCP, e.g., AWS NLB, HAProxy TCP) vs L7 (HTTP, e.g., ALB, Envoy, Nginx). | |
| - Algorithms: round-robin, weighted RR, least connections, least response time, IP hash, consistent hash, power-of-two-choices. | |
| - Health checks, sticky sessions, draining, blue/green & canary deployments. | |
| - Reverse proxy vs API gateway vs service mesh. | |
| - Rate limiting algorithms: token bucket, leaky bucket, fixed window, sliding window log, sliding window counter. | |
| - Throttling, circuit breakers (Hystrix pattern), bulkheads, retries with backoff + jitter, timeouts. | |
| ### 2.6 Storage | |
| - Block (EBS), file (EFS, NFS), object (S3, GCS) — semantics and use cases. | |
| - S3 internals at high level: consistency model (now strong read-after-write), multipart upload, lifecycle policies, storage classes. | |
| - HDFS, Ceph, GlusterFS. | |
| - Data lakes vs warehouses vs lakehouses (Iceberg, Delta Lake, Hudi). | |
| ### 2.7 Microservices & architecture patterns | |
| - Monolith → modular monolith → microservices — *don't start with microservices*. | |
| - Service discovery (client-side: Eureka; server-side: AWS ELB/Consul). | |
| - API gateway responsibilities: auth, rate limit, routing, aggregation, protocol translation. | |
| - Service mesh (Istio, Linkerd) — sidecar proxies, mTLS, traffic shaping, observability. | |
| - Patterns: BFF (backend-for-frontend), CQRS, Event Sourcing, Saga, Outbox, Strangler Fig, Sidecar, Ambassador, Anti-Corruption Layer. | |
| - Synchronous vs asynchronous communication trade-offs. | |
| ### 2.8 Reliability & operations | |
| - SLI / SLO / SLA / error budgets. | |
| - High availability: redundancy, replication, multi-AZ, multi-region (active-active vs active-passive). | |
| - DR: RTO (recovery time objective), RPO (recovery point objective), backup strategies (full, incremental, differential). | |
| - Chaos engineering (Chaos Monkey, Gremlin). | |
| - Graceful degradation, feature flags (LaunchDarkly, Unleash, OpenFeature). | |
| - Deployment strategies: rolling, blue/green, canary, shadow, A/B. | |
| ### 2.9 Observability | |
| - Three pillars: logs, metrics, traces. | |
| - Metrics: Prometheus (pull-based), Datadog, CloudWatch; PromQL basics; RED (Rate, Errors, Duration) and USE (Utilization, Saturation, Errors) methods. | |
| - Logs: structured logging (JSON), centralized (ELK / Loki / OpenSearch). | |
| - Distributed tracing: OpenTelemetry, Jaeger, Zipkin, Tempo. Trace ID propagation, spans. | |
| - On-call, runbooks, post-mortems (blameless), incident severity levels. | |
| ### 2.10 Security | |
| - AuthN vs AuthZ. | |
| - OAuth2 flows (auth code, PKCE, client credentials), OIDC, SAML, JWT (signing vs encryption, JWS vs JWE), session cookies vs tokens. | |
| - RBAC vs ABAC vs ReBAC; Open Policy Agent (OPA). | |
| - Secrets management (Vault, AWS Secrets Manager, KMS). | |
| - Encryption: at rest (envelope encryption), in transit (TLS), in use (confidential compute, homomorphic — rare). | |
| - OWASP Top 10; SQLi, XSS, CSRF, SSRF, RCE. | |
| - DDoS mitigation: WAF, rate limiting, anti-bot, scrubbing centers. | |
| - Zero-trust architecture, network segmentation, VPCs, security groups. | |
| - Supply-chain security (SBOM, dependency scanning, signed images — Sigstore, Cosign). | |
| ### 2.11 Containers & orchestration | |
| - Docker fundamentals: images, layers, registries, multi-stage builds, BuildKit. | |
| - Kubernetes core: Pods, Deployments, StatefulSets, DaemonSets, Jobs/CronJobs, Services (ClusterIP/NodePort/LoadBalancer), Ingress, HPA/VPA, ConfigMaps, Secrets, PV/PVC, namespaces, RBAC. | |
| - Operators, CRDs; Helm; Kustomize. | |
| - Networking: CNI, kube-proxy, service mesh integration. | |
| - GPU scheduling: device plugins, NVIDIA Operator, KubeRay, MIG partitioning. | |
| ## Phase 3 — Classic System Design Practice | |
| Drill these 15 problems. For each: (a) clarify scope, (b) capacity & QPS estimates, (c) API design, (d) data model, (e) high-level architecture, (f) deep-dive on 2 components, (g) failure handling, (h) scaling next step. | |
| | # | Problem | Key concepts | | |
| |---|---|---| | |
| | 1 | URL shortener (TinyURL/bit.ly) | Hashing, base62, DB choice, cache, analytics pipeline | | |
| | 2 | Pastebin | Storage tier, expiry, CDN | | |
| | 3 | Rate limiter | Token bucket in Redis, distributed counters | | |
| | 4 | Web crawler | BFS/DFS, politeness, dedup with Bloom filter, robots.txt, frontier | | |
| | 5 | Notification system | Fan-out, queues, retries, priority | | |
| | 6 | News feed / Twitter | Fan-out on write vs read, hybrid, timeline cache | | |
| | 7 | Chat (WhatsApp/Slack) | WebSocket, presence, message ordering, E2EE basics | | |
| | 8 | Video streaming (YouTube/Netflix) | CDN, adaptive bitrate, HLS/DASH, encoding pipeline | | |
| | 9 | Ride sharing (Uber) | Geohash/H3 indexing, matching, surge pricing | | |
| | 10 | Food delivery (DoorDash) | Geo, ETA, dispatch optimization | | |
| | 11 | Distributed file store (Dropbox/Drive) | Chunking, dedup, sync, conflict resolution | | |
| | 12 | Search autocomplete (typeahead) | Trie, ranking, prefix cache | | |
| | 13 | Distributed counter / metrics (analytics) | Sharded counters, approximate algorithms | | |
| | 14 | Payment system | Idempotency, double-entry ledger, exactly-once, reconciliation | | |
| | 15 | Ticketing (Ticketmaster) | High-contention seat locking, queues, anti-bot | | |
| **Practice resource priority**: | |
| 1. Alex Xu — *System Design Interview* Vol. 1 & 2 (read end-to-end). | |
| 2. Designing Data-Intensive Applications (Martin Kleppmann) — *the* book; chapters 1–9, 11. | |
| 3. ByteByteGo channel, *System Design Primer* GitHub repo, *Hello Interview* on YouTube. | |
| 4. Engineering blogs: Uber, Netflix, Stripe, Discord, Cloudflare, Meta, LinkedIn, Pinterest, Shopify. | |
| --- | |
| # PART 2 — ML SYSTEM DESIGN | |
| ## Phase 4 — ML System Design Framework | |
| ### 4.1 The 7-step interview framework (use this every time) | |
| 1. **Clarify the problem** — business goal, success metric, constraints (latency, cost, freshness, scale). | |
| 2. **Frame as ML** — supervised/unsupervised/RL; classification/regression/ranking/generation; online vs batch. | |
| 3. **Metrics** — offline (AUC, NDCG, MAP, BLEU, perplexity); online (CTR, engagement, revenue); guardrail (latency p99, fairness, hallucination rate). | |
| 4. **Data** — sources, volume, labels (human, semi-supervised, weak, self-supervised), privacy/PII, class imbalance, leakage. | |
| 5. **Features & model** — feature engineering, embeddings, baselines first (logistic regression, GBDT), then DL; tune for bias-variance. | |
| 6. **System architecture** — training pipeline, feature store, model registry, serving stack, monitoring. | |
| 7. **Iteration & scale** — A/B testing, drift detection, retraining cadence, cost optimization. | |
| ### 4.2 Problem framing pitfalls | |
| - Solving the wrong problem (proxy metric divorced from business). | |
| - Label leakage (future info in features). | |
| - Train/serve skew (different code paths). | |
| - Sampling bias (selection effects). | |
| - Survivorship bias. | |
| - Concept of *counterfactual evaluation* for recommender/ads systems. | |
| ## Phase 5 — ML Production Infrastructure | |
| ### 5.1 Data engineering for ML | |
| - Batch vs streaming ingestion; CDC (Debezium); ETL vs ELT. | |
| - Orchestration: Airflow, Prefect, Dagster, Argo Workflows. | |
| - Data warehouse (Snowflake, BigQuery, Redshift) vs lakehouse (Databricks, Iceberg, Delta). | |
| - Data quality: Great Expectations, Soda, Deequ; schema enforcement; data contracts. | |
| - Labeling: Label Studio, Scale AI, Snorkel (weak supervision), active learning loops. | |
| - Privacy: PII detection, anonymization, differential privacy, k-anonymity, federated learning (when relevant). | |
| ### 5.2 Feature stores | |
| - Why: solves online/offline parity, reuse, point-in-time correctness. | |
| - Components: offline store (warehouse), online store (Redis/DynamoDB), registry, materialization engine. | |
| - Tools: Feast (OSS), Tecton, Hopsworks, Vertex AI Feature Store, SageMaker Feature Store. | |
| - Point-in-time joins / time-travel — critical to avoid leakage. | |
| - Feature freshness, TTL, on-demand vs precomputed features. | |
| ### 5.3 Training infrastructure | |
| - Single-GPU → multi-GPU (DataParallel, DistributedDataParallel) → multi-node. | |
| - Parallelism strategies: | |
| - Data parallelism | |
| - Tensor / model parallelism (Megatron) | |
| - Pipeline parallelism (GPipe, PipeDream) | |
| - Sequence parallelism (long context) | |
| - 3D parallelism (combine all) | |
| - FSDP / ZeRO (DeepSpeed) — sharded optimizer/gradient/parameter. | |
| - Mixed precision (FP16, BF16, FP8), gradient checkpointing, gradient accumulation. | |
| - Distributed training infra: Ray, Horovod, NCCL, RDMA, InfiniBand. | |
| - Hyperparameter tuning: grid, random, Bayesian (Optuna), population-based (Ray Tune), Hyperband/ASHA. | |
| - Reproducibility: seed everything, pin versions, log artifacts. | |
| ### 5.4 Model serving patterns | |
| - **Batch inference** — Spark/Beam pipelines, write to DB; cheap, simple, high-latency. | |
| - **Online / real-time** — gRPC/REST endpoint, <100 ms latency. | |
| - **Near-real-time / streaming** — Kafka → consumer with model → output topic. | |
| - **Edge** — on-device (TFLite, Core ML, ONNX Runtime Mobile). | |
| - Patterns: shadow mode, canary, multi-armed bandit, champion–challenger. | |
| - Model servers: TorchServe, TensorFlow Serving, Triton Inference Server (NVIDIA), BentoML, KServe, Ray Serve, Seldon. | |
| - Triton specifics: dynamic batching, model ensembles, multiple backends (TRT, ONNX, PyTorch, Python). | |
| ### 5.5 Inference optimization | |
| - Hardware: GPU vs CPU vs TPU vs Inferentia; FP16/INT8/INT4 tensor cores. | |
| - Graph compilation: TorchScript, ONNX, TensorRT, OpenVINO, XLA, Apache TVM. | |
| - Quantization: post-training (static, dynamic), quantization-aware training (QAT); INT8/INT4. | |
| - Pruning (structured vs unstructured), distillation (teacher-student). | |
| - Operator fusion, kernel autotuning. | |
| - Caching: result cache, embedding cache. | |
| - Batching: static vs dynamic vs continuous (covered under LLM). | |
| ### 5.6 MLOps & lifecycle | |
| - **Experiment tracking**: MLflow, Weights & Biases, Neptune, Comet. | |
| - **Model registry & versioning**: MLflow registry, SageMaker, Vertex; data versioning with DVC, LakeFS, Pachyderm. | |
| - **CI/CD for ML**: GitHub Actions/GitLab CI + DVC + tests; data tests, model tests (slice metrics, behavioral tests à la Checklist). | |
| - **Pipeline orchestration**: Kubeflow Pipelines, Vertex AI Pipelines, Metaflow, ZenML, SageMaker Pipelines. | |
| - **Reproducibility**: deterministic builds, container images per experiment, lockfiles. | |
| - **Cost ops (FinOps for ML)**: spot/preemptible instances, autoscaling, right-sizing GPUs, fractional GPU sharing. | |
| ### 5.7 Monitoring & observability for ML | |
| - **Operational**: latency, throughput, error rate, GPU/CPU/memory. | |
| - **Data quality**: missing values, schema drift, range violations. | |
| - **Data drift**: covariate shift (input distribution) — PSI, KL, KS test, Wasserstein. | |
| - **Concept drift**: P(y|x) changes — tracked via labeled feedback or proxy metrics. | |
| - **Label drift**: P(y) changes. | |
| - **Performance**: live metric vs ground truth (with lag). | |
| - **Fairness & bias**: subgroup metrics, disparate impact. | |
| - Tools: Evidently, Arize, WhyLabs, Fiddler, Aporia, Datadog ML Monitoring. | |
| - Retraining triggers: scheduled, drift-based, performance-based. | |
| ### 5.8 ML system case studies (drill these) | |
| | Problem | Anchor company / paper | | |
| |---|---| | |
| | YouTube video recommendation | Two-tower retrieval + ranking, *YouTube DNN* paper | | |
| | Netflix recommendations | Personalized ranking + contextual bandits | | |
| | Instagram/TikTok feed ranking | Multi-task DNN, real-time features | | |
| | Twitter/X "For You" | Open-sourced algo (Heavy Ranker, SimClusters) | | |
| | Ad CTR prediction | DCN-v2, DeepFM, DLRM (Meta) | | |
| | Fraud detection | GBDT + graph features + rules | | |
| | Search ranking | LTR (LambdaMART), neural rerankers | | |
| | Spam / abuse | Two-stage (cheap filter + heavy model) | | |
| | Image classification at scale | Distributed training, edge deploy | | |
| | Document understanding | OCR + layout LM | | |
| | Speech recognition | Streaming ASR, RNN-T / Conformer | | |
| | Machine translation | NMT, beam search | | |
| | Personalization (Spotify) | Embedding-based retrieval | | |
| | Anomaly detection | Statistical + isolation forest + autoencoders | | |
| | Forecasting (Uber demand) | Hierarchical, Prophet/DeepAR | | |
| --- | |
| # PART 3 — LLM, RAG, AGENTIC & VLM SYSTEM DESIGN | |
| ## Phase 6 — LLM System Design | |
| ### 6.1 LLM fundamentals (what designers must know) | |
| - Transformer recap: attention, multi-head, KV cache, positional encodings (RoPE, ALiBi). | |
| - Decoder-only (GPT family) vs encoder-decoder (T5, mT5) vs encoder-only (BERT, used for embeddings). | |
| - Tokenizers: BPE, SentencePiece, tiktoken; token cost implications. | |
| - Context window: techniques to extend (RoPE scaling, YaRN, sliding window, sparse attention, Mamba/SSMs as alternative). | |
| - Sampling: temperature, top-k, top-p, repetition penalty, beam search vs greedy vs sampling, logit bias. | |
| ### 6.2 LLM serving stack (deep) | |
| - **vLLM** — paged attention, continuous (in-flight) batching, prefix caching; default OSS choice. | |
| - **TensorRT-LLM** — NVIDIA-optimized, highest throughput; supports speculative decoding, in-flight batching. | |
| - **TGI (Text Generation Inference)** — HuggingFace; production-ready. | |
| - **SGLang** — fast for structured generation and agents. | |
| - **Triton + TensorRT-LLM backend** — production pattern at scale. | |
| - **llama.cpp** / Ollama — CPU/edge. | |
| - **Ray Serve + vLLM** — orchestrated multi-replica. | |
| - Key concepts: | |
| - **KV cache** — memory dominates inference; managed by paged attention. | |
| - **Prefill vs decode** — prefill is compute-bound, decode is memory-bound → different scaling. | |
| - **Continuous batching** — replaces static batching, dramatic throughput gains. | |
| - **Speculative decoding** — small draft model proposes, large verifies. | |
| - **Prefix caching / prompt caching** — reuse KV for shared system prompts. | |
| - **Chunked prefill** — overlap prefill and decode to smooth tail latency. | |
| - **Disaggregated serving** — separate prefill and decode pools (Splitwise pattern). | |
| ### 6.3 Inference optimization for LLMs | |
| - Quantization: GPTQ, AWQ, SmoothQuant, GGUF (llama.cpp), FP8. | |
| - KV-cache quantization (INT8/FP8 KV). | |
| - Flash Attention v1/v2/v3, Flash Decoding. | |
| - LoRA at inference: dynamic LoRA loading (vLLM supports multi-LoRA). | |
| - Speculative decoding variants: Medusa, EAGLE, lookahead decoding. | |
| - Distillation to smaller model (e.g., 70B → 8B). | |
| - Structured output: outlines, lm-format-enforcer, JSON-mode, grammar-constrained decoding. | |
| ### 6.4 Training & fine-tuning (for designers, not researchers) | |
| - **Pre-training** — usually not done in product teams; know the cost (millions of $). | |
| - **Continued pre-training** — domain adaptation on raw corpus. | |
| - **SFT (Supervised Fine-Tuning)** — instruction tuning on (prompt, response) pairs. | |
| - **PEFT methods**: LoRA, QLoRA, DoRA, prefix-tuning, P-tuning, IA³. LoRA/QLoRA is the default. | |
| - **Preference optimization**: RLHF (PPO), DPO, IPO, KTO, ORPO, SimPO. | |
| - **Reasoning RL**: GRPO (DeepSeek), RLAIF. | |
| - Data pipeline: deduplication (MinHash), quality filtering, decontamination, balancing. | |
| - Eval harness: lm-eval-harness, HELM, MT-Bench, Arena-Hard, custom domain evals. | |
| ### 6.5 Cost & latency math for LLM systems | |
| - Tokens/sec per GPU per model size — memorize ballparks for Llama-8B/70B on H100. | |
| - Cost = (input_tokens × input_$/1M) + (output_tokens × output_$/1M). | |
| - Latency = TTFT (time-to-first-token, dominated by prefill) + (output_tokens × time-per-output-token). | |
| - Levers: smaller model, distillation, quantization, caching, batching, regional deploy, model routing. | |
| - Throughput vs latency trade-off curve — design choice. | |
| ## Phase 7 — RAG System Design | |
| ### 7.1 RAG architecture stages | |
| 1. **Ingestion** — load, parse, clean, chunk, embed, index, store metadata. | |
| 2. **Retrieval** — query understanding, embedding, search, filter, rerank. | |
| 3. **Generation** — prompt assembly, LLM call, citation, post-processing. | |
| 4. **Evaluation** — offline + online. | |
| ### 7.2 Document processing | |
| - Parsing: PDFs (PyMuPDF, Unstructured, LlamaParse, Marker, Docling), HTML, Office docs, images (OCR via Tesseract, PaddleOCR, TrOCR). | |
| - Layout-aware parsing critical for tables, multi-column. | |
| - Cleaning: remove boilerplate, normalize whitespace, dedup. | |
| ### 7.3 Chunking strategies | |
| - Fixed-size (token / char) with overlap. | |
| - Recursive character splitter. | |
| - Sentence/paragraph aware (NLTK, spaCy). | |
| - Semantic chunking (embedding-similarity-based splits). | |
| - Document-structure aware (markdown headers, code blocks). | |
| - Late chunking (embed whole doc, then chunk the embeddings). | |
| - Hierarchical (parent-child): retrieve small chunks, return larger context. | |
| - Choose per content type — code, tables, prose differ. | |
| ### 7.4 Embedding models | |
| - Closed: OpenAI text-embedding-3, Cohere embed-v3, Voyage. | |
| - Open: BGE, E5, GTE, Nomic, Jina, mxbai, Stella, Snowflake Arctic, Mistral embed. | |
| - Dimensions (Matryoshka representations let you truncate). | |
| - Specialized: code (CodeBERT, Voyage-code), multilingual, multimodal (CLIP, SigLIP). | |
| - Eval: MTEB benchmark — but build a *domain* eval set; MTEB rank does not transfer. | |
| - Fine-tuning embeddings on your data (contrastive, hard negatives) often beats picking a better OSS model. | |
| ### 7.5 Vector databases & indexes | |
| - Algorithms: HNSW (most common), IVF, IVF-PQ, ScaNN, DiskANN. | |
| - Trade-offs: recall vs latency vs memory vs build cost. | |
| - Products: | |
| - Pinecone (managed, serverless) | |
| - Weaviate | |
| - Qdrant | |
| - Milvus / Zilliz | |
| - Chroma (dev / small) | |
| - pgvector (Postgres extension) — *strong default if already on Postgres* | |
| - Elasticsearch / OpenSearch (BM25 + dense in one) | |
| - Vespa (production search at scale) | |
| - LanceDB, Turbopuffer (cheap object-storage-backed) | |
| - Filters: pre-filter vs post-filter vs hybrid filter — performance differs sharply. | |
| - Multi-tenancy: namespace per tenant; collection per tenant; metadata filter — pick based on tenant count. | |
| ### 7.6 Retrieval techniques | |
| - **Sparse**: BM25, SPLADE (learned sparse). | |
| - **Dense**: bi-encoder embeddings. | |
| - **Hybrid**: combine via RRF (reciprocal rank fusion) or weighted sum. | |
| - **Multi-vector**: ColBERT / ColPali — token-level late interaction; higher quality, more storage. | |
| - **Query transformation**: | |
| - HyDE (hypothetical document embeddings) | |
| - Multi-query (generate N variants) | |
| - Step-back prompting | |
| - Query decomposition for multi-hop | |
| - Routing (which index / source) | |
| - **Reranking**: cross-encoder (e.g., bge-reranker, Cohere Rerank, Voyage rerank); LLM-as-reranker (cost). | |
| - **Contextual retrieval** (Anthropic): prepend chunk-specific context using LLM at ingest. | |
| - **GraphRAG** (Microsoft): build entity/community graph, summarize, retrieve by community. | |
| - **Agentic RAG**: agent loops over retrieval, reads, decides whether to keep searching. | |
| ### 7.7 Advanced RAG patterns | |
| - Naive RAG → Advanced RAG (pre+post retrieval) → Modular RAG. | |
| - Self-RAG, CRAG (corrective RAG), Self-correcting loops. | |
| - Long-context RAG vs RAG vs fine-tuning — when to choose each. | |
| - Lost-in-the-middle problem; reorder retrieved chunks. | |
| - Citation enforcement (return spans, validate). | |
| - Multi-modal RAG (text + image, ColPali for screenshots/PDFs). | |
| ### 7.8 RAG evaluation | |
| - Components to eval independently: retrieval (recall@k, MRR, NDCG, hit rate), generation (faithfulness, answer relevance), end-to-end. | |
| - Frameworks: Ragas, TruLens, DeepEval, Arize Phoenix, LangSmith. | |
| - LLM-as-judge — calibrate vs human labels; use pairwise > pointwise. | |
| - Build a *golden eval set* (50–200 questions with reference answers) before scaling anything. | |
| - Online: thumbs up/down, downstream task completion, edit distance from user edits. | |
| ### 7.9 RAG production concerns | |
| - Freshness: incremental ingestion, change data capture from source. | |
| - Permissions: row/document-level ACLs in vector DB filters. | |
| - PII handling and redaction. | |
| - Caching: semantic cache (GPTCache), exact prompt cache. | |
| - Cost: embedding cost (one-time at ingest), retrieval cost (cheap), generation cost (dominant). | |
| ## Phase 8 — Agentic AI System Design | |
| ### 8.1 Agent paradigms | |
| - **ReAct** (reason + act): interleave thought / action / observation. | |
| - **Plan-and-Execute**: plan first, then execute steps. | |
| - **Reflexion / Self-critique**: agent reviews its own output. | |
| - **Tree-of-Thoughts / Graph-of-Thoughts**. | |
| - **Multi-agent**: supervisor/worker, debate, role-based (CrewAI), graph-based (LangGraph). | |
| ### 8.2 Building blocks | |
| - **Tool use / function calling**: schema (JSON), parallel calls, tool error handling, retries. | |
| - **Memory**: | |
| - Short-term: conversation buffer, summary buffer. | |
| - Long-term: vector store of past interactions, episodic memory. | |
| - Semantic memory: facts, KG. | |
| - Working memory: scratchpad. | |
| - Mem0, Letta (MemGPT), Zep, LangMem — frameworks. | |
| - **Planning**: HTN, LLM-generated DAG, code-as-plan (CodeAct). | |
| - **Reflection / verification**: separate critic LLM, deterministic verifier (unit tests, JSON-schema validation). | |
| ### 8.3 Frameworks (know strengths, not just names) | |
| - **LangChain / LangGraph** — LangGraph is the production-grade graph runtime (state, branching, human-in-loop, checkpointing). | |
| - **LlamaIndex** — data-centric, agents add-on. | |
| - **CrewAI** — role-based multi-agent. | |
| - **AutoGen** (Microsoft) — multi-agent conversation. | |
| - **Semantic Kernel** — Microsoft, .NET friendly. | |
| - **DSPy** — declarative, optimizable pipelines and prompts. | |
| - **Pydantic AI**, **Atomic Agents** — typed agent frameworks. | |
| - **OpenAI Agents SDK**, **Anthropic SDK with tool use**. | |
| - **Smolagents** (HF) — code-as-action minimalism. | |
| ### 8.4 Agent orchestration & infra | |
| - State management: graph state, checkpoints (LangGraph SqliteSaver/Postgres), replayability. | |
| - Concurrency: parallel tool calls, fan-out/fan-in. | |
| - Human-in-the-loop: approval gates, edit-and-resume. | |
| - Streaming: stream tokens + tool events to UI; SSE/WebSocket. | |
| - Timeouts, max-step limits, cost caps. | |
| - Sandboxed code execution: E2B, Modal, Daytona, Firecracker microVMs. | |
| - Browser agents: Browserbase, Playwright, computer-use models. | |
| ### 8.5 Evaluating agents | |
| - Trajectory-level vs final-answer-level metrics. | |
| - Tool-call accuracy, plan correctness, cost per task. | |
| - Benchmarks: AgentBench, GAIA, SWE-bench, WebArena, OSWorld, τ-bench, AppWorld. | |
| - Build domain-specific task suites — golden trajectories. | |
| ### 8.6 Agentic production challenges | |
| - Non-determinism — pin seeds where possible, snapshot test inputs/outputs. | |
| - Cost — agents loop; enforce step + token budgets. | |
| - Latency — parallelize tool calls; cache plan steps. | |
| - Safety — restrict tool capabilities, sandbox writes, allow-list URLs, output filters. | |
| - Observability — full trace per run (LangSmith, Langfuse, Arize, Helicone). | |
| - Versioning — versions of prompts, tools, models, schemas; replay old traces against new versions. | |
| ### 8.7 Protocols | |
| - **MCP (Model Context Protocol)** — Anthropic-led standard for tool/resource servers. | |
| - **A2A (Agent-to-Agent)** — emerging inter-agent protocols. | |
| - **OpenAPI as tool spec** — common pattern. | |
| ## Phase 9 — VLM / Multimodal System Design | |
| ### 9.1 Multimodal models | |
| - **CLIP / SigLIP** — contrastive vision-text; embeddings for search. | |
| - **BLIP-2 / InstructBLIP** — Q-Former bridge. | |
| - **LLaVA family, Qwen-VL, Llama 3.2 Vision, InternVL, Pixtral, Phi-Vision, Molmo, Idefics** — open VLMs. | |
| - **GPT-4o, Claude (vision), Gemini** — closed. | |
| - **Video VLMs**: VideoLLaMA, Video-LLaVA, Gemini. | |
| - **Document VLMs**: Donut, Nougat, ColPali, GOT-OCR. | |
| ### 9.2 Use cases & patterns | |
| - Visual question answering, document understanding, chart/table extraction. | |
| - Multimodal RAG (retrieve images by text/embedding; pass images into VLM). | |
| - OCR-free document QA (ColPali + VLM). | |
| - UI agents / screen understanding (Anthropic computer use, OmniParser). | |
| - Robotics / embodied AI is its own world — out of scope. | |
| ### 9.3 Multimodal infra specifics | |
| - Image preprocessing: resize, tile/patch (high-res), normalization; influences token count drastically. | |
| - Storage: object store + CDN; signed URLs; dedup by perceptual hash. | |
| - Inference cost: image tokens often dominate; bound max images, downsample. | |
| - Caching image embeddings. | |
| - Safety: NSFW filters, CSAM detection (hash matching, e.g., PhotoDNA), face redaction. | |
| --- | |
| # PART 4 — CROSS-CUTTING: PRODUCTION-GRADE AI | |
| ### 10.1 Evaluation (broader) | |
| - Offline: dataset → metric. Build *golden sets* per use case. | |
| - Online: A/B, interleaving, guardrail metrics. | |
| - Human eval: pairwise > Likert; calibrate raters; inter-rater agreement (Cohen's kappa). | |
| - LLM-as-judge: known biases (position, verbosity, self-preference) — mitigate with randomization & rubric. | |
| - Continuous evals: run on every release; alert on regressions. | |
| ### 10.2 Safety, guardrails, alignment-in-product | |
| - Input filters: PII detection, prompt-injection detection, jailbreak detection (Lakera, Prompt Guard, Llama Guard). | |
| - Output filters: toxicity (Perspective API, Detoxify), hallucination check (faithfulness vs sources), PII leakage. | |
| - Constrained generation: JSON schema, regex grammars, function calling. | |
| - Tools: NeMo Guardrails, Guardrails AI, Llama Guard, ShieldGemma. | |
| - Policy/red-teaming process; abuse monitoring; rate limiting & per-user quotas. | |
| - Prompt-injection defenses: separate trusted and untrusted contexts, sandbox tools, never let model execute arbitrary tools on untrusted input. | |
| ### 10.3 Cost & latency engineering for AI | |
| - Model routing / cascading (cheap → expensive based on confidence). | |
| - Distillation to smaller in-house model after collecting traffic. | |
| - Semantic cache for repeated queries. | |
| - Prompt caching (Anthropic, OpenAI, Gemini all support). | |
| - Batched embedding ingestion. | |
| - Async / streaming UX to hide latency. | |
| - Region-local serving to cut RTT. | |
| - Speculative decoding, smaller draft models. | |
| - Right-size GPUs (A10G < L4 < L40S < A100 < H100 < H200 < B200); fractional MIG for small models. | |
| ### 10.4 Data flywheel / continuous learning | |
| - Capture user feedback (explicit + implicit). | |
| - Curate logs into training/eval data with PII scrubbing. | |
| - Active learning: surface uncertain or edge-case examples for labeling. | |
| - Closed-loop: prod logs → eval set → fine-tune → A/B → ship. | |
| - Synthetic data: use stronger model to bootstrap weaker model's data. | |
| ### 10.5 LLMOps stack (concrete) | |
| - Gateway: LiteLLM, Portkey, OpenRouter, Kong AI Gateway (multi-provider, fallback, cost tracking). | |
| - Tracing/Observability: LangSmith, Langfuse, Arize Phoenix, Helicone, Honeyhive, Braintrust. | |
| - Eval: Ragas, DeepEval, Promptfoo, Inspect, Braintrust. | |
| - Vector DB + Embedding service. | |
| - Prompt versioning: PromptLayer, Helicone, Langfuse, in-repo + git. | |
| - Feature flags & experimentation: LaunchDarkly, Statsig. | |
| ### 10.6 Common production reference architectures (memorize these blueprints) | |
| **(a) Enterprise RAG chatbot** | |
| - Ingest: connectors (S3, SharePoint, Confluence, Slack) → parser → chunker → embedder → vector DB + metadata. | |
| - Auth & ACL propagation into retriever filters. | |
| - Query: query rewrite → hybrid search → rerank → prompt assembly → LLM (with prompt cache) → citations → post-filter. | |
| - Observability + feedback capture + eval harness. | |
| **(b) Agentic copilot** | |
| - LangGraph state machine; tools (DB query, API calls, code exec). | |
| - Per-user memory store. | |
| - Sandbox for code; allow-list for HTTP. | |
| - Streaming + interrupts for human approval. | |
| - Trace store + replay. | |
| **(c) Real-time recommendation system** | |
| - Candidate generation (two-tower, ANN) → ranking (DLRM-style) → re-ranking with business rules. | |
| - Feature store (online + offline parity). | |
| - Logging for counterfactual eval. | |
| - Online learning / hourly retrain. | |
| **(d) Fraud / abuse detection** | |
| - Streaming features (Flink) → low-latency model (GBDT/DL) + rules engine → action service → human review queue. | |
| **(e) Search with semantic + lexical** | |
| - Indexer (BM25 + dense embeddings). | |
| - Multi-stage: retrieval → light ranker → heavy cross-encoder. | |
| - Personalization layer. | |
| --- | |
| # PART 5 — INTERVIEW PREPARATION | |
| ## Phase 10 — Interview Drilling | |
| ### 11.1 ML system design interview template (use this verbatim) | |
| 1. Clarify: business problem, scope, scale (DAU, QPS, data volume), latency budget, freshness, cost. | |
| 2. Metrics: business → ML → guardrail. Trade-offs. | |
| 3. Data: sources, labeling, volume, biases, privacy. | |
| 4. Architecture overview: pipelines (data, training, serving, monitoring). | |
| 5. Modeling: baseline → enhanced; features; loss; eval. | |
| 6. Serving: latency, throughput, scaling, fallback. | |
| 7. Monitoring & feedback. | |
| 8. Trade-offs and what you'd improve with more time. | |
| ### 11.2 Common ML/LLM interview questions to drill | |
| - Design YouTube recommendations. | |
| - Design ad CTR prediction at Meta. | |
| - Design fraud detection for Stripe. | |
| - Design semantic search across enterprise docs. | |
| - Design ChatGPT-style assistant with citations over a private corpus (RAG). | |
| - Design an agent that books travel end-to-end. | |
| - Design a code assistant (Copilot-like). | |
| - Design a multimodal product-search system (image+text). | |
| - Design hallucination-detection / fact-checking pipeline. | |
| - Design model serving for a 70B LLM with <1s p50 latency and 100 QPS. | |
| - Design A/B test framework for LLM products (ratios, novelty effects, guardrails). | |
| - Build a personalization system with cold-start. | |
| - Design content moderation at scale. | |
| ### 11.3 Behavioral / staff-level signals | |
| - Talk about *trade-offs*, not just choices. | |
| - Lead with metrics and constraints. | |
| - Surface failure modes proactively. | |
| - Distinguish "what I'd build now" vs "what I'd build at 100× scale." | |
| - Discuss organizational and rollout aspects: ownership, on-call, ramp-up, rollback. | |
| ### 11.4 Practice cadence | |
| - Solo: 1 problem/day, 60 min timed, then 30 min review. | |
| - Peer mocks: 2/week with feedback. | |
| - Record yourself once a week and rewatch. | |
| - After each problem: write a 1-page "would do differently" note. | |
| --- | |
| ## Resources (curated, in order) | |
| **Books** | |
| 1. *Designing Data-Intensive Applications* — Kleppmann (mandatory). | |
| 2. *System Design Interview Vol. 1 & 2* — Alex Xu. | |
| 3. *Designing Machine Learning Systems* — Chip Huyen (mandatory). | |
| 4. *Machine Learning System Design Interview* — Ali Aminian & Alex Xu. | |
| 5. *AI Engineering* — Chip Huyen (2024). | |
| 6. *Building LLMs for Production* — Louis-François Bouchard. | |
| 7. *Hands-On Large Language Models* — Jay Alammar & Maarten Grootendorst. | |
| 8. *Site Reliability Engineering* — Google (free online). | |
| **Courses / channels** | |
| - ByteByteGo (YouTube + newsletter). | |
| - Hello Interview (YouTube). | |
| - Stanford CS329S — ML Systems Design. | |
| - Full Stack Deep Learning. | |
| - DeepLearning.AI short courses (RAG, agents, evals, function calling). | |
| - LangChain Academy (LangGraph). | |
| **Engineering blogs to read weekly** | |
| - Anthropic Engineering ("Building effective agents", "Contextual retrieval"). | |
| - OpenAI cookbook, Google Research, Meta AI, Microsoft Research blogs. | |
| - Netflix Tech Blog, Uber Engineering, Stripe, Cloudflare, Discord, Pinterest, LinkedIn, Shopify, DoorDash, Airbnb. | |
| - Eugene Yan, Chip Huyen, Hamel Husain, Jason Liu, Lilian Weng, Sebastian Raschka. | |
| **Papers (read at least the abstracts of these)** | |
| - *Attention Is All You Need*, *GPT-3*, *LLaMA*, *RAG (Lewis et al.)*, *ReAct*, *Toolformer*, *Self-RAG*, *ColBERT/ColPali*, *FlashAttention*, *vLLM/PagedAttention*, *LoRA*, *DPO*, *DLRM*, *Wide & Deep*, *YouTube DNN*, *Two Tower*, *DCN-v2*. | |
| --- | |
| ## Final Tracker (check off as you go) | |
| - [ ] Phase 0 — Foundations of computing & networks | |
| - [ ] Phase 1 — Building blocks of distributed systems | |
| - [ ] Phase 2 — Core components in depth | |
| - [ ] Phase 3 — 15 classic system design problems | |
| - [ ] Phase 4 — ML system design framework | |
| - [ ] Phase 5 — ML production infra (data, training, serving, MLOps, monitoring) | |
| - [ ] Phase 6 — LLM system design + serving | |
| - [ ] Phase 7 — RAG system design | |
| - [ ] Phase 8 — Agentic AI system design | |
| - [ ] Phase 9 — VLM / multimodal | |
| - [ ] Phase 10 — Cross-cutting production AI (cost, safety, evals, LLMOps) | |
| - [ ] Phase 11 — Interview drilling (20+ mocks) | |
| When you can sketch any of the reference architectures in section 10.6 on a whiteboard in 10 minutes and defend every choice — you're ready. | |
Xet Storage Details
- Size:
- 38.4 kB
- Xet hash:
- cc903ba652b0d33b853a10abc7ae48c4c51c9deecb52da701ea69d0a61297973
·
Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.