Vector Search at Scale — system design interview guide

Vector Search at Scale is a depth lesson in the AI Infrastructure family of the Thita system design course.

LLM serving, inference batching, RAG pipelines, and vector search at scale.

It sits on the Deep Dive path — worth reading once the Interview Soon lessons are solid, or when a role names this area explicitly.

The rest of AI Infrastructure

• LLM Serving (Design ChatGPT) — Depth lesson.
• Inference Batching — Depth lesson.
• RAG Pipeline — Depth lesson.
• Vector Search at Scale — Depth lesson.
• Distributed Model Training — Depth lesson.
• Design a Recommendation System — Depth lesson, worked case study, about 60 minutes, asked at Netflix, Amazon, Spotify.

Other system design pattern families

• Foundations — The 45-minute interview framework, estimation with modern numbers, and CAP theorem. 6 lessons · about 2 hours · Interview Soon · 1 worked case study.
• Networking & APIs — Protocols (REST, gRPC, WebSocket, SSE), API design, pagination, and idempotency. 5 lessons · about 2 hours · Interview Soon · 1 worked case study.
• Databases — SQL vs NoSQL decision tree, replication, sharding, and indexing strategies. 7 lessons · about 3 hours · Interview Soon · 2 worked case studies.
• Caching — Cache-aside, write-through, failure modes, and when NOT to cache. 5 lessons · about 1.5 hours · Interview Soon · 2 worked case studies.
• Queues & Async — Kafka vs SQS, delivery guarantees, idempotent consumers, and event-driven patterns. 10 lessons · about 1.5 hours · Interview Soon · 5 worked case studies.
• Scalability — Consistent hashing, read vs write scaling, fan-out strategies, and circuit breakers. 7 lessons · about 1.5 hours · Interview Soon · 4 worked case studies.
• Coordination & Transactions — Raft consensus, distributed locking, saga pattern, and conflict resolution. 9 lessons · about 1.5 hours · Interview Soon · 5 worked case studies.
• Search, Geo & Aggregation — Full-text search, geospatial indexing, stream processing, and probabilistic structures. 9 lessons · about 1.5 hours · Interview Soon · 5 worked case studies.
• Reliability & Operations — Monitoring, observability, deployment strategies, and graceful degradation. 2 lessons · about 3 hours · Deep Dive.
• Security & Auth — OAuth 2.0, JWT vs sessions, API security, and encryption. 2 lessons · about 3 hours · Deep Dive.
• Real-World Architecture Studies — Deep dives into Discord, Uber, Stripe, Netflix, and Kafka production architectures. 5 lessons · about 4 hours · Deep Dive.

Related interview preparation

• Looking for the practice sheet? — Track your progress against every family as a checklist, with company tags and AI interview practice.
• Low Level Design course — The object-level companion: OOP, SOLID, the design patterns and the machine-coding round.
• DSA patterns course — The coding round: the algorithmic patterns behind the problems asked alongside system design.
• Latency and capacity numbers — The reference table of modern latency, throughput and storage figures used in estimation.
• AI mock interviews — Practise the spoken answer against an interviewer that pushes back on the trade-offs.