AI Infrastructure — system design interview guide

LLM serving, inference batching, RAG pipelines, and vector search at scale.

6 lessons · about 6 hours · Deep Dive · 1 worked case study

AI Infrastructure is family 11 of 12 in the Thita system design course. It holds 5 lessons and 1 worked case study, and it is marked Deep Dive — depth to reach for once the Interview Soon families are solid.

Lessons in AI Infrastructure

• LLM Serving (Design ChatGPT) — Depth lesson.
• Inference Batching — Depth lesson.
• RAG Pipeline — Depth lesson.
• Vector Search at Scale — Depth lesson.
• Distributed Model Training — Depth lesson.
• Design a Recommendation System — Depth lesson, worked case study, about 60 minutes, asked at Netflix, Amazon, Spotify.

Case studies in AI Infrastructure

Each one takes a product brief from this family through the whole worked answer — requirements, estimation, API, data model and trade-offs.

• Design a Recommendation System — about 60 minutes, asked at Netflix, Amazon, Spotify.

The rest of the system design course

• Foundations — The 45-minute interview framework, estimation with modern numbers, and CAP theorem. 6 lessons · about 2 hours · Interview Soon · 1 worked case study.
• Networking & APIs — Protocols (REST, gRPC, WebSocket, SSE), API design, pagination, and idempotency. 5 lessons · about 2 hours · Interview Soon · 1 worked case study.
• Databases — SQL vs NoSQL decision tree, replication, sharding, and indexing strategies. 7 lessons · about 3 hours · Interview Soon · 2 worked case studies.
• Caching — Cache-aside, write-through, failure modes, and when NOT to cache. 5 lessons · about 1.5 hours · Interview Soon · 2 worked case studies.
• Queues & Async — Kafka vs SQS, delivery guarantees, idempotent consumers, and event-driven patterns. 10 lessons · about 1.5 hours · Interview Soon · 5 worked case studies.
• Scalability — Consistent hashing, read vs write scaling, fan-out strategies, and circuit breakers. 7 lessons · about 1.5 hours · Interview Soon · 4 worked case studies.
• Coordination & Transactions — Raft consensus, distributed locking, saga pattern, and conflict resolution. 9 lessons · about 1.5 hours · Interview Soon · 5 worked case studies.
• Search, Geo & Aggregation — Full-text search, geospatial indexing, stream processing, and probabilistic structures. 9 lessons · about 1.5 hours · Interview Soon · 5 worked case studies.
• Reliability & Operations — Monitoring, observability, deployment strategies, and graceful degradation. 2 lessons · about 3 hours · Deep Dive.
• Security & Auth — OAuth 2.0, JWT vs sessions, API security, and encryption. 2 lessons · about 3 hours · Deep Dive.
• Real-World Architecture Studies — Deep dives into Discord, Uber, Stripe, Netflix, and Kafka production architectures. 5 lessons · about 4 hours · Deep Dive.

Related interview preparation

• Looking for the practice sheet? — Track your progress against every family as a checklist, with company tags and AI interview practice.
• Low Level Design course — The object-level companion: OOP, SOLID, the design patterns and the machine-coding round.
• DSA patterns course — The coding round: the algorithmic patterns behind the problems asked alongside system design.
• Latency and capacity numbers — The reference table of modern latency, throughput and storage figures used in estimation.
• AI mock interviews — Practise the spoken answer against an interviewer that pushes back on the trade-offs.