System Design

Handle more users without slowing down.

Already live in other tracks: Streams and backpressure

1. Interactive

   Live

   Vertical vs horizontal scaling

   Make one server bigger or add more servers, and see where each one hits a wall.

   Open session →

2. Interactive

   Live

   Back-of-the-envelope estimates

   Turn daily users into requests per second, storage and bandwidth.

   Open session →

3. Interactive

   Live

   Load balancing strategies

   Queues, health checks, and timeouts — not just which server is next.

   Open session →

4. Interactive

   Live

   Autoscaling

   Add servers when traffic climbs and remove them when it drops, without flapping.

   Open session →

5. Interactive

   Live

   Caching and eviction

   Watch a cache hit, miss and evict, then predict the next request.

   Open session →

6. Interactive

   Live

   Cache invalidation

   Write-through, write-behind, TTLs and explicit purges: keeping cached data fresh.

   Open session →

7. Interactive

   Live

   CDNs: caching at the edge

   Pick a TTL, ship a new logo, and see which cities still get the old one.

   Open session →

8. Interactive

   Live

   Latency, throughput and concurrency

   Why a faster request and more requests per second are different goals.

   Open session →

9. Interactive

   Coming soon

   Database bottlenecks

   Connection pools, slow queries, N+1 calls and read replicas.

10. Interactive

    Live

    Rate limiting

    Fixed window, sliding window and token bucket on the same burst.

    Open session →

11. Interactive

    Live

    Backpressure and load shedding

    Slow the sender or drop work on purpose before the whole system falls over.

    Open session →

12. Interactive

    Live

    Counting at scale: HyperLogLog and count-min sketch

    Count unique visitors and frequent items in kilobytes, by accepting a small error.

    Open session →

13. Interactive

    Live

    Top-K, trending and leaderboards

    Rank millions of scores live, and find what's trending right now.

    Open session →

Choose where data lives and how it stays correct.

1. Interactive

   Coming soon

   SQL, document and graph databases

   Same data in three models: joins, flexible documents, or relationships first.

2. Interactive

   Coming soon

   Data modeling: normalize or denormalize

   Design a schema, then trade duplicate data for faster reads.

3. Interactive

   Coming soon

   Indexes and query optimization

   Find one row in a million with a B-tree instead of a full scan, and pay for it on writes.

4. Interactive

   Coming soon

   Transactions, ACID and isolation levels

   Two people book the last seat at once: see which isolation level stops the double booking.

5. Interactive

   Live

   Replication and stale reads

   Write to the leader, read from a follower, and crash the leader mid-write.

   Open session →

6. Interactive

   Live

   Sharding: choosing the shard key

   Hash, country or date: see hot shards and queries that ask every shard.

   Open session →

7. Interactive

   Live

   Bloom filters

   Answer "definitely not here" without touching the database, with a tunable false-positive rate.

   Open session →

8. Interactive

   Coming soon

   B-trees vs LSM trees and the write-ahead log

   How databases lay data on disk, and why one is faster for reads and the other for writes.

9. Interactive

   Live

   Change data capture

   Stream every database change to search, caches and analytics without dual writes.

   Open session →

10. Interactive

    Live

    Zero-downtime schema migrations

    Change a table under live traffic: expand, migrate, then contract.

    Open session →

11. Interactive

    Coming soon

    Time-series databases

    Metrics and sensor data: append-only writes, downsampling and retention.

How services find and talk to each other.

1. Interactive

   Live

   DNS, TCP, TLS and HTTP: one request

   Follow a request from a domain name to a response, one layer at a time.

   Open session →

2. Interactive

   Coming soon

   REST, GraphQL, gRPC and WebSockets

   Four ways to shape an API, and what each one is good at.

3. Interactive

   Coming soon

   Synchronous vs asynchronous communication

   Wait for the answer, or hand off the work and get told later.

4. Interactive

   Live

   API gateways and service discovery

   One front door for auth, routing and limits, and how it finds services that move.

   Open session →

5. Interactive

   Live

   Idempotency and request deduplication

   Send the same payment twice and charge it once.

   Open session →

6. Interactive

   Live

   Webhooks

   Call another system when something happens: signatures, retries and replay.

   Open session →

7. Interactive

   Coming soon

   Service mesh

   Sidecars handle retries, TLS and traffic splits between services.

8. Interactive

   Coming soon

   HTTP/2, HTTP/3 and QUIC

   Multiplexing, head-of-line blocking, and why HTTP/3 moved to UDP.

Keep working when parts fail.

1. Interactive

   Live

   Availability and the nines

   What 99.9% really allows, and how redundancy buys more nines.

   Open session →

2. Interactive

   Live

   Fault tolerance and graceful degradation

   Turn off the extras so the core feature keeps working.

   Open session →

3. Interactive

   Live

   Retries and circuit breakers

   Turn a retry storm into a fast failure and let the broken service recover.

   Open session →

4. Interactive

   Live

   Bulkheads

   Split resources so one slow dependency can't use up every thread.

   Open session →

5. Interactive

   Live

   Failover, disaster recovery, RTO and RPO

   Lose a region and measure how long you're down and how much data you lose.

   Open session →

6. Interactive

   Live

   CAP theorem and quorums

   Split the network and choose: answer with stale data, or refuse until it heals.

   Open session →

7. Interactive

   Coming soon

   Distributed transactions and two-phase commit

   Commit across two databases, and see what happens when the coordinator dies.

8. Interactive

   Live

   Hedged requests and retry budgets

   Send a backup request to cut slow tails, without doubling the load.

   Open session →

Many machines acting like one system.

1. Interactive

   Coming soon

   Event-driven architecture

   Services react to events instead of calling each other directly.

2. Interactive

   Live

   Message queues

   Slow consumers, duplicate charges and a dead-letter queue: at-least-once in action.

   Open session →

3. Interactive

   Coming soon

   Pub/sub and event streaming

   Publish once; many consumers read a partitioned log at their own pace.

4. Interactive

   Live

   Ordering, delivery guarantees and duplicates

   At-most-once, at-least-once, exactly-once, and keeping events in order.

   Open session →

5. Interactive

   Live

   Eventual consistency

   Replicas disagree for a moment, then converge. When is that acceptable?

   Open session →

6. Interactive

   Live

   CQRS and event sourcing

   Store every change as an event and build separate read models from them.

   Open session →

7. Interactive

   Live

   Sagas and the transactional outbox

   Run a multi-service order without a global transaction, and never lose the event.

   Open session →

8. Interactive

   Live

   Leader election and consensus

   Nodes vote in a leader with Raft; crash it and watch them elect another.

   Open session →

9. Interactive

   Live

   Distributed locks

   Only one worker may run the job: leases, fencing tokens and expired locks.

   Open session →

10. Interactive

    Live

    Consistent hashing

    Add and remove nodes and see how few keys move.

    Open session →

11. Interactive

    Live

    Unique IDs at scale

    Auto-increment, UUIDs and Snowflake IDs: ordering, collisions and coordination.

    Open session →

12. Interactive

    Live

    Clocks and ordering

    Machine clocks drift: Lamport and vector clocks order events without them.

    Open session →

13. Interactive

    Live

    Gossip protocols

    Spread membership and state through a cluster by talking to a few peers at a time.

    Open session →

14. Interactive

    Live

    Merkle trees and anti-entropy

    Find which data two replicas disagree on without comparing every row.

    Open session →

How to split a codebase into parts.

1. Interactive

   Coming soon

   Monolith, modular monolith or microservices

   Where to draw service boundaries, and what each split costs.

2. Interactive

   Coming soon

   Layered, clean and hexagonal architecture

   Keep business logic apart from the database and the web framework: ports and adapters.

3. Interactive

   Coming soon

   Domain-driven design

   Bounded contexts, aggregates and a shared language with the business.

4. Interactive

   Coming soon

   Architecture decision records

   Write down why a choice was made so the next person doesn't undo it.

Who can do what, and keeping data safe.

1. Interactive

   Coming soon

   Authentication vs authorization

   Who you are versus what you're allowed to do.

2. Interactive

   Coming soon

   OAuth 2.0, OpenID Connect and JWTs

   Sign in with another provider and pass tokens between services safely.

3. Interactive

   Coming soon

   Role- and attribute-based access control

   RBAC and ABAC: permissions by role, or by rules about the user and the data.

4. Interactive

   Coming soon

   Encryption at rest and in transit

   TLS on the wire, encrypted disks and keys you can rotate.

5. Interactive

   Coming soon

   Secrets management

   Keep API keys and passwords out of code, and rotate them without downtime.

6. Interactive

   Coming soon

   Audit logs

   Record who did what and when, in a log nobody can quietly edit.

7. Interactive

   Coming soon

   Threat modeling and security boundaries

   Map where untrusted input enters and what an attacker could reach.

8. Interactive

   Coming soon

   XSS, CSRF and SQL injection

   The classic web attacks from the OWASP Top 10, and what stops each one.

9. Interactive

   Coming soon

   DDoS protection and WAFs

   Absorb floods of traffic and filter bad requests before they reach your servers.

Run it, watch it and pay for it.

Already live in other tracks: A CI/CD pipeline, stage by stage, Containers vs virtual machines, Kubernetes rolling deploys

1. Interactive

   Coming soon

   Logs, metrics and distributed tracing

   Follow one slow request across five services.

2. Interactive

   Coming soon

   Monitoring, SLOs and alerting

   Alert on what users feel, not on every CPU spike.

3. Interactive

   Live

   Blue-green and canary deployments

   Ship to a small slice of traffic first, and roll back in seconds.

   Open session →

4. Interactive

   Coming soon

   Infrastructure as code

   Describe servers and networks in files you can review and repeat.

5. Interactive

   Coming soon

   Capacity planning and cost optimization

   Forecast growth, size for the peak, and stop paying for idle machines.

6. Interactive

   Live

   Feature flags

   Ship code dark, turn it on for 1% of users, and switch it off without a deploy.

   Open session →

7. Interactive

   Coming soon

   Chaos engineering

   Break things on purpose in a controlled way to find weak spots first.

8. Interactive

   Coming soon

   Incident response and postmortems

   Roles, communication and a blameless write-up when production goes down.

Putting it together for real products.

1. Interactive

   Coming soon

   Frontend architecture

   Component boundaries, state, data fetching and rendering choices.

2. Interactive

   Coming soon

   Backend architecture

   API boundaries, service layers and where business rules live.

3. Interactive

   Coming soon

   API design: pagination, versioning and errors

   Resource modeling, cursors vs offsets, validation and useful error responses.

4. Interactive

   Coming soon

   Sessions, tokens and refresh flows

   Cookies or tokens, short-lived access and safe refresh.

5. Interactive

   Live

   Real-time: WebSockets, SSE and presence

   Push live updates and show who is online, and what each approach costs.

   Open session →

6. Interactive

   Coming soon

   File uploads and object storage

   Signed URLs, multipart uploads and processing media after upload.

7. Interactive

   Live

   Background jobs and scheduling

   Workers, retries, cron and dead-letter queues for work users shouldn't wait on.

   Open session →

8. Interactive

   Live

   Multi-tenancy

   Keep each customer's data apart, with per-tenant settings and limits.

   Open session →

9. Interactive

   Coming soon

   Search: indexing, filtering and ranking

   Full-text search that returns the right results first.

10. Interactive

    Coming soon

    Mobile and offline sync

    Local databases, syncing back, resolving conflicts and push notifications.

11. Interactive

    Coming soon

    Collaborative editing

    Two people type in the same document: operational transforms and CRDTs.

12. Interactive

    Coming soon

    Email and notification delivery

    Queues, providers, retries, preferences and not sending the same alert twice.

The parts around the model.

Already live in other tracks: How an LLM picks the next token, The context window, turn by turn, Force valid JSON

1. Interactive

   Coming soon

   LLM gateways: routing, fallbacks and versions

   Pick a model per request, fall back when a provider fails, and pin versions.

2. Interactive

   Coming soon

   Prompt templates and versioning

   Treat prompts like code: templates, versions and safe rollouts.

3. Interactive

   Coming soon

   Context windows and token budgets

   Decide what fits in the prompt when everything can't.

4. Interactive

   Live

   Streaming responses

   Send tokens as they're generated, through every hop to the browser.

   Open session →

5. Interactive

   Coming soon

   Inference latency and token cost

   Where the seconds and the money go, and how to cut both.

6. Interactive

   Coming soon

   Prompting vs RAG vs fine-tuning

   Which one to reach for: missing knowledge, missing behavior, or missing format.

7. Interactive

   Coming soon

   Semantic caching

   Reuse an answer when a new question means the same thing as an old one.

Give the model the right documents.

Already live in other tracks: Embeddings and similarity search, Hybrid search and reranking, RAG, step by step

1. Interactive

   Coming soon

   Document ingestion and chunking

   Parse PDFs and pages, then split them so each chunk still makes sense.

2. Interactive

   Coming soon

   Embeddings and vector databases

   Store vectors, search them fast, and filter by metadata.

3. Interactive

   Coming soon

   Retrieval evaluation and reranking

   Measure whether the right chunks come back, then reorder them.

4. Interactive

   Coming soon

   Citations and source attribution

   Show which document each part of the answer came from.

5. Interactive

   Coming soon

   Incremental indexing

   Update the index when one document changes, without rebuilding everything.

6. Interactive

   Coming soon

   Changing embedding models

   Swap the embedding model without breaking search: dual indexes and backfills.

Models that take actions.

Already live in other tracks: Call a tool, then answer, Agent patterns: workflow or agent?, Pause for a person, MCP: tools the model can call, A coding agent's loop

1. Interactive

   Coming soon

   Tool execution and sandboxing

   Run the tool the model asked for, safely, with limits.

2. Interactive

   Coming soon

   Agent loops and state machines

   Model an agent as explicit states so you can see and resume where it is.

3. Interactive

   Coming soon

   Workflows vs autonomous agents

   Fixed steps you control, or a model that decides the next step.

4. Interactive

   Coming soon

   Planning and task decomposition

   Break a big request into steps, and replan when one fails.

5. Interactive

   Coming soon

   Multi-agent coordination

   Several agents share work: hand-offs, shared state and conflicts.

6. Interactive

   Coming soon

   Durable execution and recovery

   Crash mid-task and resume from the last step instead of starting over.

What the model remembers between turns.

1. Interactive

   Coming soon

   Conversation and session state

   Store the chat so it survives reloads, devices and server restarts.

2. Interactive

   Coming soon

   Short-term and long-term memory

   Working memory, semantic facts and episodes: what to keep and for how long.

3. Interactive

   Coming soon

   Memory extraction, updates and conflicts

   Pull facts from chats, fix wrong ones, and resolve contradictions.

4. Interactive

   Coming soon

   Context selection and compression

   Summarize and pick what matters when the history is too long.

5. Interactive

   Coming soon

   Per-user memory and isolation

   One user's memories must never reach another user.

Know whether it actually works.

Already live in other tracks: Evals: is the new prompt better?

1. Interactive

   Coming soon

   Golden datasets and regression tests

   A fixed set of questions with known good answers, run on every change.

2. Interactive

   Coming soon

   Retrieval quality vs answer quality

   Faithfulness, relevance and correctness: find which stage caused a bad answer.

3. Interactive

   Coming soon

   Online evals, A/B tests and feedback

   Measure quality on real traffic and feed it back into the next version.

4. Interactive

   Coming soon

   Tracing and debugging agents

   See every model call, tool call and decision in one timeline.

Running the models themselves.

1. Interactive

   Coming soon

   Hosted APIs vs self-hosted models

   Pay per token or run your own GPUs: cost, control and effort.

2. Interactive

   Coming soon

   GPU capacity, batching and queueing

   Group requests to keep GPUs busy without making users wait.

3. Interactive

   Coming soon

   Quantization and serving optimization

   Smaller numbers, smaller models, faster answers, and what you lose.

4. Interactive

   Coming soon

   Provider rate limits and quotas

   Stay under tokens-per-minute limits across many users and keys.

New ways for AI systems to go wrong.

1. Interactive

   Coming soon

   Prompt injection, direct and indirect

   Instructions hidden in user input or in a retrieved document take over the model.

2. Interactive

   Coming soon

   Tool permissions and least privilege

   Give an agent only the tools and scopes this task needs.

3. Interactive

   Coming soon

   Sensitive-data leakage

   Stop secrets and personal data from reaching prompts, logs or other users.

4. Interactive

   Coming soon

   Tenant-aware retrieval

   Only retrieve documents the current user is allowed to see.

5. Interactive

   Coming soon

   Output validation and action approval

   Check what the model produced before the backend acts on it.

6. Interactive

   Coming soon

   AI auditability and data retention

   Keep a record of what the model saw and did, for as long as you need it, and no longer.

7. Interactive

   Coming soon

   Guardrails and content moderation

   Check inputs and outputs for unsafe content, off-topic requests and policy breaks.

Common product types that combine the above.

1. Interactive

   Coming soon

   Payments and ledgers

   Double-entry ledgers, reconciliation and never losing a cent.

2. Interactive

   Coming soon

   Event pipelines and data warehouses

   Move events into analytics: batch vs streaming, and late data.

3. Interactive

   Coming soon

   Distributed object storage

   How systems like S3 store and replicate huge numbers of files.

4. Interactive

   Coming soon

   Multi-region systems

   Serve users from several regions: latency, failover and data residency.

5. Interactive

   Coming soon

   Edge and IoT systems

   Devices with patchy connections: telemetry, sync and updates.

Put the pieces together: one product, end to end.

1. Interactive

   Coming soon

   Design a URL shortener

   Short codes, redirects at scale, and counting clicks.

2. Interactive

   Coming soon

   Design a rate limiter

   A shared limiter for many servers: where counters live and what happens when it fails.

3. Interactive

   Coming soon

   Design a notification system

   Push, email and SMS to millions of users, with preferences and retries.

4. Interactive

   Coming soon

   Design a chat app

   Messages, delivery receipts, presence and group chats.

5. Interactive

   Coming soon

   Design a news feed

   Fan-out on write vs fan-out on read, and what to do about celebrities.

6. Interactive

   Coming soon

   Design search autocomplete

   Suggestions in under 100 ms as the user types.

7. Interactive

   Coming soon

   Design file sync (like Dropbox)

   Chunked uploads, deduplication and syncing edits across devices.

8. Interactive

   Coming soon

   Design video streaming

   Upload, transcode, and adaptive bitrate playback through a CDN.

9. Interactive

   Coming soon

   Design ride sharing (like Uber)

   Find nearby drivers with geohashing and match them in real time.

10. Interactive

    Coming soon

    Design a web crawler

    Crawl billions of pages politely, without fetching the same page twice.

11. Interactive

    Coming soon

    Design a payment system

    Charge exactly once, keep a ledger, and reconcile with the bank.

12. Interactive

    Coming soon

    Design a document assistant

    RAG with citations that only shows each user documents they're allowed to read.

13. Interactive

    Coming soon

    Design a coding agent

    Read the repo, propose a fix, run the tests, and report the evidence.