Design a URL Shortener

Research rank: #1 most asked
Asked at: Amazon, Google, Microsoft, Uber, Coinbase
Estimated interview time: 60 minutes
Concepts tested: hashing, database choice, caching, estimation
Related patterns: foundations, databases, caching

---

1. Problem Statement

Design a URL shortener like Bitly.

Users submit a long URL and receive a short URL. When someone opens the short URL, the system redirects them to the original URL.

Assume:

• 100M new short URLs per day
• 100B redirects per day

Core flow:

Create unique ID -> encode with base62 -> store mapping -> redirect by key lookup

---

2. Requirements Gathering

Start by separating writes from reads.

URL creation is the write path. Redirects are the read path. Redirects dominate the system by several orders of magnitude.

Functional requirements

• Create a short URL from a long URL.
• Redirect a short URL to its long URL.
• Support optional custom aliases, such as bit.ly/my-product.
• Support optional expiration time.
• Support disabling or deleting a short URL.
• Track analytics:
  • total clicks
  • clicks over time
  • referrer
  • country or device when available
• Return errors for invalid, expired, deleted, or unknown short codes.

Non-functional requirements

• Redirects should complete under 50 ms server-side before network time.
• Redirect availability has higher priority than analytics accuracy.
• URL creation can tolerate higher latency than redirect.
• Analytics must run outside the redirect critical path.
• The system should handle:
  • 100M URL creates/day
  • 100B redirects/day
• Short codes should usually be 7–8 characters.
• URL mappings must be durable.
• Analytics can be eventually consistent.

Interview framing

Say:

“The redirect path is the hot path. I’ll make it a key-value lookup backed by Redis and Postgres read replicas. URL creation goes to one Postgres primary because 1.2K writes/sec fits on one primary.”

---

3. API Design

Keep the API small. Spend time on creation, redirect, and analytics. Skip low-value admin endpoints unless asked.

Create short URL

POST /v1/urls
Content-Type: application/json
Authorization: Bearer <token>

Request:

{
  "long_url": "https://www.example.com/products/123?source=email",
  "custom_alias": "spring-sale",
  "expires_at": "2026-12-31T23:59:59Z"
}

Response:

{
  "id": 9384729384,
  "short_code": "b7Xk29Q",
  "short_url": "https://sho.rt/b7Xk29Q",
  "long_url": "https://www.example.com/products/123?source=email",
  "created_at": "2026-05-14T12:00:00Z",
  "expires_at": "2026-12-31T23:59:59Z"
}

If custom_alias is already taken:

409 Conflict

{
  "error": "custom_alias_already_exists"
}

Redirect short URL

GET /{short_code}

Successful response:

302 Found
Location: https://www.example.com/products/123?source=email
Cache-Control: no-store

Unknown code:

404 Not Found

Expired or disabled code:

410 Gone

Fetch analytics

GET /v1/urls/{short_code}/analytics?from=2026-05-01&to=2026-05-14
Authorization: Bearer <token>

Response:

{
  "short_code": "b7Xk29Q",
  "total_clicks": 184920,
  "daily_clicks": [
    { "date": "2026-05-13", "clicks": 10321 },
    { "date": "2026-05-14", "clicks": 12902 }
  ],
  "top_referrers": [
    { "referrer": "twitter.com", "clicks": 43000 }
  ],
  "top_countries": [
    { "country": "US", "clicks": 82000 }
  ]
}

Update or disable URL

PATCH /v1/urls/{short_code}
Authorization: Bearer <token>

Request:

{
  "status": "disabled",
  "expires_at": "2026-06-01T00:00:00Z"
}

Use the admin endpoint for status and expiration changes. Keep admin logic out of the redirect endpoint.

---

4. Data Model

Use Postgres as the source of truth.

The core record is a durable mapping from short_code to long_url.

short_urls

CREATE TABLE short_urls (
  id BIGSERIAL PRIMARY KEY,
  short_code VARCHAR(16) NOT NULL UNIQUE,
  long_url TEXT NOT NULL,
  user_id BIGINT,
  status VARCHAR(16) NOT NULL DEFAULT 'active',
  created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
  expires_at TIMESTAMPTZ,
  updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);

Indexes:

CREATE UNIQUE INDEX idx_short_urls_code ON short_urls(short_code);
CREATE INDEX idx_short_urls_user_created ON short_urls(user_id, created_at DESC);
CREATE INDEX idx_short_urls_expires_at ON short_urls(expires_at);

Redirect lookups use short_code.

User dashboards use (user_id, created_at).

Custom aliases

Store custom aliases in short_urls.short_code.

Operationally, an alias is another short code with a uniqueness constraint. Add a separate alias table only if aliases need separate ownership, moderation, or billing rules.

Analytics event

Do not write every click into Postgres. At 100B redirects/day, row-by-row click writes will overload the database.

Emit clicks as events:

{
  "event_id": "01JZ...",
  "short_code": "b7Xk29Q",
  "url_id": 9384729384,
  "ts": "2026-05-14T12:00:01Z",
  "ip_hash": "7f83...",
  "user_agent": "Mozilla/5.0 ...",
  "referrer": "https://twitter.com/...",
  "country": "US"
}

Send events to Kafka. Consumers write to ClickHouse, Druid, BigQuery, or a warehouse-backed aggregation pipeline.

Aggregated analytics table

For dashboard reads, store aggregates:

CREATE TABLE url_daily_stats (
  url_id BIGINT NOT NULL,
  date DATE NOT NULL,
  clicks BIGINT NOT NULL,
  unique_visitors_estimate BIGINT,
  PRIMARY KEY (url_id, date)
);

Postgres can hold daily aggregates if dashboard load is modest. Keep raw click events outside Postgres.

---

5. High-Level Design

Clients
  |
  | POST /v1/urls
  v
API Gateway
  |
  v
URL Creation Service
  |
  v
Postgres Primary
  |
  v
Postgres Read Replicas


Clients
  |
  | GET /{short_code}
  v
Edge / Load Balancer
  |
  v
Redirect Service
  |
  |-- GET short_code --> Redis Cluster
  |                       |
  |                       | cache miss
  |                       v
  |                  Postgres Read Replicas
  |
  |-- async click event --> Kafka
                            |
                            v
                     Analytics Consumers
                            |
                            v
                ClickHouse / Druid / BigQuery
                            |
                            v
                    Analytics API Service

Main components

• API Gateway

  • Handles authentication.
  • Applies rate limits.
  • Routes requests.

• URL Creation Service

  • Validates long URLs.
  • Allocates IDs.
  • Encodes IDs into base62 short codes.
  • Writes mappings to Postgres.

• Postgres Primary

  • Stores URL mappings.
  • Handles URL creation and admin updates.

• Postgres Read Replicas

  • Serve cache misses from the redirect path.

• Redis Cluster

  • Stores hot short_code -> redirect metadata mappings.
  • Protects Postgres from redirect traffic.

• Redirect Service

  • Resolves short codes.
  • Returns 302 responses.
  • Emits click events asynchronously.

• Kafka

  • Buffers click events.
  • Keeps analytics work outside redirects.

• Analytics Consumers

  • Aggregate events by URL, time, referrer, country, and device.

• Analytics Store

  • Stores raw or semi-aggregated click data.

• Analytics API Service

  • Serves dashboards and reports.

Recommended design

Use:

• one Postgres primary for writes
• Postgres read replicas for cache misses
• Redis for hot redirects
• Kafka for analytics

Skip database sharding in the first version. The write load is too small to justify sharding.

---

6. Deep Dives

ID Generation: Auto-Increment, Snowflake, or Pre-Generated Pool

The short code must be unique, compact, and fast to generate.

Default design:

Postgres BIGSERIAL id -> base62(id) -> short_code

Base62 alphabet:

0-9, a-z, A-Z

Capacity:

62^6 ≈ 56.8B
62^7 ≈ 3.5T
62^8 ≈ 218T

At 100M new URLs/day:

100M/day * 365 = 36.5B/year

A 6-character code lasts about 1.5 years.

A 7-character code lasts about 96 years at this traffic.

Use 7 characters by default.

Default answer

Use an auto-incrementing database ID and encode it with base62.

Example:

id = 125
base62(id) = "21"
short_url = https://sho.rt/21

In production, left-pad short codes or apply a reversible permutation so early URLs do not look short or sequential.

The ID must be unique before encoding.

Hashing the long URL

Avoid hash(long_url) as the main ID strategy.

Hashing adds collision handling. It also forces a product decision: should the same long URL always return the same short URL?

Most real systems treat each shortened URL as a separate resource. Two users may shorten the same URL and need separate analytics, ownership, expiration, and abuse controls.

Use hashing only for deduplication checks or abuse detection.

Auto-increment

Pros:

• Simple.
• No collisions.
• Easy to reason about.
• One Postgres primary can handle 1.2K writes/sec.

Cons:

• Sequential IDs reveal approximate creation volume.
• One writer owns ID allocation.

Pick auto-increment by default.

If exposed volume is a concern, apply a reversible permutation before base62 encoding:

id -> permuted_id -> base62(permuted_id)

The permutation preserves uniqueness and avoids random collision handling.

Snowflake IDs

Snowflake-style IDs combine timestamp, machine ID, and sequence number.

Pros:

• Multiple machines can generate IDs without calling the database.
• Works for multi-region active-active writes.

Cons:

• Adds operational complexity.
• Produces larger IDs.
• Requires clock-skew handling.
• The stated 1.2K writes/sec load does not need it.

Use Snowflake only if the interviewer asks for independent ID generation across multiple write regions.

Pre-generated ID pool

A pre-generation service creates random unused codes and stores them in a pool.

Pros:

• URL creation does not wait on ID generation.
• Codes can look random.
• Vanity ranges are easy to reserve.

Cons:

• Adds another durable system.
• Pool exhaustion becomes a failure mode.
• Uniqueness still needs strict enforcement.

Skip this as the default. Use it only when the product requires random-looking codes or reserved code ranges from day one.

Senior answer

Say:

“I’ll use a database sequence and base62 encoding because the write load is small. If enumeration is a concern, I’ll permute the ID before encoding. I’ll avoid hashing the long URL because ownership and analytics make identical long URLs separate resources.”

---

301 vs 302 Redirect

A redirect response affects caching, analytics, and future edits.

301 permanent redirect

A 301 Moved Permanently tells browsers and intermediaries they can cache the redirect.

Pros:

• Faster repeat visits.
• Less load on the redirect service.
• Good for immutable links with best-effort analytics.

Cons:

• The browser may skip your service on later visits.
• Click analytics can be missed.
• Destination changes may not reach users who cached the old redirect.

302 temporary redirect

A 302 Found tells the client to request the short URL again next time.

Pros:

• Every click reaches your service.
• Analytics are accurate.
• Destination changes take effect immediately.
• Expiration and abuse controls work reliably.

Cons:

• More load on the redirect service.
• Less client-side caching.

Default answer

Use 302.

For a Bitly-style system, analytics, abuse handling, expiration, and editable destinations matter more than browser-side caching. Redis and replicas handle the read load.

Response:

302 Found
Location: https://www.example.com/products/123?source=email
Cache-Control: no-store

Use 301 only for an explicitly immutable link mode where reduced repeat traffic matters more than exact click tracking.

Senior answer

Say:

“I’ll default to 302 because it preserves analytics and allows destination changes. I’ll consider 301 only for immutable links where reducing repeat redirect traffic matters more than accurate click tracking.”

---

Analytics via Async Event Pipeline

Redirects should never wait on analytics storage. The user is waiting for the destination page.

Redirect flow:

1. Resolve short_code.
2. Return 302.
3. Emit click event asynchronously.

The service may submit the event before the response is flushed, but the Kafka call needs a tight timeout. Downstream analytics must never block redirect completion.

graph TD
  subgraph Create["Create Short URL"]
    C1["Client"] -->|"POST /shorten"| API["API Server"]
    API --> IDGen["ID Generator\n(auto-increment or snowflake)"]
    IDGen --> Encode["base62(id) = short code"]
    Encode --> PG["Postgres\n(short_code, long_url, created)"]
    API -->|"201"| C1
  end

  subgraph Redirect["Redirect Flow"]
    C2["Browser"] -->|"GET /abc123"| LB["Load Balancer"]
    LB --> API2["API Server"]
    API2 --> Redis["Redis Cache\nhot short codes"]
    Redis -->|"MISS"| PG2["Postgres"]
    API2 -->|"302 redirect"| C2
  end

  subgraph Analytics["Analytics (async)"]
    API2 -->|"click event"| Kafka["Kafka"]
    Kafka --> Workers["Analytics Workers"]
    Workers --> Store["ClickHouse / TSDB"]
  end

Default answer

Use Kafka for click events.

Redirect Service -> Kafka -> Analytics Consumers -> ClickHouse/Druid/BigQuery

At 100B redirects/day:

100B / 86,400 ≈ 1.16M events/sec average
Peak ≈ 2.3M to 3.5M events/sec

Kafka can handle this with a properly sized cluster. Size the cluster from peak traffic, replication factor, retention, and event size.

If each event is about 200 bytes:

100B * 200 bytes = 20 TB/day before replication

With replication factor 3:

≈ 60 TB/day written across brokers

Postgres row-by-row inserts cannot handle this click volume.

Event contents

Keep the event small:

{
  "event_id": "01JZ7VZ3K7...",
  "url_id": 9384729384,
  "short_code": "b7Xk29Q",
  "timestamp": 1778760001000,
  "referrer": "twitter.com",
  "user_agent_hash": "abc123",
  "ip_prefix": "203.0.113.0/24",
  "country": "US"
}

Avoid storing full IP addresses unless the product and privacy requirements require them. Prefer hashes or prefixes.

Aggregation

Consumers produce time-bucketed aggregates:

(url_id, minute) -> clicks
(url_id, day) -> clicks
(url_id, country, day) -> clicks
(url_id, referrer, day) -> clicks

Dashboards should read aggregates.

Backpressure behavior

If Kafka is slow or unavailable, redirects continue.

Recommended behavior:

• Use a short produce timeout, such as 5 ms.
• Keep a small in-memory buffer.
• Drop analytics events when buffers are full.
• Emit metrics on dropped events.
• Alert when the drop rate exceeds a threshold.

For a URL shortener, losing a small percentage of analytics during an incident is better than delaying or failing redirects.

If the business sells strict analytics accuracy, add a local durable log on redirect hosts. That design adds complexity, so skip it unless the interviewer asks for stronger analytics guarantees.

Senior answer

Say:

“Analytics are async. The redirect succeeds even if Kafka is down. I’ll track dropped events and backfill only if the product requires strict analytics accuracy.”

---

Redis Cache for Hot Short Codes

The redirect path is a read-heavy key-value lookup:

short_code -> long_url, status, expires_at

At 100B redirects/day:

100B / 86,400 ≈ 1.16M redirects/sec average
Peak ≈ 2.3M to 3.5M redirects/sec

Postgres should not serve this read load directly. Redis protects the database and keeps redirect latency low.

Default answer

Use Redis Cluster for hot mappings.

Cache key:

url:b7Xk29Q

Cache value:

{
  "url_id": 9384729384,
  "long_url": "https://www.example.com/products/123?source=email",
  "status": "active",
  "expires_at": "2026-12-31T23:59:59Z"
}

Set TTL to 1–24 hours. If the URL is updated or disabled, delete the cache key.

Cache read path

1. Redirect Service receives GET /b7Xk29Q.
2. Check in-process LRU cache.
3. Check Redis.
4. On Redis miss, query Postgres read replica.
5. Populate Redis.
6. Return 302.
7. Emit analytics event.

Use a small in-process LRU cache before Redis. Extremely hot links can route all traffic for one key to one Redis shard. Local caching reduces that hot-key pressure.

Redis sizing

Redis can handle roughly 100K–500K ops/sec per instance, depending on command mix, value size, networking, and CPU.

At peak 3M redirects/sec, use multiple Redis shards and replicas.

A reasonable interview answer:

Redis Cluster with 12–24 nodes total:
- several primary shards for throughput
- replicas for read scaling and failover

Exact node count is less important than sizing Redis by throughput and hot-key behavior.

Cache misses

Cache misses go to Postgres read replicas.

Use negative caching for unknown short codes:

url:not_found:b7Xk29Q -> true, TTL 1-5 minutes

Negative caching prevents random-code scanning from hammering Postgres.

Cache invalidation

For updates:

1. Write new status or destination to Postgres primary.
2. Delete Redis key.
3. Future redirect reloads from Postgres.

TTL alone is too slow for admin changes. If a malicious URL is disabled, redirects must stop quickly.

Senior answer

Say:

“Redis handles hot redirects, Postgres replicas handle misses, and I’ll add in-process caching for hot-key protection. Updates delete cache keys so disabled URLs stop working quickly.”

---

7. Estimation

Spend two or three minutes on estimation in the interview.

Writes

Given:

100M new URLs/day

Average write QPS:

100,000,000 / 86,400 ≈ 1,157 writes/sec

Peak write QPS at 3x:

≈ 3,500 writes/sec

A single Postgres primary can handle 10K–50K TPS on modern hardware. One primary is enough for URL creation.

Reads

Given:

100B redirects/day

Average read QPS:

100,000,000,000 / 86,400 ≈ 1.16M reads/sec

Peak read QPS at 3x:

≈ 3.5M reads/sec

Postgres should not handle this directly. Redis and in-process caches serve the hot path.

Short code length

62^7 ≈ 3.5T codes

At 36.5B URLs/year, 7 characters lasts around 96 years.

Use 7 characters.

Storage for URL mappings

Assume one short_urls row averages about 1 KB after row overhead, indexes, metadata, and long URL storage.

100M rows/day * 1 KB = 100 GB/day
36.5B rows/year * 1 KB = 36.5 TB/year

36.5 TB/year is large, but a carefully operated Postgres deployment can handle it when writes are modest and reads mostly hit Redis.

Use table partitioning by creation time for maintenance, backups, and retention workflows.

Do not shard at 100 GB. Do not shard because the table sounds large.

Shard only when you hit a concrete limit:

• write throughput
• index maintenance
• failover time
• backup time
• storage growth beyond one primary’s safe operating range

For this scale, say:

“I’ll use one Postgres primary with partitioning and read replicas. I won’t shard the write path yet.”

Redirect bandwidth

Assume each redirect response is 500 bytes to 1 KB including headers.

Average:

1.16M/sec * 1 KB ≈ 1.16 GB/sec

Peak:

3.5M/sec * 1 KB ≈ 3.5 GB/sec ≈ 28 Gbps

Modern datacenter networking often supports 25–100 Gbps per machine class. You will run many redirect service instances behind a load balancer. Bandwidth is manageable.

Analytics volume

Assume 200 bytes per click event:

100B events/day * 200 bytes = 20 TB/day

With Kafka replication factor 3:

≈ 60 TB/day written across brokers

Use Kafka retention for short-term buffering. Compact or aggregate into the analytics store.

---

8. Failure Modes and Mitigations

Postgres primary fails

Impact:

• New URL creation fails or stalls.
• Redirects continue if Redis and read replicas still have data.

Mitigation:

• Use managed Postgres or automated failover.
• Keep read replicas in separate availability zones.
• Make URL creation retry-safe with idempotency keys.

Recommendation:

“Redirect availability should not depend on the primary database being up.”

---

Redis outage

Impact:

• Redirect Service falls back to Postgres read replicas.
• Postgres may overload if all cache traffic misses at once.

Mitigation:

• Use Redis Cluster with replicas and automatic failover.
• Keep an in-process LRU cache for the hottest links.
• Rate-limit suspicious traffic.
• Shed analytics before shedding redirects.
• Use circuit breakers to protect Postgres during cache collapse.

Recommendation:

“If Redis is degraded, serve from local cache and replicas. Protect Postgres with rate limits and circuit breakers.”

---

Kafka outage

Impact:

• Analytics events may be delayed or dropped.
• Redirects continue.

Mitigation:

• Use short Kafka timeouts.
• Buffer briefly in memory.
• Drop events under sustained backpressure.
• Alert on drop rate.

Recommendation:

“Kafka failure must not break redirects.”

---

Hot short code

Impact:

• One viral URL can overload one Redis shard or one Redirect Service instance if traffic is uneven.

Mitigation:

• Load balance across Redirect Service instances.
• Add in-process LRU cache.
• Replicate hot keys in local memory.
• Cache immutable redirect metadata at the edge if analytics requirements allow it.

Recommendation:

“Use local cache for hot-key protection. Redis Cluster alone does not solve a single-key hotspot.”

---

Cache stampede

Impact:

• Many requests miss Redis for the same code and hit Postgres at the same time.

Mitigation:

• Use request coalescing per short code inside Redirect Service.
• Use short negative cache entries for misses.
• Add jitter to TTLs.

Recommendation:

“Coalesce misses so only one request reloads a key from Postgres.”

---

Malicious or unsafe URLs

Impact:

• The shortener can become a phishing or malware distribution tool.

Mitigation:

• Allow only http and https.
• Block internal IP ranges to prevent SSRF-style abuse.
• Scan for malware and phishing asynchronously.
• Support fast disable and cache invalidation.

Recommendation:

“Build abuse controls into URL creation, scanning, and fast disable flows.”

---

Code enumeration

Impact:

• Sequential codes allow attackers to discover recently created URLs.

Mitigation:

• Use a reversible permutation before base62.
• Rate-limit unknown-code requests.
• Add private or unlisted access controls if required.

Recommendation:

“If enumeration matters, permute IDs. Keep the database-sequence design and add collision-free obfuscation.”

---

Region outage

Impact:

• A full regional outage can break redirects if all services run in one region.

Mitigation:

• Start with single-region, multi-AZ deployment.
• Move to active-passive multi-region when redirect resilience needs it.
• Replicate Postgres backups and Redis warm data where practical.
• Use DNS or global load balancing for failover.
• Accept disabled URL creation during failover until multi-region writes are required.

Recommendation:

“Start single-region with multi-AZ. Add active-passive multi-region for redirect resilience when needed.”

---

9. What Makes a Strong Answer

L5 signal

An L5 candidate should produce a clean, correct design:

• Uses base62 encoding over a unique numeric ID.
• Avoids hashing as the main uniqueness mechanism.
• Separates URL creation from redirect serving.
• Uses Postgres as source of truth.
• Uses Redis for hot redirect lookups.
• Uses 302 redirects by default.
• Sends analytics through Kafka asynchronously.
• Does basic QPS estimation:
  • 1.2K writes/sec
  • 1.2M reads/sec
• Understands that redirects dominate the system.
• Avoids premature sharding.

A good L5 answer:

“Writes are small enough for one Postgres primary. Reads are huge, so redirects go through Redis and replicas. Analytics are async and must not block the redirect.”

---

L6 signal

An L6 candidate adds operational judgment:

• Explains why 7-character base62 codes are enough.
• Discusses sequential ID exposure and proposes reversible permutation.
• Handles custom alias uniqueness.
• Picks 302 because analytics and destination changes matter.
• Mentions cache invalidation for disabled or updated URLs.
• Adds in-process caching for hot-key protection.
• Handles Redis outage without crushing Postgres.
• Protects the system from cache stampedes.
• Sizes Kafka for 100B click events/day.
• Separates durable mappings from lossy analytics.
• Avoids active-active multi-region writes unless required.
• Talks about abuse, phishing, and fast disable flows.

A strong L6 answer:

“The durable path is simple: Postgres sequence to base62. The hot path is Redis-backed redirect lookup with local caching for viral links. Analytics go through Kafka with backpressure rules that prefer dropping events over delaying redirects. I won’t shard Postgres at this scale; I’ll partition for operations and add replicas for read misses.”