What is Caching?

Caching means storing frequently accessed data in a faster storage layer so that future requests can be served faster without repeatedly accessing the primary database or another slower system.

The simplest idea is:

Without Cache:

User → API → Database → Response

With Cache:

User → API → Cache → Response
              ↓
           Database
           only on miss

For system design interviews, remember:

Cache = faster reads + lower database load + better scalability.

1. Why Do We Need Caching?

Suppose an e-commerce application receives 10,000 requests per second.

Assume users frequently request the same product:

GET /products/1001

Without caching, every request may hit SQL Server.

flowchart LR
    U[Users] --> API[ASP.NET Core API]
    API --> DB[SQL Server]

    U1[Request 1] --> DB
    U2[Request 2] --> DB
    U3[Request 3] --> DB
    U4[Request 4] --> DB

This can cause:

• High database CPU
• High database connections
• Higher response latency
• More disk I/O
• Reduced scalability

2. With Cache

We place a cache between the application and database.

flowchart LR
    U[User] --> API[ASP.NET Core API]
    API --> C[Cache]
    C -->|Cache Miss| DB[SQL Server]

    C -->|Cache Hit| API
    DB --> C

The first request may go to SQL Server.

The following requests can come directly from cache.

Request 1
API → Cache → MISS → Database
                  ↓
                Store
                in Cache

Request 2
API → Cache → HIT → Response

Request 3
API → Cache → HIT → Response

3. Cache Hit vs Cache Miss

Cache Hit

The requested data already exists in cache.

flowchart LR
    A[API] --> B[Cache]
    B -->|HIT| C[Return Cached Data]
    C --> A

Example:

GET /products/1001

Cache:
product:1001 → Product data

Result:
CACHE HIT

Cache Miss

The requested data does not exist in cache.

flowchart LR
    A[API] --> B[Cache]
    B -->|MISS| C[SQL Server]
    C --> D[Store in Cache]
    D --> A

Example:

GET /products/2000

Cache:
product:2000 → Not Found

Result:
CACHE MISS

4. Complete SQL Server Example

Let's create a simple product database.

CREATE DATABASE CacheDemo;
GO

USE CacheDemo;
GO

CREATE TABLE Products
(
    ProductId INT PRIMARY KEY,
    ProductName VARCHAR(100),
    Price DECIMAL(10,2),
    Stock INT
);
GO

INSERT INTO Products
(
    ProductId,
    ProductName,
    Price,
    Stock
)
VALUES
(1, 'Laptop', 75000, 20),
(2, 'Mobile', 30000, 50),
(3, 'Headphones', 3000, 100);
GO

SELECT ProductId, ProductName, Price, Stock
FROM Products
ORDER BY ProductId;
GO

Expected Output

ProductId  ProductName  Price      Stock
---------  -----------  ---------  -----
1          Laptop       75000.00   20
2          Mobile       30000.00   50
3          Headphones   3000.00    100

5. ASP.NET Core Cache Example

For a simple application, we can use IMemoryCache.

builder.Services.AddMemoryCache();

var app = builder.Build();

app.MapGet("/products/{id}", async (
    int id,
    IMemoryCache cache,
    AppDbContext db) =>
{
    string key = $"product:{id}";

    if (cache.TryGetValue(key, out Product? product))
    {
        return Results.Ok(product);
    }

    product = await db.Products
        .FirstOrDefaultAsync(x => x.ProductId == id);

    if (product == null)
        return Results.NotFound();

    cache.Set(
        key,
        product,
        TimeSpan.FromMinutes(10));

    return Results.Ok(product);
});

app.Run();

Request:

GET /products/1

First request:

Cache MISS
    ↓
SQL Server
    ↓
Product loaded
    ↓
Stored in cache
    ↓
Response

Second request:

Cache HIT
    ↓
Return cached product

6. Why Not Cache Everything?

Cache is not a replacement for the database.

flowchart LR
    A[Application] --> C[Cache]
    A --> DB[Primary Database]

    C -->|Fast temporary data| A
    DB -->|Durable source of truth| A

The database normally remains the source of truth.

Cache is used for data that is:

• Frequently read
• Expensive to calculate or retrieve
• Acceptable to temporarily serve from cache
• Not required to be perfectly fresh for every request

7. What Should We Cache?

Good candidates include:

• Product catalog
• User profile information
• Configuration
• Frequently accessed reference data
• Popular articles
• Frequently requested API responses
• Session-related data where appropriate

Usually Avoid Caching Without Careful Design

• Highly sensitive information
• Frequently changing data
• Data requiring strict real-time consistency
• Very large objects
• Data with poor cache reuse

8. Cache Read Flow

sequenceDiagram
    participant U as User
    participant A as API
    participant C as Cache
    participant D as SQL Server

    U->>A: GET /products/1
    A->>C: GET product:1

    alt Cache Hit
        C-->>A: Product
        A-->>U: Response
    else Cache Miss
        C-->>A: Not Found
        A->>D: SELECT Product
        D-->>A: Product
        A->>C: SET product:1
        A-->>U: Response
    end

9. Cache Key

Every cached item needs a key.

product:1
product:2
product:3

user:100
user:101

order:5001

A good key should be:

• Unique
• Predictable
• Consistent
• Easy to invalidate

For example:

product:{productId}

10. Cache Can Reduce Database Load

Suppose we have:

10,000 requests/second

Without cache:
10,000 database reads/second

With 90% cache hit ratio:

10,000 × 10% = 1,000 database reads/second

The cache absorbs approximately 9,000 requests.

flowchart LR
    U[10000 Requests/sec] --> C[Cache]

    C -->|90% Hit| H[9000 Cached Responses]
    C -->|10% Miss| DB[1000 Database Reads]

This is one of the main reasons caching is important in system design.

11. Cache Hit Ratio

Cache Hit Ratio tells us how many requests are served from cache.

Cache Hit Ratio =
Cache Hits / Total Requests × 100

Example:

Total requests = 10,000
Cache hits      = 9,000

Hit Ratio = 9000 / 10000 × 100

Hit Ratio = 90%

A high hit ratio generally means the cache is effectively serving repeated requests, although the desired ratio depends heavily on the workload.

12. In-Memory Cache vs Distributed Cache

In-Memory Cache

Cache exists inside the application server.

flowchart LR
    U[User] --> A1[API Server 1]
    U --> A2[API Server 2]

    A1 --> C1[Local Memory Cache]
    A2 --> C2[Local Memory Cache]

    A1 --> DB[Database]
    A2 --> DB

Problem:

Each server has its own cache.

Server 1:
product:1 → cached

Server 2:
product:1 → not cached

Distributed Cache

A shared cache is used by multiple application servers.

flowchart LR
    U[Users] --> LB[Load Balancer]

    LB --> A1[API Server 1]
    LB --> A2[API Server 2]
    LB --> A3[API Server 3]

    A1 --> R[Distributed Cache]
    A2 --> R
    A3 --> R

    R --> DB[SQL Server]

Now all API servers can access the same cached data.

13. Redis

Redis is a popular in-memory data store frequently used as a distributed cache.

flowchart LR
    U[Users] --> LB[Load Balancer]
    LB --> API1[API Server 1]
    LB --> API2[API Server 2]
    LB --> API3[API Server 3]

    API1 --> R[Redis]
    API2 --> R
    API3 --> R

    R --> DB[SQL Server]

Example Redis keys:

product:1
product:2
user:100
configuration:payment

Example logical operation:

SET product:1 "{...}" EX 600

GET product:1

EX 600 means the key expires after 600 seconds.

14. ASP.NET Core + Redis Example

A typical ASP.NET Core application can use Redis through IDistributedCache.

builder.Services.AddStackExchangeRedisCache(options =>
{
    options.Configuration = "localhost:6379";
    options.InstanceName = "MyApp:";
});

app.MapGet("/products/{id}", async (
    int id,
    IDistributedCache cache,
    AppDbContext db) =>
{
    string key = $"product:{id}";

    var cached = await cache.GetStringAsync(key);

    if (cached != null)
    {
        return Results.Content(
            cached,
            "application/json");
    }

    var product = await db.Products
        .FirstOrDefaultAsync(x => x.ProductId == id);

    if (product == null)
        return Results.NotFound();

    var json = JsonSerializer.Serialize(product);

    await cache.SetStringAsync(
        key,
        json,
        new DistributedCacheEntryOptions
        {
            AbsoluteExpirationRelativeToNow =
                TimeSpan.FromMinutes(10)
        });

    return Results.Content(
        json,
        "application/json");
});

15. Important Cache Problems

Caching improves performance, but introduces additional system-design problems.

flowchart TD
    A[Caching] --> B[Cache Invalidation]
    A --> C[TTL]
    A --> D[Eviction]
    A --> E[Cache Stampede]
    A --> F[Cache Penetration]
    A --> G[Cache Avalanche]
    A --> H[Consistency]
    A --> I[Distributed Lock]

These topics are covered individually in the following caching articles.

16. Cache vs Database

17. Real-World Scenario — Product Catalog

Suppose an e-commerce site has:

10 million products
100,000 requests/sec
Most users view popular products

Putting every request directly on SQL Server would create unnecessary database load.

Instead:

flowchart LR
    U[Users] --> CDN[CDN]
    CDN --> API[ASP.NET Core API]

    API --> R[Redis Cache]

    R -->|Hit| API
    R -->|Miss| DB[SQL Server]

    DB --> R

Popular products remain in Redis and are served quickly.

SQL Server handles cache misses and remains the source of truth.

18. Cache Is Not Always Better

Adding a cache introduces another component.

flowchart LR
    A[Application] --> C[Cache]
    C --> DB[Database]

    C --> X[Cache Failure]
    X --> Y[Fallback / Database Load]
    Y --> DB

If Redis fails and all traffic suddenly reaches SQL Server, the database can become overloaded.

Therefore a production design should consider:

• Cache availability
• Cache cluster/replication
• Fallback behavior
• Database protection
• TTL
• Eviction
• Monitoring

19. Interview Scenario

Interviewer: Your API receives 50,000 requests per second, but 80% of requests ask for the same product information. What would you do?

Answer:

Use a cache.

Request
   ↓
Redis
   ↓
Cache Hit → Return immediately

Cache Miss
   ↓
SQL Server
   ↓
Store result in Redis
   ↓
Return response

This reduces repeated database reads and improves response latency.

20. Common Interview Questions

Q1. What is caching?

Caching stores frequently accessed data in a faster storage layer so future requests can be served faster.

Q2. Why use cache?

To reduce latency, reduce database load, and improve application scalability.

Q3. What is cache hit?

The requested data exists in cache and can be returned directly.

Q4. What is cache miss?

The requested data does not exist in cache, so the application retrieves it from the source and may populate the cache.

Q5. Is cache the source of truth?

Usually no. The primary database or another authoritative system generally remains the source of truth.

Q6. In-memory cache vs distributed cache?

In-memory cache exists inside an application instance. Distributed cache is shared across multiple application instances.

Q7. Why use Redis?

Redis provides fast in-memory data access and supports distributed caching, expiration, replication, clustering, and other useful capabilities.

Q8. What happens if cache goes down?

The application needs a fallback strategy. It may read from the database, but the design must prevent a sudden cache-miss storm from overwhelming the database.

21. Key Takeaways

• Cache = faster access to frequently used data.
• Cache reduces database load and application latency.
• Cache Hit means data was found in cache.
• Cache Miss means data must be retrieved from the source.
• The database normally remains the source of truth.
• In-memory cache belongs to one application instance.
• Distributed cache can be shared by multiple application instances.
• Redis is a common distributed cache.
• Cache keys should be unique and predictable.
• Cache hit ratio is an important performance metric.
• Caching introduces challenges such as invalidation, TTL, eviction, stampede, penetration, and avalanche.

Conclusion

The simplest caching architecture to remember for interviews is:

flowchart LR
    U[User] --> API[API]
    API --> C[Cache]

    C -->|HIT| R[Fast Response]

    C -->|MISS| DB[Database]
    DB --> C
    C --> R

Key interview sentence:

“Caching stores frequently accessed data in a faster storage layer such as Redis. The application checks the cache first; on a cache hit it returns the cached value, while on a cache miss it reads from the database and populates the cache. This reduces database load, improves latency, and helps the application scale.”