Scalability in System Design

Scalability means the ability of a system to handle increasing users, requests and data without a major degradation in performance.

For example, an application may initially support 1,000 users. Later, the number of users may increase to 100,000 or 10 million.

A scalable system should be able to handle this growth by adding or improving resources.

Why Do We Need Scalability?

Imagine we build an ASP.NET Core application with one server and one SQL Server database.

flowchart LR
    Users --> API
    API --> Database

This may work well when the application has a small number of users.

Now imagine that traffic increases significantly.

The API server may start using high CPU and memory. Database connections may increase. Response times may become slower.

Eventually, requests may start failing.

This is where scalability becomes important.

What Can Increase?

Scalability is not only about users.

Several things can increase:

• Number of users
• Requests per second
• Database records
• File storage
• Background jobs
• Network traffic

Our architecture should be prepared for these changes.

Two Main Ways to Scale

There are two common approaches:

• Vertical Scaling
• Horizontal Scaling

Vertical Scaling

Vertical scaling means increasing the capacity of an existing server.

For example, suppose our application server has:

• 4 CPU cores
• 16 GB RAM

We can increase it to:

• 16 CPU cores
• 64 GB RAM

flowchart LR
    Users --> Server
    Server --> Database

The number of servers has not changed. We have increased the capacity of the existing server.

Horizontal Scaling

Horizontal scaling means adding more servers.

Instead of having one API server, we can have multiple API servers.

flowchart LR
    Users --> LoadBalancer
    LoadBalancer --> API1
    LoadBalancer --> API2
    LoadBalancer --> API3
    API1 --> Database
    API2 --> Database
    API3 --> Database

The Load Balancer distributes incoming requests across the API servers.

If traffic increases, we can add more API servers.

Real-World Example

Imagine an e-commerce application.

During a normal day, it receives 1,000 requests per second.

During a major sale, traffic increases to 20,000 requests per second.

If we have only one application server, it may become overloaded.

With horizontal scaling, we can run multiple API servers.

flowchart LR
    Users --> LoadBalancer
    LoadBalancer --> API1
    LoadBalancer --> API2
    LoadBalancer --> API3
    LoadBalancer --> API4
    API1 --> Database
    API2 --> Database
    API3 --> Database
    API4 --> Database

Now requests can be distributed across multiple servers.

ASP.NET Core Example

Suppose we have a simple API:

[ApiController]
[Route("api/products")]
public class ProductsController : ControllerBase
{
    [HttpGet]
    public IActionResult GetProducts()
    {
        return Ok(new[]
        {
            "Laptop",
            "Mobile",
            "Monitor"
        });
    }
}

This API can run on one server.

With horizontal scaling, the same application can run on multiple servers:

flowchart LR
    User --> LoadBalancer
    LoadBalancer --> Server1
    LoadBalancer --> Server2
    LoadBalancer --> Server3

All three servers can run the same ASP.NET Core application.

What About SQL Server?

Scaling the API servers does not automatically solve the database problem.

For example:

flowchart LR
    LoadBalancer --> API1
    LoadBalancer --> API2
    LoadBalancer --> API3

    API1 --> Database
    API2 --> Database
    API3 --> Database

Now three API servers are sending requests to the same database.

If the database is already under heavy load, adding more API servers may make the database problem worse.

This is an important System Design concept:

Scaling one component can move the bottleneck to another component.

Database Scalability

For SQL Server, depending on the workload, we may consider approaches such as:

• Proper indexing
• Query optimization
• Connection management
• Read replicas
• Partitioning
• Archiving old data
• Caching frequently accessed data

For example, suppose we frequently query employees by Department.

SELECT Id, Name, Department
FROM Employees
WHERE Department = 'IT';

An appropriate index can reduce the amount of data SQL Server needs to scan.

CREATE INDEX IX_Employees_Department
ON Employees(Department);

This is one simple way to improve database performance as data grows.

Scaling With Cache

Suppose thousands of users request the same product information.

If every request goes directly to SQL Server, the database receives unnecessary repeated queries.

A cache can reduce this load.

flowchart LR
    User --> API
    API --> Cache
    Cache --> Database

The API first checks the cache.

If the data exists, it can return it without querying SQL Server.

If the data does not exist, the API reads it from SQL Server and stores it in the cache.

Scaling Is Not Just Adding Servers

A common mistake is to think:

"If traffic increases, just add more servers."

In a real system, we need to identify the bottleneck.

The bottleneck could be:

• CPU
• Memory
• Database
• Network
• External API
• Disk
• Connection pool
• Slow queries

For example, adding 10 API servers will not solve a database query that takes 5 seconds.

How to Think About Scalability

Consider this system:

flowchart LR
    Users --> LoadBalancer
    LoadBalancer --> API
    API --> Cache
    API --> Database
    API --> ExternalService

We should ask:

• Can the API scale horizontally?
• Can the cache handle more requests?
• Can the database handle more traffic?
• What happens if the external service becomes slow?
• What happens when the number of users increases ten times?

These questions help us identify scalability requirements.

Simple Example

Suppose our application currently handles:

• 10,000 users
• 1,000 requests per second

After six months, we expect:

• 1 million users
• 10,000 requests per second

We cannot simply assume that the current architecture will handle this growth.

We need to design the system so that capacity can increase as demand increases.

Key Takeaways

• Scalability means handling increasing workload without unacceptable degradation.
• Vertical scaling increases the capacity of an existing server.
• Horizontal scaling adds more servers.
• Load Balancers are commonly used with horizontally scaled application servers.
• Scaling the application layer does not automatically solve database bottlenecks.
• Caching can reduce repeated database requests.
• Indexes and query optimization are important for database scalability.
• Always identify the actual bottleneck before deciding how to scale.

Conclusion

A scalable system is designed with future growth in mind.

When users and traffic increase, the system should have a way to increase capacity without completely redesigning the application.

In real-world systems, scalability usually involves multiple layers such as the API, database, cache, storage and external services.

The next important question is: What happens if one of these components goes down?

That leads us to the next System Design concept: Availability.