LATEST 🚀 Welcome to Upgradix by Shrash Studio – Your destination for free technology learning and career growth.
🕘 Updated:
Python

Python - System Design & Performance Tuning

System design and performance tuning involve finding slow bottlenecks with profilers, storing fast responses in Redis caches, and scaling backend services horizontally to handle high traffic.

Python System Design and Performance Tuning – Profiling, Caching, and Scaling Backends

Quick Answer: System design and performance tuning involve finding slow bottlenecks with profilers, storing fast responses in Redis caches, and scaling backend services horizontally to handle high traffic.

Imagine Ravi manages the online ticket booking portal for a major festival in Bengaluru.

Suddenly, 50,000 users hit the website at once, and the backend server starts freezing up.

How do we profile our Python code, find the slow queries, and scale the architecture to handle the rush?

1. Why Do We Need Profiling and Performance Tuning?

The problem with unoptimized backend code

We could write simple Python endpoints, but without profiling, we do not know why requests take three seconds to respond.

Example 1: A slow unoptimized database query endpoint

This endpoint runs fine for one user, but crashes under high concurrent load.

from fastapi import FastAPI
import time

app = FastAPI()

@app.get("/tickets")
def get_tickets():
    # Simulating a slow blocking database query
    time.sleep(2.0)
    return {"status": "success", "count": 150}

Expected output

{"status": "success", "count": 150}

This code works for a single user, but it has major production limitations.

  1. Blocking I/O pauses the request thread for two full seconds.
  2. Database hits repeat on every single request instead of using cache.
  3. Server concurrency drops rapidly under high traffic loads.

We need a systematic way to measure performance and cache repeated data.

The solution: profiling and caching

Bottleneck = a slow part of code that restricts overall system speed.

Caching = storing frequently accessed data in fast memory like Redis.

Profiling = measuring execution time and memory usage of functions.

High-Performance Request Flow in Ravi's Portal
How caching and profiling protect the backend from traffic spikes
Incoming Client Request 50,000 concurrent users
Checked Via
Redis Cache Fast in-memory lookup
Profiler Tools cProfile & timeit
Database PostgreSQL / MS SQL
Client Entry Active Check Fast Response

Figure 1: Traffic is intercepted by Redis cache before hitting heavy backend databases.

2. Profiling Python Code with cProfile

Profiling = measuring function execution times to pinpoint exact slow lines.

Let us use Python's built-in cProfile module to analyze execution bottlenecks.

Example 2: Running cProfile on a data processing function

This script profiles execution time and outputs function call statistics.

import cProfile
import time

def heavy_computation():
    total = 0
    for i in range(1000000):
        total += i
    return total

def main():
    heavy_computation()

if __name__ == "__main__":
    cProfile.run("main()")

Expected output

      2 function calls in 0.065 seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.065    0.065    0.065    0.065 script.py:4(heavy_computation)
        1    0.000    0.000    0.065    0.065 script.py:10(main)

Notice how cProfile shows exactly how many seconds each function took to run.

We can optimize only the specific functions flagged as slow.

3. Implementing Caching Strategies

Caching = saving results of expensive operations so future requests load instantly.

Let us implement an in-memory LRU cache using Python's functools module.

Example 3: Using lru_cache for fast function results

This decorator prevents recalculating results for identical inputs.

from functools import lru_cache
import time

@lru_cache(maxsize=128)
def get_festival_pass(pass_id):
    time.sleep(1.0) # Simulate slow DB fetch
    return {"pass_id": pass_id, "status": "Confirmed"}

if __name__ == "__main__":
    start = time.time()
    print(get_festival_pass(101)) # Slow first call
    print($"First call time: {time.time() - start:.2f}s")
    
    start2 = time.time()
    print(get_festival_pass(101)) # Instant cached call
    print($"Second call time: {time.time() - start2:.2f}s")

Expected output

{'pass_id': 101, 'status': 'Confirmed'}
First call time: 1.00s
{'pass_id': 101, 'status': 'Confirmed'}
Second call time: 0.00s

The first call takes one second, while the second call returns instantly from memory.

Comparing Unoptimized Code vs. Optimized Caching & Profiling
Unoptimized System Optimized System Design
Blocking database queries on every request In-memory LRU and Redis distributed caching
Unknown bottlenecks slowing down endpoints Precise function profiling with cProfile
Single server bottlenecking under heavy load Horizontal scaling with asynchronous workers

The important terms to remember

  • Profiling = measuring execution speed and resource consumption.
  • Caching = storing computed results for rapid retrieval.
  • Horizontal Scaling = adding more servers to distribute user traffic.
  • Asynchronous I/O = handling multiple requests concurrently without blocking.

Design tip: Always profile your code before optimizing, because guessing bottlenecks leads to wasted effort.

4. Common Mistakes in Backend Optimization

1. Caching data that changes constantly

Caching highly volatile data like live seat availability leads to outdated user information.

Better approach: Set short Time-To-Live (TTL) expiration rules on fast-moving cache keys.

5. Senior-Level System Design Interview Questions

Question 1: How would you scale a Python API to handle 100,000 requests per second?

What the interviewer expects: A complete architectural breakdown including load balancing, caching layers, asynchronous workers, and database read replicas.

Strong answer: "To handle 100,000 requests per second, we deploy an Nginx load balancer in front of multiple FastAPI application instances running asynchronously. We place a Redis cluster in front of our databases to cache frequent reads, and we use message queues like RabbitMQ or Celery for heavy background tasks."

What interviewers expect at different levels

  • Junior developer: Suggests buying a bigger server with more RAM.
  • Senior developer: Proposes caching, database indexing, and asynchronous workers.
  • Technical architect: Designs distributed caching layers, rate limiting, and fault-tolerant microservices.

6. Practice Exercises

Beginner: Function timer decorator

Write a custom Python decorator that measures and prints the execution time of any function.

Expected result: Console outputs execution duration in milliseconds.

Intermediate: Redis cache integration

Connect a Python script to a local Redis instance and cache user profile lookups with a 60-second expiration.

Hint: Use the redis-py library and set expiration using ex=60.

Advanced: Rate limiter middleware

Build custom FastAPI middleware that restricts clients to a maximum of 10 requests per minute using Redis counters.

Challenge: Return an HTTP 429 Too Many Requests status when the limit is exceeded.

7. Frequently Asked Questions (FAQ)

Does Python's GIL prevent multi-threading performance gains?

Yes. The Global Interpreter Lock (GIL) allows only one thread to execute Python bytecode at a time, making multi-threading ineffective for CPU-bound tasks. Use multiprocessing or async instead.

When should I choose Redis over in-memory LRU cache?

Use in-memory LRU cache for single-process function results, and use Redis when caching needs to be shared across multiple horizontal server instances.

8. Next Steps

Now that you have mastered system design and performance tuning, you are ready to explore automated testing and production deployment in the next module.

Comments & Ratings (0)

Log in or create a student account to comment and rate this article.

Be the first to comment on this article.