Retry Pattern 🔁 — Handling Temporary Failures

The Retry Pattern automatically attempts a failed operation again when the failure may be temporary, such as a network interruption or a temporarily unavailable service.

sequenceDiagram
    participant A as Service A
    participant B as Service B

    A->>B: Request
    B-->>A: Temporary Failure

    A->>B: Retry
    B-->>A: Success

1. Why Do We Need Retry?

Distributed systems can experience temporary failures caused by network issues, transient service failures, connection problems, or short periods of overload.

flowchart LR
    A["Service A"] --> B["Service B"]
    B --> X["Temporary Failure"]
    A --> R["Retry"]
    R --> B

2. Simple Retry

A failed request can be attempted again a limited number of times.

flowchart LR
    R["Request"] --> B["Service"]

    B -->|"Failure"| R1["Retry 1"]
    R1 -->|"Failure"| R2["Retry 2"]
    R2 -->|"Success"| S["Success"]

Retries should have a maximum attempt limit. Retrying forever can make the problem worse.

3. Exponential Backoff

Instead of retrying immediately every time, the delay increases after each failure.

Retry 1 → 100 ms
Retry 2 → 200 ms
Retry 3 → 400 ms
Retry 4 → 800 ms

flowchart LR
    F["Failure"] --> D1["100 ms"]
    D1 --> R1["Retry"]
    R1 --> D2["200 ms"]
    D2 --> R2["Retry"]
    R2 --> D3["400 ms"]
    D3 --> R3["Retry"]

4. Jitter

If thousands of clients retry at exactly the same time, they can create another traffic spike. Jitter adds randomness to the retry delay.

flowchart LR
    F["Failure"] --> B["Exponential Backoff"]
    B --> J["Add Jitter"]
    J --> R["Retry at Variable Time"]

5. Which Errors Should Be Retried?

Retry is generally appropriate for transient failures. Permanent failures should normally not be retried.

6. Retry + Circuit Breaker

Retry and Circuit Breaker are often used together. Retry handles temporary failures, while Circuit Breaker prevents repeated calls when a dependency is consistently failing.

flowchart LR
    A["Service A"] --> R["Retry"]
    R --> CB["Circuit Breaker"]
    CB --> B["Service B"]

    B --> X["Repeated Failure"]

    CB --> F["Fast Failure"]

7. Retry Storm

Aggressive retries from many clients can increase traffic against an already struggling service. This is sometimes called a retry storm.

flowchart LR
    C1["Client 1"] --> R["Retries"]
    C2["Client 2"] --> R
    C3["Client 3"] --> R
    CN["Many Clients"] --> R

    R --> B["Overloaded Service"]

Exponential backoff, jitter, retry limits, timeouts, and circuit breakers help control this problem.

8. Idempotency

Before retrying an operation that changes data, consider whether executing it multiple times can create duplicate effects.

sequenceDiagram
    participant C as Client
    participant API as Payment API

    C->>API: Payment + Idempotency Key
    API-->>C: Timeout

    C->>API: Retry with Same Key
    API-->>C: Existing Result

For operations such as payments or order creation, an idempotency key can prevent duplicate processing.

Important Interview Questions

• What is the Retry Pattern?
• Why should we not retry every failure?
• What is exponential backoff?
• What is jitter and why is it needed?
• What is a retry storm?
• How many retries should you allow?
• Retry vs Circuit Breaker?
• Why is idempotency important when retrying payments?
• Which failures should be retried?

Key Takeaways

• Retry is useful for temporary failures.
• Always use a maximum retry count.
• Use exponential backoff instead of immediate repeated retries.
• Use jitter to avoid synchronized retry traffic.
• Do not blindly retry permanent failures.
• Use idempotency for retryable operations that modify data.
• Retry and Circuit Breaker work well together.

Conclusion

For interviews, remember: Temporary Failure → Retry → Backoff → Jitter → Success or Stop.