Automating Background Work: Jobs & CronJobs
Most of the time, when we deploy an application to Kubernetes using a Deployment, we want it to run forever. We want our web servers and APIs to stay alive, listening for user requests. But what happens when you need to run a task that is meant to finish? Things like database migrations, nightly data backups, sending out a batch of emails, or generating a daily report.
If you use a standard Deployment for these tasks, Kubernetes will see the container stop when it finishes its work, assume it crashed, and immediately restart it. This creates an endless loop of completed tasks restarting. To solve this, Kubernetes provides two specific resources designed purely for run-to-completion workloads: Jobs and CronJobs.
- The architectural difference between Deployments, Jobs, and CronJobs.
- How to write and deploy a one-time Kubernetes Job for background processing.
- How to schedule recurring tasks using Kubernetes CronJobs.
- How to manage concurrency, failure limits, and execution deadlines to prevent runaway processes.
1. Deployments vs. Jobs vs. CronJobs
Before we write any YAML, we need to understand exactly which tool to use for the job. Using the wrong resource type can lead to system instability, such as a database backup script running hundreds of times a minute because Kubernetes thinks it is supposed to be a permanent web server.
| Resource Type | Primary Use Case | Expected Behavior on Completion |
|---|---|---|
| Deployment | Web servers, APIs, always-on microservices. | RestartPolicy: Always. If it stops, it is considered broken and gets restarted immediately. |
| Job | One-off scripts, database migrations, manual batch processing. | RestartPolicy: Never / OnFailure. It runs its logic, succeeds, and enters a "Completed" state permanently. |
| CronJob | Nightly backups, scheduled reports, routine maintenance. | It sits quietly and acts like an alarm clock. On a schedule, it creates a Job (which then creates a Pod). |
2. Creating a One-Time Job
Let's create a simple Job. Imagine we have a script that computes the value of pi to 2,000 decimal places and then stops. We want Kubernetes to run this calculation once.
The Basic Job YAML
Example 1: math-job.yaml
apiVersion: batch/v1
kind: Job
metadata:
name: pi-calculator
spec:
template:
spec:
containers:
- name: pi
image: perl:5.34.0
command: ["perl", "-Mbignum=bpi", "-wle", "print bpi(2000)"]
restartPolicy: Never
backoffLimit: 4
Expected output when checking the cluster
$ kubectl apply -f math-job.yaml job.batch/pi-calculator created $ kubectl get jobs NAME COMPLETIONS DURATION AGE pi-calculator 1/1 4s 10s $ kubectl get pods NAME READY STATUS RESTARTS AGE pi-calculator-8abc1 0/1 Completed 0 10s
restartPolicy: Never. This is mandatory for Jobs. Also, backoffLimit: 4 tells Kubernetes to retry the job up to 4 times if the script crashes before permanently marking the Job as failed.3. Scheduling Tasks with CronJobs
A Job is great for one-off tasks, but usually, background work needs to happen on a schedule. A CronJob is simply a controller that creates Jobs based on a time schedule using standard Unix Cron format.
The CronJob YAML
Example 2: backup-cronjob.yaml
apiVersion: batch/v1
kind: CronJob
metadata:
name: daily-db-backup
spec:
schedule: "0 2 * * *" # Runs at 2:00 AM every day
concurrencyPolicy: Forbid
successfulJobsHistoryLimit: 3
failedJobsHistoryLimit: 1
jobTemplate:
spec:
template:
spec:
containers:
- name: backup
image: my-backup-image:v1
command: ["/bin/sh", "-c", "echo 'Backing up database...'; sleep 10"]
restartPolicy: OnFailure
concurrencyPolicy: Forbid is a lifesaver. If your 2:00 AM backup takes an unusually long time and is still running at 2:00 AM the next day, this setting prevents Kubernetes from starting a second backup at the same time, which could lock your database.4. The Architecture / Workflow Visualized
To truly master Kubernetes, you need to understand the controller hierarchy. A CronJob does not run your code. A CronJob manages Jobs. A Job does not run your code. A Job manages Pods. The Pod runs your code.
Kubernetes CronJob Execution Flow
The architectural flow: The system clock triggers the CronJob controller, which generates a unique Job resource. The Job spins up a Pod to do the heavy lifting, monitors it for success, and leaves it in a "Completed" state for logging.
5. Step-by-Step Execution Breakdown
Let's take a granular look at exactly what happens inside the cluster when a CronJob triggers.
daily-db-backup-1698710400).jobTemplate. The Pod pulls the image and runs the script.successfulJobsHistoryLimit (which we set to 3), Kubernetes will automatically delete older completed Jobs to keep your cluster clean.backoffLimit.6. Hands-On Practice
The best way to understand Jobs is to run them in a real or local cluster (like Minikube, kind, or Docker Desktop).
Write a simple Job YAML that spins up an Alpine Linux container, echoes "Hello Kubernetes Jobs", and exits.
Hint: Remember to use `command: ["/bin/sh", "-c", "echo 'Hello Kubernetes Jobs'"]` and set `restartPolicy: Never`.
Create a CronJob that runs every 5 minutes and prints the current date to the console.
Expected result: You should see a new Job generated every 5 minutes when you run kubectl get jobs. The schedule expression you need is "*/5 * * * *".
Sometimes background tasks get stuck in infinite loops. Add the activeDeadlineSeconds parameter to a Job specification so that Kubernetes forcefully terminates the job if it runs for more than 30 seconds.
Hint: activeDeadlineSeconds: 30 is placed at the same indentation level as your template inside the Job spec.
7. Interview Questions
If you are interviewing for a DevOps, Site Reliability Engineer (SRE), or Backend Engineering role, you will likely encounter questions about background processing in Kubernetes.
Answer: A Deployment ensures a specific number of replicas are always running for stateless, continuous services. A DaemonSet ensures exactly one copy of a Pod runs on every single node in the cluster (useful for logging or monitoring agents). A Job runs one or more Pods to successfully complete a specific, finite task and then stops.
Answer: You must configure the concurrencyPolicy in the CronJob spec. By default, it is set to Allow. I would change it to Forbid. This tells the CronJob controller to skip the new schedule run if the previous Job hasn't finished. Alternatively, you could use Replace to kill the old, hanging job and start the new one.
Conclusion
Automating background work is a cornerstone of maintaining healthy, scalable systems. By leveraging Kubernetes Jobs and CronJobs, you shift the burden of scheduling, retrying failures, and logging task completions from your application code directly onto the cluster's control plane.
Always remember to define your restartPolicy, strictly manage your concurrencyPolicy for CronJobs, and clean up your history limits. Mastering these nuances separates junior operators from senior architects.
Official Documentation
Upgradix by Shrash Studio – Learn Technology. Build Your Future.