Kubernetes CronJobs and Batch Processing

Kubernetes CronJobs allow you to run tasks on a recurring schedule, similar to traditional cron but with the benefits of containerization, logging, and resource management.

Creating a CronJob

Define a CronJob manifest with a standard cron schedule expression:

apiVersion: batch/v1
kind: CronJob
metadata:
  name: db-backup
spec:
  schedule: "0 2 * * *"
  concurrencyPolicy: Forbid
  successfulJobsHistoryLimit: 3
  failedJobsHistoryLimit: 1
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: backup
            image: postgres:16
            command: ["pg_dump", "-h", "db-host", "-U", "admin", "-d", "mydb", "-f", "/backups/dump.sql"]
            volumeMounts:
            - name: backup-vol
              mountPath: /backups
          restartPolicy: OnFailure
          volumes:
          - name: backup-vol
            persistentVolumeClaim:
              claimName: backup-storage

Apply and Monitor

kubectl apply -f cronjob-backup.yaml
kubectl get cronjobs
kubectl get jobs --watch

One-Off Batch Jobs

For single-run tasks, use a Job instead of a CronJob:

apiVersion: batch/v1
kind: Job
metadata:
  name: data-migration
spec:
  backoffLimit: 3
  template:
    spec:
      containers:
      - name: migrate
        image: myapp:latest
        command: ["python", "migrate.py"]
      restartPolicy: Never

Best Practices

• Set concurrencyPolicy: Forbid to prevent overlapping runs
• Use activeDeadlineSeconds to kill jobs that run too long
• Check logs with kubectl logs job/data-migration
• Set resource requests and limits to prevent runaway batch jobs