ImagePullBackOff

What Is ImagePullBackOff

ImagePullBackOff means:

• Kubernetes tried to pull the container image
• It failed
• It will retry again, but with increasing delay

This is not a registry outage in most cases. Kubernetes is working fine — the image cannot be pulled.

What Happens Internally

1. Pod gets scheduled to a node
2. kubelet tries to pull the image
3. Image pull fails
4. Kubernetes retries with increasing delay
5. Pod shows ImagePullBackOff

Kubernetes is healthy. Image access is the problem.

---

1. First Thing to Check (Always)

kubectl get pods

You will see:

STATUS: ImagePullBackOff

or

STATUS: ErrImagePull

For all namespaces:

kubectl get pods -A | grep ImagePullBackOff

If many pods fail together, it's usually configuration, not Docker Hub.

---

2. Read Pod Events (Most Important Step)

kubectl describe pod <pod-name>

Look only at Events. You will see exact reasons, for example:

• pull access denied
• unauthorized
• manifest unknown
• no such host
• repository does not exist

Read the error literally. Kubernetes is not guessing.

---

3. The Most Common Root Causes

3.1 Wrong Image Name or Tag (Very Common)

Example (Broken)

image: myapp:v1.0.0

But v1.0.0 does not exist.

How to confirm:

docker pull myapp:v1.0.0

If it fails locally → it will fail in Kubernetes.

---

3.2 Private Registry Without Credentials

Registry is private, but Kubernetes has no login details.

Event message:

pull access denied
unauthorized

Fix: Create an image pull secret

kubectl create secret docker-registry regcred \
  --docker-server=<registry-url> \
  --docker-username=<username> \
  --docker-password=<password>

---

3.3 Secret Exists but NOT Attached

This is very common. The secret exists:

kubectl get secret regcred

But the pod does not reference it.

Broken Deployment:

containers:
- name: app
  image: private-registry/myapp:latest

Fixed Deployment:

containers:
- name: app
  image: private-registry/myapp:latest
imagePullSecrets:
- name: regcred

No reference = no authentication.

---

3.4 Wrong Registry URL

Small mistakes break everything.

Examples:

• docker.io/myapp vs myapp
• Missing cloud region
• Wrong hostname

Kubernetes will not auto-correct registry URLs.

---

3.5 Node Has No Internet / Registry Access

This is infrastructure related.

Common causes:

• No outbound NAT
• Firewall blocking registry
• Proxy misconfiguration

To test from inside the cluster, run a temporary debug pod:

kubectl run net-test \
  --image=curlimages/curl:latest \
  --restart=Never \
  --command -- sleep 3600

Exec into it:

kubectl exec -it net-test -- sh

Test registry access:

curl -v https://registry-1.docker.io

Expected success:

• HTTP response (401 / 200 is OK)
• DNS resolves
• TLS handshake succeeds

---

Hands-On Demo

The following examples deliberately introduce failures to show how ImagePullBackOff occurs.

Example 1: Wrong Image Tag

apiVersion: apps/v1
kind: Deployment
metadata:
  name: imagepull-demo
spec:
  replicas: 1
  selector:
    matchLabels:
      app: imagepull-demo
  template:
    metadata:
      labels:
        app: imagepull-demo
    spec:
      containers:
      - name: app
        image: nginx:app

Result: ImagePullBackOff — the tag app does not exist for the nginx image.

---

Example 2: Private Image Without Secret

apiVersion: apps/v1
kind: Deployment
metadata:
  name: private-image-demo
spec:
  replicas: 1
  selector:
    matchLabels:
      app: private-demo
  template:
    metadata:
      labels:
        app: private-demo
    spec:
      containers:
      - name: app
        image: private-registry/myapp:latest
        ports:
        - containerPort: 3000

Result: ImagePullBackOff — pull access denied

Fix: Add imagePullSecrets to the deployment:

spec:
  containers:
  - name: app
    image: private-registry/myapp:latest
    ports:
    - containerPort: 3000
  imagePullSecrets:
  - name: regcred

---

About imagePullPolicy

imagePullPolicy: Always

This forces Kubernetes to pull the image every restart.

In unstable setups:

• Makes failures noisy
• Slows recovery

Use carefully.

---

Quick Debug Checklist

Ask these in sequence:

1. Does the image exist?
2. Can I pull it locally?
3. Is the registry private?
4. Is imagePullSecret created?
5. Is imagePullSecret attached?
6. Can the node reach the registry?

90% of ImagePullBackOff issues are solved by this checklist.

---

How to Recover

After fixing the issue:

kubectl rollout restart deployment/<name>

Verify:

kubectl rollout status deployment/<name>

Status should move: ImagePullBackOff → Running

---

Important Clarification

Status	Meaning
ErrImagePull	Pull failed (first attempt)
ImagePullBackOff	Pull failed repeatedly

They are the same problem, different stages.

ImagePullBackOff means Kubernetes cannot fetch your image — not that the registry is down.

---

Final Truth

• Not a Kubernetes bug
• Usually not a registry outage
• Almost always:
  • Wrong image
  • Missing secret
  • Wrong registry
  • No node internet

Kubernetes is strict — and that's a good thing.