IT 3300 : Virtualization

Kubernetes — Observability & Autoscaling

You can't fix what you can't see

  • Clusters are dynamic — pods move, scale, restart
  • Observability = knowing what's happening and why
  • Three pillars: logs, metrics, traces

Built-in debugging

  • kubectl get pods — status, restarts
  • kubectl describe pod <p> — events (why it won't start)
  • kubectl logs <p> — app output (-f to follow, --previous for a crash)
  • kubectl exec -it <p> -- sh — poke around inside

Reading pod status

  • Pending — can't be scheduled (resources? node?)
  • CrashLoopBackOff — starts, crashes, repeats -> check logs
  • ImagePullBackOff — can't pull the image (name/creds?)
  • The status usually tells you where to look

metrics-server

  • Lightweight CPU/memory metrics for the cluster

  • Powers kubectl top and autoscaling

      kubectl top nodes
  kubectl top pods

Prometheus + Grafana

  • Prometheus — scrapes and stores time-series metrics
  • Grafana — dashboards and visualization on top
  • The de facto open-source monitoring stack for Kubernetes
  • Often installed together via a Helm chart (next deck)

Horizontal Pod Autoscaler (HPA)

  • Automatically scale replicas based on load (e.g. CPU)

      kubectl autoscale deployment web --min=2 --max=10 --cpu-percent=70

  • Needs metrics-server and resource requests set

  • Desired state that adjusts itself to demand

Logging at scale

  • kubectl logs is fine for one pod, not a fleet
  • Aggregate logs centrally (e.g. Loki, ELK/EFK stacks)
  • Ship logs off the node so they survive pod deletion

Lab goals

  • Enable metrics-server; use kubectl top
  • Add an HPA and drive load to watch it scale
  • (Stretch) Install Prometheus + Grafana and open a dashboard