RUNNING, READY… AND RETURNING 502
Kipper / Under the hood / 03

Read the tutorial and run each step individually:
https://getkipper.com/learn/troubleshooting-with-kipper/

Use a test cluster with monitoring enabled and an unused routing-lab project.
Replace YOUR_TEST_CLUSTER and YOUR_APP_HOST. Keep the same terminal open.
Wait for each rollout to finish before making requests.
Run the queries in Grafana Explore. Use the shell for the commands.
Cleanup deletes the dedicated tutorial project and all its environments.

Choose a cluster and create the lab
===================================
kip --version
kip cluster list
# Use the exact name of your test cluster from the list.
export KIP_CLUSTER='YOUR_TEST_CLUSTER'
kip platform status
kip project create routing-lab --environments test

Deploy the working app
======================
kip app deploy --name web \
  --image nginx:1.28.0-alpine --port 80 \
  --health http --health-path / --health-port 80 \
  --project routing-lab --environment test

Request the baseline URL
========================
# Copy the app's HTTPS URL from Kipper without a trailing slash.
export APP_URL='https://YOUR_APP_HOST'
curl --silent --show-error --max-time 10 \
  --output /dev/null --write-out 'HTTP %{http_code}\n' \
  "$APP_URL/?lab=baseline"

Introduce the wrong port
========================
kip app deploy --name web \
  --image nginx:1.28.0-alpine --port 8080 \
  --health http --health-path / --health-port 80 \
  --project routing-lab --environment test

Record time and reproduce
=========================
date -u '+%Y-%m-%dT%H:%M:%SZ'
for i in $(seq 1 12); do
  curl --silent --show-error --max-time 10 \
    --output /dev/null --write-out 'HTTP %{http_code}\n' \
    "$APP_URL/?lab=wrong-port"
  sleep 5
done

Inspect in Kipper
=================
kip app list --project routing-lab --environment test
kip app info web --project routing-lab --environment test
kip app logs web --tail 30 --project routing-lab --environment test

Loki: all app containers
========================
{namespace="routing-lab-test", app="web"}

Loki: instance-proxy errors
===========================
{namespace="routing-lab-test", app="web", container="kipper-instance-proxy"} |= "proxy error"

Loki: marked nginx requests
===========================
{namespace="routing-lab-test", app="web", container="web"} |= "lab="

Prometheus: status counts
=========================
sum by (code) (
  increase(traefik_service_requests_total{
    exported_service=~"routing-lab-test-web-.*"
  }[5m])
)

Prometheus: alternative label layout
====================================
sum by (code) (
  increase(traefik_service_requests_total{
    service=~"routing-lab-test-web-.*"
  }[5m])
)

Compare listeners
=================
kip exec web --project routing-lab --environment test -- \
  sh -c 'wget -S -O /dev/null http://127.0.0.1:80/; wget -S -O /dev/null http://127.0.0.1:8080/'

Restore the correct port
========================
kip app deploy --name web \
  --image nginx:1.28.0-alpine --port 80 \
  --health http --health-path / --health-port 80 \
  --project routing-lab --environment test

Verify recovery
===============
for i in $(seq 1 12); do
  curl --silent --show-error --max-time 10 \
    --output /dev/null --write-out 'HTTP %{http_code}\n' \
    "$APP_URL/?lab=recovered"
  sleep 5
done

Delete the tutorial project
===========================
kip project delete routing-lab