Make the
model real.
Break a workload. Trace a request. Recover the data. Take each idea into a real cluster you can explore, reset, and understand.
Your local workbench
Set up a three-node lab cluster on macOS or Windows with WSL 2.
Your local workbench
Set up a three-node lab cluster on macOS or Windows with WSL 2.Install the tools for your machine first. Both setups use Docker Desktop, Bash, Python 3, kind v0.33.0, and kubectl v1.37.x.
macOS
- Install and start Docker Desktop for Mac. Choose Apple Silicon or Intel to match your Mac.
- Open Terminal. Check
python3 --versionand install Python 3 for macOS if needed. - Install kind v0.33.0 and kubectl v1.37.x on your PATH. Use the Darwin ARM64 binaries for Apple Silicon or Darwin AMD64 for Intel. Use the linked kubectl instructions for an explicit release such as v1.37.0.
Windows with WSL 2
- In administrator PowerShell, run
wsl --install -d Ubuntu. Restart if prompted and open Ubuntu to finish setup. Runwsl -l -vin PowerShell to confirm Ubuntu shows version 2. Microsoft’s WSL setup guide covers existing installs and upgrades. - Install Docker Desktop for Windows and start it in Linux containers mode. Turn on the WSL 2 engine and enable Ubuntu under Settings → Resources → WSL Integration. Docker’s WSL setup guide shows the settings.
- Switch to the Ubuntu terminal for every command below. In Ubuntu, run
sudo apt updateandsudo apt install curl python3 unzip, then install the Linux builds of kind v0.33.0 and kubectl v1.37.x on your PATH. Use Linux AMD64 for an x64 PC or Linux ARM64 for an ARM PC. Use the linked kubectl instructions for an explicit release such as v1.37.0.
The full suite passed on an Apple Silicon Mac. Windows with WSL 2 is the recommended Windows setup and still needs an end-to-end lab run.
1. Check your tools
Run these in Terminal on Mac or Ubuntu on Windows. Look for a Docker Server section, kind v0.33.0, a v1.37.x kubectl client, and a Python 3 version.
docker version
kind version
kubectl version --client
python3 --version
bash --versionLeave several GiB of free disk space for images and the cluster. If you configure Docker’s memory allocation, 6–8 GiB is a starting point for three nodes.
2. Download the labs and create the cluster
Download the lab bundle and extract it. On Windows, extract it inside your Ubuntu home directory, for example with unzip /path/to/kv-labs.zip -d ~/kv-labs. In your terminal, change to the extracted directory containing kind.yaml. Keep this terminal open so the labs keep using its dedicated kubeconfig.
export KUBECONFIG="$PWD/kv-lab.kubeconfig"
kind create cluster --name kv-lab --config kind.yaml --wait 180s
kubectl --context kind-kv-lab wait --for=condition=Ready nodes --all --timeout=180s
kubectl --context kind-kv-lab get nodesExpect one control-plane node and two workers, all Ready. If kv-lab already exists, use its original lab kubeconfig or finish and clean up that cluster before creating another one.
3. Run your first exercise
From the extracted directory, run the reconciliation lab. Its runner creates the lab namespace and shows the ReplicaSet replacing a deleted Pod. The subshell leaves you in the bundle directory for the next exercise.
(
cd m02 && bash run.sh && bash cleanup.sh
)Each investigation includes its own requirements, expected results, and cleanup. Keep all files for an exercise together.
4. Remove the lab cluster
When you finish all exercises, save any lab data you want to keep, then delete the disposable cluster from the same terminal.
kind delete cluster --name kv-labChoose an investigation.
20 exercisesNo matching exercises. Try a workload, resource, or another level.
Before the cluster
Compare an image, a container process, and a listening port
Requirements & expected result
Requires: Docker Desktop or Docker Engine running; kind v0.33.0; kubectl v1.37.x; Python 3; named disposable cluster kv-lab and namespace kv-lab. Download every file into one directory, then run bash run.sh. See /labs/kind.yaml and the lab guide for cluster setup.
Expected: A short-lived container prints its hostname and UID. A Kubernetes Pod has its own IP; the HTTP request reaches port 80 and returns an nginx page.
Docker and Kubernetes use separate lifecycle controls here; docker run does not create a Kubernetes object. A Pod IP is not a public internet address.
Run, inspect & clean up
Commands
bash run.shDiagnose
kubectl --context kind-kv-lab -n kv-lab get pods -l lab=m01 -o wide
kubectl --context kind-kv-lab -n kv-lab get events --sort-by=.metadata.creationTimestamp
kubectl --context kind-kv-lab -n kv-lab describe pods -l lab=m01Clean up
bash cleanup.sh
# When ALL labs are finished, destroy only the named disposable cluster:
kind delete cluster --name kv-labExercise files 5 files
A cluster that reconciles
Follow Deployment reconciliation through a ReplicaSet to replacement Pods
Requirements & expected result
Requires: Docker Desktop or Docker Engine running; kind v0.33.0; kubectl v1.37.x; Python 3; named disposable cluster kv-lab and namespace kv-lab. Download every file into one directory, then run bash run.sh. See /labs/kind.yaml and the lab guide for cluster setup.
Expected: Two replicas become available. Deleting one Pod changes its UID; the ReplicaSet returns to two ready replicas while the Deployment UID stays the same.
Local mechanism exercise; no claim of production equivalence.
Run, inspect & clean up
Commands
bash run.shDiagnose
kubectl --context kind-kv-lab -n kv-lab get pods -l lab=m02 -o wide
kubectl --context kind-kv-lab -n kv-lab get events --sort-by=.metadata.creationTimestamp
kubectl --context kind-kv-lab -n kv-lab describe pods -l lab=m02Clean up
bash cleanup.sh
# When ALL labs are finished, destroy only the named disposable cluster:
kind delete cluster --name kv-labExercise files 5 files
The language of Kubernetes
Address objects with explicit contexts, labels, annotations and apply
Requirements & expected result
Requires: Docker Desktop or Docker Engine running; kind v0.33.0; kubectl v1.37.x; Python 3; named disposable cluster kv-lab and namespace kv-lab. Download every file into one directory, then run bash run.sh. See /labs/kind.yaml and the lab guide for cluster setup.
Expected: A namespaced ConfigMap is selected by its lab label. Its UID stays stable after a unique annotation update; the script verifies that resourceVersion changes without arithmetic or an event-count inference.
kubectl config current-context is read-only. The displayed active context may differ; every lab command explicitly selects kind-kv-lab. Apply updates declared fields, while unowned annotations can remain.
Run, inspect & clean up
Commands
bash run.shDiagnose
kubectl --context kind-kv-lab -n kv-lab get pods -l lab=m03 -o wide
kubectl --context kind-kv-lab -n kv-lab get events --sort-by=.metadata.creationTimestamp
kubectl --context kind-kv-lab -n kv-lab describe pods -l lab=m03Clean up
bash cleanup.sh
# When ALL labs are finished, destroy only the named disposable cluster:
kind delete cluster --name kv-labExercise files 5 files
Life inside a Pod
Watch init ordering and share configuration inside a Pod
Requirements & expected result
Requires: Docker Desktop or Docker Engine running; kind v0.33.0; kubectl v1.37.x; Python 3; named disposable cluster kv-lab and namespace kv-lab. Download every file into one directory, then run bash run.sh. See /labs/kind.yaml and the lab guide for cluster setup.
Expected: The init container completes before app containers start. App logs contain prepared and the ConfigMap greeting. Both app and companion see the same emptyDir file.
The companion is a regular second container, not a native restartable init sidecar. emptyDir survives a container restart but disappears with its Pod. The disposable Secret is plaintext in this download and is not a credential. Environment values do not refresh when a ConfigMap changes.
Run, inspect & clean up
Commands
bash run.shDiagnose
kubectl --context kind-kv-lab -n kv-lab get pods -l lab=m04 -o wide
kubectl --context kind-kv-lab -n kv-lab get events --sort-by=.metadata.creationTimestamp
kubectl --context kind-kv-lab -n kv-lab describe pods -l lab=m04Clean up
bash cleanup.sh
# When ALL labs are finished, destroy only the named disposable cluster:
kind delete cluster --name kv-labExercise files 5 files
Keep the application running
Remove and restore a ready endpoint without restarting the container
Requirements & expected result
Requires: Docker Desktop or Docker Engine running; kind v0.33.0; kubectl v1.37.x; Python 3; named disposable cluster kv-lab and namespace kv-lab. Download every file into one directory, then run bash run.sh. See /labs/kind.yaml and the lab guide for cluster setup.
Expected: Selected Pods appear in EndpointSlices with readiness conditions. Removing /tmp/ready sets one endpoint ready=false without removing its address record. Restoring the file makes it ready again; a template annotation triggers a rollout and undo restores the previous revision.
Readiness affects endpoint eligibility; it does not restart the container. Liveness failures can restart it. This healthy image plus a template annotation demonstrates a rollout without relying on a deliberately broken pull.
Run, inspect & clean up
Commands
bash run.shDiagnose
kubectl --context kind-kv-lab -n kv-lab get pods -l lab=m05 -o wide
kubectl --context kind-kv-lab -n kv-lab get events --sort-by=.metadata.creationTimestamp
kubectl --context kind-kv-lab -n kv-lab describe pods -l lab=m05Clean up
bash cleanup.sh
# When ALL labs are finished, destroy only the named disposable cluster:
kind delete cluster --name kv-labExercise files 5 files
Connect and investigate
Trace DNS through a Service selector to EndpointSlices
Requirements & expected result
Requires: Docker Desktop or Docker Engine running; kind v0.33.0; kubectl v1.37.x; Python 3; named disposable cluster kv-lab and namespace kv-lab. Download every file into one directory, then run bash run.sh. See /labs/kind.yaml and the lab guide for cluster setup.
Expected: The client resolves m06.kv-lab.svc.cluster.local and downloads nginx HTML. A deliberately wrong selector produces no usable endpoints; the script restores the selector.
ClusterIP and cluster DNS are proven locally. NodePort needs host routing/port mapping; a LoadBalancer needs a controller or cloud implementation. These are not installed by this exercise.
Run, inspect & clean up
Commands
bash run.shDiagnose
kubectl --context kind-kv-lab -n kv-lab get pods -l lab=m06 -o wide
kubectl --context kind-kv-lab -n kv-lab get events --sort-by=.metadata.creationTimestamp
kubectl --context kind-kv-lab -n kv-lab describe pods -l lab=m06Clean up
bash cleanup.sh
# When ALL labs are finished, destroy only the named disposable cluster:
kind delete cluster --name kv-labExercise files 5 files
Placement and resources
Explain a Pending Pod from its impossible placement constraint
Requirements & expected result
Requires: Docker Desktop or Docker Engine running; kind v0.33.0; kubectl v1.37.x; Python 3; named disposable cluster kv-lab and namespace kv-lab. Download every file into one directory, then run bash run.sh. See /labs/kind.yaml and the lab guide for cluster setup.
Expected: m07-allowed runs. m07-pending remains Pending with PodScheduled=False and an Unschedulable reason because no node has the requested label.
The failure is deterministic placement, not artificial node exhaustion. Burstable QoS follows unequal request/limit values. Affinity, topology spread, quotas and taints need separate constraints; this does not claim their behavior is identical.
Run, inspect & clean up
Commands
bash run.shDiagnose
kubectl --context kind-kv-lab -n kv-lab get pods -l lab=m07 -o wide
kubectl --context kind-kv-lab -n kv-lab get events --sort-by=.metadata.creationTimestamp
kubectl --context kind-kv-lab -n kv-lab describe pods -l lab=m07Clean up
bash cleanup.sh
# When ALL labs are finished, destroy only the named disposable cluster:
kind delete cluster --name kv-labExercise files 5 files
Networking beyond discovery
Measure whether this cluster enforces a selected ingress policy
Requirements & expected result
Requires: Docker Desktop or Docker Engine running; kind v0.33.0; kubectl v1.37.x; Python 3; named disposable cluster kv-lab and namespace kv-lab. Download every file into one directory, then run bash run.sh. See /labs/kind.yaml and the lab guide for cluster setup.
Expected: Baseline HTTP succeeds. With deny-ingress, a new TCP request fails only when the dataplane enforces NetworkPolicy. An explicit allow rule permits the labeled client again. The script reports unsupported enforcement clearly.
NetworkPolicy requires an enforcing CNI or policy controller. Older kind configurations did not enforce it; newer kind releases advertise embedded kube-network-policies. Do not infer enforcement from version or object acceptance: this lab measures it. If the default runtime does not enforce, recreate a separate disposable cluster with a supported enforcing implementation before using policy as a security boundary. Ingress/Gateway API controllers and TLS certificates are separate optional add-ons; a policy is not an ingress route.
Run, inspect & clean up
Commands
bash run.shDiagnose
kubectl --context kind-kv-lab -n kv-lab get pods -l lab=m08 -o wide
kubectl --context kind-kv-lab -n kv-lab get events --sort-by=.metadata.creationTimestamp
kubectl --context kind-kv-lab -n kv-lab describe pods -l lab=m08Clean up
bash cleanup.sh
# When ALL labs are finished, destroy only the named disposable cluster:
kind delete cluster --name kv-labExercise files 7 files
Data that outlives a Pod
Verify a PVC survives replacement of its consuming Pod
Requirements & expected result
Requires: Docker Desktop or Docker Engine running; kind v0.33.0; kubectl v1.37.x; Python 3; named disposable cluster kv-lab and namespace kv-lab. Download every file into one directory, then run bash run.sh. See /labs/kind.yaml and the lab guide for cluster setup.
Expected: The default StorageClass provisions and binds a volume after the consumer is scheduled. A file written into /data remains after deleting and recreating the Pod.
kind local-path storage is tied to the node container and is not distributed or replicated. ReadWriteOnce means one node, not one Pod. Snapshot APIs require VolumeSnapshot CRDs, a snapshot controller and a CSI driver that supports snapshots; the default local-path provisioner is not a CSI snapshot backend. This lab proves Pod-level persistence, not snapshot recovery or node-loss durability.
Run, inspect & clean up
Commands
bash run.shDiagnose
kubectl --context kind-kv-lab -n kv-lab get pods -l lab=m09 -o wide
kubectl --context kind-kv-lab -n kv-lab get events --sort-by=.metadata.creationTimestamp
kubectl --context kind-kv-lab -n kv-lab describe pods -l lab=m09Clean up
bash cleanup.sh
# When ALL labs are finished, destroy only the named disposable cluster:
kind delete cluster --name kv-labExercise files 6 files
Choose the workload
Compare stable identity, per-node placement, and completion
Requirements & expected result
Requires: Docker Desktop or Docker Engine running; kind v0.33.0; kubectl v1.37.x; Python 3; named disposable cluster kv-lab and namespace kv-lab. Download every file into one directory, then run bash run.sh. See /labs/kind.yaml and the lab guide for cluster setup.
Expected: The StatefulSet creates m10-0, the DaemonSet places an agent on schedulable matching nodes, and the Job completes. The suspended CronJob records a schedule without launching periodic work.
A one-replica PDB with minAvailable=1 blocks voluntary eviction; it cannot prevent machine failure. This script does not drain the shared lab node. StatefulSet identity alone does not create persistent storage; m09 covers claims.
Run, inspect & clean up
Commands
bash run.shDiagnose
kubectl --context kind-kv-lab -n kv-lab get pods -l lab=m10 -o wide
kubectl --context kind-kv-lab -n kv-lab get events --sort-by=.metadata.creationTimestamp
kubectl --context kind-kv-lab -n kv-lab describe pods -l lab=m10Clean up
bash cleanup.sh
# When ALL labs are finished, destroy only the named disposable cluster:
kind delete cluster --name kv-labExercise files 5 files
Trust and boundaries
Prove a ServiceAccount can read Pods but cannot read Secrets
Requirements & expected result
Requires: Docker Desktop or Docker Engine running; kind v0.33.0; kubectl v1.37.x; Python 3; named disposable cluster kv-lab and namespace kv-lab. Download every file into one directory, then run bash run.sh. See /labs/kind.yaml and the lab guide for cluster setup.
Expected: RBAC impersonation permits list pods and denies get secrets in kv-lab. A non-root Pod uses a read-only filesystem, dropped capabilities and no automatic ServiceAccount token.
Impersonation requires the current kind administrator permission. It tests authorization, not login or token verification. Pod securityContext is not an admission policy. Enable Pod Security Admission labels on a separate disposable namespace to explore rejection; do not relabel this shared lab namespace mid-course.
Run, inspect & clean up
Commands
bash run.shDiagnose
kubectl --context kind-kv-lab -n kv-lab get pods -l lab=m11 -o wide
kubectl --context kind-kv-lab -n kv-lab get events --sort-by=.metadata.creationTimestamp
kubectl --context kind-kv-lab -n kv-lab describe pods -l lab=m11Clean up
bash cleanup.sh
# When ALL labs are finished, destroy only the named disposable cluster:
kind delete cluster --name kv-labExercise files 5 files
Observe and diagnose
Collect evidence for a failing process and a broken Service selector
Requirements & expected result
Requires: Docker Desktop or Docker Engine running; kind v0.33.0; kubectl v1.37.x; Python 3; named disposable cluster kv-lab and namespace kv-lab. Download every file into one directory, then run bash run.sh. See /labs/kind.yaml and the lab guide for cluster setup.
Expected: The crash Pod emits a useful error and restarts. The web Deployment is healthy, while the Service selector initially points nowhere; restoring its selector makes client HTTP succeed.
Events, container exit state, current/previous logs, DNS and EndpointSlices answer different questions. Logs are bounded local evidence, not a Prometheus/Grafana/OpenTelemetry installation. Ephemeral debugging requires an admitted debug image and permission; not injected automatically here.
Run, inspect & clean up
Commands
bash run.shDiagnose
kubectl --context kind-kv-lab -n kv-lab get pods -l lab=m12 -o wide
kubectl --context kind-kv-lab -n kv-lab get events --sort-by=.metadata.creationTimestamp
kubectl --context kind-kv-lab -n kv-lab describe pods -l lab=m12Clean up
bash cleanup.sh
# When ALL labs are finished, destroy only the named disposable cluster:
kind delete cluster --name kv-labExercise files 5 files
Deliver and scale
Render configuration and inspect HPA prerequisites
Requirements & expected result
Requires: Docker Desktop or Docker Engine running; kind v0.33.0; kubectl v1.37.x; Python 3; named disposable cluster kv-lab and namespace kv-lab. Download every file into one directory, then run bash run.sh. See /labs/kind.yaml and the lab guide for cluster setup.
Expected: kubectl kustomize renders two replicas with a generated ConfigMap. HPA exists but cannot compute CPU utilization until Metrics Server supplies metrics and requests are set.
Metrics Server is required for resource HPA. The default kind cluster does not include it. No load-based autoscaling success is claimed. VPA and node autoscaling require their own controllers; kind does not provision cloud nodes. Helm templating is optional and separate from Kustomize. The baseline generated ConfigMap has name hashing disabled and is not consumed by the Deployment; configuration-triggered rollout is an optional extension.
Run, inspect & clean up
Commands
bash run.shDiagnose
kubectl --context kind-kv-lab -n kv-lab get pods -l lab=m13 -o wide
kubectl --context kind-kv-lab -n kv-lab get events --sort-by=.metadata.creationTimestamp
kubectl --context kind-kv-lab -n kv-lab describe pods -l lab=m13Clean up
bash cleanup.sh
# When ALL labs are finished, destroy only the named disposable cluster:
kind delete cluster --name kv-labExercise files 6 files
Inside the API
Observe a server-side apply ownership conflict and a finalizer
Requirements & expected result
Requires: Docker Desktop or Docker Engine running; kind v0.33.0; kubectl v1.37.x; Python 3; named disposable cluster kv-lab and namespace kv-lab. Download every file into one directory, then run bash run.sh. See /labs/kind.yaml and the lab guide for cluster setup.
Expected: A second field manager cannot change a field owned by kv-gitops without an explicit takeover. A ConfigMap with a finalizer remains terminating until that finalizer is removed.
The finalizer is a deliberate manual hold; no controller cleans external state. Remove it only because this disposable ConfigMap has no external resource. This exercise does not install an admission webhook or prove conversion compatibility.
Run, inspect & clean up
Commands
bash run.shDiagnose
kubectl --context kind-kv-lab -n kv-lab get pods -l lab=m14 -o wide
kubectl --context kind-kv-lab -n kv-lab get events --sort-by=.metadata.creationTimestamp
kubectl --context kind-kv-lab -n kv-lab describe pods -l lab=m14Clean up
bash cleanup.sh
# When ALL labs are finished, destroy only the named disposable cluster:
kind delete cluster --name kv-labExercise files 6 files
Control loops under the hood
Read a leader Lease and repeat a convergent reconciliation
Requirements & expected result
Requires: Docker Desktop or Docker Engine running; kind v0.33.0; kubectl v1.37.x; Python 3; named disposable cluster kv-lab and namespace kv-lab. Download every file into one directory, then run bash run.sh. See /labs/kind.yaml and the lab guide for cluster setup.
Expected: The lab creates a coordination Lease and lists the control-plane leader Leases in kube-system. Reapplying the same desired ConfigMap preserves its UID and content.
A manually written Lease is not leader election: there is no renewal or contender loop here. Repeated apply demonstrates convergence at the API boundary, not informer caches/workqueue implementation. The bounded watch may time out normally. m16 supplies a real repeated controller loop.
Run, inspect & clean up
Commands
bash run.shDiagnose
kubectl --context kind-kv-lab -n kv-lab get pods -l lab=m15 -o wide
kubectl --context kind-kv-lab -n kv-lab get events --sort-by=.metadata.creationTimestamp
kubectl --context kind-kv-lab -n kv-lab describe pods -l lab=m15Clean up
bash cleanup.sh
# When ALL labs are finished, destroy only the named disposable cluster:
kind delete cluster --name kv-labExercise files 5 files
Extend the control plane
Run a schema-validated API and an idempotent reconciler
Requirements & expected result
Requires: Docker Desktop or Docker Engine running; kind v0.33.0; kubectl v1.37.x; Python 3; named disposable cluster kv-lab and namespace kv-lab. Download every file into one directory, then run bash run.sh. See /labs/kind.yaml and the lab guide for cluster setup.
Expected: The WebApp creates an owned ConfigMap and Deployment. Repeated reconciliation keeps the Deployment UID; deletion of an owned ConfigMap is repaired. A replica change in the custom resource converges to two ready Pods.
The Python process uses your disposable-cluster credentials and polls; it is a real controller but not a production operator. It owns only generated objects and refuses to adopt an unowned collision. Owner references drive garbage collection after deleting the WebApp. No conversion webhook, leader election or finalizer is implemented. The shared WebApp CRD is used by m20; delete it only after both labs.
Run, inspect & clean up
Commands
bash run.shDiagnose
kubectl --context kind-kv-lab -n kv-lab get pods -l lab=m16 -o wide
kubectl --context kind-kv-lab -n kv-lab get events --sort-by=.metadata.creationTimestamp
kubectl --context kind-kv-lab -n kv-lab describe pods -l lab=m16Clean up
bash cleanup.sh
# When ALL labs are finished, destroy only the named disposable cluster:
kind delete cluster --name kv-labExercise files 7 files
Performance and pressure
Distinguish a container memory limit from node pressure
Requirements & expected result
Requires: Docker Desktop or Docker Engine running; kind v0.33.0; kubectl v1.37.x; Python 3; named disposable cluster kv-lab and namespace kv-lab. Download every file into one directory, then run bash run.sh. See /labs/kind.yaml and the lab guide for cluster setup.
Expected: The bounded process writes to a memory-backed volume until its 16Mi container limit is exceeded; the container terminates with reason OOMKilled. Other Pods remain schedulable.
This bounded test consumes at most a small container limit; it deliberately avoids exhausting Docker Desktop or a node. OOMKilled is not an eviction. CPU limits throttle; the presence of a CPU limit alone does not measure throttling. Metrics and a load experiment are required for capacity decisions.
Run, inspect & clean up
Commands
bash run.shDiagnose
kubectl --context kind-kv-lab -n kv-lab get pods -l lab=m17 -o wide
kubectl --context kind-kv-lab -n kv-lab get events --sort-by=.metadata.creationTimestamp
kubectl --context kind-kv-lab -n kv-lab describe pods -l lab=m17Clean up
bash cleanup.sh
# When ALL labs are finished, destroy only the named disposable cluster:
kind delete cluster --name kv-labExercise files 5 files
Operate the control plane
Capture an etcd snapshot and rehearse offline restoration separately
Requirements & expected result
Requires: Docker Desktop or Docker Engine running; kind v0.33.0; kubectl v1.37.x; Python 3; named disposable cluster kv-lab and namespace kv-lab. Download every file into one directory, then run bash run.sh. See /labs/kind.yaml and the lab guide for cluster setup.
Expected: The snapshot command writes a consistent etcd snapshot to kv-lab.db in the private directory printed by the script, outside the cluster and published assets. The optional restore script creates only kv-recovery, stops its API/etcd, restores into a fresh data directory with a revision bump, and checks the saved marker.
A one-control-plane kind cluster cannot prove HA, quorum resilience, failure-domain isolation or production recovery RTO/RPO. Snapshot restore rewinds all API data; it does not restore external volumes or cloud resources. Revision bump and compaction invalidate stale watch caches. The offline rehearsal is optional and must be verified on the exact etcd image; preserve source-cluster PKI and encryption configuration in a real recovery plan. Snapshot files contain Secrets. Runtime outputs default to a private user-specific OS temporary directory, outside the extracted lab and the checkout: run `python3 backup-path.py m18` to print the exact selected location. Replace m18 with this lab's ID. Both capture and recovery use the same default. Set `KV_LAB_BACKUP_DIR` to an explicit private directory to override it; preserve that value for later recovery. The helper requires an owned directory with mode 0700, rejects artifact symlinks, and resolves paths before refusing any destination under this checkout's public/ or dist/ directories. Temporary storage can be cleared by the OS: copy wanted backups to a durable private destination before relying on them.
Run, inspect & clean up
Commands
bash run.shDiagnose
kubectl --context kind-kv-lab -n kv-lab get pods -l lab=m18 -o wide
kubectl --context kind-kv-lab -n kv-lab get events --sort-by=.metadata.creationTimestamp
kubectl --context kind-kv-lab -n kv-lab describe pods -l lab=m18Clean up
bash cleanup.sh
# When ALL labs are finished, destroy only the named disposable cluster:
kind delete cluster --name kv-labPlatform patterns
Reconcile declared configuration and identify delivery boundaries
Requirements & expected result
Requires: Docker Desktop or Docker Engine running; kind v0.33.0; kubectl v1.37.x; Python 3; named disposable cluster kv-lab and namespace kv-lab. Download every file into one directory, then run bash run.sh. See /labs/kind.yaml and the lab guide for cluster setup.
Expected: The declared ConfigMap is applied by a GitOps field manager. Manual drift changes the live value. Reapplying the declaration restores it; managedFields show ownership. No Argo CD runtime is implied by this local replay.
The baseline is manual declarative reconciliation. The Argo CD extension is compatibility research only: select an upstream-supported release and a Kubernetes minor in its tested matrix, then verify the pair in a separate disposable cluster. No compatible 1.37 installation is claimed here. A digest identifies bytes but does not by itself verify signatures or attestations. Service mesh and multi-tenant enforcement require separately installed controllers and trust configuration.
Run, inspect & clean up
Commands
bash run.shDiagnose
kubectl --context kind-kv-lab -n kv-lab get pods -l lab=m19 -o wide
kubectl --context kind-kv-lab -n kv-lab get events --sort-by=.metadata.creationTimestamp
kubectl --context kind-kv-lab -n kv-lab describe pods -l lab=m19Clean up
bash cleanup.sh
# When ALL labs are finished, destroy only the named disposable cluster:
kind delete cluster --name kv-labExercise files 6 files
Put the system together
Complete stateless diagnosis, file recovery, and GitOps/operator capstones
Requirements & expected result
Requires: Docker Desktop or Docker Engine running; kind v0.33.0; kubectl v1.37.x; Python 3; named disposable cluster kv-lab and namespace kv-lab. Download every file into one directory, then run bash run.sh. See /labs/kind.yaml and the lab guide for cluster setup.
Expected: Capstone 1 diagnoses and repairs a wrong Service selector. Capstone 2 backs up a PVC file externally, deletes it, restores it, and checks persistence. Capstone 3 proves CR field ownership conflicts, repeated reconciliation, repair and owner-reference garbage collection.
All three capstones run against kind-kv-lab and kv-lab. The GitOps capstone uses explicit SSA apply as the declarative writer; Argo CD is optional in m19. The file backup is not a transaction-consistent database backup. Neither kind storage nor a one-member control plane proves production HA. A real operator must also solve RBAC, status conditions, retry/backoff, deletion semantics and leader election. Runtime outputs default to a private user-specific OS temporary directory, outside the extracted lab and the checkout: run `python3 backup-path.py m20` to print the exact selected location. Replace m20 with this lab's ID. Both capture and recovery use the same default. Set `KV_LAB_BACKUP_DIR` to an explicit private directory to override it; preserve that value for later recovery. The helper requires an owned directory with mode 0700, rejects artifact symlinks, and resolves paths before refusing any destination under this checkout's public/ or dist/ directories. Temporary storage can be cleared by the OS: copy wanted backups to a durable private destination before relying on them.
Run, inspect & clean up
Commands
bash run.shDiagnose
kubectl --context kind-kv-lab -n kv-lab get pods -l lab=m20 -o wide
kubectl --context kind-kv-lab -n kv-lab get events --sort-by=.metadata.creationTimestamp
kubectl --context kind-kv-lab -n kv-lab describe pods -l lab=m20Clean up
bash cleanup.sh
# When ALL labs are finished, destroy only the named disposable cluster:
kind delete cluster --name kv-lab