Before we start:
Conformance tests in Kubernetes are a set of automated tests designed to verify that a Kubernetes cluster adheres to the official Kubernetes API and behaves according to the expected standards. These tests ensure that a Kubernetes deployment is functioning correctly, is compatible with the broader Kubernetes ecosystem, and can interact seamlessly with other components.
By passing conformance tests, Kubernetes distributions and cloud providers can demonstrate that their systems are certified to meet the essential features of the platform, such as resource scheduling, networking, and security policies. The advantages of conformance tests include improved reliability, compatibility, and trust, enabling developers and organizations to deploy Kubernetes in production with confidence. They also foster a consistent experience across different environments and facilitate the identification of issues early in the development process.
A Kind (Kubernetes IN Docker) cluster is a tool for running local Kubernetes clusters using Docker containers as nodes. It is primarily designed for testing Kubernetes clusters and applications on Kubernetes in a lightweight, isolated environment. Kind creates a Kubernetes cluster in Docker containers, making it easy to set up and tear down clusters quickly without the need for dedicated virtual machines or cloud resources.
TL;DR
– Create a KIND cluster
– Create E2E test binary
– Run Conformance tests by setting the context to KIND cluster
Lets run Kubernetes Conformance Tests in KIND:
KIND Cluster preparation:
Build Node Image, Define config.yaml, Set KubeContext
In your kubernetes branch:
Create your kind node image:
kind build node-image
Create your kind e2e cluster config kind-config-yaml:
# necessary for conformance
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
networking:
ipFamily: ipv4
nodes:
# the control plane node
- role: control-plane
- role: worker
- role: worker
Set your KUBECONFIG env variable (KIND generates the conf based on it):
export KUBECONFIG="${HOME}/.kube/kind-test-config"
Use the previous config to create your cluster:
kind create cluster --config kind-config.yaml --image kindest/node:latest -v4
E2E Binary Preparation:
Create your e2e Kubernetes binary (from your Kubernetes src code):
make WHAT="test/e2e/e2e.test"
Execute your tests:
./_output/bin/e2e.test -context kind-kind -ginkgo.focus="\[sig-network\].*Conformance" -num-nodes 2
Long Version from my setup;
To expand on this and for completeness, let’s say that you have a change in kube-proxy, you don’t have to redo all the steps, you just need to:
build the new image:
bazel build //build:docker-artifacts
load it in your current kind cluster:
kind load image-archive bazel-bin/build/kube-proxy.tar
check that it has been loaded:
docker exec -it kind-control-plane crictl images
IMAGE TAG IMAGE ID SIZE
docker.io/kindest/kindnetd 0.5.4 2186a1a396deb 113MB
docker.io/rancher/local-path-provisioner v0.0.11 9d12f9848b99f 36.5MB
k8s.gcr.io/coredns 1.6.5 70f311871ae12 41.7MB
k8s.gcr.io/debian-base v2.0.0 9bd6154724425 53.9MB
k8s.gcr.io/etcd 3.4.3-0 303ce5db0e90d 290MB
k8s.gcr.io/kube-apiserver v1.17.0 4a0e3a87a5e22 144MB
k8s.gcr.io/kube-controller-manager v1.17.0 fa313a582b872 131MB
k8s.gcr.io/kube-proxy-amd64 v1.18.0-alpha.1.633_8428af6fd79e1f-dirty 5dd06b2bb1290 124MB
k8s.gcr.io/kube-proxy v1.17.0 e3dd0e2bea53a 132MB
k8s.gcr.io/kube-scheduler v1.17.0 cb8feb1d83dd3 112MB
k8s.gcr.io/pause 3.1 da86e6ba6ca19 746kB
and modify the kubernetes deployment to pick the new image, in this case you can follow these instructions to switch the container image of the daemonset
https://kubernetes.io/docs/tasks/manage-daemon/update-daemon-set/ , please refer https://github.com/kubernetes-sigs/kind/issues/1181
In my setup, it was first failing due to the lack of space as shown below
github.com/coredns/caddy/caddyfile: mkdir /tmp.k8s/go-build2193697665/b2501/: no space left on device
k8s.io/system-validators/validators: mkdir /tmp.k8s/go-build2193697665/b2525/: no space left on device
!!! [0429 13:28:44] Call tree:
!!! [0429 13:28:44] 1: /go/src/k8s.io/kubernetes/hack/lib/golang.sh:771 kube::golang::build_some_binaries(...)
!!! [0429 13:28:44] 2: /go/src/k8s.io/kubernetes/hack/lib/golang.sh:941 kube::golang::build_binaries_for_platform(...)
!!! [0429 13:28:44] 3: hack/make-rules/build.sh:27 kube::golang::build_binaries(...)
!!! [0429 13:28:44] Call tree:
!!! [0429 13:28:44] 1: hack/make-rules/build.sh:27 kube::golang::build_binaries(...)
!!! [0429 13:28:44] Call tree:
!!! [0429 13:28:44] 1: hack/make-rules/build.sh:27 kube::golang::build_binaries(...)
make: *** [Makefile:92: all] Error 1
!!! [0429 13:28:44] Call tree:
!!! [0429 13:28:44] 1: build/../build/common.sh:488 kube::build::run_build_command_ex(...)
!!! [0429 13:28:44] 2: build/release-images.sh:40 kube::build::run_build_command(...)
make: *** [quick-release-images] Error 1
The error reported here is : “NO SPACE LEFT ON DEVICE’ which can be resolved by adjusting the resoures in DOCKER DESKTOP IN MAC.
chumble2TR91:kubernetes chumble$ kind build node-image
Starting to build Kubernetes
+++ [0429 13:31:56] Verifying Prerequisites....
+++ [0429 13:31:56] Using docker on macOS
+++ [0429 13:31:58] Building Docker image kube-build:build-45f9a540d8-5-v1.27.0-go1.20.3-bullseye.0
+++ [0429 13:32:03] Syncing sources to container
+++ [0429 13:32:07] Running build command...
+++ [0429 13:32:09] Setting GOMAXPROCS: 10
+++ [0429 13:32:09] Building go targets for linux/arm64
k8s.io/kubernetes/cmd/kube-apiserver (static)
k8s.io/kubernetes/cmd/kube-controller-manager (static)
k8s.io/kubernetes/cmd/kube-scheduler (static)
k8s.io/kubernetes/cmd/kube-proxy (static)
k8s.io/kubernetes/cmd/kubectl (static)
k8s.io/kubernetes/cmd/kubeadm (static)
k8s.io/kubernetes/cmd/kubectl (static)
k8s.io/kubernetes/cmd/kubelet (non-static)
+++ [0429 13:33:21] Syncing out of container
+++ [0429 13:33:31] Building images: linux-arm64
+++ [0429 13:33:31] Starting docker build for image: kube-apiserver-arm64
+++ [0429 13:33:31] Starting docker build for image: kube-controller-manager-arm64
+++ [0429 13:33:31] Starting docker build for image: kube-scheduler-arm64
+++ [0429 13:33:31] Starting docker build for image: kube-proxy-arm64
+++ [0429 13:33:31] Starting docker build for image: kubectl-arm64
+++ [0429 13:33:47] Deleting docker image registry.k8s.io/kubectl-arm64:v1.28.0-alpha.0.530_d8bdddcab42932-dirty
+++ [0429 13:33:47] Deleting docker image registry.k8s.io/kube-scheduler-arm64:v1.28.0-alpha.0.530_d8bdddcab42932-dirty
+++ [0429 13:33:48] Deleting docker image registry.k8s.io/kube-controller-manager-arm64:v1.28.0-alpha.0.530_d8bdddcab42932-dirty
+++ [0429 13:33:54] Deleting docker image registry.k8s.io/kube-proxy-arm64:v1.28.0-alpha.0.530_d8bdddcab42932-dirty
+++ [0429 13:34:10] Deleting docker image registry.k8s.io/kube-apiserver-arm64:v1.28.0-alpha.0.530_d8bdddcab42932-dirty
+++ [0429 13:34:10] Docker builds done
Finished building Kubernetes
Building node image ...
Building in container: kind-build-1682755456-1540799027
Image "kindest/node:latest" build completed.
I have already built KIND node image as you see:
chumble2TR91:kubernetes chumble$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
kindest/node latest da5453583d1d 28 seconds ago 1.32GB
kube-build build-45f9a540d8-5-v1.27.0-go1.20.3-bullseye.0 aaffa6b7ac5f 3 minutes ago 1.26GB
kube-build build-a1b83e6b4a-5-v1.27.0-go1.20.3-bullseye.0 0a89e9dbddf9 About an hour ago 1.26GB
kindest/node
kindest/base v20230330-89a4b81b 227842213139 4 weeks ago 311MB
projects.registry.vmware.com/tce/kind v1.22.7 fd1e232b07fe 13 months ago 1.14GB
Let us create a new KIND cluster from this Node image and config file
chumble2TR91:kubernetes chumble$ kind create cluster --config kind-config.yaml --image kindest/node:latest -v4
Creating cluster "kind" ...
DEBUG: docker/images.go:58] Image: kindest/node:latest present locally
✓ Ensuring node image (kindest/node:latest) 🖼
✓ Preparing nodes 📦 📦 📦
DEBUG: config/config.go:96] Using the following kubeadm config for node kind-worker2:
apiServer:
certSANs:
- localhost
- 127.0.0.1
extraArgs:
runtime-config: ""
apiVersion: kubeadm.k8s.io/v1beta3
clusterName: kind
controlPlaneEndpoint: kind-control-plane:6443
controllerManager:
extraArgs:
enable-hostpath-provisioner: "true"
kind: ClusterConfiguration
kubernetesVersion: v1.28.0-alpha.0.530+d8bdddcab42932-dirty
networking:
podSubnet: 10.244.0.0/16
serviceSubnet: 10.96.0.0/16
scheduler:
extraArgs: null
---
apiVersion: kubeadm.k8s.io/v1beta3
bootstrapTokens:
- token: abcdef.0123456789abcdef
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 172.18.0.4
bindPort: 6443
nodeRegistration:
criSocket: unix:///run/containerd/containerd.sock
kubeletExtraArgs:
node-ip: 172.18.0.4
node-labels: ""
provider-id: kind://docker/kind/kind-worker2
---
apiVersion: kubeadm.k8s.io/v1beta3
discovery:
bootstrapToken:
apiServerEndpoint: kind-control-plane:6443
token: abcdef.0123456789abcdef
unsafeSkipCAVerification: true
kind: JoinConfiguration
nodeRegistration:
criSocket: unix:///run/containerd/containerd.sock
kubeletExtraArgs:
node-ip: 172.18.0.4
node-labels: ""
provider-id: kind://docker/kind/kind-worker2
---
apiVersion: kubelet.config.k8s.io/v1beta1
cgroupDriver: systemd
cgroupRoot: /kubelet
evictionHard:
imagefs.available: 0%
nodefs.available: 0%
nodefs.inodesFree: 0%
failSwapOn: false
imageGCHighThresholdPercent: 100
kind: KubeletConfiguration
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
conntrack:
maxPerCore: 0
iptables:
minSyncPeriod: 1s
kind: KubeProxyConfiguration
mode: iptables
DEBUG: config/config.go:96] Using the following kubeadm config for node kind-worker:
apiServer:
certSANs:
- localhost
- 127.0.0.1
extraArgs:
runtime-config: ""
apiVersion: kubeadm.k8s.io/v1beta3
clusterName: kind
controlPlaneEndpoint: kind-control-plane:6443
controllerManager:
extraArgs:
enable-hostpath-provisioner: "true"
kind: ClusterConfiguration
kubernetesVersion: v1.28.0-alpha.0.530+d8bdddcab42932-dirty
networking:
podSubnet: 10.244.0.0/16
serviceSubnet: 10.96.0.0/16
scheduler:
extraArgs: null
---
apiVersion: kubeadm.k8s.io/v1beta3
bootstrapTokens:
- token: abcdef.0123456789abcdef
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 172.18.0.5
bindPort: 6443
nodeRegistration:
criSocket: unix:///run/containerd/containerd.sock
kubeletExtraArgs:
node-ip: 172.18.0.5
node-labels: ""
provider-id: kind://docker/kind/kind-worker
---
apiVersion: kubeadm.k8s.io/v1beta3
discovery:
bootstrapToken:
apiServerEndpoint: kind-control-plane:6443
token: abcdef.0123456789abcdef
unsafeSkipCAVerification: true
kind: JoinConfiguration
nodeRegistration:
criSocket: unix:///run/containerd/containerd.sock
kubeletExtraArgs:
node-ip: 172.18.0.5
node-labels: ""
provider-id: kind://docker/kind/kind-worker
---
apiVersion: kubelet.config.k8s.io/v1beta1
cgroupDriver: systemd
cgroupRoot: /kubelet
evictionHard:
imagefs.available: 0%
nodefs.available: 0%
nodefs.inodesFree: 0%
failSwapOn: false
imageGCHighThresholdPercent: 100
kind: KubeletConfiguration
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
conntrack:
maxPerCore: 0
iptables:
minSyncPeriod: 1s
kind: KubeProxyConfiguration
mode: iptables
DEBUG: config/config.go:96] Using the following kubeadm config for node kind-control-plane:
apiServer:
certSANs:
- localhost
- 127.0.0.1
extraArgs:
runtime-config: ""
apiVersion: kubeadm.k8s.io/v1beta3
clusterName: kind
controlPlaneEndpoint: kind-control-plane:6443
controllerManager:
extraArgs:
enable-hostpath-provisioner: "true"
kind: ClusterConfiguration
kubernetesVersion: v1.28.0-alpha.0.530+d8bdddcab42932-dirty
networking:
podSubnet: 10.244.0.0/16
serviceSubnet: 10.96.0.0/16
scheduler:
extraArgs: null
---
apiVersion: kubeadm.k8s.io/v1beta3
bootstrapTokens:
- token: abcdef.0123456789abcdef
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 172.18.0.6
bindPort: 6443
nodeRegistration:
criSocket: unix:///run/containerd/containerd.sock
kubeletExtraArgs:
node-ip: 172.18.0.6
node-labels: ""
provider-id: kind://docker/kind/kind-control-plane
---
apiVersion: kubeadm.k8s.io/v1beta3
controlPlane:
localAPIEndpoint:
advertiseAddress: 172.18.0.6
bindPort: 6443
discovery:
bootstrapToken:
apiServerEndpoint: kind-control-plane:6443
token: abcdef.0123456789abcdef
unsafeSkipCAVerification: true
kind: JoinConfiguration
nodeRegistration:
criSocket: unix:///run/containerd/containerd.sock
kubeletExtraArgs:
node-ip: 172.18.0.6
node-labels: ""
provider-id: kind://docker/kind/kind-control-plane
---
apiVersion: kubelet.config.k8s.io/v1beta1
cgroupDriver: systemd
cgroupRoot: /kubelet
evictionHard:
imagefs.available: 0%
nodefs.available: 0%
nodefs.inodesFree: 0%
failSwapOn: false
imageGCHighThresholdPercent: 100
kind: KubeletConfiguration
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
conntrack:
maxPerCore: 0
iptables:
minSyncPeriod: 1s
kind: KubeProxyConfiguration
mode: iptables
✓ Writing configuration 📜
DEBUG: kubeadminit/init.go:82] I0429 08:07:18.339790 135 initconfiguration.go:255] loading configuration from "/kind/kubeadm.conf"
W0429 08:07:18.340457 135 initconfiguration.go:332] [config] WARNING: Ignored YAML document with GroupVersionKind kubeadm.k8s.io/v1beta3, Kind=JoinConfiguration
I0429 08:07:18.347847 135 common.go:128] WARNING: tolerating control plane version v1.28.0-alpha.0.530+d8bdddcab42932-dirty as a pre-release version
I0429 08:07:18.348093 135 certs.go:112] creating a new certificate authority for ca
[init] Using Kubernetes version: v1.28.0-alpha.0.530+d8bdddcab42932-dirty
[certs] Using certificateDir folder "/etc/kubernetes/pki"
[certs] Generating "ca" certificate and key
I0429 08:07:18.583289 135 certs.go:519] validating certificate period for ca certificate
[certs] Generating "apiserver" certificate and key
[certs] apiserver serving cert is signed for DNS names [kind-control-plane kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local localhost] and IPs [10.96.0.1 172.18.0.6 127.0.0.1]
[certs] Generating "apiserver-kubelet-client" certificate and key
I0429 08:07:18.856250 135 certs.go:112] creating a new certificate authority for front-proxy-ca
[certs] Generating "front-proxy-ca" certificate and key
I0429 08:07:18.975239 135 certs.go:519] validating certificate period for front-proxy-ca certificate
[certs] Generating "front-proxy-client" certificate and key
I0429 08:07:19.142521 135 certs.go:112] creating a new certificate authority for etcd-ca
[certs] Generating "etcd/ca" certificate and key
I0429 08:07:19.244257 135 certs.go:519] validating certificate period for etcd/ca certificate
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [kind-control-plane localhost] and IPs [172.18.0.6 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [kind-control-plane localhost] and IPs [172.18.0.6 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
I0429 08:07:19.636517 135 certs.go:78] creating new public/private key files for signing service account users
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I0429 08:07:19.724249 135 kubeconfig.go:103] creating kubeconfig file for admin.conf
[kubeconfig] Writing "admin.conf" kubeconfig file
I0429 08:07:19.774390 135 kubeconfig.go:103] creating kubeconfig file for kubelet.conf
[kubeconfig] Writing "kubelet.conf" kubeconfig file
I0429 08:07:19.915966 135 kubeconfig.go:103] creating kubeconfig file for controller-manager.conf
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0429 08:07:20.049706 135 kubeconfig.go:103] creating kubeconfig file for scheduler.conf
[kubeconfig] Writing "scheduler.conf" kubeconfig file
I0429 08:07:20.131770 135 kubelet.go:67] Stopping the kubelet
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
I0429 08:07:20.229196 135 manifests.go:99] [control-plane] getting StaticPodSpecs
I0429 08:07:20.229436 135 certs.go:519] validating certificate period for CA certificate
I0429 08:07:20.229486 135 manifests.go:125] [control-plane] adding volume "ca-certs" for component "kube-apiserver"
I0429 08:07:20.229496 135 manifests.go:125] [control-plane] adding volume "etc-ca-certificates" for component "kube-apiserver"
I0429 08:07:20.229499 135 manifests.go:125] [control-plane] adding volume "k8s-certs" for component "kube-apiserver"
I0429 08:07:20.229501 135 manifests.go:125] [control-plane] adding volume "usr-local-share-ca-certificates" for component "kube-apiserver"
I0429 08:07:20.229504 135 manifests.go:125] [control-plane] adding volume "usr-share-ca-certificates" for component "kube-apiserver"
I0429 08:07:20.230909 135 manifests.go:154] [control-plane] wrote static Pod manifest for component "kube-apiserver" to "/etc/kubernetes/manifests/kube-apiserver.yaml"
I0429 08:07:20.230927 135 manifests.go:99] [control-plane] getting StaticPodSpecs
[control-plane] Creating static Pod manifest for "kube-controller-manager"
I0429 08:07:20.231037 135 manifests.go:125] [control-plane] adding volume "ca-certs" for component "kube-controller-manager"
I0429 08:07:20.231049 135 manifests.go:125] [control-plane] adding volume "etc-ca-certificates" for component "kube-controller-manager"
I0429 08:07:20.231051 135 manifests.go:125] [control-plane] adding volume "flexvolume-dir" for component "kube-controller-manager"
I0429 08:07:20.231054 135 manifests.go:125] [control-plane] adding volume "k8s-certs" for component "kube-controller-manager"
I0429 08:07:20.231056 135 manifests.go:125] [control-plane] adding volume "kubeconfig" for component "kube-controller-manager"
I0429 08:07:20.231058 135 manifests.go:125] [control-plane] adding volume "usr-local-share-ca-certificates" for component "kube-controller-manager"
I0429 08:07:20.231061 135 manifests.go:125] [control-plane] adding volume "usr-share-ca-certificates" for component "kube-controller-manager"
I0429 08:07:20.231409 135 manifests.go:154] [control-plane] wrote static Pod manifest for component "kube-controller-manager" to "/etc/kubernetes/manifests/kube-controller-manager.yaml"
I0429 08:07:20.231425 135 manifests.go:99] [control-plane] getting StaticPodSpecs
[control-plane] Creating static Pod manifest for "kube-scheduler"
I0429 08:07:20.231520 135 manifests.go:125] [control-plane] adding volume "kubeconfig" for component "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0429 08:07:20.231741 135 manifests.go:154] [control-plane] wrote static Pod manifest for component "kube-scheduler" to "/etc/kubernetes/manifests/kube-scheduler.yaml"
W0429 08:07:20.231897 135 images.go:80] could not find officially supported version of etcd for Kubernetes v1.28.0-alpha.0.530+d8bdddcab42932-dirty, falling back to the nearest etcd version (3.5.7-0)
I0429 08:07:20.232222 135 local.go:65] [etcd] wrote Static Pod manifest for a local etcd member to "/etc/kubernetes/manifests/etcd.yaml"
I0429 08:07:20.232242 135 waitcontrolplane.go:83] [wait-control-plane] Waiting for the API server to be healthy
I0429 08:07:20.232603 135 loader.go:373] Config loaded from file: /etc/kubernetes/admin.conf
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
I0429 08:07:20.236145 135 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s in 1 milliseconds
I0429 08:07:20.737046 135 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s in 0 milliseconds
I0429 08:07:21.237061 135 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s in 0 milliseconds
I0429 08:07:21.737819 135 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s in 0 milliseconds
I0429 08:07:22.238113 135 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s in 0 milliseconds
I0429 08:07:22.737134 135 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s in 0 milliseconds
I0429 08:07:24.203201 135 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s 500 Internal Server Error in 966 milliseconds
I0429 08:07:24.237852 135 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s 500 Internal Server Error in 0 milliseconds
I0429 08:07:24.738486 135 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s 500 Internal Server Error in 0 milliseconds
I0429 08:07:25.237609 135 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s 500 Internal Server Error in 0 milliseconds
[apiclient] All control plane components are healthy after 5.503154 seconds
[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
I0429 08:07:25.737825 135 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s 200 OK in 0 milliseconds
I0429 08:07:25.737889 135 uploadconfig.go:112] [upload-config] Uploading the kubeadm ClusterConfiguration to a ConfigMap
I0429 08:07:25.742197 135 round_trippers.go:553] POST https://kind-control-plane:6443/api/v1/namespaces/kube-system/configmaps?timeout=10s 201 Created in 3 milliseconds
I0429 08:07:25.745200 135 round_trippers.go:553] POST https://kind-control-plane:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/roles?timeout=10s 201 Created in 2 milliseconds
I0429 08:07:25.747917 135 round_trippers.go:553] POST https://kind-control-plane:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/rolebindings?timeout=10s 201 Created in 2 milliseconds
I0429 08:07:25.748062 135 uploadconfig.go:126] [upload-config] Uploading the kubelet component config to a ConfigMap
[kubelet] Creating a ConfigMap "kubelet-config" in namespace kube-system with the configuration for the kubelets in the cluster
I0429 08:07:25.750423 135 round_trippers.go:553] POST https://kind-control-plane:6443/api/v1/namespaces/kube-system/configmaps?timeout=10s 201 Created in 1 milliseconds
I0429 08:07:25.753871 135 round_trippers.go:553] POST https://kind-control-plane:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/roles?timeout=10s 201 Created in 3 milliseconds
I0429 08:07:25.756081 135 round_trippers.go:553] POST https://kind-control-plane:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/rolebindings?timeout=10s 201 Created in 2 milliseconds
I0429 08:07:25.756187 135 uploadconfig.go:131] [upload-config] Preserving the CRISocket information for the control-plane node
I0429 08:07:25.756227 135 patchnode.go:31] [patchnode] Uploading the CRI Socket information "unix:///run/containerd/containerd.sock" to the Node API object "kind-control-plane" as an annotation
I0429 08:07:26.258847 135 round_trippers.go:553] GET https://kind-control-plane:6443/api/v1/nodes/kind-control-plane?timeout=10s 200 OK in 2 milliseconds
I0429 08:07:26.263093 135 round_trippers.go:553] PATCH https://kind-control-plane:6443/api/v1/nodes/kind-control-plane?timeout=10s 200 OK in 3 milliseconds
[upload-certs] Skipping phase. Please see --upload-certs
[mark-control-plane] Marking the node kind-control-plane as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]
[mark-control-plane] Marking the node kind-control-plane as control-plane by adding the taints [node-role.kubernetes.io/control-plane:NoSchedule]
I0429 08:07:26.766339 135 round_trippers.go:553] GET https://kind-control-plane:6443/api/v1/nodes/kind-control-plane?timeout=10s 200 OK in 1 milliseconds
I0429 08:07:26.770489 135 round_trippers.go:553] PATCH https://kind-control-plane:6443/api/v1/nodes/kind-control-plane?timeout=10s 200 OK in 3 milliseconds
[bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
I0429 08:07:26.771988 135 round_trippers.go:553] GET https://kind-control-plane:6443/api/v1/namespaces/kube-system/secrets/bootstrap-token-abcdef?timeout=10s 404 Not Found in 1 milliseconds
I0429 08:07:26.774246 135 round_trippers.go:553] POST https://kind-control-plane:6443/api/v1/namespaces/kube-system/secrets?timeout=10s 201 Created in 1 milliseconds
[bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes
I0429 08:07:26.776345 135 round_trippers.go:553] POST https://kind-control-plane:6443/apis/rbac.authorization.k8s.io/v1/clusterroles?timeout=10s 201 Created in 1 milliseconds
I0429 08:07:26.778282 135 round_trippers.go:553] POST https://kind-control-plane:6443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings?timeout=10s 201 Created in 1 milliseconds
[bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
I0429 08:07:26.780015 135 round_trippers.go:553] POST https://kind-control-plane:6443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings?timeout=10s 201 Created in 1 milliseconds
[bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
I0429 08:07:26.781571 135 round_trippers.go:553] POST https://kind-control-plane:6443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings?timeout=10s 201 Created in 1 milliseconds
[bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
I0429 08:07:26.783762 135 round_trippers.go:553] POST https://kind-control-plane:6443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings?timeout=10s 201 Created in 2 milliseconds
[bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
I0429 08:07:26.783842 135 clusterinfo.go:47] [bootstrap-token] loading admin kubeconfig
I0429 08:07:26.784135 135 loader.go:373] Config loaded from file: /etc/kubernetes/admin.conf
I0429 08:07:26.784146 135 clusterinfo.go:58] [bootstrap-token] copying the cluster from admin.conf to the bootstrap kubeconfig
I0429 08:07:26.784288 135 clusterinfo.go:70] [bootstrap-token] creating/updating ConfigMap in kube-public namespace
I0429 08:07:26.786057 135 round_trippers.go:553] POST https://kind-control-plane:6443/api/v1/namespaces/kube-public/configmaps?timeout=10s 201 Created in 1 milliseconds
I0429 08:07:26.786158 135 clusterinfo.go:84] creating the RBAC rules for exposing the cluster-info ConfigMap in the kube-public namespace
I0429 08:07:26.788162 135 round_trippers.go:553] POST https://kind-control-plane:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-public/roles?timeout=10s 201 Created in 1 milliseconds
I0429 08:07:26.790140 135 round_trippers.go:553] POST https://kind-control-plane:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-public/rolebindings?timeout=10s 201 Created in 1 milliseconds
I0429 08:07:26.790241 135 kubeletfinalize.go:90] [kubelet-finalize] Assuming that kubelet client certificate rotation is enabled: found "/var/lib/kubelet/pki/kubelet-client-current.pem"
[kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key
I0429 08:07:26.790488 135 loader.go:373] Config loaded from file: /etc/kubernetes/kubelet.conf
I0429 08:07:26.790717 135 kubeletfinalize.go:134] [kubelet-finalize] Restarting the kubelet to enable client certificate rotation
I0429 08:07:26.926544 135 round_trippers.go:553] GET https://kind-control-plane:6443/apis/apps/v1/namespaces/kube-system/deployments?labelSelector=k8s-app%3Dkube-dns 200 OK in 2 milliseconds
I0429 08:07:26.930089 135 round_trippers.go:553] GET https://kind-control-plane:6443/api/v1/namespaces/kube-system/configmaps/coredns?timeout=10s 404 Not Found in 1 milliseconds
I0429 08:07:26.932878 135 round_trippers.go:553] POST https://kind-control-plane:6443/api/v1/namespaces/kube-system/configmaps?timeout=10s 201 Created in 2 milliseconds
I0429 08:07:26.935097 135 round_trippers.go:553] POST https://kind-control-plane:6443/apis/rbac.authorization.k8s.io/v1/clusterroles?timeout=10s 201 Created in 1 milliseconds
I0429 08:07:26.937274 135 round_trippers.go:553] POST https://kind-control-plane:6443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings?timeout=10s 201 Created in 1 milliseconds
I0429 08:07:26.939560 135 round_trippers.go:553] POST https://kind-control-plane:6443/api/v1/namespaces/kube-system/serviceaccounts?timeout=10s 201 Created in 2 milliseconds
I0429 08:07:26.946747 135 round_trippers.go:553] POST https://kind-control-plane:6443/apis/apps/v1/namespaces/kube-system/deployments?timeout=10s 201 Created in 6 milliseconds
I0429 08:07:26.951967 135 round_trippers.go:553] POST https://kind-control-plane:6443/api/v1/namespaces/kube-system/services?timeout=10s 201 Created in 4 milliseconds
[addons] Applied essential addon: CoreDNS
I0429 08:07:26.955728 135 round_trippers.go:553] POST https://kind-control-plane:6443/api/v1/namespaces/kube-system/configmaps?timeout=10s 201 Created in 2 milliseconds
I0429 08:07:26.959698 135 round_trippers.go:553] POST https://kind-control-plane:6443/apis/apps/v1/namespaces/kube-system/daemonsets?timeout=10s 201 Created in 3 milliseconds
I0429 08:07:26.968211 135 round_trippers.go:553] POST https://kind-control-plane:6443/api/v1/namespaces/kube-system/serviceaccounts?timeout=10s 201 Created in 3 milliseconds
I0429 08:07:26.970298 135 round_trippers.go:553] POST https://kind-control-plane:6443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings?timeout=10s 201 Created in 1 milliseconds
I0429 08:07:26.978224 135 round_trippers.go:553] POST https://kind-control-plane:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/roles?timeout=10s 201 Created in 2 milliseconds
I0429 08:07:27.175175 135 request.go:628] Waited for 196.734625ms due to client-side throttling, not priority and fairness, request: POST:https://kind-control-plane:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/rolebindings?timeout=10s
I0429 08:07:27.177065 135 round_trippers.go:553] POST https://kind-control-plane:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/rolebindings?timeout=10s 201 Created in 1 milliseconds
[addons] Applied essential addon: kube-proxy
I0429 08:07:27.177411 135 loader.go:373] Config loaded from file: /etc/kubernetes/admin.conf
I0429 08:07:27.177698 135 loader.go:373] Config loaded from file: /etc/kubernetes/admin.conf
Your Kubernetes control-plane has initialized successfully!
To start using your cluster, you need to run the following as a regular user:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
Alternatively, if you are the root user, you can run:
export KUBECONFIG=/etc/kubernetes/admin.conf
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
https://kubernetes.io/docs/concepts/cluster-administration/addons/
You can now join any number of control-plane nodes by copying certificate authorities
and service account keys on each node and then running the following as root:
kubeadm join kind-control-plane:6443 --token
--discovery-token-ca-cert-hash sha256:0e7645f3f223d6bf59519b6f28c98dc12f93af56637a8e70f613117f834802c8 \
--control-plane
Then you can join any number of worker nodes by running the following on each as root:
kubeadm join kind-control-plane:6443 --token
--discovery-token-ca-cert-hash sha256:0e7645f3f223d6bf59519b6f28c98dc12f93af56637a8e70f613117f834802c8
✓ Starting control-plane 🕹️
✓ Installing CNI 🔌
✓ Installing StorageClass 💾
DEBUG: kubeadmjoin/join.go:133] I0429 08:07:30.050482 135 join.go:412] [preflight] found NodeName empty; using OS hostname as NodeName
I0429 08:07:30.050537 135 joinconfiguration.go:76] loading configuration from "/kind/kubeadm.conf"
I0429 08:07:30.050923 135 controlplaneprepare.go:225] [download-certs] Skipping certs download
I0429 08:07:30.050947 135 join.go:529] [preflight] Discovering cluster-info
I0429 08:07:30.050952 135 token.go:80] [discovery] Created cluster-info discovery client, requesting info from "kind-control-plane:6443"
I0429 08:07:30.058360 135 round_trippers.go:553] GET https://kind-control-plane:6443/api/v1/namespaces/kube-public/configmaps/cluster-info?timeout=10s 200 OK in 6 milliseconds
I0429 08:07:30.058522 135 token.go:223] [discovery] The cluster-info ConfigMap does not yet contain a JWS signature for token ID "abcdef", will try again
I0429 08:07:35.762641 135 round_trippers.go:553] GET https://kind-control-plane:6443/api/v1/namespaces/kube-public/configmaps/cluster-info?timeout=10s 200 OK in 2 milliseconds
I0429 08:07:35.762896 135 token.go:223] [discovery] The cluster-info ConfigMap does not yet contain a JWS signature for token ID "abcdef", will try again
I0429 08:07:42.009998 135 round_trippers.go:553] GET https://kind-control-plane:6443/api/v1/namespaces/kube-public/configmaps/cluster-info?timeout=10s 200 OK in 4 milliseconds
I0429 08:07:42.010916 135 token.go:105] [discovery] Cluster info signature and contents are valid and no TLS pinning was specified, will use API Server "kind-control-plane:6443"
I0429 08:07:42.010950 135 discovery.go:52] [discovery] Using provided TLSBootstrapToken as authentication credentials for the join process
I0429 08:07:42.010964 135 join.go:543] [preflight] Fetching init configuration
I0429 08:07:42.010970 135 join.go:589] [preflight] Retrieving KubeConfig objects
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
I0429 08:07:42.019661 135 round_trippers.go:553] GET https://kind-control-plane:6443/api/v1/namespaces/kube-system/configmaps/kubeadm-config?timeout=10s 200 OK in 8 milliseconds
I0429 08:07:42.021846 135 round_trippers.go:553] GET https://kind-control-plane:6443/api/v1/namespaces/kube-system/configmaps/kube-proxy?timeout=10s 200 OK in 1 milliseconds
I0429 08:07:42.022721 135 kubelet.go:74] attempting to download the KubeletConfiguration from ConfigMap "kubelet-config"
I0429 08:07:42.023904 135 round_trippers.go:553] GET https://kind-control-plane:6443/api/v1/namespaces/kube-system/configmaps/kubelet-config?timeout=10s 200 OK in 1 milliseconds
I0429 08:07:42.025071 135 interface.go:432] Looking for default routes with IPv4 addresses
I0429 08:07:42.025083 135 interface.go:437] Default route transits interface "eth0"
I0429 08:07:42.025166 135 interface.go:209] Interface eth0 is up
I0429 08:07:42.025209 135 interface.go:257] Interface "eth0" has 3 addresses :[172.18.0.5/16 fc00:f853:ccd:e793::5/64 fe80::42:acff:fe12:5/64].
I0429 08:07:42.025226 135 interface.go:224] Checking addr 172.18.0.5/16.
I0429 08:07:42.025231 135 interface.go:231] IP found 172.18.0.5
I0429 08:07:42.025244 135 interface.go:263] Found valid IPv4 address 172.18.0.5 for interface "eth0".
I0429 08:07:42.025248 135 interface.go:443] Found active IP 172.18.0.5
I0429 08:07:42.036005 135 common.go:128] WARNING: tolerating control plane version v1.28.0-alpha.0.530+d8bdddcab42932-dirty as a pre-release version
I0429 08:07:42.036029 135 kubelet.go:121] [kubelet-start] writing bootstrap kubelet config file at /etc/kubernetes/bootstrap-kubelet.conf
I0429 08:07:42.036504 135 kubelet.go:136] [kubelet-start] writing CA certificate at /etc/kubernetes/pki/ca.crt
I0429 08:07:42.036756 135 loader.go:373] Config loaded from file: /etc/kubernetes/bootstrap-kubelet.conf
I0429 08:07:42.036940 135 kubelet.go:157] [kubelet-start] Checking for an existing Node in the cluster with name "kind-worker" and status "Ready"
I0429 08:07:42.038536 135 round_trippers.go:553] GET https://kind-control-plane:6443/api/v1/nodes/kind-worker?timeout=10s 404 Not Found in 1 milliseconds
I0429 08:07:42.038739 135 kubelet.go:172] [kubelet-start] Stopping the kubelet
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...
I0429 08:07:43.129088 135 loader.go:373] Config loaded from file: /etc/kubernetes/kubelet.conf
I0429 08:07:43.129611 135 cert_rotation.go:137] Starting client certificate rotation controller
I0429 08:07:43.129684 135 loader.go:373] Config loaded from file: /etc/kubernetes/kubelet.conf
I0429 08:07:43.129820 135 kubelet.go:220] [kubelet-start] preserving the crisocket information for the node
I0429 08:07:43.129848 135 patchnode.go:31] [patchnode] Uploading the CRI Socket information "unix:///run/containerd/containerd.sock" to the Node API object "kind-worker" as an annotation
I0429 08:07:43.635180 135 round_trippers.go:553] GET https://kind-control-plane:6443/api/v1/nodes/kind-worker?timeout=10s 404 Not Found in 4 milliseconds
I0429 08:07:44.132681 135 round_trippers.go:553] GET https://kind-control-plane:6443/api/v1/nodes/kind-worker?timeout=10s 200 OK in 1 milliseconds
I0429 08:07:44.136895 135 round_trippers.go:553] PATCH https://kind-control-plane:6443/api/v1/nodes/kind-worker?timeout=10s 200 OK in 3 milliseconds
This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.
Run 'kubectl get nodes' on the control-plane to see this node join the cluster.
DEBUG: kubeadmjoin/join.go:133] I0429 08:07:30.050342 133 join.go:412] [preflight] found NodeName empty; using OS hostname as NodeName
I0429 08:07:30.050414 133 joinconfiguration.go:76] loading configuration from "/kind/kubeadm.conf"
I0429 08:07:30.050917 133 controlplaneprepare.go:225] [download-certs] Skipping certs download
I0429 08:07:30.050930 133 join.go:529] [preflight] Discovering cluster-info
I0429 08:07:30.050937 133 token.go:80] [discovery] Created cluster-info discovery client, requesting info from "kind-control-plane:6443"
I0429 08:07:30.058360 133 round_trippers.go:553] GET https://kind-control-plane:6443/api/v1/namespaces/kube-public/configmaps/cluster-info?timeout=10s 200 OK in 6 milliseconds
I0429 08:07:30.058522 133 token.go:223] [discovery] The cluster-info ConfigMap does not yet contain a JWS signature for token ID "abcdef", will try again
I0429 08:07:36.512197 133 round_trippers.go:553] GET https://kind-control-plane:6443/api/v1/namespaces/kube-public/configmaps/cluster-info?timeout=10s 200 OK in 1 milliseconds
I0429 08:07:36.512330 133 token.go:223] [discovery] The cluster-info ConfigMap does not yet contain a JWS signature for token ID "abcdef", will try again
I0429 08:07:42.040704 133 round_trippers.go:553] GET https://kind-control-plane:6443/api/v1/namespaces/kube-public/configmaps/cluster-info?timeout=10s 200 OK in 1 milliseconds
I0429 08:07:42.041639 133 token.go:105] [discovery] Cluster info signature and contents are valid and no TLS pinning was specified, will use API Server "kind-control-plane:6443"
I0429 08:07:42.041750 133 discovery.go:52] [discovery] Using provided TLSBootstrapToken as authentication credentials for the join process
I0429 08:07:42.041837 133 join.go:543] [preflight] Fetching init configuration
I0429 08:07:42.041877 133 join.go:589] [preflight] Retrieving KubeConfig objects
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
I0429 08:07:42.047750 133 round_trippers.go:553] GET https://kind-control-plane:6443/api/v1/namespaces/kube-system/configmaps/kubeadm-config?timeout=10s 200 OK in 5 milliseconds
I0429 08:07:42.049491 133 round_trippers.go:553] GET https://kind-control-plane:6443/api/v1/namespaces/kube-system/configmaps/kube-proxy?timeout=10s 200 OK in 1 milliseconds
I0429 08:07:42.050134 133 kubelet.go:74] attempting to download the KubeletConfiguration from ConfigMap "kubelet-config"
I0429 08:07:42.051420 133 round_trippers.go:553] GET https://kind-control-plane:6443/api/v1/namespaces/kube-system/configmaps/kubelet-config?timeout=10s 200 OK in 1 milliseconds
I0429 08:07:42.052380 133 interface.go:432] Looking for default routes with IPv4 addresses
I0429 08:07:42.052391 133 interface.go:437] Default route transits interface "eth0"
I0429 08:07:42.052465 133 interface.go:209] Interface eth0 is up
I0429 08:07:42.052505 133 interface.go:257] Interface "eth0" has 3 addresses :[172.18.0.4/16 fc00:f853:ccd:e793::4/64 fe80::42:acff:fe12:4/64].
I0429 08:07:42.052520 133 interface.go:224] Checking addr 172.18.0.4/16.
I0429 08:07:42.052524 133 interface.go:231] IP found 172.18.0.4
I0429 08:07:42.052531 133 interface.go:263] Found valid IPv4 address 172.18.0.4 for interface "eth0".
I0429 08:07:42.052535 133 interface.go:443] Found active IP 172.18.0.4
I0429 08:07:42.056901 133 common.go:128] WARNING: tolerating control plane version v1.28.0-alpha.0.530+d8bdddcab42932-dirty as a pre-release version
I0429 08:07:42.056926 133 kubelet.go:121] [kubelet-start] writing bootstrap kubelet config file at /etc/kubernetes/bootstrap-kubelet.conf
I0429 08:07:42.057374 133 kubelet.go:136] [kubelet-start] writing CA certificate at /etc/kubernetes/pki/ca.crt
I0429 08:07:42.057616 133 loader.go:373] Config loaded from file: /etc/kubernetes/bootstrap-kubelet.conf
I0429 08:07:42.057802 133 kubelet.go:157] [kubelet-start] Checking for an existing Node in the cluster with name "kind-worker2" and status "Ready"
I0429 08:07:42.059305 133 round_trippers.go:553] GET https://kind-control-plane:6443/api/v1/nodes/kind-worker2?timeout=10s 404 Not Found in 1 milliseconds
I0429 08:07:42.059468 133 kubelet.go:172] [kubelet-start] Stopping the kubelet
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...
I0429 08:07:43.134993 133 loader.go:373] Config loaded from file: /etc/kubernetes/kubelet.conf
I0429 08:07:43.135354 133 cert_rotation.go:137] Starting client certificate rotation controller
I0429 08:07:43.135454 133 loader.go:373] Config loaded from file: /etc/kubernetes/kubelet.conf
I0429 08:07:43.135626 133 kubelet.go:220] [kubelet-start] preserving the crisocket information for the node
I0429 08:07:43.135651 133 patchnode.go:31] [patchnode] Uploading the CRI Socket information "unix:///run/containerd/containerd.sock" to the Node API object "kind-worker2" as an annotation
I0429 08:07:43.640349 133 round_trippers.go:553] GET https://kind-control-plane:6443/api/v1/nodes/kind-worker2?timeout=10s 404 Not Found in 4 milliseconds
I0429 08:07:44.137322 133 round_trippers.go:553] GET https://kind-control-plane:6443/api/v1/nodes/kind-worker2?timeout=10s 200 OK in 1 milliseconds
I0429 08:07:44.141827 133 round_trippers.go:553] PATCH https://kind-control-plane:6443/api/v1/nodes/kind-worker2?timeout=10s 200 OK in 3 milliseconds
This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.
Run 'kubectl get nodes' on the control-plane to see this node join the cluster.
✓ Joining worker nodes 🚜
Set kubectl context to "kind-kind"
You can now use your cluster with:
kubectl cluster-info --context kind-kind
Not sure what to do next? 😅 Check out https://kind.sigs.k8s.io/docs/user/quick-start/
chumble2TR91:kubernetes chumble$ kubectl cluster-info --context kind-kind
Kubernetes control plane is running at https://127.0.0.1:56623
CoreDNS is running at https://127.0.0.1:56623/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
Inspect the cluster details:
chumble2TR91:kubernetes chumble$ kubectl version
WARNING: This version information is deprecated and will be replaced with the output from kubectl version --short. Use --output=yaml|json to get the full version.
Client Version: version.Info{Major:"1", Minor:"25", GitVersion:"v1.25.4", GitCommit:"872a965c6c6526caa949f0c6ac028ef7aff3fb78", GitTreeState:"clean", BuildDate:"2022-11-09T13:36:36Z", GoVersion:"go1.19.3", Compiler:"gc", Platform:"darwin/arm64"}
Kustomize Version: v4.5.7
Server Version: version.Info{Major:"1", Minor:"28+", GitVersion:"v1.28.0-alpha.0.530+d8bdddcab42932-dirty", GitCommit:"d8bdddcab4293284ce9f11b12f37fb827fc56f7c", GitTreeState:"dirty", BuildDate:"2023-04-29T08:02:09Z", GoVersion:"go1.20.3", Compiler:"gc", Platform:"linux/arm64"}
WARNING: version difference between client (1.25) and server (1.28) exceeds the supported minor version skew of +/-1
chumble2TR91:kubernetes chumble$ /usr/local/bin/kubectl version
WARNING: This version information is deprecated and will be replaced with the output from kubectl version --short. Use --output=yaml|json to get the full version.
Client Version: version.Info{Major:"1", Minor:"25", GitVersion:"v1.25.4", GitCommit:"872a965c6c6526caa949f0c6ac028ef7aff3fb78", GitTreeState:"clean", BuildDate:"2022-11-09T13:36:36Z", GoVersion:"go1.19.3", Compiler:"gc", Platform:"darwin/arm64"}
Kustomize Version: v4.5.7
Server Version: version.Info{Major:"1", Minor:"28+", GitVersion:"v1.28.0-alpha.0.530+d8bdddcab42932-dirty", GitCommit:"d8bdddcab4293284ce9f11b12f37fb827fc56f7c", GitTreeState:"dirty", BuildDate:"2023-04-29T08:02:09Z", GoVersion:"go1.20.3", Compiler:"gc", Platform:"linux/arm64"}
WARNING: version difference between client (1.25) and server (1.28) exceeds the supported minor version skew of +/-1
cumble2TR91:kubernetes chumble$ _output/bin/kubectl version
WARNING: This version information is deprecated and will be replaced with the output from kubectl version --short. Use --output=yaml|json to get the full version.
Client Version: version.Info{Major:"1", Minor:"28+", GitVersion:"v1.28.0-alpha.0.476+81076233e71cf5-dirty", GitCommit:"81076233e71cf5ffa7e8041be4289fd54bde1527", GitTreeState:"dirty", BuildDate:"2023-04-29T05:20:40Z", GoVersion:"go1.20.3", Compiler:"gc", Platform:"darwin/arm64"}
Kustomize Version: v5.0.1
Server Version: version.Info{Major:"1", Minor:"28+", GitVersion:"v1.28.0-alpha.0.530+d8bdddcab42932-dirty", GitCommit:"d8bdddcab4293284ce9f11b12f37fb827fc56f7c", GitTreeState:"dirty", BuildDate:"2023-04-29T08:02:09Z", GoVersion:"go1.20.3", Compiler:"gc", Platform:"linux/arm64"}
chumble2TR91:kubernetes chumble$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
kind-control-plane Ready control-plane 23m v1.28.0-alpha.0.530+d8bdddcab42932-dirty
kind-worker Ready
kind-worker2 Ready
chumble2TR91:kubernetes chumble$ make WHAT="test/e2e/e2e.test"
go version go1.20.3 darwin/arm64
+++ [0429 14:01:30] Setting GOMAXPROCS: 10
+++ [0429 14:01:32] Building go targets for darwin/arm64
k8s.io/kubernetes/test/e2e/e2e.test (test)
chumble2TR91:kubernetes chumble$ ll test/e2e/e2e
e2e-example-config.json e2e.go e2e_test.go
chumble2TR91:kubernetes chumble$ ls -lh _output/bin/e2e.test
-rwxr-xr-x@ 1 chumble staff 142M Apr 29 14:02 _output/bin/e2e.test
chumble2TR91:kubernetes chumble$ ./_output/bin/e2e.test -context kind-kind -ginkgo.focus="\[sig-network\].*Conformance" -num-nodes 2
Apr 29 14:04:50.815: INFO: The --provider flag is not set. Continuing as if --provider=skeleton had been used.
I0429 14:04:50.816778 89284 e2e.go:117] Starting e2e run "4f955303-8f34-42ee-aed0-c2ba14b657e0" on Ginkgo node 1
Running Suite: Kubernetes e2e suite - /Users/chumble/gospace/src/k8s.io/kubernetes
==================================================================================
Random Seed: 1682757290 - will randomize all specs
Will run 40 of 7340 specs
SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS•SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS•SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS•SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS•SSSSSSSSSSSSSSSSSSSSSSSSSSSSS•SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS•SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS•SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS•SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS•SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS•SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS•SSSSSSSSSSSSSSSSSS•SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS•SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS•SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS•SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS•SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS•SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS•SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS•SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS•SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS•SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS•SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
Ref #
https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/conformance-tests.md#running-conformance-tests