Kubernetes Default Service CIDR ReconfigurationFEATURE STATE: Kubernetes v1.33 [stable] (enabled by default: true)This document shares how to reconfigure the default Service IP range(s) assigned to a cluster.Before you beginYou need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. It is recommended to run this tutorial on a cluster with at least two nodes that are not acting as control plane hosts. If you do not already have a cluster, you can create one by using minikube or you can use one of these Kubernetes playgrounds:iximiuz LabsKillercodaKodeKloudPlay with KubernetesYour Kubernetes server must be at or later than version v1.33.To check the version, enter kubectl version.Kubernetes Default Service CIDR ReconfigurationThis document explains how to manage the Service IP address range within a Kubernetes cluster, which also influences the cluster's supported IP families for Services.The IP families available for Service ClusterIPs are determined by the --service-cluster-ip-range flag to kube-apiserver. For a better understanding of Service IP address allocation, refer to the Services IP address allocation tracking documentation.Since Kubernetes 1.33, the Service IP families configured for the cluster are reflected by the ServiceCIDR object named kubernetes. The kubernetes ServiceCIDR object is created by the first kube-apiserver instance that starts, based on its configured --service-cluster-ip-range flag. To ensure consistent cluster behavior, all kube-apiserver instances must be configured with the same --service-cluster-ip-range values, which must match the default kubernetes ServiceCIDR object.Kubernetes Service CIDR Reconfiguration CategoriesWe can categorize Service CIDR reconfiguration into the following scenarios:Extending the existing Service CIDRs: This can be done dynamically by adding new ServiceCIDR objects without the need of reconfiguration of the kube-apiserver. Please refer to the dedicated documentation on Extending Service IP Ranges.Single-to-dual-stack conversion preserving the primary service CIDR: This involves introducing a secondary IP family (IPv6 to an IPv4-only cluster, or IPv4 to an IPv6-only cluster) while keeping the original IP family as primary. This requires an update to the kube-apiserver configuration and a corresponding modification of various cluster components that need to handle this additional IP family. These components include, but are not limited to, kube-proxy, the CNI or network plugin, service mesh implementations, and DNS services.Dual-to-single conversion preserving the primary service CIDR: This involves removing the secondary IP family from a dual-stack cluster, reverting to a single IP family while retaining the original primary IP family. In addition to the reconfiguration of the components to match the new IP family, you might need to address Services that were explicitly configured to use the removed IP family.Anything that results in changing the primary service CIDR: Completely replacing the default ServiceCIDR is a complex operation. If the new ServiceCIDR does not overlap with the existing one, it will require renumbering all existing Services and changing the kubernetes.default service. The case where the primary IP family also changes is even more complicated, and may require to change multiple cluster components (kubelet, network plugins, etc.) to match the new primary IP family.Manual Operations for Replacing the Default Service CIDRReconfiguring the default Service CIDR necessitates manual steps performed by the cluster operator, administrator, or the software managing the cluster lifecycle. These typically include:Updating the kube-apiserver configuration: Modify the --service-cluster-ip-range flag with the new IP range(s).Reconfiguring the network components: This is a critical step and the specific procedure depends on the different networking components in use. It might involve updating configuration files, restarting agent pods, or updating the components to manage the new Service CIDR(s) and the desired IP family configuration for Pods. Typical components can be the implementation of Kubernetes Services, such as kube-proxy, and the configured networking plugin, and potentially other networking components like service mesh controllers and DNS servers, to ensure they can correctly handle traffic and perform service discovery with the new IP family configuration.Managing existing Services: Services with IPs from the old CIDR need to be addressed if they are not within the new configured ranges. Options include recreation (leading to downtime and new IP assignments) or potentially more complex reconfiguration strategies.Recreating internal Kubernetes services: The kubernetes.default service must be deleted and recreated to obtain an IP address from the new Service CIDR if the primary IP family is changed or replaced by a different network.Illustrative Reconfiguration StepsThe following steps describe a controlled reconfiguration focusing on the completely replacement of the default Service CIDR and the recreation of the kubernetes.default Service:Start the kube-apiserver with the initial --service-cluster-ip-range.Create initial Services that obtain IPs from this range.Introduce a new Service CIDR as a temporary target for reconfiguration.Mark the kubernetes default Service CIDR for deletion (it will remain pending due to existing IPs and finalizers). This prevents new allocations from the old range.Recreate existing Services. They should now be allocated IPs from the new, temporary Service CIDR.Restart the kube-apiserver with the new Service CIDR(s) configured and shut down the old instance.Delete the kubernetes.default service. The new kube-apiserver will recreate it within the new Service CIDR.What's nextKubernetes Networking Concepts: https://kubernetes.io/docs/concepts/cluster-administration/networking/Kubernetes Dual-Stack Services: https://kubernetes.io/docs/concepts/services-networking/dual-stack/Extending Kubernetes Service IP Ranges: https://kubernetes.io/docs/tasks/network/extend-service-ip-ranges/


ServiceAccountServiceAccount binds together: * a name, understood by users, and perhaps by peripheral systems, for an identity * a principal that can be authenticated and authorized * a set of secrets.apiVersion: v1import "k8s.io/api/core/v1"ServiceAccountServiceAccount binds together: * a name, understood by users, and perhaps by peripheral systems, for an identity * a principal that can be authenticated and authorized * a set of secretsapiVersion: v1kind: ServiceAccountmetadata (ObjectMeta)Standard object's metadata. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadataautomountServiceAccountToken (boolean)AutomountServiceAccountToken indicates whether pods running as this service account should have an API token automatically mounted. Can be overridden at the pod level.imagePullSecrets ([]LocalObjectReference)Atomic: will be replaced during a mergeImagePullSecrets is a list of references to secrets in the same namespace to use for pulling any images in pods that reference this ServiceAccount. ImagePullSecrets are distinct from Secrets because Secrets can be mounted in the pod, but ImagePullSecrets are only accessed by the kubelet. More info: https://kubernetes.io/docs/concepts/containers/images/#specifying-imagepullsecrets-on-a-podsecrets ([]ObjectReference)Patch strategy: merge on key nameMap: unique values on key name will be kept during a mergeSecrets is a list of the secrets in the same namespace that pods running using this ServiceAccount are allowed to use. Pods are only limited to this list if this service account has a "kubernetes.io/enforce-mountable-secrets" annotation set to "true". The "kubernetes.io/enforce-mountable-secrets" annotation is deprecated since v1.32. Prefer separate namespaces to isolate access to mounted secrets. This field should not be used to find auto-generated service account token secrets for use outside of pods. Instead, tokens can be requested directly using the TokenRequest API, or service account token secrets can be manually created. More info: https://kubernetes.io/docs/concepts/configuration/secretServiceAccountListServiceAccountList is a list of ServiceAccount objectsapiVersion: v1kind: ServiceAccountListmetadata (ListMeta)Standard list metadata. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kindsitems ([]ServiceAccount), requiredList of ServiceAccounts. More info: https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/Operationsget read the specified ServiceAccountHTTP RequestGET /api/v1/namespaces/{namespace}/serviceaccounts/{name}Parametersname (in path): string, requiredname of the ServiceAccountnamespace (in path): string, requirednamespacepretty (in query): stringprettyResponse200 (ServiceAccount): OK401: Unauthorizedlist list or watch objects of kind ServiceAccountHTTP RequestGET /api/v1/namespaces/{namespace}/serviceaccountsParametersnamespace (in path): string, requirednamespaceallowWatchBookmarks (in query): booleanallowWatchBookmarkscontinue (in query): stringcontinuefieldSelector (in query): stringfieldSelectorlabelSelector (in query): stringlabelSelectorlimit (in query): integerlimitpretty (in query): stringprettyresourceVersion (in query): stringresourceVersionresourceVersionMatch (in query): stringresourceVersionMatchsendInitialEvents (in query): booleansendInitialEventstimeoutSeconds (in query): integertimeoutSecondswatch (in query): booleanwatchResponse200 (ServiceAccountList): OK401: Unauthorizedlist list or watch objects of kind ServiceAccountHTTP RequestGET /api/v1/serviceaccountsParametersallowWatchBookmarks (in query): booleanallowWatchBookmarkscontinue (in query): stringcontinuefieldSelector (in query): stringfieldSelectorlabelSelector (in query): stringlabelSelectorlimit (in query): integerlimitpretty (in query): stringprettyresourceVersion (in query): stringresourceVersionresourceVersionMatch (in query): stringresourceVersionMatchsendInitialEvents (in query): booleansendInitialEventstimeoutSeconds (in query): integertimeoutSecondswatch (in query): booleanwatchResponse200 (ServiceAccountList): OK401: Unauthorizedcreate create a ServiceAccountHTTP RequestPOST /api/v1/namespaces/{namespace}/serviceaccountsParametersnamespace (in path): string, requirednamespacebody: ServiceAccount, requireddryRun (in query): stringdryRunfieldManager (in query): stringfieldManagerfieldValidation (in query): stringfieldValidationpretty (in query): stringprettyResponse200 (ServiceAccount): OK201 (ServiceAccount): Created202 (ServiceAccount): Accepted401: Unauthorizedupdate replace the specified ServiceAccountHTTP RequestPUT /api/v1/namespaces/{namespace}/serviceaccounts/{name}Parametersname (in path): string, requiredname of the ServiceAccountnamespace (in path): string, requirednamespacebody: ServiceAccount, requireddryRun (in query): stringdryRunfieldManager (in query): stringfieldManagerfieldValidation (in query): stringfieldValidationpretty (in query): stringprettyResponse200 (ServiceAccount): OK201 (ServiceAccount): Created401: Unauthorizedpatch partially update the specified ServiceAccountHTTP RequestPATCH /api/v1/namespaces/{namespace}/serviceaccounts/{name}Parametersname (in path): string, requiredname of the ServiceAccountnamespace (in path): string, requirednamespacebody: Patch, requireddryRun (in query): stringdryRunfieldManager (in query): stringfieldManagerfieldValidation (in query): stringfieldValidationforce (in query): booleanforcepretty (in query): stringprettyResponse200 (ServiceAccount): OK201 (ServiceAccount): Created401: Unauthorizeddelete delete a ServiceAccountHTTP RequestDELETE /api/v1/namespaces/{namespace}/serviceaccounts/{name}Parametersname (in path): string, requiredname of the ServiceAccountnamespace (in path): string, requirednamespacebody: DeleteOptionsdryRun (in query): stringdryRungracePeriodSeconds (in query): integergracePeriodSecondsignoreStoreReadErrorWithClusterBreakingPotential (in query): booleanignoreStoreReadErrorWithClusterBreakingPotentialpretty (in query): stringprettypropagationPolicy (in query): stringpropagationPolicyResponse200 (ServiceAccount): OK202 (ServiceAccount): Accepted401: Unauthorizeddeletecollection delete collection of ServiceAccountHTTP RequestDELETE /api/v1/namespaces/{namespace}/serviceaccountsParametersnamespace (in path): string, requirednamespacebody: DeleteOptionscontinue (in query): stringcontinuedryRun (in query): stringdryRunfieldSelector (in query): stringfieldSelectorgracePeriodSeconds (in query): integergracePeriodSecondsignoreStoreReadErrorWithClusterBreakingPotential (in query): booleanignoreStoreReadErrorWithClusterBreakingPotentiallabelSelector (in query): stringlabelSelectorlimit (in query): integerlimitpretty (in query): stringprettypropagationPolicy (in query): stringpropagationPolicyresourceVersion (in query): stringresourceVersionresourceVersionMatch (in query): stringresourceVersionMatchsendInitialEvents (in query): booleansendInitialEventstimeoutSeconds (in query): integertimeoutSecondsResponse200 (Status): OK401: UnauthorizedThis page is automatically generated.If you plan to report an issue with this page, mention that the page is auto-generated in your issue description. The fix may need to happen elsewhere in the Kubernetes project.

ServiceAccount


Summary: CertificateSigningRequestCertificateSigningRequest objects provide a mechanism to obtain x509 certificates by submitting a certificate signing request, and having it asynchronously approved and issued.apiVersion: certificates.k8s.io/v1import "k8s.io/api/certificates/v1"CertificateSigningRequestCertificateSigningRequest objects provide a mechanism to obtain x509 certificates by submitting a certificate signing request, and having it asynchronously approved and issued.Kubelets use this API to obta

CertificateSigningRequest


Communicate Between Containers in the Same Pod Using a Shared VolumeThis page shows how to use a Volume to communicate between two Containers running in the same Pod. See also how to allow processes to communicate by sharing process namespace between containers.Before you beginYou need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. It is recommended to run this tutorial on a cluster with at least two nodes that are not acting as control plane hosts. If you do not already have a cluster, you can create one by using minikube or you can use one of these Kubernetes playgrounds:iximiuz LabsKillercodaKodeKloudPlay with KubernetesTo check the version, enter kubectl version.Creating a Pod that runs two ContainersIn this exercise, you create a Pod that runs two Containers. The two containers share a Volume that they can use to communicate. Here is the configuration file for the Pod:pods/two-container-pod.yaml apiVersion: v1 kind: Pod metadata: name: two-containers spec: restartPolicy: Never volumes: - name: shared-data emptyDir: {} containers: - name: nginx-container image: nginx volumeMounts: - name: shared-data mountPath: /usr/share/nginx/html - name: debian-container image: debian volumeMounts: - name: shared-data mountPath: /pod-data command: ["/bin/sh"] args: ["-c", "echo Hello from the debian container > /pod-data/index.html"] In the configuration file, you can see that the Pod has a Volume named shared-data.The first container listed in the configuration file runs an nginx server. The mount path for the shared Volume is /usr/share/nginx/html. The second container is based on the debian image, and has a mount path of /pod-data. The second container runs the following command and then terminates.echo Hello from the debian container > /pod-data/index.html Notice that the second container writes the index.html file in the root directory of the nginx server.Create the Pod and the two Containers:kubectl apply -f https://k8s.io/examples/pods/two-container-pod.yaml View information about the Pod and the Containers:kubectl get pod two-containers --output=yaml Here is a portion of the output:apiVersion: v1 kind: Pod metadata: ... name: two-containers namespace: default ... spec: ... containerStatuses: - containerID: docker://c1d8abd1 ... image: debian ... lastState: terminated: ... name: debian-container ... - containerID: docker://96c1ff2c5bb ... image: nginx ... name: nginx-container ... state: running: ... You can see that the debian Container has terminated, and the nginx Container is still running.Get a shell to nginx Container:kubectl exec -it two-containers -c nginx-container -- /bin/bash In your shell, verify that nginx is running:root@two-containers:/# apt-get update root@two-containers:/# apt-get install curl procps root@two-containers:/# ps aux The output is similar to this:USER PID ... STAT START TIME COMMAND root 1 ... Ss 21:12 0:00 nginx: master process nginx -g daemon off; Recall that the debian Container created the index.html file in the nginx root directory. Use curl to send a GET request to the nginx server:root@two-containers:/# curl localhost The output shows that nginx serves a web page written by the debian container:Hello from the debian container DiscussionThe primary reason that Pods can have multiple containers is to support helper applications that assist a primary application. Typical examples of helper applications are data pullers, data pushers, and proxies. Helper and primary applications often need to communicate with each other. Typically this is done through a shared filesystem, as shown in this exercise, or through the loopback network interface, localhost. An example of this pattern is a web server along with a helper program that polls a Git repository for new updates.The Volume in this exercise provides a way for Containers to communicate during the life of the Pod. If the Pod is deleted and recreated, any data stored in the shared Volume is lost.What's nextLearn more about patterns for composite containers.Learn about composite containers for modular architecture.See Configuring a Pod to Use a Volume for Storage.See Configure a Pod to share process namespace between containers in a PodSee Volume.See Pod.


kubectl getSynopsisDisplay one or many resources.Prints a table of the most important information about the specified resources. You can filter the list using a label selector and the --selector flag. If the desired resource type is namespaced you will only see results in the current namespace if you don't specify any namespace.By specifying the output as 'template' and providing a Go template as the value of the --template flag, you can filter the attributes of the fetched resources.Use "kubectl api-resources" for a complete list of supported resources.kubectl get [(-o|--output=)json|yaml|name|go-template|go-template-file|template|templatefile|jsonpath|jsonpath-as-json|jsonpath-file|custom-columns|custom-columns-file|wide] (TYPE[.VERSION][.GROUP] [NAME | -l label] | TYPE[.VERSION][.GROUP]/NAME ...) [flags] Examples # List all pods in ps output format kubectl get pods # List all pods in ps output format with more information (such as node name) kubectl get pods -o wide # List a single replication controller with specified NAME in ps output format kubectl get replicationcontroller web # List deployments in JSON output format, in the "v1" version of the "apps" API group kubectl get deployments.v1.apps -o json # List a single pod in JSON output format kubectl get -o json pod web-pod-13je7 # List a pod identified by type and name specified in "pod.yaml" in JSON output format kubectl get -f pod.yaml -o json # List resources from a directory with kustomization.yaml - e.g. dir/kustomization.yaml kubectl get -k dir/ # Return only the phase value of the specified pod kubectl get -o template pod/web-pod-13je7 --template={{.status.phase}} # List resource information in custom columns kubectl get pod test-pod -o custom-columns=CONTAINER:.spec.containers[0].name,IMAGE:.spec.containers[0].image # List all replication controllers and services together in ps output format kubectl get rc,services # List one or more resources by their type and names kubectl get rc/web service/frontend pods/web-pod-13je7 # List the 'status' subresource for a single pod kubectl get pod web-pod-13je7 --subresource status # List all deployments in namespace 'backend' kubectl get deployments.apps --namespace backend # List all pods existing in all namespaces kubectl get pods --all-namespaces Options-A, --all-namespacesIf present, list the requested object(s) across all namespaces. Namespace in current context is ignored even if specified with --namespace.--allow-missing-template-keys Default: trueIf true, ignore any errors in templates when a field or map key is missing in the template. Only applies to golang and jsonpath output formats.--chunk-size int Default: 500Return large lists in chunks rather than all at once. Pass 0 to disable. This flag is beta and may change in the future.--field-selector stringSelector (field query) to filter on, supports '=', '==', and '!='.(e.g. --field-selector key1=value1,key2=value2). The server only supports a limited number of field queries per type.-f, --filename stringsFilename, directory, or URL to files identifying the resource to get from a server.-h, --helphelp for get--ignore-not-foundIf set to true, suppresses NotFound error for specific objects that do not exist. Using this flag with commands that query for collections of resources has no effect when no resources are found.-k, --kustomize stringProcess the kustomization directory. This flag can't be used together with -f or -R.-L, --label-columns stringsAccepts a comma separated list of labels that are going to be presented as columns. Names are case-sensitive. You can also use multiple flag options like -L label1 -L label2...--no-headersWhen using the default or custom-column output format, don't print headers (default print headers).-o, --output stringOutput format. One of: (json, yaml, name, go-template, go-template-file, template, templatefile, jsonpath, jsonpath-as-json, jsonpath-file, custom-columns, custom-columns-file, wide). See custom columns [https://kubernetes.io/docs/reference/kubectl/#custom-columns], golang template [http://golang.org/pkg/text/template/#pkg-overview] and jsonpath template [https://kubernetes.io/docs/reference/kubectl/jsonpath/].--output-watch-eventsOutput watch event objects when --watch or --watch-only is used. Existing objects are output as initial ADDED events.--raw stringRaw URI to request from the server. Uses the transport specified by the kubeconfig file.-R, --recursiveProcess the directory used in -f, --filename recursively. Useful when you want to manage related manifests organized within the same directory.-l, --selector stringSelector (label query) to filter on, supports '=', '==', '!=', 'in', 'notin'.(e.g. -l key1=value1,key2=value2,key3 in (value3)). Matching objects must satisfy all of the specified label constraints.--server-print Default: trueIf true, have the server return the appropriate table output. Supports extension APIs and CRDs.--show-kindIf present, list the resource type for the requested object(s).--show-labelsWhen printing, show all labels as the last column (default hide labels column)--show-managed-fieldsIf true, keep the managedFields when printing objects in JSON or YAML format.--sort-by stringIf non-empty, sort list types using this field specification. The field specification is expressed as a JSONPath expression (e.g. '{.metadata.name}'). The field in the API resource specified by this JSONPath expression must be an integer or a string.--subresource stringIf specified, gets the subresource of the requested object.--template stringTemplate string or path to template file to use when -o=go-template, -o=go-template-file. The template format is golang templates [http://golang.org/pkg/text/template/#pkg-overview].-w, --watchAfter listing/getting the requested object, watch for changes.--watch-onlyWatch for changes to the requested object(s), without listing/getting first.Parent Options Inherited--as stringUsername to impersonate for the operation. User could be a regular user or a service account in a namespace.--as-group stringsGroup to impersonate for the operation, this flag can be repeated to specify multiple groups.--as-uid stringUID to impersonate for the operation.--cache-dir string Default: "$HOME/.kube/cache"Default cache directory--certificate-authority stringPath to a cert file for the certificate authority--client-certificate stringPath to a client certificate file for TLS--client-key stringPath to a client key file for TLS--cluster stringThe name of the kubeconfig cluster to use--context stringThe name of the kubeconfig context to use--disable-compressionIf true, opt-out of response compression for all requests to the server--insecure-skip-tls-verifyIf true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure--kubeconfig stringPath to the kubeconfig file to use for CLI requests.--kuberc stringPath to the kuberc file to use for preferences. This can be disabled by exporting KUBECTL_KUBERC=false feature gate or turning off the feature KUBERC=off.--match-server-versionRequire server version to match client version-n, --namespace stringIf present, the namespace scope for this CLI request--password stringPassword for basic authentication to the API server--profile string Default: "none"Name of profile to capture. One of (none|cpu|heap|goroutine|threadcreate|block|mutex)--profile-output string Default: "profile.pprof"Name of the file to write the profile to--request-timeout string Default: "0"The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests.-s, --server stringThe address and port of the Kubernetes API server--storage-driver-buffer-duration duration Default: 1m0sWrites in the storage driver will be buffered for this duration, and committed to the non memory backends as a single transaction--storage-driver-db string Default: "cadvisor"database name--storage-driver-host string Default: "localhost:8086"database host:port--storage-driver-password string Default: "root"database password--storage-driver-secureuse secure connection with database--storage-driver-table string Default: "stats"table name--storage-driver-user string Default: "root"database username--tls-server-name stringServer name to use for server certificate validation. If it is not provided, the hostname used to contact the server is used--token stringBearer token for authentication to the API server--user stringThe name of the kubeconfig user to use--username stringUsername for basic authentication to the API server--version version[=true]--version, --version=raw prints version information and quits; --version=vX.Y.Z... sets the reported version--warnings-as-errorsTreat warnings received from the server as errors and exit with a non-zero exit codeSee Alsokubectl - kubectl controls the Kubernetes cluster managerThis page is automatically generated.If you plan to report an issue with this page, mention that the page is auto-generated in your issue description. The fix may need to happen elsewhere in the Kubernetes project.

kubectl get


Kubernetes Metrics (v1beta1)Package v1beta1 is the v1beta1 version of the metrics API.Resource TypesNodeMetricsNodeMetricsListPodMetricsPodMetricsListNodeMetricsAppears in:NodeMetricsListNodeMetrics sets resource usage metrics of a node.FieldDescriptionapiVersionstringmetrics.k8s.io/v1beta1kindstringNodeMetricsmetadatameta/v1.ObjectMetaStandard object's metadata. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadataRefer to the Kubernetes API documentation for the fields of the metadata field.timestamp [Required]meta/v1.TimeThe following fields define time interval from which metrics were collected from the interval [Timestamp-Window, Timestamp].window [Required]meta/v1.DurationNo description provided.usage [Required]core/v1.ResourceListThe memory usage is the memory working set.NodeMetricsListNodeMetricsList is a list of NodeMetrics.FieldDescriptionapiVersionstringmetrics.k8s.io/v1beta1kindstringNodeMetricsListmetadata [Required]meta/v1.ListMetaStandard list metadata. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kindsitems [Required][]NodeMetricsList of node metrics.PodMetricsAppears in:PodMetricsListPodMetrics sets resource usage metrics of a pod.FieldDescriptionapiVersionstringmetrics.k8s.io/v1beta1kindstringPodMetricsmetadatameta/v1.ObjectMetaStandard object's metadata. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadataRefer to the Kubernetes API documentation for the fields of the metadata field.timestamp [Required]meta/v1.TimeThe following fields define time interval from which metrics were collected from the interval [Timestamp-Window, Timestamp].window [Required]meta/v1.DurationNo description provided.containers [Required][]ContainerMetricsMetrics for all containers are collected within the same time window.PodMetricsListPodMetricsList is a list of PodMetrics.FieldDescriptionapiVersionstringmetrics.k8s.io/v1beta1kindstringPodMetricsListmetadata [Required]meta/v1.ListMetaStandard list metadata. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kindsitems [Required][]PodMetricsList of pod metrics.ContainerMetricsAppears in:PodMetricsContainerMetrics sets resource usage metrics of a container.FieldDescriptionname [Required]stringContainer name corresponding to the one from pod.spec.containers.usage [Required]core/v1.ResourceListThe memory usage is the memory working set.This page is automatically generated.If you plan to report an issue with this page, mention that the page is auto-generated in your issue description. The fix may need to happen elsewhere in the Kubernetes project.

Kubernetes Metrics (v1beta1)


kubectl set imageSynopsisUpdate existing container image(s) of resources.Possible resources include (case insensitive): pod (po), replicationcontroller (rc), deployment (deploy), daemonset (ds), statefulset (sts), cronjob (cj), replicaset (rs) kubectl set image (-f FILENAME | TYPE NAME) CONTAINER_NAME_1=CONTAINER_IMAGE_1 ... CONTAINER_NAME_N=CONTAINER_IMAGE_N Examples # Set a deployment's nginx container image to 'nginx:1.9.1', and its busybox container image to 'busybox' kubectl set image deployment/nginx busybox=busybox nginx=nginx:1.9.1 # Update all deployments' and rc's nginx container's image to 'nginx:1.9.1' kubectl set image deployments,rc nginx=nginx:1.9.1 --all # Update image of all containers of daemonset abc to 'nginx:1.9.1' kubectl set image daemonset abc *=nginx:1.9.1 # Print result (in yaml format) of updating nginx container image from local file, without hitting the server kubectl set image -f path/to/file.yaml nginx=nginx:1.9.1 --local -o yaml Options--allSelect all resources, in the namespace of the specified resource types--allow-missing-template-keys Default: trueIf true, ignore any errors in templates when a field or map key is missing in the template. Only applies to golang and jsonpath output formats.--dry-run string[="unchanged"] Default: "none"Must be "none", "server", or "client". If client strategy, only print the object that would be sent, without sending it. If server strategy, submit server-side request without persisting the resource.--field-manager string Default: "kubectl-set"Name of the manager used to track field ownership.-f, --filename stringsFilename, directory, or URL to files identifying the resource to get from a server.-h, --helphelp for image-k, --kustomize stringProcess the kustomization directory. This flag can't be used together with -f or -R.--localIf true, set image will NOT contact api-server but run locally.-o, --output stringOutput format. One of: (json, yaml, name, go-template, go-template-file, template, templatefile, jsonpath, jsonpath-as-json, jsonpath-file).-R, --recursiveProcess the directory used in -f, --filename recursively. Useful when you want to manage related manifests organized within the same directory.-l, --selector stringSelector (label query) to filter on, supports '=', '==', '!=', 'in', 'notin'.(e.g. -l key1=value1,key2=value2,key3 in (value3)). Matching objects must satisfy all of the specified label constraints.--show-managed-fieldsIf true, keep the managedFields when printing objects in JSON or YAML format.--template stringTemplate string or path to template file to use when -o=go-template, -o=go-template-file. The template format is golang templates [http://golang.org/pkg/text/template/#pkg-overview].Parent Options Inherited--as stringUsername to impersonate for the operation. User could be a regular user or a service account in a namespace.--as-group stringsGroup to impersonate for the operation, this flag can be repeated to specify multiple groups.--as-uid stringUID to impersonate for the operation.--cache-dir string Default: "$HOME/.kube/cache"Default cache directory--certificate-authority stringPath to a cert file for the certificate authority--client-certificate stringPath to a client certificate file for TLS--client-key stringPath to a client key file for TLS--cluster stringThe name of the kubeconfig cluster to use--context stringThe name of the kubeconfig context to use--disable-compressionIf true, opt-out of response compression for all requests to the server--insecure-skip-tls-verifyIf true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure--kubeconfig stringPath to the kubeconfig file to use for CLI requests.--kuberc stringPath to the kuberc file to use for preferences. This can be disabled by exporting KUBECTL_KUBERC=false feature gate or turning off the feature KUBERC=off.--match-server-versionRequire server version to match client version-n, --namespace stringIf present, the namespace scope for this CLI request--password stringPassword for basic authentication to the API server--profile string Default: "none"Name of profile to capture. One of (none|cpu|heap|goroutine|threadcreate|block|mutex)--profile-output string Default: "profile.pprof"Name of the file to write the profile to--request-timeout string Default: "0"The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests.-s, --server stringThe address and port of the Kubernetes API server--storage-driver-buffer-duration duration Default: 1m0sWrites in the storage driver will be buffered for this duration, and committed to the non memory backends as a single transaction--storage-driver-db string Default: "cadvisor"database name--storage-driver-host string Default: "localhost:8086"database host:port--storage-driver-password string Default: "root"database password--storage-driver-secureuse secure connection with database--storage-driver-table string Default: "stats"table name--storage-driver-user string Default: "root"database username--tls-server-name stringServer name to use for server certificate validation. If it is not provided, the hostname used to contact the server is used--token stringBearer token for authentication to the API server--user stringThe name of the kubeconfig user to use--username stringUsername for basic authentication to the API server--version version[=true]--version, --version=raw prints version information and quits; --version=vX.Y.Z... sets the reported version--warnings-as-errorsTreat warnings received from the server as errors and exit with a non-zero exit codeSee Alsokubectl set - Set specific features on objectsThis page is automatically generated.If you plan to report an issue with this page, mention that the page is auto-generated in your issue description. The fix may need to happen elsewhere in the Kubernetes project.

kubectl set image


kubectl Usage ConventionsRecommended usage conventions for kubectl.Using kubectl in Reusable ScriptsFor a stable output in a script:Request one of the machine-oriented output forms, such as -o name, -o json, -o yaml, -o go-template, or -o jsonpath.Fully-qualify the version. For example, jobs.v1.batch/myjob. This will ensure that kubectl does not use its default version that can change over time.Don't rely on context, preferences, or other implicit states.SubresourcesYou can use the --subresource argument for kubectl subcommands such as get, patch, edit, apply and replace to fetch and update subresources for all resources that support them. In Kubernetes version 1.34, only the status, scale and resize subresources are supported.For kubectl edit, the scale subresource is not supported. If you use --subresource with kubectl edit and specify scale as the subresource, the command will error out.The API contract against a subresource is identical to a full resource. While updating the status subresource to a new value, keep in mind that the subresource could be potentially reconciled by a controller to a different value.Best Practiceskubectl runFor kubectl run to satisfy infrastructure as code:Tag the image with a version-specific tag and don't move that tag to a new version. For example, use :v1234, v1.2.3, r03062016-1-4, rather than :latest (For more information, see Best Practices for Configuration).Check in the script for an image that is heavily parameterized.Switch to configuration files checked into source control for features that are needed, but not expressible via kubectl run flags.You can use the --dry-run=client flag to preview the object that would be sent to your cluster, without really submitting it.kubectl applyYou can use kubectl apply to create or update resources. For more information about using kubectl apply to update resources, see Kubectl Book.

kubectl Usage Conventions


Pod Topology Spread ConstraintsYou can use topology spread constraints to control how Pods are spread across your cluster among failure-domains such as regions, zones, nodes, and other user-defined topology domains. This can help to achieve high availability as well as efficient resource utilization.You can set cluster-level constraints as a default, or configure topology spread constraints for individual workloads.MotivationImagine that you have a cluster of up to twenty nodes, and you want to run a workload that automatically scales how many replicas it uses. There could be as few as two Pods or as many as fifteen. When there are only two Pods, you'd prefer not to have both of those Pods run on the same node: you would run the risk that a single node failure takes your workload offline.In addition to this basic usage, there are some advanced usage examples that enable your workloads to benefit on high availability and cluster utilization.As you scale up and run more Pods, a different concern becomes important. Imagine that you have three nodes running five Pods each. The nodes have enough capacity to run that many replicas; however, the clients that interact with this workload are split across three different datacenters (or infrastructure zones). Now you have less concern about a single node failure, but you notice that latency is higher than you'd like, and you are paying for network costs associated with sending network traffic between the different zones.You decide that under normal operation you'd prefer to have a similar number of replicas scheduled into each infrastructure zone, and you'd like the cluster to self-heal in the case that there is a problem.Pod topology spread constraints offer you a declarative way to configure that.topologySpreadConstraints fieldThe Pod API includes a field, spec.topologySpreadConstraints. The usage of this field looks like the following:--- apiVersion: v1 kind: Pod metadata: name: example-pod spec: # Configure a topology spread constraint topologySpreadConstraints: - maxSkew: <integer> minDomains: <integer> # optional topologyKey: <string> whenUnsatisfiable: <string> labelSelector: <object> matchLabelKeys: <list> # optional; beta since v1.27 nodeAffinityPolicy: [Honor|Ignore] # optional; beta since v1.26 nodeTaintsPolicy: [Honor|Ignore] # optional; beta since v1.26 ### other Pod fields go here You can read more about this field by running kubectl explain Pod.spec.topologySpreadConstraints or refer to the scheduling section of the API reference for Pod.Spread constraint definitionYou can define one or multiple topologySpreadConstraints entries to instruct the kube-scheduler how to place each incoming Pod in relation to the existing Pods across your cluster. Those fields are:maxSkew describes the degree to which Pods may be unevenly distributed. You must specify this field and the number must be greater than zero. Its semantics differ according to the value of whenUnsatisfiable:if you select whenUnsatisfiable: DoNotSchedule, then maxSkew defines the maximum permitted difference between the number of matching pods in the target topology and the global minimum (the minimum number of matching pods in an eligible domain or zero if the number of eligible domains is less than MinDomains). For example, if you have 3 zones with 2, 2 and 1 matching pods respectively, MaxSkew is set to 1 then the global minimum is 1.if you select whenUnsatisfiable: ScheduleAnyway, the scheduler gives higher precedence to topologies that would help reduce the skew.minDomains indicates a minimum number of eligible domains. This field is optional. A domain is a particular instance of a topology. An eligible domain is a domain whose nodes match the node selector.Note:Before Kubernetes v1.30, the minDomains field was only available if the MinDomainsInPodTopologySpread feature gate was enabled (default since v1.28). In older Kubernetes clusters it might be explicitly disabled or the field might not be available.The value of minDomains must be greater than 0, when specified. You can only specify minDomains in conjunction with whenUnsatisfiable: DoNotSchedule.When the number of eligible domains with match topology keys is less than minDomains, Pod topology spread treats global minimum as 0, and then the calculation of skew is performed. The global minimum is the minimum number of matching Pods in an eligible domain, or zero if the number of eligible domains is less than minDomains.When the number of eligible domains with matching topology keys equals or is greater than minDomains, this value has no effect on scheduling.If you do not specify minDomains, the constraint behaves as if minDomains is 1.topologyKey is the key of node labels. Nodes that have a label with this key and identical values are considered to be in the same topology. We call each instance of a topology (in other words, a <key, value> pair) a domain. The scheduler will try to put a balanced number of pods into each domain. Also, we define an eligible domain as a domain whose nodes meet the requirements of nodeAffinityPolicy and nodeTaintsPolicy.whenUnsatisfiable indicates how to deal with a Pod if it doesn't satisfy the spread constraint:DoNotSchedule (default) tells the scheduler not to schedule it.ScheduleAnyway tells the scheduler to still schedule it while prioritizing nodes that minimize the skew.labelSelector is used to find matching Pods. Pods that match this label selector are counted to determine the number of Pods in their corresponding topology domain. See Label Selectors for more details.matchLabelKeys is a list of pod label keys to select the group of pods over which the spreading skew will be calculated. At a pod creation, the kube-apiserver uses those keys to lookup values from the incoming pod labels, and those key-value labels will be merged with any existing labelSelector. The same key is forbidden to exist in both matchLabelKeys and labelSelector. matchLabelKeys cannot be set when labelSelector isn't set. Keys that don't exist in the pod labels will be ignored. A null or empty list means only match against the labelSelector.Caution:It's not recommended to use matchLabelKeys with labels that might be updated directly on pods. Even if you edit the pod's label that is specified at matchLabelKeys directly, (that is, you edit the Pod and not a Deployment), kube-apiserver doesn't reflect the label update onto the merged labelSelector.With matchLabelKeys, you don't need to update the pod.spec between different revisions. The controller/operator just needs to set different values to the same label key for different revisions. For example, if you are configuring a Deployment, you can use the label keyed with pod-template-hash, which is added automatically by the Deployment controller, to distinguish between different revisions in a single Deployment. topologySpreadConstraints: - maxSkew: 1 topologyKey: kubernetes.io/hostname whenUnsatisfiable: DoNotSchedule labelSelector: matchLabels: app: foo matchLabelKeys: - pod-template-hash Note:The matchLabelKeys field is a beta-level field and enabled by default in 1.27. You can disable it by disabling the MatchLabelKeysInPodTopologySpread feature gate.Before v1.34, matchLabelKeys was handled implicitly. Since v1.34, key-value labels corresponding to matchLabelKeys are explicitly merged into labelSelector. You can disable it and revert to the previous behavior by disabling the MatchLabelKeysInPodTopologySpreadSelectorMerge feature gate of kube-apiserver.nodeAffinityPolicy indicates how we will treat Pod's nodeAffinity/nodeSelector when calculating pod topology spread skew. Options are:Honor: only nodes matching nodeAffinity/nodeSelector are included in the calculations.Ignore: nodeAffinity/nodeSelector are ignored. All nodes are included in the calculations.If this value is null, the behavior is equivalent to the Honor policy.Note:The nodeAffinityPolicy became beta in 1.26 and graduated to GA in 1.33. It's enabled by default in beta, you can disable it by disabling the NodeInclusionPolicyInPodTopologySpread feature gate.nodeTaintsPolicy indicates how we will treat node taints when calculating pod topology spread skew. Options are:Honor: nodes without taints, along with tainted nodes for which the incoming pod has a toleration, are included.Ignore: node taints are ignored. All nodes are included.If this value is null, the behavior is equivalent to the Ignore policy.Note:The nodeTaintsPolicy became beta in 1.26 and graduated to GA in 1.33. It's enabled by default in beta, you can disable it by disabling the NodeInclusionPolicyInPodTopologySpread feature gate.When a Pod defines more than one topologySpreadConstraint, those constraints are combined using a logical AND operation: the kube-scheduler looks for a node for the incoming Pod that satisfies all the configured constraints.Node labelsTopology spread constraints rely on node labels to identify the topology domain(s) that each node is in. For example, a node might have labels: region: us-east-1 zone: us-east-1a Note:For brevity, this example doesn't use the well-known label keys topology.kubernetes.io/zone and topology.kubernetes.io/region. However, those registered label keys are nonetheless recommended rather than the private (unqualified) label keys region and zone that are used here.You can't make a reliable assumption about the meaning of a private label key between different contexts.Suppose you have a 4-node cluster with the following labels:NAME STATUS ROLES AGE VERSION LABELS node1 Ready <none> 4m26s v1.16.0 node=node1,zone=zoneA node2 Ready <none> 3m58s v1.16.0 node=node2,zone=zoneA node3 Ready <none> 3m17s v1.16.0 node=node3,zone=zoneB node4 Ready <none> 2m43s v1.16.0 node=node4,zone=zoneB Then the cluster is logically viewed as below:graph TB subgraph "zoneB" n3(Node3) n4(Node4) end subgraph "zoneA" n1(Node1) n2(Node2) end classDef plain fill:#ddd,stroke:#fff,stroke-width:4px,color:#000; classDef k8s fill:#326ce5,stroke:#fff,stroke-width:4px,color:#fff; classDef cluster fill:#fff,stroke:#bbb,stroke-width:2px,color:#326ce5; class n1,n2,n3,n4 k8s; class zoneA,zoneB cluster;JavaScript must be enabled to view this contentConsistencyYou should set the same Pod topology spread constraints on all pods in a group.Usually, if you are using a workload controller such as a Deployment, the pod template takes care of this for you. If you mix different spread constraints then Kubernetes follows the API definition of the field; however, the behavior is more likely to become confusing and troubleshooting is less straightforward.You need a mechanism to ensure that all the nodes in a topology domain (such as a cloud provider region) are labeled consistently. To avoid you needing to manually label nodes, most clusters automatically populate well-known labels such as kubernetes.io/hostname. Check whether your cluster supports this.Topology spread constraint examplesExample: one topology spread constraintSuppose you have a 4-node cluster where 3 Pods labeled foo: bar are located in node1, node2 and node3 respectively:graph BT subgraph "zoneB" p3(Pod) --> n3(Node3) n4(Node4) end subgraph "zoneA" p1(Pod) --> n1(Node1) p2(Pod) --> n2(Node2) end classDef plain fill:#ddd,stroke:#fff,stroke-width:4px,color:#000; classDef k8s fill:#326ce5,stroke:#fff,stroke-width:4px,color:#fff; classDef cluster fill:#fff,stroke:#bbb,stroke-width:2px,color:#326ce5; class n1,n2,n3,n4,p1,p2,p3 k8s; class zoneA,zoneB cluster;JavaScript must be enabled to view this contentIf you want an incoming Pod to be evenly spread with existing Pods across zones, you can use a manifest similar to:pods/topology-spread-constraints/one-constraint.yaml kind: Pod apiVersion: v1 metadata: name: mypod labels: foo: bar spec: topologySpreadConstraints: - maxSkew: 1 topologyKey: zone whenUnsatisfiable: DoNotSchedule labelSelector: matchLabels: foo: bar containers: - name: pause image: registry.k8s.io/pause:3.1From that manifest, topologyKey: zone implies the even distribution will only be applied to nodes that are labeled zone: <any value> (nodes that don't have a zone label are skipped). The field whenUnsatisfiable: DoNotSchedule tells the scheduler to let the incoming Pod stay pending if the scheduler can't find a way to satisfy the constraint.If the scheduler placed this incoming Pod into zone A, the distribution of Pods would become [3, 1]. That means the actual skew is then 2 (calculated as 3 - 1), which violates maxSkew: 1. To satisfy the constraints and context for this example, the incoming Pod can only be placed onto a node in zone B:graph BT subgraph "zoneB" p3(Pod) --> n3(Node3) p4(mypod) --> n4(Node4) end subgraph "zoneA" p1(Pod) --> n1(Node1) p2(Pod) --> n2(Node2) end classDef plain fill:#ddd,stroke:#fff,stroke-width:4px,color:#000; classDef k8s fill:#326ce5,stroke:#fff,stroke-width:4px,color:#fff; classDef cluster fill:#fff,stroke:#bbb,stroke-width:2px,color:#326ce5; class n1,n2,n3,n4,p1,p2,p3 k8s; class p4 plain; class zoneA,zoneB cluster;JavaScript must be enabled to view this contentORgraph BT subgraph "zoneB" p3(Pod) --> n3(Node3) p4(mypod) --> n3 n4(Node4) end subgraph "zoneA" p1(Pod) --> n1(Node1) p2(Pod) --> n2(Node2) end classDef plain fill:#ddd,stroke:#fff,stroke-width:4px,color:#000; classDef k8s fill:#326ce5,stroke:#fff,stroke-width:4px,color:#fff; classDef cluster fill:#fff,stroke:#bbb,stroke-width:2px,color:#326ce5; class n1,n2,n3,n4,p1,p2,p3 k8s; class p4 plain; class zoneA,zoneB cluster;JavaScript must be enabled to view this contentYou can tweak the Pod spec to meet various kinds of requirements:Change maxSkew to a bigger value - such as 2 - so that the incoming Pod can be placed into zone A as well.Change topologyKey to node so as to distribute the Pods evenly across nodes instead of zones. In the above example, if maxSkew remains 1, the incoming Pod can only be placed onto the node node4.Change whenUnsatisfiable: DoNotSchedule to whenUnsatisfiable: ScheduleAnyway to ensure the incoming Pod to be always schedulable (suppose other scheduling APIs are satisfied). However, it's preferred to be placed into the topology domain which has fewer matching Pods. (Be aware that this preference is jointly normalized with other internal scheduling priorities such as resource usage ratio).Example: multiple topology spread constraintsThis builds upon the previous example. Suppose you have a 4-node cluster where 3 existing Pods labeled foo: bar are located on node1, node2 and node3 respectively:graph BT subgraph "zoneB" p3(Pod) --> n3(Node3) n4(Node4) end subgraph "zoneA" p1(Pod) --> n1(Node1) p2(Pod) --> n2(Node2) end classDef plain fill:#ddd,stroke:#fff,stroke-width:4px,color:#000; classDef k8s fill:#326ce5,stroke:#fff,stroke-width:4px,color:#fff; classDef cluster fill:#fff,stroke:#bbb,stroke-width:2px,color:#326ce5; class n1,n2,n3,n4,p1,p2,p3 k8s; class p4 plain; class zoneA,zoneB cluster;JavaScript must be enabled to view this contentYou can combine two topology spread constraints to control the spread of Pods both by node and by zone:pods/topology-spread-constraints/two-constraints.yaml kind: Pod apiVersion: v1 metadata: name: mypod labels: foo: bar spec: topologySpreadConstraints: - maxSkew: 1 topologyKey: zone whenUnsatisfiable: DoNotSchedule labelSelector: matchLabels: foo: bar - maxSkew: 1 topologyKey: node whenUnsatisfiable: DoNotSchedule labelSelector: matchLabels: foo: bar containers: - name: pause image: registry.k8s.io/pause:3.1In this case, to match the first constraint, the incoming Pod can only be placed onto nodes in zone B; while in terms of the second constraint, the incoming Pod can only be scheduled to the node node4. The scheduler only considers options that satisfy all defined constraints, so the only valid placement is onto node node4.Example: conflicting topology spread constraintsMultiple constraints can lead to conflicts. Suppose you have a 3-node cluster across 2 zones:graph BT subgraph "zoneB" p4(Pod) --> n3(Node3) p5(Pod) --> n3 end subgraph "zoneA" p1(Pod) --> n1(Node1) p2(Pod) --> n1 p3(Pod) --> n2(Node2) end classDef plain fill:#ddd,stroke:#fff,stroke-width:4px,color:#000; classDef k8s fill:#326ce5,stroke:#fff,stroke-width:4px,color:#fff; classDef cluster fill:#fff,stroke:#bbb,stroke-width:2px,color:#326ce5; class n1,n2,n3,n4,p1,p2,p3,p4,p5 k8s; class zoneA,zoneB cluster;JavaScript must be enabled to view this contentIf you were to apply two-constraints.yaml (the manifest from the previous example) to this cluster, you would see that the Pod mypod stays in the Pending state. This happens because: to satisfy the first constraint, the Pod mypod can only be placed into zone B; while in terms of the second constraint, the Pod mypod can only schedule to node node2. The intersection of the two constraints returns an empty set, and the scheduler cannot place the Pod.To overcome this situation, you can either increase the value of maxSkew or modify one of the constraints to use whenUnsatisfiable: ScheduleAnyway. Depending on circumstances, you might also decide to delete an existing Pod manually - for example, if you are troubleshooting why a bug-fix rollout is not making progress.Interaction with node affinity and node selectorsThe scheduler will skip the non-matching nodes from the skew calculations if the incoming Pod has spec.nodeSelector or spec.affinity.nodeAffinity defined.Example: topology spread constraints with node affinitySuppose you have a 5-node cluster ranging across zones A to C:graph BT subgraph "zoneB" p3(Pod) --> n3(Node3) n4(Node4) end subgraph "zoneA" p1(Pod) --> n1(Node1) p2(Pod) --> n2(Node2) end classDef plain fill:#ddd,stroke:#fff,stroke-width:4px,color:#000; classDef k8s fill:#326ce5,stroke:#fff,stroke-width:4px,color:#fff; classDef cluster fill:#fff,stroke:#bbb,stroke-width:2px,color:#326ce5; class n1,n2,n3,n4,p1,p2,p3 k8s; class p4 plain; class zoneA,zoneB cluster;JavaScript must be enabled to view this contentgraph BT subgraph "zoneC" n5(Node5) end classDef plain fill:#ddd,stroke:#fff,stroke-width:4px,color:#000; classDef k8s fill:#326ce5,stroke:#fff,stroke-width:4px,color:#fff; classDef cluster fill:#fff,stroke:#bbb,stroke-width:2px,color:#326ce5; class n5 k8s; class zoneC cluster;JavaScript must be enabled to view this contentand you know that zone C must be excluded. In this case, you can compose a manifest as below, so that Pod mypod will be placed into zone B instead of zone C. Similarly, Kubernetes also respects spec.nodeSelector.pods/topology-spread-constraints/one-constraint-with-nodeaffinity.yaml kind: Pod apiVersion: v1 metadata: name: mypod labels: foo: bar spec: topologySpreadConstraints: - maxSkew: 1 topologyKey: zone whenUnsatisfiable: DoNotSchedule labelSelector: matchLabels: foo: bar affinity: nodeAffinity: requiredDuringSchedulingIgnoredDuringExecution: nodeSelectorTerms: - matchExpressions: - key: zone operator: NotIn values: - zoneC containers: - name: pause image: registry.k8s.io/pause:3.1Implicit conventionsThere are some implicit conventions worth noting here:Only the Pods holding the same namespace as the incoming Pod can be matching candidates.The scheduler only considers nodes that have all topologySpreadConstraints[*].topologyKey present at the same time. Nodes missing any of these topologyKeys are bypassed. This implies that:any Pods located on those bypassed nodes do not impact maxSkew calculation - in the above example, suppose the node node1 does not have a label "zone", then the 2 Pods will be disregarded, hence the incoming Pod will be scheduled into zone A.the incoming Pod has no chances to be scheduled onto this kind of nodes - in the above example, suppose a node node5 has the mistyped label zone-typo: zoneC (and no zone label set). After node node5 joins the cluster, it will be bypassed and Pods for this workload aren't scheduled there.Be aware of what will happen if the incoming Pod's topologySpreadConstraints[*].labelSelector doesn't match its own labels. In the above example, if you remove the incoming Pod's labels, it can still be placed onto nodes in zone B, since the constraints are still satisfied. However, after that placement, the degree of imbalance of the cluster remains unchanged - it's still zone A having 2 Pods labeled as foo: bar, and zone B having 1 Pod labeled as foo: bar. If this is not what you expect, update the workload's topologySpreadConstraints[*].labelSelector to match the labels in the pod template.Cluster-level default constraintsIt is possible to set default topology spread constraints for a cluster. Default topology spread constraints are applied to a Pod if, and only if:It doesn't define any constraints in its .spec.topologySpreadConstraints.It belongs to a Service, ReplicaSet, StatefulSet or ReplicationController.Default constraints can be set as part of the PodTopologySpread plugin arguments in a scheduling profile. The constraints are specified with the same API above, except that labelSelector must be empty. The selectors are calculated from the Services, ReplicaSets, StatefulSets or ReplicationControllers that the Pod belongs to.An example configuration might look like follows:apiVersion: kubescheduler.config.k8s.io/v1 kind: KubeSchedulerConfiguration profiles: - schedulerName: default-scheduler pluginConfig: - name: PodTopologySpread args: defaultConstraints: - maxSkew: 1 topologyKey: topology.kubernetes.io/zone whenUnsatisfiable: ScheduleAnyway defaultingType: List Built-in default constraintsFEATURE STATE: Kubernetes v1.24 [stable]If you don't configure any cluster-level default constraints for pod topology spreading, then kube-scheduler acts as if you specified the following default topology constraints:defaultConstraints: - maxSkew: 3 topologyKey: "kubernetes.io/hostname" whenUnsatisfiable: ScheduleAnyway - maxSkew: 5 topologyKey: "topology.kubernetes.io/zone" whenUnsatisfiable: ScheduleAnyway Also, the legacy SelectorSpread plugin, which provides an equivalent behavior, is disabled by default.Note:The PodTopologySpread plugin does not score the nodes that don't have the topology keys specified in the spreading constraints. This might result in a different default behavior compared to the legacy SelectorSpread plugin when using the default topology constraints.If your nodes are not expected to have both kubernetes.io/hostname and topology.kubernetes.io/zone labels set, define your own constraints instead of using the Kubernetes defaults.If you don't want to use the default Pod spreading constraints for your cluster, you can disable those defaults by setting defaultingType to List and leaving empty defaultConstraints in the PodTopologySpread plugin configuration:apiVersion: kubescheduler.config.k8s.io/v1 kind: KubeSchedulerConfiguration profiles: - schedulerName: default-scheduler pluginConfig: - name: PodTopologySpread args: defaultConstraints: [] defaultingType: List Comparison with podAffinity and podAntiAffinityIn Kubernetes, inter-Pod affinity and anti-affinity control how Pods are scheduled in relation to one another - either more packed or more scattered.podAffinityattracts Pods; you can try to pack any number of Pods into qualifying topology domain(s).podAntiAffinityrepels Pods. If you set this to requiredDuringSchedulingIgnoredDuringExecution mode then only a single Pod can be scheduled into a single topology domain; if you choose preferredDuringSchedulingIgnoredDuringExecution then you lose the ability to enforce the constraint.For finer control, you can specify topology spread constraints to distribute Pods across different topology domains - to achieve either high availability or cost-saving. This can also help on rolling update workloads and scaling out replicas smoothly.For more context, see the Motivation section of the enhancement proposal about Pod topology spread constraints.Known limitationsThere's no guarantee that the constraints remain satisfied when Pods are removed. For example, scaling down a Deployment may result in imbalanced Pods distribution.You can use a tool such as the Descheduler to rebalance the Pods distribution.Pods matched on tainted nodes are respected. See Issue 80921.The scheduler doesn't have prior knowledge of all the zones or other topology domains that a cluster has. They are determined from the existing nodes in the cluster. This could lead to a problem in autoscaled clusters, when a node pool (or node group) is scaled to zero nodes, and you're expecting the cluster to scale up, because, in this case, those topology domains won't be considered until there is at least one node in them.You can work around this by using a Node autoscaler that is aware of Pod topology spread constraints and is also aware of the overall set of topology domains.What's nextThe blog article Introducing PodTopologySpread explains maxSkew in some detail, as well as covering some advanced usage examples.Read the scheduling section of the API reference for Pod.

Pod Topology Spread Constraints


DeviceClassDeviceClass is a vendor- or admin-provided resource that contains device configuration and selectors.apiVersion: resource.k8s.io/v1import "k8s.io/api/resource/v1"DeviceClassDeviceClass is a vendor- or admin-provided resource that contains device configuration and selectors. It can be referenced in the device requests of a claim to apply these presets. Cluster scoped.This is an alpha type and requires enabling the DynamicResourceAllocation feature gate.apiVersion: resource.k8s.io/v1kind: DeviceClassmetadata (ObjectMeta)Standard object metadataspec (DeviceClassSpec), requiredSpec defines what can be allocated and how to configure it.This is mutable. Consumers have to be prepared for classes changing at any time, either because they get updated or replaced. Claim allocations are done once based on whatever was set in classes at the time of allocation.Changing the spec automatically increments the metadata.generation number.DeviceClassSpecDeviceClassSpec is used in a [DeviceClass] to define what can be allocated and how to configure it.config ([]DeviceClassConfiguration)Atomic: will be replaced during a mergeConfig defines configuration parameters that apply to each device that is claimed via this class. Some classes may potentially be satisfied by multiple drivers, so each instance of a vendor configuration applies to exactly one driver.They are passed to the driver, but are not considered while allocating the claim.DeviceClassConfiguration is used in DeviceClass.config.opaque (OpaqueDeviceConfiguration)Opaque provides driver-specific configuration parameters.OpaqueDeviceConfiguration contains configuration parameters for a driver in a format defined by the driver vendor.config.opaque.driver (string), requiredDriver is used to determine which kubelet plugin needs to be passed these configuration parameters.An admission policy provided by the driver developer could use this to decide whether it needs to validate them.Must be a DNS subdomain and should end with a DNS domain owned by the vendor of the driver.config.opaque.parameters (RawExtension), requiredParameters can contain arbitrary data. It is the responsibility of the driver developer to handle validation and versioning. Typically this includes self-identification and a version ("kind" + "apiVersion" for Kubernetes types), with conversion between different versions.The length of the raw data must be smaller or equal to 10 Ki.*RawExtension is used to hold extensions in external versions.To use this, make a field which has RawExtension as its type in your external, versioned struct, and Object in your internal struct. You also need to register your various plugin types.// Internal package:type MyAPIObject struct { runtime.TypeMeta json:",inline" MyPlugin runtime.Object json:"myPlugin" }type PluginA struct { AOption string json:"aOption" }// External package:type MyAPIObject struct { runtime.TypeMeta json:",inline" MyPlugin runtime.RawExtension json:"myPlugin" }type PluginA struct { AOption string json:"aOption" }// On the wire, the JSON will look something like this:{ "kind":"MyAPIObject", "apiVersion":"v1", "myPlugin": { "kind":"PluginA", "aOption":"foo", }, }So what happens? Decode first uses json or yaml to unmarshal the serialized data into your external MyAPIObject. That causes the raw JSON to be stored, but not unpacked. The next step is to copy (using pkg/conversion) into the internal struct. The runtime package's DefaultScheme has conversion functions installed which will unpack the JSON stored in RawExtension, turning it into the correct object type, and storing it in the Object. (TODO: In the case where the object is of an unknown type, a runtime.Unknown object will be created and stored.)*extendedResourceName (string)ExtendedResourceName is the extended resource name for the devices of this class. The devices of this class can be used to satisfy a pod's extended resource requests. It has the same format as the name of a pod's extended resource. It should be unique among all the device classes in a cluster. If two device classes have the same name, then the class created later is picked to satisfy a pod's extended resource requests. If two classes are created at the same time, then the name of the class lexicographically sorted first is picked.This is an alpha field.selectors ([]DeviceSelector)Atomic: will be replaced during a mergeEach selector must be satisfied by a device which is claimed via this class.DeviceSelector must have exactly one field set.selectors.cel (CELDeviceSelector)CEL contains a CEL expression for selecting a device.CELDeviceSelector contains a CEL expression for selecting a device.selectors.cel.expression (string), requiredExpression is a CEL expression which evaluates a single device. It must evaluate to true when the device under consideration satisfies the desired criteria, and false when it does not. Any other result is an error and causes allocation of devices to abort.The expression's input is an object named "device", which carries the following properties:driver (string): the name of the driver which defines this device.attributes (map[string]object): the device's attributes, grouped by prefix (e.g. device.attributes["dra.example.com"] evaluates to an object with all of the attributes which were prefixed by "dra.example.com".capacity (map[string]object): the device's capacities, grouped by prefix.allowMultipleAllocations (bool): the allowMultipleAllocations property of the device (v1.34+ with the DRAConsumableCapacity feature enabled).Example: Consider a device with driver="dra.example.com", which exposes two attributes named "model" and "ext.example.com/family" and which exposes one capacity named "modules". This input to this expression would have the following fields:device.driver device.attributes["dra.example.com"].model device.attributes["ext.example.com"].family device.capacity["dra.example.com"].modules The device.driver field can be used to check for a specific driver, either as a high-level precondition (i.e. you only want to consider devices from this driver) or as part of a multi-clause expression that is meant to consider devices from different drivers.The value type of each attribute is defined by the device definition, and users who write these expressions must consult the documentation for their specific drivers. The value type of each capacity is Quantity.If an unknown prefix is used as a lookup in either device.attributes or device.capacity, an empty map will be returned. Any reference to an unknown field will cause an evaluation error and allocation to abort.A robust expression should check for the existence of attributes before referencing them.For ease of use, the cel.bind() function is enabled, and can be used to simplify expressions that access multiple attributes with the same domain. For example:cel.bind(dra, device.attributes["dra.example.com"], dra.someBool && dra.anotherBool) The length of the expression must be smaller or equal to 10 Ki. The cost of evaluating it is also limited based on the estimated number of logical steps.DeviceClassListDeviceClassList is a collection of classes.apiVersion: resource.k8s.io/v1kind: DeviceClassListmetadata (ListMeta)Standard list metadataitems ([]DeviceClass), requiredItems is the list of resource classes.Operationsget read the specified DeviceClassHTTP RequestGET /apis/resource.k8s.io/v1/deviceclasses/{name}Parametersname (in path): string, requiredname of the DeviceClasspretty (in query): stringprettyResponse200 (DeviceClass): OK401: Unauthorizedlist list or watch objects of kind DeviceClassHTTP RequestGET /apis/resource.k8s.io/v1/deviceclassesParametersallowWatchBookmarks (in query): booleanallowWatchBookmarkscontinue (in query): stringcontinuefieldSelector (in query): stringfieldSelectorlabelSelector (in query): stringlabelSelectorlimit (in query): integerlimitpretty (in query): stringprettyresourceVersion (in query): stringresourceVersionresourceVersionMatch (in query): stringresourceVersionMatchsendInitialEvents (in query): booleansendInitialEventstimeoutSeconds (in query): integertimeoutSecondswatch (in query): booleanwatchResponse200 (DeviceClassList): OK401: Unauthorizedcreate create a DeviceClassHTTP RequestPOST /apis/resource.k8s.io/v1/deviceclassesParametersbody: DeviceClass, requireddryRun (in query): stringdryRunfieldManager (in query): stringfieldManagerfieldValidation (in query): stringfieldValidationpretty (in query): stringprettyResponse200 (DeviceClass): OK201 (DeviceClass): Created202 (DeviceClass): Accepted401: Unauthorizedupdate replace the specified DeviceClassHTTP RequestPUT /apis/resource.k8s.io/v1/deviceclasses/{name}Parametersname (in path): string, requiredname of the DeviceClassbody: DeviceClass, requireddryRun (in query): stringdryRunfieldManager (in query): stringfieldManagerfieldValidation (in query): stringfieldValidationpretty (in query): stringprettyResponse200 (DeviceClass): OK201 (DeviceClass): Created401: Unauthorizedpatch partially update the specified DeviceClassHTTP RequestPATCH /apis/resource.k8s.io/v1/deviceclasses/{name}Parametersname (in path): string, requiredname of the DeviceClassbody: Patch, requireddryRun (in query): stringdryRunfieldManager (in query): stringfieldManagerfieldValidation (in query): stringfieldValidationforce (in query): booleanforcepretty (in query): stringprettyResponse200 (DeviceClass): OK201 (DeviceClass): Created401: Unauthorizeddelete delete a DeviceClassHTTP RequestDELETE /apis/resource.k8s.io/v1/deviceclasses/{name}Parametersname (in path): string, requiredname of the DeviceClassbody: DeleteOptionsdryRun (in query): stringdryRungracePeriodSeconds (in query): integergracePeriodSecondsignoreStoreReadErrorWithClusterBreakingPotential (in query): booleanignoreStoreReadErrorWithClusterBreakingPotentialpretty (in query): stringprettypropagationPolicy (in query): stringpropagationPolicyResponse200 (DeviceClass): OK202 (DeviceClass): Accepted401: Unauthorizeddeletecollection delete collection of DeviceClassHTTP RequestDELETE /apis/resource.k8s.io/v1/deviceclassesParametersbody: DeleteOptionscontinue (in query): stringcontinuedryRun (in query): stringdryRunfieldSelector (in query): stringfieldSelectorgracePeriodSeconds (in query): integergracePeriodSecondsignoreStoreReadErrorWithClusterBreakingPotential (in query): booleanignoreStoreReadErrorWithClusterBreakingPotentiallabelSelector (in query): stringlabelSelectorlimit (in query): integerlimitpretty (in query): stringprettypropagationPolicy (in query): stringpropagationPolicyresourceVersion (in query): stringresourceVersionresourceVersionMatch (in query): stringresourceVersionMatchsendInitialEvents (in query): booleansendInitialEventstimeoutSeconds (in query): integertimeoutSecondsResponse200 (Status): OK401: UnauthorizedThis page is automatically generated.If you plan to report an issue with this page, mention that the page is auto-generated in your issue description. The fix may need to happen elsewhere in the Kubernetes project.

DeviceClass


Traces For Kubernetes System ComponentsFEATURE STATE: Kubernetes v1.27 [beta]System component traces record the latency of and relationships between operations in the cluster.Kubernetes components emit traces using the OpenTelemetry Protocol with the gRPC exporter and can be collected and routed to tracing backends using an OpenTelemetry Collector.Trace CollectionKubernetes components have built-in gRPC exporters for OTLP to export traces, either with an OpenTelemetry Collector, or without an OpenTelemetry Collector.For a complete guide to collecting traces and using the collector, see Getting Started with the OpenTelemetry Collector. However, there are a few things to note that are specific to Kubernetes components.By default, Kubernetes components export traces using the grpc exporter for OTLP on the IANA OpenTelemetry port, 4317. As an example, if the collector is running as a sidecar to a Kubernetes component, the following receiver configuration will collect spans and log them to standard output:receivers: otlp: protocols: grpc: exporters: # Replace this exporter with the exporter for your backend exporters: debug: verbosity: detailed service: pipelines: traces: receivers: [otlp] exporters: [debug] To directly emit traces to a backend without utilizing a collector, specify the endpoint field in the Kubernetes tracing configuration file with the desired trace backend address. This method negates the need for a collector and simplifies the overall structure.For trace backend header configuration, including authentication details, environment variables can be used with OTEL_EXPORTER_OTLP_HEADERS, see OTLP Exporter Configuration.Additionally, for trace resource attribute configuration such as Kubernetes cluster name, namespace, Pod name, etc., environment variables can also be used with OTEL_RESOURCE_ATTRIBUTES, see OTLP Kubernetes Resource.Component traceskube-apiserver tracesThe kube-apiserver generates spans for incoming HTTP requests, and for outgoing requests to webhooks, etcd, and re-entrant requests. It propagates the W3C Trace Context with outgoing requests but does not make use of the trace context attached to incoming requests, as the kube-apiserver is often a public endpoint.Enabling tracing in the kube-apiserverTo enable tracing, provide the kube-apiserver with a tracing configuration file with --tracing-config-file=<path-to-config>. This is an example config that records spans for 1 in 10000 requests, and uses the default OpenTelemetry endpoint:apiVersion: apiserver.config.k8s.io/v1 kind: TracingConfiguration # default value #endpoint: localhost:4317 samplingRatePerMillion: 100 For more information about the TracingConfiguration struct, see API server config API (v1).kubelet tracesFEATURE STATE: Kubernetes v1.34 [stable] (enabled by default: true)The kubelet CRI interface and authenticated http servers are instrumented to generate trace spans. As with the apiserver, the endpoint and sampling rate are configurable. Trace context propagation is also configured. A parent span's sampling decision is always respected. A provided tracing configuration sampling rate will apply to spans without a parent. Enabled without a configured endpoint, the default OpenTelemetry Collector receiver address of "localhost:4317" is set.Enabling tracing in the kubeletTo enable tracing, apply the tracing configuration. This is an example snippet of a kubelet config that records spans for 1 in 10000 requests, and uses the default OpenTelemetry endpoint:apiVersion: kubelet.config.k8s.io/v1beta1 kind: KubeletConfiguration tracing: # default value #endpoint: localhost:4317 samplingRatePerMillion: 100 If the samplingRatePerMillion is set to one million (1000000), then every span will be sent to the exporter.The kubelet in Kubernetes v1.34 collects spans from the garbage collection, pod synchronization routine as well as every gRPC method. The kubelet propagates trace context with gRPC requests so that container runtimes with trace instrumentation, such as CRI-O and containerd, can associate their exported spans with the trace context from the kubelet. The resulting traces will have parent-child links between kubelet and container runtime spans, providing helpful context when debugging node issues.Please note that exporting spans always comes with a small performance overhead on the networking and CPU side, depending on the overall configuration of the system. If there is any issue like that in a cluster which is running with tracing enabled, then mitigate the problem by either reducing the samplingRatePerMillion or disabling tracing completely by removing the configuration.StabilityTracing instrumentation is still under active development, and may change in a variety of ways. This includes span names, attached attributes, instrumented endpoints, etc. Until this feature graduates to stable, there are no guarantees of backwards compatibility for tracing instrumentation.What's nextRead about Getting Started with the OpenTelemetry CollectorRead about OTLP Exporter Configuration

Traces For Kubernetes System Components


kubectl pluginSynopsisProvides utilities for interacting with plugins.Plugins provide extended functionality that is not part of the major command-line distribution. Please refer to the documentation and examples for more information about how write your own plugins.The easiest way to discover and install plugins is via the kubernetes sub-project krew: [krew.sigs.k8s.io]. To install krew, visit https://krew.sigs.k8s.io/docs/user-guide/setup/installkubectl plugin [flags] Examples # List all available plugins kubectl plugin list # List only binary names of available plugins without paths kubectl plugin list --name-only Options-h, --helphelp for pluginParent Options Inherited--as stringUsername to impersonate for the operation. User could be a regular user or a service account in a namespace.--as-group stringsGroup to impersonate for the operation, this flag can be repeated to specify multiple groups.--as-uid stringUID to impersonate for the operation.--cache-dir string Default: "$HOME/.kube/cache"Default cache directory--certificate-authority stringPath to a cert file for the certificate authority--client-certificate stringPath to a client certificate file for TLS--client-key stringPath to a client key file for TLS--cluster stringThe name of the kubeconfig cluster to use--context stringThe name of the kubeconfig context to use--disable-compressionIf true, opt-out of response compression for all requests to the server--insecure-skip-tls-verifyIf true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure--kubeconfig stringPath to the kubeconfig file to use for CLI requests.--kuberc stringPath to the kuberc file to use for preferences. This can be disabled by exporting KUBECTL_KUBERC=false feature gate or turning off the feature KUBERC=off.--match-server-versionRequire server version to match client version-n, --namespace stringIf present, the namespace scope for this CLI request--password stringPassword for basic authentication to the API server--profile string Default: "none"Name of profile to capture. One of (none|cpu|heap|goroutine|threadcreate|block|mutex)--profile-output string Default: "profile.pprof"Name of the file to write the profile to--request-timeout string Default: "0"The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests.-s, --server stringThe address and port of the Kubernetes API server--storage-driver-buffer-duration duration Default: 1m0sWrites in the storage driver will be buffered for this duration, and committed to the non memory backends as a single transaction--storage-driver-db string Default: "cadvisor"database name--storage-driver-host string Default: "localhost:8086"database host:port--storage-driver-password string Default: "root"database password--storage-driver-secureuse secure connection with database--storage-driver-table string Default: "stats"table name--storage-driver-user string Default: "root"database username--tls-server-name stringServer name to use for server certificate validation. If it is not provided, the hostname used to contact the server is used--token stringBearer token for authentication to the API server--user stringThe name of the kubeconfig user to use--username stringUsername for basic authentication to the API server--version version[=true]--version, --version=raw prints version information and quits; --version=vX.Y.Z... sets the reported version--warnings-as-errorsTreat warnings received from the server as errors and exit with a non-zero exit codeSee Alsokubectl - kubectl controls the Kubernetes cluster managerkubectl plugin list - List all visible plugin executables on a user's PATHThis page is automatically generated.If you plan to report an issue with this page, mention that the page is auto-generated in your issue description. The fix may need to happen elsewhere in the Kubernetes project.

kubectl plugin


Expose Pod Information to Containers Through FilesThis page shows how a Pod can use a downwardAPI volume, to expose information about itself to containers running in the Pod. A downwardAPI volume can expose Pod fields and container fields.In Kubernetes, there are two ways to expose Pod and container fields to a running container:Environment variablesVolume files, as explained in this taskTogether, these two ways of exposing Pod and container fields are called the downward API.Before you beginYou need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. It is recommended to run this tutorial on a cluster with at least two nodes that are not acting as control plane hosts. If you do not already have a cluster, you can create one by using minikube or you can use one of these Kubernetes playgrounds:iximiuz LabsKillercodaKodeKloudPlay with KubernetesStore Pod fieldsIn this part of exercise, you create a Pod that has one container, and you project Pod-level fields into the running container as files. Here is the manifest for the Pod:pods/inject/dapi-volume.yaml apiVersion: v1 kind: Pod metadata: name: kubernetes-downwardapi-volume-example labels: zone: us-est-coast cluster: test-cluster1 rack: rack-22 annotations: build: two builder: john-doe spec: containers: - name: client-container image: registry.k8s.io/busybox:1.27.2 command: ["sh", "-c"] args: - while true; do if [[ -e /etc/podinfo/labels ]]; then echo -en '\n\n'; cat /etc/podinfo/labels; fi; if [[ -e /etc/podinfo/annotations ]]; then echo -en '\n\n'; cat /etc/podinfo/annotations; fi; sleep 5; done; volumeMounts: - name: podinfo mountPath: /etc/podinfo volumes: - name: podinfo downwardAPI: items: - path: "labels" fieldRef: fieldPath: metadata.labels - path: "annotations" fieldRef: fieldPath: metadata.annotations In the manifest, you can see that the Pod has a downwardAPI Volume, and the container mounts the volume at /etc/podinfo.Look at the items array under downwardAPI. Each element of the array defines a downwardAPI volume. The first element specifies that the value of the Pod's metadata.labels field should be stored in a file named labels. The second element specifies that the value of the Pod's annotations field should be stored in a file named annotations.Note:The fields in this example are Pod fields. They are not fields of the container in the Pod.Create the Pod:kubectl apply -f https://k8s.io/examples/pods/inject/dapi-volume.yaml Verify that the container in the Pod is running:kubectl get pods View the container's logs:kubectl logs kubernetes-downwardapi-volume-example The output shows the contents of the labels file and the annotations file:cluster="test-cluster1" rack="rack-22" zone="us-est-coast" build="two" builder="john-doe" Get a shell into the container that is running in your Pod:kubectl exec -it kubernetes-downwardapi-volume-example -- sh In your shell, view the labels file:/# cat /etc/podinfo/labels The output shows that all of the Pod's labels have been written to the labels file:cluster="test-cluster1" rack="rack-22" zone="us-est-coast" Similarly, view the annotations file:/# cat /etc/podinfo/annotations View the files in the /etc/podinfo directory:/# ls -laR /etc/podinfo In the output, you can see that the labels and annotations files are in a temporary subdirectory: in this example, ..2982_06_02_21_47_53.299460680. In the /etc/podinfo directory, ..data is a symbolic link to the temporary subdirectory. Also in the /etc/podinfo directory, labels and annotations are symbolic links.drwxr-xr-x ... Feb 6 21:47 ..2982_06_02_21_47_53.299460680 lrwxrwxrwx ... Feb 6 21:47 ..data -> ..2982_06_02_21_47_53.299460680 lrwxrwxrwx ... Feb 6 21:47 annotations -> ..data/annotations lrwxrwxrwx ... Feb 6 21:47 labels -> ..data/labels /etc/..2982_06_02_21_47_53.299460680: total 8 -rw-r--r-- ... Feb 6 21:47 annotations -rw-r--r-- ... Feb 6 21:47 labels Using symbolic links enables dynamic atomic refresh of the metadata; updates are written to a new temporary directory, and the ..data symlink is updated atomically using rename(2).Note:A container using Downward API as a subPath volume mount will not receive Downward API updates.Exit the shell:/# exit Store container fieldsThe preceding exercise, you made Pod-level fields accessible using the downward API. In this next exercise, you are going to pass fields that are part of the Pod definition, but taken from the specific container rather than from the Pod overall. Here is a manifest for a Pod that again has just one container:pods/inject/dapi-volume-resources.yaml apiVersion: v1 kind: Pod metadata: name: kubernetes-downwardapi-volume-example-2 spec: containers: - name: client-container image: registry.k8s.io/busybox:1.27.2 command: ["sh", "-c"] args: - while true; do echo -en '\n'; if [[ -e /etc/podinfo/cpu_limit ]]; then echo -en '\n'; cat /etc/podinfo/cpu_limit; fi; if [[ -e /etc/podinfo/cpu_request ]]; then echo -en '\n'; cat /etc/podinfo/cpu_request; fi; if [[ -e /etc/podinfo/mem_limit ]]; then echo -en '\n'; cat /etc/podinfo/mem_limit; fi; if [[ -e /etc/podinfo/mem_request ]]; then echo -en '\n'; cat /etc/podinfo/mem_request; fi; sleep 5; done; resources: requests: memory: "32Mi" cpu: "125m" limits: memory: "64Mi" cpu: "250m" volumeMounts: - name: podinfo mountPath: /etc/podinfo volumes: - name: podinfo downwardAPI: items: - path: "cpu_limit" resourceFieldRef: containerName: client-container resource: limits.cpu divisor: 1m - path: "cpu_request" resourceFieldRef: containerName: client-container resource: requests.cpu divisor: 1m - path: "mem_limit" resourceFieldRef: containerName: client-container resource: limits.memory divisor: 1Mi - path: "mem_request" resourceFieldRef: containerName: client-container resource: requests.memory divisor: 1Mi In the manifest, you can see that the Pod has a downwardAPI volume, and that the single container in that Pod mounts the volume at /etc/podinfo.Look at the items array under downwardAPI. Each element of the array defines a file in the downward API volume.The first element specifies that in the container named client-container, the value of the limits.cpu field in the format specified by 1m should be published as a file named cpu_limit. The divisor field is optional and has the default value of 1. A divisor of 1 means cores for cpu resources, or bytes for memory resources.Create the Pod:kubectl apply -f https://k8s.io/examples/pods/inject/dapi-volume-resources.yaml Get a shell into the container that is running in your Pod:kubectl exec -it kubernetes-downwardapi-volume-example-2 -- sh In your shell, view the cpu_limit file:# Run this in a shell inside the container cat /etc/podinfo/cpu_limit You can use similar commands to view the cpu_request, mem_limit and mem_request files.Project keys to specific paths and file permissionsYou can project keys to specific paths and specific permissions on a per-file basis. For more information, see Secrets.What's nextRead the spec API definition for Pod. This includes the definition of Container (part of Pod).Read the list of available fields that you can expose using the downward API.Read about volumes in the legacy API reference:Check the Volume API definition which defines a generic volume in a Pod for containers to access.Check the DownwardAPIVolumeSource API definition which defines a volume that contains Downward API information.Check the DownwardAPIVolumeFile API definition which contains references to object or resource fields for populating a file in the Downward API volume.Check the ResourceFieldSelector API definition which specifies the container resources and their output format.