kubectl rollout historySynopsisView previous rollout revisions and configurations.kubectl rollout history (TYPE NAME | TYPE/NAME) [flags] Examples # View the rollout history of a deployment kubectl rollout history deployment/abc # View the details of daemonset revision 3 kubectl rollout history daemonset/abc --revision=3 Options--allow-missing-template-keys Default: trueIf true, ignore any errors in templates when a field or map key is missing in the template. Only applies to golang and jsonpath output formats.-f, --filename stringsFilename, directory, or URL to files identifying the resource to get from a server.-h, --helphelp for history-k, --kustomize stringProcess the kustomization directory. This flag can't be used together with -f or -R.-o, --output stringOutput format. One of: (json, yaml, name, go-template, go-template-file, template, templatefile, jsonpath, jsonpath-as-json, jsonpath-file).-R, --recursiveProcess the directory used in -f, --filename recursively. Useful when you want to manage related manifests organized within the same directory.--revision intSee the details, including podTemplate of the revision specified-l, --selector stringSelector (label query) to filter on, supports '=', '==', '!=', 'in', 'notin'.(e.g. -l key1=value1,key2=value2,key3 in (value3)). Matching objects must satisfy all of the specified label constraints.--show-managed-fieldsIf true, keep the managedFields when printing objects in JSON or YAML format.--template stringTemplate string or path to template file to use when -o=go-template, -o=go-template-file. The template format is golang templates [http://golang.org/pkg/text/template/#pkg-overview].Parent Options Inherited--as stringUsername to impersonate for the operation. User could be a regular user or a service account in a namespace.--as-group stringsGroup to impersonate for the operation, this flag can be repeated to specify multiple groups.--as-uid stringUID to impersonate for the operation.--cache-dir string Default: "$HOME/.kube/cache"Default cache directory--certificate-authority stringPath to a cert file for the certificate authority--client-certificate stringPath to a client certificate file for TLS--client-key stringPath to a client key file for TLS--cluster stringThe name of the kubeconfig cluster to use--context stringThe name of the kubeconfig context to use--disable-compressionIf true, opt-out of response compression for all requests to the server--insecure-skip-tls-verifyIf true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure--kubeconfig stringPath to the kubeconfig file to use for CLI requests.--kuberc stringPath to the kuberc file to use for preferences. This can be disabled by exporting KUBECTL_KUBERC=false feature gate or turning off the feature KUBERC=off.--match-server-versionRequire server version to match client version-n, --namespace stringIf present, the namespace scope for this CLI request--password stringPassword for basic authentication to the API server--profile string Default: "none"Name of profile to capture. One of (none|cpu|heap|goroutine|threadcreate|block|mutex)--profile-output string Default: "profile.pprof"Name of the file to write the profile to--request-timeout string Default: "0"The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests.-s, --server stringThe address and port of the Kubernetes API server--storage-driver-buffer-duration duration Default: 1m0sWrites in the storage driver will be buffered for this duration, and committed to the non memory backends as a single transaction--storage-driver-db string Default: "cadvisor"database name--storage-driver-host string Default: "localhost:8086"database host:port--storage-driver-password string Default: "root"database password--storage-driver-secureuse secure connection with database--storage-driver-table string Default: "stats"table name--storage-driver-user string Default: "root"database username--tls-server-name stringServer name to use for server certificate validation. If it is not provided, the hostname used to contact the server is used--token stringBearer token for authentication to the API server--user stringThe name of the kubeconfig user to use--username stringUsername for basic authentication to the API server--version version[=true]--version, --version=raw prints version information and quits; --version=vX.Y.Z... sets the reported version--warnings-as-errorsTreat warnings received from the server as errors and exit with a non-zero exit codeSee Alsokubectl rollout - Manage the rollout of a resourceThis page is automatically generated.If you plan to report an issue with this page, mention that the page is auto-generated in your issue description. The fix may need to happen elsewhere in the Kubernetes project.

kubectl rollout history


Limit Storage ConsumptionThis example demonstrates how to limit the amount of storage consumed in a namespace.The following resources are used in the demonstration: ResourceQuota, LimitRange, and PersistentVolumeClaim.Before you beginYou need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. It is recommended to run this tutorial on a cluster with at least two nodes that are not acting as control plane hosts. If you do not already have a cluster, you can create one by using minikube or you can use one of these Kubernetes playgrounds:iximiuz LabsKillercodaKodeKloudPlay with KubernetesTo check the version, enter kubectl version.Scenario: Limiting Storage ConsumptionThe cluster-admin is operating a cluster on behalf of a user population and the admin wants to control how much storage a single namespace can consume in order to control cost.The admin would like to limit:The number of persistent volume claims in a namespaceThe amount of storage each claim can requestThe amount of cumulative storage the namespace can haveLimitRange to limit requests for storageAdding a LimitRange to a namespace enforces storage request sizes to a minimum and maximum. Storage is requested via PersistentVolumeClaim. The admission controller that enforces limit ranges will reject any PVC that is above or below the values set by the admin.In this example, a PVC requesting 10Gi of storage would be rejected because it exceeds the 2Gi max.apiVersion: v1 kind: LimitRange metadata: name: storagelimits spec: limits: - type: PersistentVolumeClaim max: storage: 2Gi min: storage: 1Gi Minimum storage requests are used when the underlying storage provider requires certain minimums. For example, AWS EBS volumes have a 1Gi minimum requirement.ResourceQuota to limit PVC count and cumulative storage capacityAdmins can limit the number of PVCs in a namespace as well as the cumulative capacity of those PVCs. New PVCs that exceed either maximum value will be rejected.In this example, a 6th PVC in the namespace would be rejected because it exceeds the maximum count of 5. Alternatively, a 5Gi maximum quota when combined with the 2Gi max limit above, cannot have 3 PVCs where each has 2Gi. That would be 6Gi requested for a namespace capped at 5Gi.apiVersion: v1 kind: ResourceQuota metadata: name: storagequota spec: hard: persistentvolumeclaims: "5" requests.storage: "5Gi" SummaryA limit range can put a ceiling on how much storage is requested while a resource quota can effectively cap the storage consumed by a namespace through claim counts and cumulative storage capacity. The allows a cluster-admin to plan their cluster's storage budget without risk of any one project going over their allotment.

Limit Storage Consumption


kubectl eventsSynopsisDisplay events.Prints a table of the most important information about events. You can request events for a namespace, for all namespace, or filtered to only those pertaining to a specified resource.kubectl events [(-o|--output=)json|yaml|name|go-template|go-template-file|template|templatefile|jsonpath|jsonpath-as-json|jsonpath-file] [--for TYPE/NAME] [--watch] [--types=Normal,Warning] Examples # List recent events in the default namespace kubectl events # List recent events in all namespaces kubectl events --all-namespaces # List recent events for the specified pod, then wait for more events and list them as they arrive kubectl events --for pod/web-pod-13je7 --watch # List recent events in YAML format kubectl events -oyaml # List recent only events of type 'Warning' or 'Normal' kubectl events --types=Warning,Normal Options-A, --all-namespacesIf present, list the requested object(s) across all namespaces. Namespace in current context is ignored even if specified with --namespace.--allow-missing-template-keys Default: trueIf true, ignore any errors in templates when a field or map key is missing in the template. Only applies to golang and jsonpath output formats.--chunk-size int Default: 500Return large lists in chunks rather than all at once. Pass 0 to disable. This flag is beta and may change in the future.--for stringFilter events to only those pertaining to the specified resource.-h, --helphelp for events--no-headersWhen using the default output format, don't print headers.-o, --output stringOutput format. One of: (json, yaml, name, go-template, go-template-file, template, templatefile, jsonpath, jsonpath-as-json, jsonpath-file).--show-managed-fieldsIf true, keep the managedFields when printing objects in JSON or YAML format.--template stringTemplate string or path to template file to use when -o=go-template, -o=go-template-file. The template format is golang templates [http://golang.org/pkg/text/template/#pkg-overview].--types stringsOutput only events of given types.-w, --watchAfter listing the requested events, watch for more events.Parent Options Inherited--as stringUsername to impersonate for the operation. User could be a regular user or a service account in a namespace.--as-group stringsGroup to impersonate for the operation, this flag can be repeated to specify multiple groups.--as-uid stringUID to impersonate for the operation.--cache-dir string Default: "$HOME/.kube/cache"Default cache directory--certificate-authority stringPath to a cert file for the certificate authority--client-certificate stringPath to a client certificate file for TLS--client-key stringPath to a client key file for TLS--cluster stringThe name of the kubeconfig cluster to use--context stringThe name of the kubeconfig context to use--disable-compressionIf true, opt-out of response compression for all requests to the server--insecure-skip-tls-verifyIf true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure--kubeconfig stringPath to the kubeconfig file to use for CLI requests.--kuberc stringPath to the kuberc file to use for preferences. This can be disabled by exporting KUBECTL_KUBERC=false feature gate or turning off the feature KUBERC=off.--match-server-versionRequire server version to match client version-n, --namespace stringIf present, the namespace scope for this CLI request--password stringPassword for basic authentication to the API server--profile string Default: "none"Name of profile to capture. One of (none|cpu|heap|goroutine|threadcreate|block|mutex)--profile-output string Default: "profile.pprof"Name of the file to write the profile to--request-timeout string Default: "0"The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests.-s, --server stringThe address and port of the Kubernetes API server--storage-driver-buffer-duration duration Default: 1m0sWrites in the storage driver will be buffered for this duration, and committed to the non memory backends as a single transaction--storage-driver-db string Default: "cadvisor"database name--storage-driver-host string Default: "localhost:8086"database host:port--storage-driver-password string Default: "root"database password--storage-driver-secureuse secure connection with database--storage-driver-table string Default: "stats"table name--storage-driver-user string Default: "root"database username--tls-server-name stringServer name to use for server certificate validation. If it is not provided, the hostname used to contact the server is used--token stringBearer token for authentication to the API server--user stringThe name of the kubeconfig user to use--username stringUsername for basic authentication to the API server--version version[=true]--version, --version=raw prints version information and quits; --version=vX.Y.Z... sets the reported version--warnings-as-errorsTreat warnings received from the server as errors and exit with a non-zero exit codeSee Alsokubectl - kubectl controls the Kubernetes cluster managerThis page is automatically generated.If you plan to report an issue with this page, mention that the page is auto-generated in your issue description. The fix may need to happen elsewhere in the Kubernetes project.

kubectl events


Find Out What Container Runtime is Used on a NodeThis page outlines steps to find out what container runtime the nodes in your cluster use.Depending on the way you run your cluster, the container runtime for the nodes may have been pre-configured or you need to configure it. If you're using a managed Kubernetes service, there might be vendor-specific ways to check what container runtime is configured for the nodes. The method described on this page should work whenever the execution of kubectl is allowed.Before you beginInstall and configure kubectl. See Install Tools section for details.Find out the container runtime used on a NodeUse kubectl to fetch and show node information:kubectl get nodes -o wide The output is similar to the following. The column CONTAINER-RUNTIME outputs the runtime and its version.For Docker Engine, the output is similar to this:NAME STATUS VERSION CONTAINER-RUNTIME node-1 Ready v1.16.15 docker://19.3.1 node-2 Ready v1.16.15 docker://19.3.1 node-3 Ready v1.16.15 docker://19.3.1 If your runtime shows as Docker Engine, you still might not be affected by the removal of dockershim in Kubernetes v1.24. Check the runtime endpoint to see if you use dockershim. If you don't use dockershim, you aren't affected.For containerd, the output is similar to this:NAME STATUS VERSION CONTAINER-RUNTIME node-1 Ready v1.19.6 containerd://1.4.1 node-2 Ready v1.19.6 containerd://1.4.1 node-3 Ready v1.19.6 containerd://1.4.1 Find out more information about container runtimes on Container Runtimes page.Find out what container runtime endpoint you useThe container runtime talks to the kubelet over a Unix socket using the CRI protocol, which is based on the gRPC framework. The kubelet acts as a client, and the runtime acts as the server. In some cases, you might find it useful to know which socket your nodes use. For example, with the removal of dockershim in Kubernetes v1.24 and later, you might want to know whether you use Docker Engine with dockershim.Note:If you currently use Docker Engine in your nodes with cri-dockerd, you aren't affected by the dockershim removal.You can check which socket you use by checking the kubelet configuration on your nodes.Read the starting commands for the kubelet process:tr \\0 ' ' < /proc/"$(pgrep kubelet)"/cmdline If you don't have tr or pgrep, check the command line for the kubelet process manually.In the output, look for the --container-runtime flag and the --container-runtime-endpoint flag.If your nodes use Kubernetes v1.23 and earlier and these flags aren't present or if the --container-runtime flag is not remote, you use the dockershim socket with Docker Engine. The --container-runtime command line argument is not available in Kubernetes v1.27 and later.If the --container-runtime-endpoint flag is present, check the socket name to find out which runtime you use. For example, unix:///run/containerd/containerd.sock is the containerd endpoint.If you want to change the Container Runtime on a Node from Docker Engine to containerd, you can find out more information on migrating from Docker Engine to containerd, or, if you want to continue using Docker Engine in Kubernetes v1.24 and later, migrate to a CRI-compatible adapter like cri-dockerd.


Service ClusterIP allocationIn Kubernetes, Services are an abstract way to expose an application running on a set of Pods. Services can have a cluster-scoped virtual IP address (using a Service of type: ClusterIP). Clients can connect using that virtual IP address, and Kubernetes then load-balances traffic to that Service across the different backing Pods.How Service ClusterIPs are allocated?When Kubernetes needs to assign a virtual IP address for a Service, that assignment happens one of two ways:dynamicallythe cluster's control plane automatically picks a free IP address from within the configured IP range for type: ClusterIP Services.staticallyyou specify an IP address of your choice, from within the configured IP range for Services.Across your whole cluster, every Service ClusterIP must be unique. Trying to create a Service with a specific ClusterIP that has already been allocated will return an error.Why do you need to reserve Service Cluster IPs?Sometimes you may want to have Services running in well-known IP addresses, so other components and users in the cluster can use them.The best example is the DNS Service for the cluster. As a soft convention, some Kubernetes installers assign the 10th IP address from the Service IP range to the DNS service. Assuming you configured your cluster with Service IP range 10.96.0.0/16 and you want your DNS Service IP to be 10.96.0.10, you'd have to create a Service like this:apiVersion: v1 kind: Service metadata: labels: k8s-app: kube-dns kubernetes.io/cluster-service: "true" kubernetes.io/name: CoreDNS name: kube-dns namespace: kube-system spec: clusterIP: 10.96.0.10 ports: - name: dns port: 53 protocol: UDP targetPort: 53 - name: dns-tcp port: 53 protocol: TCP targetPort: 53 selector: k8s-app: kube-dns type: ClusterIP But, as it was explained before, the IP address 10.96.0.10 has not been reserved. If other Services are created before or in parallel with dynamic allocation, there is a chance they can allocate this IP. Hence, you will not be able to create the DNS Service because it will fail with a conflict error.How can you avoid Service ClusterIP conflicts?The allocation strategy implemented in Kubernetes to allocate ClusterIPs to Services reduces the risk of collision.The ClusterIP range is divided, based on the formula min(max(16, cidrSize / 16), 256), described as never less than 16 or more than 256 with a graduated step between them.Dynamic IP assignment uses the upper band by default, once this has been exhausted it will use the lower range. This will allow users to use static allocations on the lower band with a low risk of collision.ExamplesExample 1This example uses the IP address range: 10.96.0.0/24 (CIDR notation) for the IP addresses of Services.Range Size: 28 - 2 = 254Band Offset: min(max(16, 256/16), 256) = min(16, 256) = 16Static band start: 10.96.0.1Static band end: 10.96.0.16Range end: 10.96.0.254pie showData title 10.96.0.0/24 "Static" : 16 "Dynamic" : 238JavaScript must be enabled to view this contentExample 2This example uses the IP address range: 10.96.0.0/20 (CIDR notation) for the IP addresses of Services.Range Size: 212 - 2 = 4094Band Offset: min(max(16, 4096/16), 256) = min(256, 256) = 256Static band start: 10.96.0.1Static band end: 10.96.1.0Range end: 10.96.15.254pie showData title 10.96.0.0/20 "Static" : 256 "Dynamic" : 3838JavaScript must be enabled to view this contentExample 3This example uses the IP address range: 10.96.0.0/16 (CIDR notation) for the IP addresses of Services.Range Size: 216 - 2 = 65534Band Offset: min(max(16, 65536/16), 256) = min(4096, 256) = 256Static band start: 10.96.0.1Static band ends: 10.96.1.0Range end: 10.96.255.254pie showData title 10.96.0.0/16 "Static" : 256 "Dynamic" : 65278JavaScript must be enabled to view this contentWhat's nextRead about Service External Traffic PolicyRead about Connecting Applications with ServicesRead about Services

Service ClusterIP allocation


Kubernetes SchedulerIn Kubernetes, scheduling refers to making sure that Pods are matched to Nodes so that Kubelet can run them.Scheduling overviewA scheduler watches for newly created Pods that have no Node assigned. For every Pod that the scheduler discovers, the scheduler becomes responsible for finding the best Node for that Pod to run on. The scheduler reaches this placement decision taking into account the scheduling principles described below.If you want to understand why Pods are placed onto a particular Node, or if you're planning to implement a custom scheduler yourself, this page will help you learn about scheduling.kube-schedulerkube-scheduler is the default scheduler for Kubernetes and runs as part of the control plane. kube-scheduler is designed so that, if you want and need to, you can write your own scheduling component and use that instead.Kube-scheduler selects an optimal node to run newly created or not yet scheduled (unscheduled) pods. Since containers in pods - and pods themselves - can have different requirements, the scheduler filters out any nodes that don't meet a Pod's specific scheduling needs. Alternatively, the API lets you specify a node for a Pod when you create it, but this is unusual and is only done in special cases.In a cluster, Nodes that meet the scheduling requirements for a Pod are called feasible nodes. If none of the nodes are suitable, the pod remains unscheduled until the scheduler is able to place it.The scheduler finds feasible Nodes for a Pod and then runs a set of functions to score the feasible Nodes and picks a Node with the highest score among the feasible ones to run the Pod. The scheduler then notifies the API server about this decision in a process called binding.Factors that need to be taken into account for scheduling decisions include individual and collective resource requirements, hardware / software / policy constraints, affinity and anti-affinity specifications, data locality, inter-workload interference, and so on.Node selection in kube-schedulerkube-scheduler selects a node for the pod in a 2-step operation:FilteringScoringThe filtering step finds the set of Nodes where it's feasible to schedule the Pod. For example, the PodFitsResources filter checks whether a candidate Node has enough available resources to meet a Pod's specific resource requests. After this step, the node list contains any suitable Nodes; often, there will be more than one. If the list is empty, that Pod isn't (yet) schedulable.In the scoring step, the scheduler ranks the remaining nodes to choose the most suitable Pod placement. The scheduler assigns a score to each Node that survived filtering, basing this score on the active scoring rules.Finally, kube-scheduler assigns the Pod to the Node with the highest ranking. If there is more than one node with equal scores, kube-scheduler selects one of these at random.There are two supported ways to configure the filtering and scoring behavior of the scheduler:Scheduling Policies allow you to configure Predicates for filtering and Priorities for scoring.Scheduling Profiles allow you to configure Plugins that implement different scheduling stages, including: QueueSort, Filter, Score, Bind, Reserve, Permit, and others. You can also configure the kube-scheduler to run different profiles.What's nextRead about scheduler performance tuningRead about Pod topology spread constraintsRead the reference documentation for kube-schedulerRead the kube-scheduler config (v1) referenceLearn about configuring multiple schedulersLearn about topology management policiesLearn about Pod OverheadLearn about scheduling of Pods that use volumes in:Volume Topology SupportStorage Capacity TrackingNode-specific Volume Limits

Kubernetes Scheduler


Get a Shell to a Running ContainerThis page shows how to use kubectl exec to get a shell to a running container.Before you beginYou need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. It is recommended to run this tutorial on a cluster with at least two nodes that are not acting as control plane hosts. If you do not already have a cluster, you can create one by using minikube or you can use one of these Kubernetes playgrounds:iximiuz LabsKillercodaKodeKloudPlay with KubernetesGetting a shell to a containerIn this exercise, you create a Pod that has one container. The container runs the nginx image. Here is the configuration file for the Pod:application/shell-demo.yaml apiVersion: v1 kind: Pod metadata: name: shell-demo spec: volumes: - name: shared-data emptyDir: {} containers: - name: nginx image: nginx volumeMounts: - name: shared-data mountPath: /usr/share/nginx/html hostNetwork: true dnsPolicy: Default Create the Pod:kubectl apply -f https://k8s.io/examples/application/shell-demo.yaml Verify that the container is running:kubectl get pod shell-demo Get a shell to the running container:kubectl exec --stdin --tty shell-demo -- /bin/bash Note:The double dash (--) separates the arguments you want to pass to the command from the kubectl arguments.In your shell, list the root directory:# Run this inside the container ls / In your shell, experiment with other commands. Here are some examples:# You can run these example commands inside the container ls / cat /proc/mounts cat /proc/1/maps apt-get update apt-get install -y tcpdump tcpdump apt-get install -y lsof lsof apt-get install -y procps ps aux ps aux | grep nginx Writing the root page for nginxLook again at the configuration file for your Pod. The Pod has an emptyDir volume, and the container mounts the volume at /usr/share/nginx/html.In your shell, create an index.html file in the /usr/share/nginx/html directory:# Run this inside the container echo 'Hello shell demo' > /usr/share/nginx/html/index.html In your shell, send a GET request to the nginx server:# Run this in the shell inside your container apt-get update apt-get install curl curl http://localhost/ The output shows the text that you wrote to the index.html file:Hello shell demo When you are finished with your shell, enter exit.exit # To quit the shell in the container Running individual commands in a containerIn an ordinary command window, not your shell, list the environment variables in the running container:kubectl exec shell-demo -- env Experiment with running other commands. Here are some examples:kubectl exec shell-demo -- ps aux kubectl exec shell-demo -- ls / kubectl exec shell-demo -- cat /proc/1/mounts Opening a shell when a Pod has more than one containerIf a Pod has more than one container, use --container or -c to specify a container in the kubectl exec command. For example, suppose you have a Pod named my-pod, and the Pod has two containers named main-app and helper-app. The following command would open a shell to the main-app container.kubectl exec -i -t my-pod --container main-app -- /bin/bash Note:The short options -i and -t are the same as the long options --stdin and --ttyWhat's nextRead about kubectl exec

Get a Shell to a Running Container


ListMetaListMeta describes metadata that synthetic resources must have, including lists and various status objects.import "k8s.io/apimachinery/pkg/apis/meta/v1"ListMeta describes metadata that synthetic resources must have, including lists and various status objects. A resource may have only one of {ObjectMeta, ListMeta}.continue (string)continue may be set if the user set a limit on the number of items returned, and indicates that the server has more data available. The value is opaque and may be used to issue another request to the endpoint that served this list to retrieve the next set of available objects. Continuing a consistent list may not be possible if the server configuration has changed or more than a few minutes have passed. The resourceVersion field returned when using this continue value will be identical to the value in the first response, unless you have received this token from an error message.remainingItemCount (int64)remainingItemCount is the number of subsequent items in the list which are not included in this list response. If the list request contained label or field selectors, then the number of remaining items is unknown and the field will be left unset and omitted during serialization. If the list is complete (either because it is not chunking or because this is the last chunk), then there are no more remaining items and this field will be left unset and omitted during serialization. Servers older than v1.15 do not set this field. The intended use of the remainingItemCount is estimating the size of a collection. Clients should not rely on the remainingItemCount to be set or to be exact.resourceVersion (string)String that identifies the server's internal version of this object that can be used by clients to determine when objects have changed. Value must be treated as opaque by clients and passed unmodified back to the server. Populated by the system. Read-only. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistencyselfLink (string)Deprecated: selfLink is a legacy read-only field that is no longer populated by the system.This page is automatically generated.If you plan to report an issue with this page, mention that the page is auto-generated in your issue description. The fix may need to happen elsewhere in the Kubernetes project.

ListMeta


kubectl set selectorSynopsisSet the selector on a resource. Note that the new selector will overwrite the old selector if the resource had one prior to the invocation of 'set selector'.A selector must begin with a letter or number, and may contain letters, numbers, hyphens, dots, and underscores, up to 63 characters. If --resource-version is specified, then updates will use this resource version, otherwise the existing resource-version will be used. Note: currently selectors can only be set on Service objects.kubectl set selector (-f FILENAME | TYPE NAME) EXPRESSIONS [--resource-version=version] Examples # Set the labels and selector before creating a deployment/service pair kubectl create service clusterip my-svc --clusterip="None" -o yaml --dry-run=client | kubectl set selector --local -f - 'environment=qa' -o yaml | kubectl create -f - kubectl create deployment my-dep -o yaml --dry-run=client | kubectl label --local -f - environment=qa -o yaml | kubectl create -f - Options--allSelect all resources in the namespace of the specified resource types--allow-missing-template-keys Default: trueIf true, ignore any errors in templates when a field or map key is missing in the template. Only applies to golang and jsonpath output formats.--dry-run string[="unchanged"] Default: "none"Must be "none", "server", or "client". If client strategy, only print the object that would be sent, without sending it. If server strategy, submit server-side request without persisting the resource.--field-manager string Default: "kubectl-set"Name of the manager used to track field ownership.-f, --filename stringsidentifying the resource.-h, --helphelp for selector--localIf true, annotation will NOT contact api-server but run locally.-o, --output stringOutput format. One of: (json, yaml, name, go-template, go-template-file, template, templatefile, jsonpath, jsonpath-as-json, jsonpath-file).-R, --recursive Default: trueProcess the directory used in -f, --filename recursively. Useful when you want to manage related manifests organized within the same directory.--resource-version stringIf non-empty, the selectors update will only succeed if this is the current resource-version for the object. Only valid when specifying a single resource.--show-managed-fieldsIf true, keep the managedFields when printing objects in JSON or YAML format.--template stringTemplate string or path to template file to use when -o=go-template, -o=go-template-file. The template format is golang templates [http://golang.org/pkg/text/template/#pkg-overview].Parent Options Inherited--as stringUsername to impersonate for the operation. User could be a regular user or a service account in a namespace.--as-group stringsGroup to impersonate for the operation, this flag can be repeated to specify multiple groups.--as-uid stringUID to impersonate for the operation.--cache-dir string Default: "$HOME/.kube/cache"Default cache directory--certificate-authority stringPath to a cert file for the certificate authority--client-certificate stringPath to a client certificate file for TLS--client-key stringPath to a client key file for TLS--cluster stringThe name of the kubeconfig cluster to use--context stringThe name of the kubeconfig context to use--disable-compressionIf true, opt-out of response compression for all requests to the server--insecure-skip-tls-verifyIf true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure--kubeconfig stringPath to the kubeconfig file to use for CLI requests.--kuberc stringPath to the kuberc file to use for preferences. This can be disabled by exporting KUBECTL_KUBERC=false feature gate or turning off the feature KUBERC=off.--match-server-versionRequire server version to match client version-n, --namespace stringIf present, the namespace scope for this CLI request--password stringPassword for basic authentication to the API server--profile string Default: "none"Name of profile to capture. One of (none|cpu|heap|goroutine|threadcreate|block|mutex)--profile-output string Default: "profile.pprof"Name of the file to write the profile to--request-timeout string Default: "0"The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests.-s, --server stringThe address and port of the Kubernetes API server--storage-driver-buffer-duration duration Default: 1m0sWrites in the storage driver will be buffered for this duration, and committed to the non memory backends as a single transaction--storage-driver-db string Default: "cadvisor"database name--storage-driver-host string Default: "localhost:8086"database host:port--storage-driver-password string Default: "root"database password--storage-driver-secureuse secure connection with database--storage-driver-table string Default: "stats"table name--storage-driver-user string Default: "root"database username--tls-server-name stringServer name to use for server certificate validation. If it is not provided, the hostname used to contact the server is used--token stringBearer token for authentication to the API server--user stringThe name of the kubeconfig user to use--username stringUsername for basic authentication to the API server--version version[=true]--version, --version=raw prints version information and quits; --version=vX.Y.Z... sets the reported version--warnings-as-errorsTreat warnings received from the server as errors and exit with a non-zero exit codeSee Alsokubectl set - Set specific features on objectsThis page is automatically generated.If you plan to report an issue with this page, mention that the page is auto-generated in your issue description. The fix may need to happen elsewhere in the Kubernetes project.

kubectl set selector


Example: Deploying PHP Guestbook application with RedisThis tutorial shows you how to build and deploy a simple (not production ready), multi-tier web application using Kubernetes and Docker. This example consists of the following components:A single-instance Redis to store guestbook entriesMultiple web frontend instancesObjectivesStart up a Redis leader.Start up two Redis followers.Start up the guestbook frontend.Expose and view the Frontend Service.Clean up.Before you beginYou need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. It is recommended to run this tutorial on a cluster with at least two nodes that are not acting as control plane hosts. If you do not already have a cluster, you can create one by using minikube or you can use one of these Kubernetes playgrounds:iximiuz LabsKillercodaKodeKloudPlay with KubernetesYour Kubernetes server must be at or later than version v1.14.To check the version, enter kubectl version.Start up the Redis DatabaseThe guestbook application uses Redis to store its data.Creating the Redis DeploymentThe manifest file, included below, specifies a Deployment controller that runs a single replica Redis Pod.application/guestbook/redis-leader-deployment.yaml # SOURCE: https://cloud.google.com/kubernetes-engine/docs/tutorials/guestbook apiVersion: apps/v1 kind: Deployment metadata: name: redis-leader labels: app: redis role: leader tier: backend spec: replicas: 1 selector: matchLabels: app: redis template: metadata: labels: app: redis role: leader tier: backend spec: containers: - name: leader image: "docker.io/redis:6.0.5" resources: requests: cpu: 100m memory: 100Mi ports: - containerPort: 6379Launch a terminal window in the directory you downloaded the manifest files.Apply the Redis Deployment from the redis-leader-deployment.yaml file:kubectl apply -f https://k8s.io/examples/application/guestbook/redis-leader-deployment.yaml Query the list of Pods to verify that the Redis Pod is running:kubectl get pods The response should be similar to this:NAME READY STATUS RESTARTS AGE redis-leader-fb76b4755-xjr2n 1/1 Running 0 13s Run the following command to view the logs from the Redis leader Pod:kubectl logs -f deployment/redis-leader Creating the Redis leader ServiceThe guestbook application needs to communicate to the Redis to write its data. You need to apply a Service to proxy the traffic to the Redis Pod. A Service defines a policy to access the Pods.application/guestbook/redis-leader-service.yaml # SOURCE: https://cloud.google.com/kubernetes-engine/docs/tutorials/guestbook apiVersion: v1 kind: Service metadata: name: redis-leader labels: app: redis role: leader tier: backend spec: ports: - port: 6379 targetPort: 6379 selector: app: redis role: leader tier: backendApply the Redis Service from the following redis-leader-service.yaml file:kubectl apply -f https://k8s.io/examples/application/guestbook/redis-leader-service.yaml Query the list of Services to verify that the Redis Service is running:kubectl get service The response should be similar to this:NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.0.0.1 <none> 443/TCP 1m redis-leader ClusterIP 10.103.78.24 <none> 6379/TCP 16s Note:This manifest file creates a Service named redis-leader with a set of labels that match the labels previously defined, so the Service routes network traffic to the Redis Pod.Set up Redis followersAlthough the Redis leader is a single Pod, you can make it highly available and meet traffic demands by adding a few Redis followers, or replicas.application/guestbook/redis-follower-deployment.yaml # SOURCE: https://cloud.google.com/kubernetes-engine/docs/tutorials/guestbook apiVersion: apps/v1 kind: Deployment metadata: name: redis-follower labels: app: redis role: follower tier: backend spec: replicas: 2 selector: matchLabels: app: redis template: metadata: labels: app: redis role: follower tier: backend spec: containers: - name: follower image: us-docker.pkg.dev/google-samples/containers/gke/gb-redis-follower:v2 resources: requests: cpu: 100m memory: 100Mi ports: - containerPort: 6379Apply the Redis Deployment from the following redis-follower-deployment.yaml file:kubectl apply -f https://k8s.io/examples/application/guestbook/redis-follower-deployment.yaml Verify that the two Redis follower replicas are running by querying the list of Pods:kubectl get pods The response should be similar to this:NAME READY STATUS RESTARTS AGE redis-follower-dddfbdcc9-82sfr 1/1 Running 0 37s redis-follower-dddfbdcc9-qrt5k 1/1 Running 0 38s redis-leader-fb76b4755-xjr2n 1/1 Running 0 11m Creating the Redis follower serviceThe guestbook application needs to communicate with the Redis followers to read data. To make the Redis followers discoverable, you must set up another Service.application/guestbook/redis-follower-service.yaml # SOURCE: https://cloud.google.com/kubernetes-engine/docs/tutorials/guestbook apiVersion: v1 kind: Service metadata: name: redis-follower labels: app: redis role: follower tier: backend spec: ports: # the port that this service should serve on - port: 6379 selector: app: redis role: follower tier: backendApply the Redis Service from the following redis-follower-service.yaml file:kubectl apply -f https://k8s.io/examples/application/guestbook/redis-follower-service.yaml Query the list of Services to verify that the Redis Service is running:kubectl get service The response should be similar to this:NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 3d19h redis-follower ClusterIP 10.110.162.42 <none> 6379/TCP 9s redis-leader ClusterIP 10.103.78.24 <none> 6379/TCP 6m10s Note:This manifest file creates a Service named redis-follower with a set of labels that match the labels previously defined, so the Service routes network traffic to the Redis Pod.Set up and Expose the Guestbook FrontendNow that you have the Redis storage of your guestbook up and running, start the guestbook web servers. Like the Redis followers, the frontend is deployed using a Kubernetes Deployment.The guestbook app uses a PHP frontend. It is configured to communicate with either the Redis follower or leader Services, depending on whether the request is a read or a write. The frontend exposes a JSON interface, and serves a jQuery-Ajax-based UX.Creating the Guestbook Frontend Deploymentapplication/guestbook/frontend-deployment.yaml # SOURCE: https://cloud.google.com/kubernetes-engine/docs/tutorials/guestbook apiVersion: apps/v1 kind: Deployment metadata: name: frontend spec: replicas: 3 selector: matchLabels: app: guestbook tier: frontend template: metadata: labels: app: guestbook tier: frontend spec: containers: - name: php-redis image: us-docker.pkg.dev/google-samples/containers/gke/gb-frontend:v5 env: - name: GET_HOSTS_FROM value: "dns" resources: requests: cpu: 100m memory: 100Mi ports: - containerPort: 80 Apply the frontend Deployment from the frontend-deployment.yaml file:kubectl apply -f https://k8s.io/examples/application/guestbook/frontend-deployment.yaml Query the list of Pods to verify that the three frontend replicas are running:kubectl get pods -l app=guestbook -l tier=frontend The response should be similar to this:NAME READY STATUS RESTARTS AGE frontend-85595f5bf9-5tqhb 1/1 Running 0 47s frontend-85595f5bf9-qbzwm 1/1 Running 0 47s frontend-85595f5bf9-zchwc 1/1 Running 0 47s Creating the Frontend ServiceThe Redis Services you applied is only accessible within the Kubernetes cluster because the default type for a Service is ClusterIP. ClusterIP provides a single IP address for the set of Pods the Service is pointing to. This IP address is accessible only within the cluster.If you want guests to be able to access your guestbook, you must configure the frontend Service to be externally visible, so a client can request the Service from outside the Kubernetes cluster. However a Kubernetes user can use kubectl port-forward to access the service even though it uses a ClusterIP.Note:Some cloud providers, like Google Compute Engine or Google Kubernetes Engine, support external load balancers. If your cloud provider supports load balancers and you want to use it, uncomment type: LoadBalancer.application/guestbook/frontend-service.yaml # SOURCE: https://cloud.google.com/kubernetes-engine/docs/tutorials/guestbook apiVersion: v1 kind: Service metadata: name: frontend labels: app: guestbook tier: frontend spec: # if your cluster supports it, uncomment the following to automatically create # an external load-balanced IP for the frontend service. # type: LoadBalancer #type: LoadBalancer ports: # the port that this service should serve on - port: 80 selector: app: guestbook tier: frontendApply the frontend Service from the frontend-service.yaml file:kubectl apply -f https://k8s.io/examples/application/guestbook/frontend-service.yaml Query the list of Services to verify that the frontend Service is running:kubectl get services The response should be similar to this:NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE frontend ClusterIP 10.97.28.230 <none> 80/TCP 19s kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 3d19h redis-follower ClusterIP 10.110.162.42 <none> 6379/TCP 5m48s redis-leader ClusterIP 10.103.78.24 <none> 6379/TCP 11m Viewing the Frontend Service via kubectl port-forwardRun the following command to forward port 8080 on your local machine to port 80 on the service.kubectl port-forward svc/frontend 8080:80 The response should be similar to this:Forwarding from 127.0.0.1:8080 -> 80 Forwarding from [::1]:8080 -> 80 load the page http://localhost:8080 in your browser to view your guestbook.Viewing the Frontend Service via LoadBalancerIf you deployed the frontend-service.yaml manifest with type: LoadBalancer you need to find the IP address to view your Guestbook.Run the following command to get the IP address for the frontend Service.kubectl get service frontend The response should be similar to this:NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE frontend LoadBalancer 10.51.242.136 109.197.92.229 80:32372/TCP 1m Copy the external IP address, and load the page in your browser to view your guestbook.Note:Try adding some guestbook entries by typing in a message, and clicking Submit. The message you typed appears in the frontend. This message indicates that data is successfully added to Redis through the Services you created earlier.Scale the Web FrontendYou can scale up or down as needed because your servers are defined as a Service that uses a Deployment controller.Run the following command to scale up the number of frontend Pods:kubectl scale deployment frontend --replicas=5 Query the list of Pods to verify the number of frontend Pods running:kubectl get pods The response should look similar to this:NAME READY STATUS RESTARTS AGE frontend-85595f5bf9-5df5m 1/1 Running 0 83s frontend-85595f5bf9-7zmg5 1/1 Running 0 83s frontend-85595f5bf9-cpskg 1/1 Running 0 15m frontend-85595f5bf9-l2l54 1/1 Running 0 14m frontend-85595f5bf9-l9c8z 1/1 Running 0 14m redis-follower-dddfbdcc9-82sfr 1/1 Running 0 97m redis-follower-dddfbdcc9-qrt5k 1/1 Running 0 97m redis-leader-fb76b4755-xjr2n 1/1 Running 0 108m Run the following command to scale down the number of frontend Pods:kubectl scale deployment frontend --replicas=2 Query the list of Pods to verify the number of frontend Pods running:kubectl get pods The response should look similar to this:NAME READY STATUS RESTARTS AGE frontend-85595f5bf9-cpskg 1/1 Running 0 16m frontend-85595f5bf9-l9c8z 1/1 Running 0 15m redis-follower-dddfbdcc9-82sfr 1/1 Running 0 98m redis-follower-dddfbdcc9-qrt5k 1/1 Running 0 98m redis-leader-fb76b4755-xjr2n 1/1 Running 0 109m Cleaning upDeleting the Deployments and Services also deletes any running Pods. Use labels to delete multiple resources with one command.Run the following commands to delete all Pods, Deployments, and Services.kubectl delete deployment -l app=redis kubectl delete service -l app=redis kubectl delete deployment frontend kubectl delete service frontend The response should look similar to this:deployment.apps "redis-follower" deleted deployment.apps "redis-leader" deleted deployment.apps "frontend" deleted service "frontend" deleted Query the list of Pods to verify that no Pods are running:kubectl get pods The response should look similar to this:No resources found in default namespace. What's nextComplete the Kubernetes Basics Interactive TutorialsUse Kubernetes to create a blog using Persistent Volumes for MySQL and WordpressRead more about connecting applications with servicesRead more about using labels effectively


Kubelet Checkpoint APIFEATURE STATE: Kubernetes v1.30 [beta] (enabled by default: true)Checkpointing a container is the functionality to create a stateful copy of a running container. Once you have a stateful copy of a container, you could move it to a different computer for debugging or similar purposes.If you move the checkpointed container data to a computer that's able to restore it, that restored container continues to run at exactly the same point it was checkpointed. You can also inspect the saved data, provided that you have suitable tools for doing so.Creating a checkpoint of a container might have security implications. Typically a checkpoint contains all memory pages of all processes in the checkpointed container. This means that everything that used to be in memory is now available on the local disk. This includes all private data and possibly keys used for encryption. The underlying CRI implementations (the container runtime on that node) should create the checkpoint archive to be only accessible by the root user. It is still important to remember if the checkpoint archive is transferred to another system all memory pages will be readable by the owner of the checkpoint archive.Operationspost checkpoint the specified containerTell the kubelet to checkpoint a specific container from the specified Pod.Consult the Kubelet authentication/authorization reference for more information about how access to the kubelet checkpoint interface is controlled.The kubelet will request a checkpoint from the underlying CRI implementation. In the checkpoint request the kubelet will specify the name of the checkpoint archive as checkpoint-<podFullName>-<containerName>-<timestamp>.tar and also request to store the checkpoint archive in the checkpoints directory below its root directory (as defined by --root-dir). This defaults to /var/lib/kubelet/checkpoints.The checkpoint archive is in tar format, and could be listed using an implementation of tar. The contents of the archive depend on the underlying CRI implementation (the container runtime on that node).HTTP RequestPOST /checkpoint/{namespace}/{pod}/{container}Parametersnamespace (in path): string, requiredNamespacepod (in path): string, requiredPodcontainer (in path): string, requiredContainertimeout (in query): integerTimeout in seconds to wait until the checkpoint creation is finished. If zero or no timeout is specified the default CRI timeout value will be used. Checkpoint creation time depends directly on the used memory of the container. The more memory a container uses the more time is required to create the corresponding checkpoint.Response200: OK401: Unauthorized404: Not Found (if the ContainerCheckpoint feature gate is disabled)404: Not Found (if the specified namespace, pod or container cannot be found)500: Internal Server Error (if the CRI implementation encounter an error during checkpointing (see error message for further details))500: Internal Server Error (if the CRI implementation does not implement the checkpoint CRI API (see error message for further details))

Kubelet Checkpoint API


kubectl plugin listSynopsisList all available plugin files on a user's PATH. To see plugins binary names without the full path use --name-only flag.Available plugin files are those that are: - executable - anywhere on the user's PATH - begin with "kubectl-"kubectl plugin list [flags] Examples # List all available plugins kubectl plugin list # List only binary names of available plugins without paths kubectl plugin list --name-only Options-h, --helphelp for list--name-onlyIf true, display only the binary name of each plugin, rather than its full pathParent Options Inherited--as stringUsername to impersonate for the operation. User could be a regular user or a service account in a namespace.--as-group stringsGroup to impersonate for the operation, this flag can be repeated to specify multiple groups.--as-uid stringUID to impersonate for the operation.--cache-dir string Default: "$HOME/.kube/cache"Default cache directory--certificate-authority stringPath to a cert file for the certificate authority--client-certificate stringPath to a client certificate file for TLS--client-key stringPath to a client key file for TLS--cluster stringThe name of the kubeconfig cluster to use--context stringThe name of the kubeconfig context to use--disable-compressionIf true, opt-out of response compression for all requests to the server--insecure-skip-tls-verifyIf true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure--kubeconfig stringPath to the kubeconfig file to use for CLI requests.--kuberc stringPath to the kuberc file to use for preferences. This can be disabled by exporting KUBECTL_KUBERC=false feature gate or turning off the feature KUBERC=off.--match-server-versionRequire server version to match client version-n, --namespace stringIf present, the namespace scope for this CLI request--password stringPassword for basic authentication to the API server--profile string Default: "none"Name of profile to capture. One of (none|cpu|heap|goroutine|threadcreate|block|mutex)--profile-output string Default: "profile.pprof"Name of the file to write the profile to--request-timeout string Default: "0"The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests.-s, --server stringThe address and port of the Kubernetes API server--storage-driver-buffer-duration duration Default: 1m0sWrites in the storage driver will be buffered for this duration, and committed to the non memory backends as a single transaction--storage-driver-db string Default: "cadvisor"database name--storage-driver-host string Default: "localhost:8086"database host:port--storage-driver-password string Default: "root"database password--storage-driver-secureuse secure connection with database--storage-driver-table string Default: "stats"table name--storage-driver-user string Default: "root"database username--tls-server-name stringServer name to use for server certificate validation. If it is not provided, the hostname used to contact the server is used--token stringBearer token for authentication to the API server--user stringThe name of the kubeconfig user to use--username stringUsername for basic authentication to the API server--version version[=true]--version, --version=raw prints version information and quits; --version=vX.Y.Z... sets the reported version--warnings-as-errorsTreat warnings received from the server as errors and exit with a non-zero exit codeSee Alsokubectl plugin - Provides utilities for interacting with pluginsThis page is automatically generated.If you plan to report an issue with this page, mention that the page is auto-generated in your issue description. The fix may need to happen elsewhere in the Kubernetes project.

kubectl plugin list


Networking on WindowsKubernetes supports running nodes on either Linux or Windows. You can mix both kinds of node within a single cluster. This page provides an overview to networking specific to the Windows operating system.Container networking on WindowsNetworking for Windows containers is exposed through CNI plugins. Windows containers function similarly to virtual machines in regards to networking. Each container has a virtual network adapter (vNIC) which is connected to a Hyper-V virtual switch (vSwitch). The Host Networking Service (HNS) and the Host Compute Service (HCS) work together to create containers and attach container vNICs to networks. HCS is responsible for the management of containers whereas HNS is responsible for the management of networking resources such as:Virtual networks (including creation of vSwitches)Endpoints / vNICsNamespacesPolicies including packet encapsulations, load-balancing rules, ACLs, and NAT rules.The Windows HNS and vSwitch implement namespacing and can create virtual NICs as needed for a pod or container. However, many configurations such as DNS, routes, and metrics are stored in the Windows registry database rather than as files inside /etc, which is how Linux stores those configurations. The Windows registry for the container is separate from that of the host, so concepts like mapping /etc/resolv.conf from the host into a container don't have the same effect they would on Linux. These must be configured using Windows APIs run in the context of that container. Therefore CNI implementations need to call the HNS instead of relying on file mappings to pass network details into the pod or container.Network modesWindows supports five different networking drivers/modes: L2bridge, L2tunnel, Overlay (Beta), Transparent, and NAT. In a heterogeneous cluster with Windows and Linux worker nodes, you need to select a networking solution that is compatible on both Windows and Linux. The following table lists the out-of-tree plugins are supported on Windows, with recommendations on when to use each CNI:Network DriverDescriptionContainer Packet ModificationsNetwork PluginsNetwork Plugin CharacteristicsL2bridgeContainers are attached to an external vSwitch. Containers are attached to the underlay network, although the physical network doesn't need to learn the container MACs because they are rewritten on ingress/egress.MAC is rewritten to host MAC, IP may be rewritten to host IP using HNS OutboundNAT policy.win-bridge, Azure-CNI, Flannel host-gateway uses win-bridgewin-bridge uses L2bridge network mode, connects containers to the underlay of hosts, offering best performance. Requires user-defined routes (UDR) for inter-node connectivity.L2TunnelThis is a special case of l2bridge, but only used on Azure. All packets are sent to the virtualization host where SDN policy is applied.MAC rewritten, IP visible on the underlay networkAzure-CNIAzure-CNI allows integration of containers with Azure vNET, and allows them to leverage the set of capabilities that Azure Virtual Network provides. For example, securely connect to Azure services or use Azure NSGs. See azure-cni for some examplesOverlayContainers are given a vNIC connected to an external vSwitch. Each overlay network gets its own IP subnet, defined by a custom IP prefix.The overlay network driver uses VXLAN encapsulation.Encapsulated with an outer header.win-overlay, Flannel VXLAN (uses win-overlay)win-overlay should be used when virtual container networks are desired to be isolated from underlay of hosts (e.g. for security reasons). Allows for IPs to be re-used for different overlay networks (which have different VNID tags) if you are restricted on IPs in your datacenter. This option requires KB4489899 on Windows Server 2019.Transparent (special use case for ovn-kubernetes)Requires an external vSwitch. Containers are attached to an external vSwitch which enables intra-pod communication via logical networks (logical switches and routers).Packet is encapsulated either via GENEVE or STT tunneling to reach pods which are not on the same host.Packets are forwarded or dropped via the tunnel metadata information supplied by the ovn network controller.NAT is done for north-south communication.ovn-kubernetesDeploy via ansible. Distributed ACLs can be applied via Kubernetes policies. IPAM support. Load-balancing can be achieved without kube-proxy. NATing is done without using iptables/netsh.NAT (not used in Kubernetes)Containers are given a vNIC connected to an internal vSwitch. DNS/DHCP is provided using an internal component called WinNATMAC and IP is rewritten to host MAC/IP.natIncluded here for completenessAs outlined above, the Flannel CNI plugin is also supported on Windows via the VXLAN network backend (Beta support ; delegates to win-overlay) and host-gateway network backend (stable support; delegates to win-bridge).This plugin supports delegating to one of the reference CNI plugins (win-overlay, win-bridge), to work in conjunction with Flannel daemon on Windows (Flanneld) for automatic node subnet lease assignment and HNS network creation. This plugin reads in its own configuration file (cni.conf), and aggregates it with the environment variables from the FlannelD generated subnet.env file. It then delegates to one of the reference CNI plugins for network plumbing, and sends the correct configuration containing the node-assigned subnet to the IPAM plugin (for example: host-local).For Node, Pod, and Service objects, the following network flows are supported for TCP/UDP traffic:Pod → Pod (IP)Pod → Pod (Name)Pod → Service (Cluster IP)Pod → Service (PQDN, but only if there are no ".")Pod → Service (FQDN)Pod → external (IP)Pod → external (DNS)Node → PodPod → NodeIP address management (IPAM)The following IPAM options are supported on Windows:host-localazure-vnet-ipam (for azure-cni only)Windows Server IPAM (fallback option if no IPAM is set)Direct Server Return (DSR)FEATURE STATE: Kubernetes v1.34 [stable] (enabled by default: true)Load balancing mode where the IP address fixups and the LBNAT occurs at the container vSwitch port directly; service traffic arrives with the source IP set as the originating pod IP. This provides performance optimizations by allowing the return traffic routed through load balancers to bypass the load balancer and respond directly to the client; reducing load on the load balancer and also reducing overall latency. For more information, read Direct Server Return (DSR) in a nutshell.Load balancing and ServicesA Kubernetes Service is an abstraction that defines a logical set of Pods and a means to access them over a network. In a cluster that includes Windows nodes, you can use the following types of Service:NodePortClusterIPLoadBalancerExternalNameWindows container networking differs in some important ways from Linux networking. The Microsoft documentation for Windows Container Networking provides additional details and background.On Windows, you can use the following settings to configure Services and load balancing behavior:Windows Service SettingsFeatureDescriptionMinimum Supported Windows OS buildHow to enableSession affinityEnsures that connections from a particular client are passed to the same Pod each time.Windows Server 2022Set service.spec.sessionAffinity to "ClientIP"Direct Server Return (DSR)See DSR notes above.Windows Server 2019Set the following command line argument (assuming version 1.34): --enable-dsr=truePreserve-DestinationSkips DNAT of service traffic, thereby preserving the virtual IP of the target service in packets reaching the backend Pod. Also disables node-node forwarding.Windows Server, version 1903Set "preserve-destination": "true" in service annotations and enable DSR in kube-proxy.IPv4/IPv6 dual-stack networkingNative IPv4-to-IPv4 in parallel with IPv6-to-IPv6 communications to, from, and within a clusterWindows Server 2019See IPv4/IPv6 dual-stackClient IP preservationEnsures that source IP of incoming ingress traffic gets preserved. Also disables node-node forwarding.Windows Server 2019Set service.spec.externalTrafficPolicy to "Local" and enable DSR in kube-proxyLimitationsThe following networking functionality is not supported on Windows nodes:Host networking modeLocal NodePort access from the node itself (works for other nodes or external clients)More than 64 backend pods (or unique destination addresses) for a single ServiceIPv6 communication between Windows pods connected to overlay networksLocal Traffic Policy in non-DSR modeOutbound communication using the ICMP protocol via the win-overlay, win-bridge, or using the Azure-CNI plugin. Specifically, the Windows data plane (VFP) doesn't support ICMP packet transpositions, and this means:ICMP packets directed to destinations within the same network (such as pod to pod communication via ping) work as expected;TCP/UDP packets work as expected;ICMP packets directed to pass through a remote network (e.g. pod to external internet communication via ping) cannot be transposed and thus will not be routed back to their source;Since TCP/UDP packets can still be transposed, you can substitute ping <destination> with curl <destination> when debugging connectivity with the outside world.Other limitations:Windows reference network plugins win-bridge and win-overlay do not implement CNI spec v0.4.0, due to a missing CHECK implementation.The Flannel VXLAN CNI plugin has the following limitations on Windows:Node-pod connectivity is only possible for local pods with Flannel v0.12.0 (or higher).Flannel is restricted to using VNI 4096 and UDP port 4789. See the official Flannel VXLAN backend docs for more details on these parameters.

Networking on Windows


kubectl setSynopsisConfigure application resources.These commands help you make changes to existing application resources.kubectl set SUBCOMMAND Options-h, --helphelp for setParent Options Inherited--as stringUsername to impersonate for the operation. User could be a regular user or a service account in a namespace.--as-group stringsGroup to impersonate for the operation, this flag can be repeated to specify multiple groups.--as-uid stringUID to impersonate for the operation.--cache-dir string Default: "$HOME/.kube/cache"Default cache directory--certificate-authority stringPath to a cert file for the certificate authority--client-certificate stringPath to a client certificate file for TLS--client-key stringPath to a client key file for TLS--cluster stringThe name of the kubeconfig cluster to use--context stringThe name of the kubeconfig context to use--disable-compressionIf true, opt-out of response compression for all requests to the server--insecure-skip-tls-verifyIf true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure--kubeconfig stringPath to the kubeconfig file to use for CLI requests.--kuberc stringPath to the kuberc file to use for preferences. This can be disabled by exporting KUBECTL_KUBERC=false feature gate or turning off the feature KUBERC=off.--match-server-versionRequire server version to match client version-n, --namespace stringIf present, the namespace scope for this CLI request--password stringPassword for basic authentication to the API server--profile string Default: "none"Name of profile to capture. One of (none|cpu|heap|goroutine|threadcreate|block|mutex)--profile-output string Default: "profile.pprof"Name of the file to write the profile to--request-timeout string Default: "0"The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests.-s, --server stringThe address and port of the Kubernetes API server--storage-driver-buffer-duration duration Default: 1m0sWrites in the storage driver will be buffered for this duration, and committed to the non memory backends as a single transaction--storage-driver-db string Default: "cadvisor"database name--storage-driver-host string Default: "localhost:8086"database host:port--storage-driver-password string Default: "root"database password--storage-driver-secureuse secure connection with database--storage-driver-table string Default: "stats"table name--storage-driver-user string Default: "root"database username--tls-server-name stringServer name to use for server certificate validation. If it is not provided, the hostname used to contact the server is used--token stringBearer token for authentication to the API server--user stringThe name of the kubeconfig user to use--username stringUsername for basic authentication to the API server--version version[=true]--version, --version=raw prints version information and quits; --version=vX.Y.Z... sets the reported version--warnings-as-errorsTreat warnings received from the server as errors and exit with a non-zero exit codeSee Alsokubectl - kubectl controls the Kubernetes cluster managerkubectl set env - Update environment variables on a pod templatekubectl set image - Update the image of a pod templatekubectl set resources - Update resource requests/limits on objects with pod templateskubectl set selector - Set the selector on a resourcekubectl set serviceaccount - Update the service account of a resourcekubectl set subject - Update the user, group, or service account in a role binding or cluster role bindingThis page is automatically generated.If you plan to report an issue with this page, mention that the page is auto-generated in your issue description. The fix may need to happen elsewhere in the Kubernetes project.

kubectl set


Utilizing the NUMA-aware Memory ManagerFEATURE STATE: Kubernetes v1.32 [stable] (enabled by default: true)The Kubernetes Memory Manager enables the feature of guaranteed memory (and hugepages) allocation for pods in the Guaranteed QoS class.The Memory Manager employs hint generation protocol to yield the most suitable NUMA affinity for a pod. The Memory Manager feeds the central manager (Topology Manager) with these affinity hints. Based on both the hints and Topology Manager policy, the pod is rejected or admitted to the node.Moreover, the Memory Manager ensures that the memory which a pod requests is allocated from a minimum number of NUMA nodes.The Memory Manager is only pertinent to Linux based hosts.Before you beginYou need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. It is recommended to run this tutorial on a cluster with at least two nodes that are not acting as control plane hosts. If you do not already have a cluster, you can create one by using minikube or you can use one of these Kubernetes playgrounds:iximiuz LabsKillercodaKodeKloudPlay with KubernetesYour Kubernetes server must be at or later than version v1.32.To check the version, enter kubectl version.To align memory resources with other requested resources in a Pod spec:the CPU Manager should be enabled and proper CPU Manager policy should be configured on a Node. See control CPU Management Policies;the Topology Manager should be enabled and proper Topology Manager policy should be configured on a Node. See control Topology Management Policies.Starting from v1.22, the Memory Manager is enabled by default through MemoryManager feature gate.Preceding v1.22, the kubelet must be started with the following flag:--feature-gates=MemoryManager=truein order to enable the Memory Manager feature.How does the Memory Manager Operate?The Memory Manager currently offers the guaranteed memory (and hugepages) allocation for Pods in Guaranteed QoS class. To immediately put the Memory Manager into operation follow the guidelines in the section Memory Manager configuration, and subsequently, prepare and deploy a Guaranteed pod as illustrated in the section Placing a Pod in the Guaranteed QoS class.The Memory Manager is a Hint Provider, and it provides topology hints for the Topology Manager which then aligns the requested resources according to these topology hints. On Linux, it also enforces cgroups (i.e. cpuset.mems) for pods. The complete flow diagram concerning pod admission and deployment process is illustrated in Memory Manager KEP: Design Overview and below:During this process, the Memory Manager updates its internal counters stored in Node Map and Memory Maps to manage guaranteed memory allocation.The Memory Manager updates the Node Map during the startup and runtime as follows.StartupThis occurs once a node administrator employs --reserved-memory (section Reserved memory flag). In this case, the Node Map becomes updated to reflect this reservation as illustrated in Memory Manager KEP: Memory Maps at start-up (with examples).The administrator must provide --reserved-memory flag when Static policy is configured.RuntimeReference Memory Manager KEP: Memory Maps at runtime (with examples) illustrates how a successful pod deployment affects the Node Map, and it also relates to how potential Out-of-Memory (OOM) situations are handled further by Kubernetes or operating system.Important topic in the context of Memory Manager operation is the management of NUMA groups. Each time pod's memory request is in excess of single NUMA node capacity, the Memory Manager attempts to create a group that comprises several NUMA nodes and features extend memory capacity. The problem has been solved as elaborated in Memory Manager KEP: How to enable the guaranteed memory allocation over many NUMA nodes?. Also, reference Memory Manager KEP: Simulation - how the Memory Manager works? (by examples) illustrates how the management of groups occurs.Windows SupportFEATURE STATE: Kubernetes v1.32 [alpha] (enabled by default: false)Windows support can be enabled via the WindowsCPUAndMemoryAffinity feature gate and it requires support in the container runtime. Only the BestEffort Policy is supported on Windows.Memory Manager configurationOther Managers should be first pre-configured. Next, the Memory Manager feature should be enabled and be run with Static policy (section Static policy). Optionally, some amount of memory can be reserved for system or kubelet processes to increase node stability (section Reserved memory flag).PoliciesMemory Manager supports two policies. You can select a policy via a kubelet flag --memory-manager-policy:None (default)Static (Linux only)BestEffort (Windows Only)None policyThis is the default policy and does not affect the memory allocation in any way. It acts the same as if the Memory Manager is not present at all.The None policy returns default topology hint. This special hint denotes that Hint Provider (Memory Manager in this case) has no preference for NUMA affinity with any resource.Static policyIn the case of the Guaranteed pod, the Static Memory Manager policy returns topology hints relating to the set of NUMA nodes where the memory can be guaranteed, and reserves the memory through updating the internal NodeMap object.In the case of the BestEffort or Burstable pod, the Static Memory Manager policy sends back the default topology hint as there is no request for the guaranteed memory, and does not reserve the memory in the internal NodeMap object.This policy is only supported on Linux.BestEffort policyFEATURE STATE: Kubernetes v1.32 [alpha] (enabled by default: false)This policy is only supported on Windows.On Windows, NUMA node assignment works differently than Linux. There is no mechanism to ensure that Memory access only comes from a specific NUMA node. Instead the Windows scheduler will select the most optimal NUMA node based on the CPU(s) assignments. It is possible that Windows might use other NUMA nodes if deemed optimal by the Windows scheduler.The policy does track the amount of memory available and requested through the internal NodeMap. The memory manager will make a best effort at ensuring that enough memory is available on a NUMA node before making the assignment.This means that in most cases memory assignment should function as expected.Reserved memory flagThe Node Allocatable mechanism is commonly used by node administrators to reserve K8S node system resources for the kubelet or operating system processes in order to enhance the node stability. A dedicated set of flags can be used for this purpose to set the total amount of reserved memory for a node. This pre-configured value is subsequently utilized to calculate the real amount of node's "allocatable" memory available to pods.The Kubernetes scheduler incorporates "allocatable" to optimise pod scheduling process. The foregoing flags include --kube-reserved, --system-reserved and --eviction-threshold. The sum of their values will account for the total amount of reserved memory.A new --reserved-memory flag was added to Memory Manager to allow for this total reserved memory to be split (by a node administrator) and accordingly reserved across many NUMA nodes.The flag specifies a comma-separated list of memory reservations of different memory types per NUMA node. Memory reservations across multiple NUMA nodes can be specified using semicolon as separator. This parameter is only useful in the context of the Memory Manager feature. The Memory Manager will not use this reserved memory for the allocation of container workloads.For example, if you have a NUMA node "NUMA0" with 10Gi of memory available, and the --reserved-memory was specified to reserve 1Gi of memory at "NUMA0", the Memory Manager assumes that only 9Gi is available for containers.You can omit this parameter, however, you should be aware that the quantity of reserved memory from all NUMA nodes should be equal to the quantity of memory specified by the Node Allocatable feature. If at least one node allocatable parameter is non-zero, you will need to specify --reserved-memory for at least one NUMA node. In fact, eviction-hard threshold value is equal to 100Mi by default, so if Static policy is used, --reserved-memory is obligatory.Also, avoid the following configurations:duplicates, i.e. the same NUMA node or memory type, but with a different value;setting zero limit for any of memory types;NUMA node IDs that do not exist in the machine hardware;memory type names different than memory or hugepages-<size> (hugepages of particular <size> should also exist).Syntax:--reserved-memory N:memory-type1=value1,memory-type2=value2,...N (integer) - NUMA node index, e.g. 0memory-type (string) - represents memory type:memory - conventional memoryhugepages-2Mi or hugepages-1Gi - hugepagesvalue (string) - the quantity of reserved memory, e.g. 1GiExample usage:--reserved-memory 0:memory=1Gi,hugepages-1Gi=2Gior--reserved-memory 0:memory=1Gi --reserved-memory 1:memory=2Gior--reserved-memory '0:memory=1Gi;1:memory=2Gi'When you specify values for --reserved-memory flag, you must comply with the setting that you prior provided via Node Allocatable Feature flags. That is, the following rule must be obeyed for each memory type:sum(reserved-memory(i)) = kube-reserved + system-reserved + eviction-threshold,where i is an index of a NUMA node.If you do not follow the formula above, the Memory Manager will show an error on startup.In other words, the example above illustrates that for the conventional memory (type=memory), we reserve 3Gi in total, i.e.:sum(reserved-memory(i)) = reserved-memory(0) + reserved-memory(1) = 1Gi + 2Gi = 3GiAn example of kubelet command-line arguments relevant to the node Allocatable configuration:--kube-reserved=cpu=500m,memory=50Mi--system-reserved=cpu=123m,memory=333Mi--eviction-hard=memory.available<500MiNote:The default hard eviction threshold is 100MiB, and not zero. Remember to increase the quantity of memory that you reserve by setting --reserved-memory by that hard eviction threshold. Otherwise, the kubelet will not start Memory Manager and display an error.Here is an example of a correct configuration:--kube-reserved=cpu=4,memory=4Gi --system-reserved=cpu=1,memory=1Gi --memory-manager-policy=Static --reserved-memory '0:memory=3Gi;1:memory=2148Mi' Prior to Kubernetes 1.32, you also need to add--feature-gates=MemoryManager=true Let us validate the configuration above:kube-reserved + system-reserved + eviction-hard(default) = reserved-memory(0) + reserved-memory(1)4GiB + 1GiB + 100MiB = 3GiB + 2148MiB5120MiB + 100MiB = 3072MiB + 2148MiB5220MiB = 5220MiB (which is correct)Placing a Pod in the Guaranteed QoS classIf the selected policy is anything other than None, the Memory Manager identifies pods that are in the Guaranteed QoS class. The Memory Manager provides specific topology hints to the Topology Manager for each Guaranteed pod. For pods in a QoS class other than Guaranteed, the Memory Manager provides default topology hints to the Topology Manager.The following excerpts from pod manifests assign a pod to the Guaranteed QoS class.Pod with integer CPU(s) runs in the Guaranteed QoS class, when requests are equal to limits:spec: containers: - name: nginx image: nginx resources: limits: memory: "200Mi" cpu: "2" example.com/device: "1" requests: memory: "200Mi" cpu: "2" example.com/device: "1" Also, a pod sharing CPU(s) runs in the Guaranteed QoS class, when requests are equal to limits.spec: containers: - name: nginx image: nginx resources: limits: memory: "200Mi" cpu: "300m" example.com/device: "1" requests: memory: "200Mi" cpu: "300m" example.com/device: "1" Notice that both CPU and memory requests must be specified for a Pod to lend it to Guaranteed QoS class.TroubleshootingThe following means can be used to troubleshoot the reason why a pod could not be deployed or became rejected at a node:pod status - indicates topology affinity errorssystem logs - include valuable information for debugging, e.g., about generated hintsstate file - the dump of internal state of the Memory Manager (includes Node Map and Memory Maps)starting from v1.22, the device plugin resource API can be used to retrieve information about the memory reserved for containersPod status (TopologyAffinityError)This error typically occurs in the following situations:a node has not enough resources available to satisfy the pod's requestthe pod's request is rejected due to particular Topology Manager policy constraintsThe error appears in the status of a pod:kubectl get pods NAME READY STATUS RESTARTS AGE guaranteed 0/1 TopologyAffinityError 0 113s Use kubectl describe pod <id> or kubectl get events to obtain detailed error message:Warning TopologyAffinityError 10m kubelet, dell8 Resources cannot be allocated with Topology locality System logsSearch system logs with respect to a particular pod.The set of hints that Memory Manager generated for the pod can be found in the logs. Also, the set of hints generated by CPU Manager should be present in the logs.Topology Manager merges these hints to calculate a single best hint. The best hint should be also present in the logs.The best hint indicates where to allocate all the resources. Topology Manager tests this hint against its current policy, and based on the verdict, it either admits the pod to the node or rejects it.Also, search the logs for occurrences associated with the Memory Manager, e.g. to find out information about cgroups and cpuset.mems updates.Examine the memory manager state on a nodeLet us first deploy a sample Guaranteed pod whose specification is as follows:apiVersion: v1 kind: Pod metadata: name: guaranteed spec: containers: - name: guaranteed image: consumer imagePullPolicy: Never resources: limits: cpu: "2" memory: 150Gi requests: cpu: "2" memory: 150Gi command: ["sleep","infinity"] Next, let us log into the node where it was deployed and examine the state file in /var/lib/kubelet/memory_manager_state:{ "policyName":"Static", "machineState":{ "0":{ "numberOfAssignments":1, "memoryMap":{ "hugepages-1Gi":{ "total":0, "systemReserved":0, "allocatable":0, "reserved":0, "free":0 }, "memory":{ "total":134987354112, "systemReserved":3221225472, "allocatable":131766128640, "reserved":131766128640, "free":0 } }, "nodes":[ 0, 1 ] }, "1":{ "numberOfAssignments":1, "memoryMap":{ "hugepages-1Gi":{ "total":0, "systemReserved":0, "allocatable":0, "reserved":0, "free":0 }, "memory":{ "total":135286722560, "systemReserved":2252341248, "allocatable":133034381312, "reserved":29295144960, "free":103739236352 } }, "nodes":[ 0, 1 ] } }, "entries":{ "fa9bdd38-6df9-4cf9-aa67-8c4814da37a8":{ "guaranteed":[ { "numaAffinity":[ 0, 1 ], "type":"memory", "size":161061273600 } ] } }, "checksum":4142013182 } It can be deduced from the state file that the pod was pinned to both NUMA nodes, i.e.:"numaAffinity":[ 0, 1 ], Pinned term means that pod's memory consumption is constrained (through cgroups configuration) to these NUMA nodes.This automatically implies that Memory Manager instantiated a new group that comprises these two NUMA nodes, i.e. 0 and 1 indexed NUMA nodes.Notice that the management of groups is handled in a relatively complex manner, and further elaboration is provided in Memory Manager KEP in this and this sections.In order to analyse memory resources available in a group,the corresponding entries from NUMA nodes belonging to the group must be added up.For example, the total amount of free "conventional" memory in the group can be computed by adding up the free memory available at every NUMA node in the group, i.e., in the "memory" section of NUMA node 0 ("free":0) and NUMA node 1 ("free":103739236352). So, the total amount of free "conventional" memory in this group is equal to 0 + 103739236352 bytes.The line "systemReserved":3221225472 indicates that the administrator of this node reserved 3221225472 bytes (i.e. 3Gi) to serve kubelet and system processes at NUMA node 0, by using --reserved-memory flag.Device plugin resource APIThe kubelet provides a PodResourceLister gRPC service to enable discovery of resources and associated metadata. By using its List gRPC endpoint, information about reserved memory for each container can be retrieved, which is contained in protobuf ContainerMemory message. This information can be retrieved solely for pods in Guaranteed QoS class.What's nextMemory Manager KEP: Design OverviewMemory Manager KEP: Memory Maps at start-up (with examples)Memory Manager KEP: Memory Maps at runtime (with examples)Memory Manager KEP: Simulation - how the Memory Manager works? (by examples)Memory Manager KEP: The Concept of Node Map and Memory MapsMemory Manager KEP: How to enable the guaranteed memory allocation over many NUMA nodes?

Utilizing the NUMA-aware Memory Manager


SelfSubjectAccessReviewSelfSubjectAccessReview checks whether or the current user can perform an action.apiVersion: authorization.k8s.io/v1import "k8s.io/api/authorization/v1"SelfSubjectAccessReviewSelfSubjectAccessReview checks whether or the current user can perform an action. Not filling in a spec.namespace means "in all namespaces". Self is a special case, because users should always be able to check whether they can perform an actionapiVersion: authorization.k8s.io/v1kind: SelfSubjectAccessReviewmetadata (ObjectMeta)Standard list metadata. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadataspec (SelfSubjectAccessReviewSpec), requiredSpec holds information about the request being evaluated. user and groups must be emptystatus (SubjectAccessReviewStatus)Status is filled in by the server and indicates whether the request is allowed or notSelfSubjectAccessReviewSpecSelfSubjectAccessReviewSpec is a description of the access request. Exactly one of ResourceAuthorizationAttributes and NonResourceAuthorizationAttributes must be setnonResourceAttributes (NonResourceAttributes)NonResourceAttributes describes information for a non-resource access requestNonResourceAttributes includes the authorization attributes available for non-resource requests to the Authorizer interfacenonResourceAttributes.path (string)Path is the URL path of the requestnonResourceAttributes.verb (string)Verb is the standard HTTP verbresourceAttributes (ResourceAttributes)ResourceAuthorizationAttributes describes information for a resource access requestResourceAttributes includes the authorization attributes available for resource requests to the Authorizer interfaceresourceAttributes.fieldSelector (FieldSelectorAttributes)fieldSelector describes the limitation on access based on field. It can only limit access, not broaden it.*FieldSelectorAttributes indicates a field limited access. Webhook authors are encouraged to * ensure rawSelector and requirements are not both set * consider the requirements field if set * not try to parse or consider the rawSelector field if set. This is to avoid another CVE-2022-2880 (i.e. getting different systems to agree on how exactly to parse a query is not something we want), see https://www.oxeye.io/resources/golang-parameter-smuggling-attack for more details. For the SubjectAccessReview endpoints of the kube-apiserver: * If rawSelector is empty and requirements are empty, the request is not limited. * If rawSelector is present and requirements are empty, the rawSelector will be parsed and limited if the parsing succeeds. * If rawSelector is empty and requirements are present, the requirements should be honored * If rawSelector is present and requirements are present, the request is invalid.resourceAttributes.fieldSelector.rawSelector (string)rawSelector is the serialization of a field selector that would be included in a query parameter. Webhook implementations are encouraged to ignore rawSelector. The kube-apiserver's *SubjectAccessReview will parse the rawSelector as long as the requirements are not present.resourceAttributes.fieldSelector.requirements ([]FieldSelectorRequirement)Atomic: will be replaced during a mergerequirements is the parsed interpretation of a field selector. All requirements must be met for a resource instance to match the selector. Webhook implementations should handle requirements, but how to handle them is up to the webhook. Since requirements can only limit the request, it is safe to authorize as unlimited request if the requirements are not understood.FieldSelectorRequirement is a selector that contains values, a key, and an operator that relates the key and values.resourceAttributes.fieldSelector.requirements.key (string), requiredkey is the field selector key that the requirement applies to.resourceAttributes.fieldSelector.requirements.operator (string), requiredoperator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. The list of operators may grow in the future.resourceAttributes.fieldSelector.requirements.values ([]string)Atomic: will be replaced during a mergevalues is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty.resourceAttributes.group (string)Group is the API Group of the Resource. "*" means all.resourceAttributes.labelSelector (LabelSelectorAttributes)labelSelector describes the limitation on access based on labels. It can only limit access, not broaden it.*LabelSelectorAttributes indicates a label limited access. Webhook authors are encouraged to * ensure rawSelector and requirements are not both set * consider the requirements field if set * not try to parse or consider the rawSelector field if set. This is to avoid another CVE-2022-2880 (i.e. getting different systems to agree on how exactly to parse a query is not something we want), see https://www.oxeye.io/resources/golang-parameter-smuggling-attack for more details. For the SubjectAccessReview endpoints of the kube-apiserver: * If rawSelector is empty and requirements are empty, the request is not limited. * If rawSelector is present and requirements are empty, the rawSelector will be parsed and limited if the parsing succeeds. * If rawSelector is empty and requirements are present, the requirements should be honored * If rawSelector is present and requirements are present, the request is invalid.resourceAttributes.labelSelector.rawSelector (string)rawSelector is the serialization of a field selector that would be included in a query parameter. Webhook implementations are encouraged to ignore rawSelector. The kube-apiserver's *SubjectAccessReview will parse the rawSelector as long as the requirements are not present.resourceAttributes.labelSelector.requirements ([]LabelSelectorRequirement)Atomic: will be replaced during a mergerequirements is the parsed interpretation of a label selector. All requirements must be met for a resource instance to match the selector. Webhook implementations should handle requirements, but how to handle them is up to the webhook. Since requirements can only limit the request, it is safe to authorize as unlimited request if the requirements are not understood.A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.resourceAttributes.labelSelector.requirements.key (string), requiredkey is the label key that the selector applies to.resourceAttributes.labelSelector.requirements.operator (string), requiredoperator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.resourceAttributes.labelSelector.requirements.values ([]string)Atomic: will be replaced during a mergevalues is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.resourceAttributes.name (string)Name is the name of the resource being requested for a "get" or deleted for a "delete". "" (empty) means all.resourceAttributes.namespace (string)Namespace is the namespace of the action being requested. Currently, there is no distinction between no namespace and all namespaces "" (empty) is defaulted for LocalSubjectAccessReviews "" (empty) is empty for cluster-scoped resources "" (empty) means "all" for namespace scoped resources from a SubjectAccessReview or SelfSubjectAccessReviewresourceAttributes.resource (string)Resource is one of the existing resource types. "*" means all.resourceAttributes.subresource (string)Subresource is one of the existing resource types. "" means none.resourceAttributes.verb (string)Verb is a kubernetes resource API verb, like: get, list, watch, create, update, delete, proxy. "*" means all.resourceAttributes.version (string)Version is the API Version of the Resource. "*" means all.Operationscreate create a SelfSubjectAccessReviewHTTP RequestPOST /apis/authorization.k8s.io/v1/selfsubjectaccessreviewsParametersbody: SelfSubjectAccessReview, requireddryRun (in query): stringdryRunfieldManager (in query): stringfieldManagerfieldValidation (in query): stringfieldValidationpretty (in query): stringprettyResponse200 (SelfSubjectAccessReview): OK201 (SelfSubjectAccessReview): Created202 (SelfSubjectAccessReview): Accepted401: UnauthorizedThis page is automatically generated.If you plan to report an issue with this page, mention that the page is auto-generated in your issue description. The fix may need to happen elsewhere in the Kubernetes project.

SelfSubjectAccessReview