# Use Calico for NetworkPolicy

Use Calico for NetworkPolicyThis page shows a couple of quick ways to create a Calico cluster on Kubernetes.Before you beginDecide whether you want to deploy a cloud or local cluster.Creating a Calico cluster with Google Kubernetes Engine (GKE)Prerequisite: gcloud.To launch a GKE cluster with Calico, include the --enable-network-policy flag.Syntaxgcloud container clusters create [CLUSTER_NAME] --enable-network-policy Examplegcloud container clusters create my-calico-cluster --enable-network-policy To verify the deployment, use the following command.kubectl get pods --namespace=kube-system The Calico pods begin with calico. Check to make sure each one has a status of Running.Creating a local Calico cluster with kubeadmTo get a local single-host Calico cluster in fifteen minutes using kubeadm, refer to the Calico Quickstart.What's nextOnce your cluster is running, you can follow the Declare Network Policy to try out Kubernetes NetworkPolicy.


# IPv4/IPv6 dual-stack

IPv4/IPv6 dual-stackKubernetes lets you configure single-stack IPv4 networking, single-stack IPv6 networking, or dual stack networking with both network families active. This page explains how.FEATURE STATE: Kubernetes v1.23 [stable]IPv4/IPv6 dual-stack networking enables the allocation of both IPv4 and IPv6 addresses to Pods and Services.IPv4/IPv6 dual-stack networking is enabled by default for your Kubernetes cluster starting in 1.21, allowing the simultaneous assignment of both IPv4 and IPv6 addresses.Supported FeaturesIPv4/IPv6 dual-stack on your Kubernetes cluster provides the following features:Dual-stack Pod networking (a single IPv4 and IPv6 address assignment per Pod)IPv4 and IPv6 enabled ServicesPod off-cluster egress routing (eg. the Internet) via both IPv4 and IPv6 interfacesPrerequisitesThe following prerequisites are needed in order to utilize IPv4/IPv6 dual-stack Kubernetes clusters:Kubernetes 1.20 or laterFor information about using dual-stack services with earlier Kubernetes versions, refer to the documentation for that version of Kubernetes.Provider support for dual-stack networking (Cloud provider or otherwise must be able to provide Kubernetes nodes with routable IPv4/IPv6 network interfaces)A network plugin that supports dual-stack networking.Configure IPv4/IPv6 dual-stackTo configure IPv4/IPv6 dual-stack, set dual-stack cluster network assignments:kube-apiserver:--service-cluster-ip-range=<IPv4 CIDR>,<IPv6 CIDR>kube-controller-manager:--cluster-cidr=<IPv4 CIDR>,<IPv6 CIDR>--service-cluster-ip-range=<IPv4 CIDR>,<IPv6 CIDR>--node-cidr-mask-size-ipv4|--node-cidr-mask-size-ipv6 defaults to /24 for IPv4 and /64 for IPv6kube-proxy:--cluster-cidr=<IPv4 CIDR>,<IPv6 CIDR>kubelet:--node-ip=<IPv4 IP>,<IPv6 IP>This option is required for bare metal dual-stack nodes (nodes that do not define a cloud provider with the --cloud-provider flag). If you are using a cloud provider and choose to override the node IPs chosen by the cloud provider, set the --node-ip option.(The legacy built-in cloud providers do not support dual-stack --node-ip.)Note:An example of an IPv4 CIDR: 10.244.0.0/16 (though you would supply your own address range)An example of an IPv6 CIDR: fdXY:IJKL:MNOP:15::/64 (this shows the format but is not a valid address - see RFC 4193)ServicesYou can create Services which can use IPv4, IPv6, or both.The address family of a Service defaults to the address family of the first service cluster IP range (configured via the --service-cluster-ip-range flag to the kube-apiserver).When you define a Service you can optionally configure it as dual stack. To specify the behavior you want, you set the .spec.ipFamilyPolicy field to one of the following values:SingleStack: Single-stack service. The control plane allocates a cluster IP for the Service, using the first configured service cluster IP range.PreferDualStack: Allocates both IPv4 and IPv6 cluster IPs for the Service when dual-stack is enabled. If dual-stack is not enabled or supported, it falls back to single-stack behavior.RequireDualStack: Allocates Service .spec.clusterIPs from both IPv4 and IPv6 address ranges when dual-stack is enabled. If dual-stack is not enabled or supported, the Service API object creation fails.Selects the .spec.clusterIP from the list of .spec.clusterIPs based on the address family of the first element in the .spec.ipFamilies array.If you would like to define which IP family to use for single stack or define the order of IP families for dual-stack, you can choose the address families by setting an optional field, .spec.ipFamilies, on the Service.Note:The .spec.ipFamilies field is conditionally mutable: you can add or remove a secondary IP address family, but you cannot change the primary IP address family of an existing Service.You can set .spec.ipFamilies to any of the following array values:["IPv4"]["IPv6"]["IPv4","IPv6"] (dual stack)["IPv6","IPv4"] (dual stack)The first family you list is used for the legacy .spec.clusterIP field.Dual-stack Service configuration scenariosThese examples demonstrate the behavior of various dual-stack Service configuration scenarios.Dual-stack options on new ServicesThis Service specification does not explicitly define .spec.ipFamilyPolicy. When you create this Service, Kubernetes assigns a cluster IP for the Service from the first configured service-cluster-ip-range and sets the .spec.ipFamilyPolicy to SingleStack. (Services without selectors and headless Services with selectors will behave in this same way.)service/networking/dual-stack-default-svc.yaml apiVersion: v1 kind: Service metadata: name: my-service labels: app.kubernetes.io/name: MyApp spec: selector: app.kubernetes.io/name: MyApp ports: - protocol: TCP port: 80 This Service specification explicitly defines PreferDualStack in .spec.ipFamilyPolicy. When you create this Service on a dual-stack cluster, Kubernetes assigns both IPv4 and IPv6 addresses for the service. The control plane updates the .spec for the Service to record the IP address assignments. The field .spec.clusterIPs is the primary field, and contains both assigned IP addresses; .spec.clusterIP is a secondary field with its value calculated from .spec.clusterIPs.For the .spec.clusterIP field, the control plane records the IP address that is from the same address family as the first service cluster IP range.On a single-stack cluster, the .spec.clusterIPs and .spec.clusterIP fields both only list one address.On a cluster with dual-stack enabled, specifying RequireDualStack in .spec.ipFamilyPolicy behaves the same as PreferDualStack.service/networking/dual-stack-preferred-svc.yaml apiVersion: v1 kind: Service metadata: name: my-service labels: app.kubernetes.io/name: MyApp spec: ipFamilyPolicy: PreferDualStack selector: app.kubernetes.io/name: MyApp ports: - protocol: TCP port: 80 This Service specification explicitly defines IPv6 and IPv4 in .spec.ipFamilies as well as defining PreferDualStack in .spec.ipFamilyPolicy. When Kubernetes assigns an IPv6 and IPv4 address in .spec.clusterIPs, .spec.clusterIP is set to the IPv6 address because that is the first element in the .spec.clusterIPs array, overriding the default.service/networking/dual-stack-preferred-ipfamilies-svc.yaml apiVersion: v1 kind: Service metadata: name: my-service labels: app.kubernetes.io/name: MyApp spec: ipFamilyPolicy: PreferDualStack ipFamilies: - IPv6 - IPv4 selector: app.kubernetes.io/name: MyApp ports: - protocol: TCP port: 80 Dual-stack defaults on existing ServicesThese examples demonstrate the default behavior when dual-stack is newly enabled on a cluster where Services already exist. (Upgrading an existing cluster to 1.21 or beyond will enable dual-stack.)When dual-stack is enabled on a cluster, existing Services (whether IPv4 or IPv6) are configured by the control plane to set .spec.ipFamilyPolicy to SingleStack and set .spec.ipFamilies to the address family of the existing Service. The existing Service cluster IP will be stored in .spec.clusterIPs.service/networking/dual-stack-default-svc.yaml apiVersion: v1 kind: Service metadata: name: my-service labels: app.kubernetes.io/name: MyApp spec: selector: app.kubernetes.io/name: MyApp ports: - protocol: TCP port: 80 You can validate this behavior by using kubectl to inspect an existing service.kubectl get svc my-service -o yaml apiVersion: v1 kind: Service metadata: labels: app.kubernetes.io/name: MyApp name: my-service spec: clusterIP: 10.0.197.123 clusterIPs: - 10.0.197.123 ipFamilies: - IPv4 ipFamilyPolicy: SingleStack ports: - port: 80 protocol: TCP targetPort: 80 selector: app.kubernetes.io/name: MyApp type: ClusterIP status: loadBalancer: {} When dual-stack is enabled on a cluster, existing headless Services with selectors are configured by the control plane to set .spec.ipFamilyPolicy to SingleStack and set .spec.ipFamilies to the address family of the first service cluster IP range (configured via the --service-cluster-ip-range flag to the kube-apiserver) even though .spec.clusterIP is set to None.service/networking/dual-stack-default-svc.yaml apiVersion: v1 kind: Service metadata: name: my-service labels: app.kubernetes.io/name: MyApp spec: selector: app.kubernetes.io/name: MyApp ports: - protocol: TCP port: 80 You can validate this behavior by using kubectl to inspect an existing headless service with selectors.kubectl get svc my-service -o yaml apiVersion: v1 kind: Service metadata: labels: app.kubernetes.io/name: MyApp name: my-service spec: clusterIP: None clusterIPs: - None ipFamilies: - IPv4 ipFamilyPolicy: SingleStack ports: - port: 80 protocol: TCP targetPort: 80 selector: app.kubernetes.io/name: MyApp Switching Services between single-stack and dual-stackServices can be changed from single-stack to dual-stack and from dual-stack to single-stack.To change a Service from single-stack to dual-stack, change .spec.ipFamilyPolicy from SingleStack to PreferDualStack or RequireDualStack as desired. When you change this Service from single-stack to dual-stack, Kubernetes assigns the missing address family so that the Service now has IPv4 and IPv6 addresses.Edit the Service specification updating the .spec.ipFamilyPolicy from SingleStack to PreferDualStack.Before:spec: ipFamilyPolicy: SingleStack After:spec: ipFamilyPolicy: PreferDualStack To change a Service from dual-stack to single-stack, change .spec.ipFamilyPolicy from PreferDualStack or RequireDualStack to SingleStack. When you change this Service from dual-stack to single-stack, Kubernetes retains only the first element in the .spec.clusterIPs array, and sets .spec.clusterIP to that IP address and sets .spec.ipFamilies to the address family of .spec.clusterIPs.Headless Services without selectorFor Headless Services without selectors and without .spec.ipFamilyPolicy explicitly set, the .spec.ipFamilyPolicy field defaults to RequireDualStack.Service type LoadBalancerTo provision a dual-stack load balancer for your Service:Set the .spec.type field to LoadBalancerSet .spec.ipFamilyPolicy field to PreferDualStack or RequireDualStackNote:To use a dual-stack LoadBalancer type Service, your cloud provider must support IPv4 and IPv6 load balancers.Egress trafficIf you want to enable egress traffic in order to reach off-cluster destinations (eg. the public Internet) from a Pod that uses non-publicly routable IPv6 addresses, you need to enable the Pod to use a publicly routed IPv6 address via a mechanism such as transparent proxying or IP masquerading. The ip-masq-agent project supports IP masquerading on dual-stack clusters.Note:Ensure your CNI provider supports IPv6.Windows supportKubernetes on Windows does not support single-stack "IPv6-only" networking. However, dual-stack IPv4/IPv6 networking for pods and nodes with single-family services is supported.You can use IPv4/IPv6 dual-stack networking with l2bridge networks.Note:Overlay (VXLAN) networks on Windows do not support dual-stack networking.You can read more about the different network modes for Windows within the Networking on Windows topic.What's nextValidate IPv4/IPv6 dual-stack networkingEnable dual-stack networking using kubeadm


# PersistentVolumeClaim

PersistentVolumeClaimPersistentVolumeClaim is a user's request for and claim to a persistent volume.apiVersion: v1import "k8s.io/api/core/v1"PersistentVolumeClaimPersistentVolumeClaim is a user's request for and claim to a persistent volumeapiVersion: v1kind: PersistentVolumeClaimmetadata (ObjectMeta)Standard object's metadata. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadataspec (PersistentVolumeClaimSpec)spec defines the desired characteristics of a volume requested by a pod author. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#persistentvolumeclaimsstatus (PersistentVolumeClaimStatus)status represents the current information/status of a persistent volume claim. Read-only. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#persistentvolumeclaimsPersistentVolumeClaimSpecPersistentVolumeClaimSpec describes the common attributes of storage devices and allows a Source for provider-specific attributesaccessModes ([]string)Atomic: will be replaced during a mergeaccessModes contains the desired access modes the volume should have. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#access-modes-1selector (LabelSelector)selector is a label query over volumes to consider for binding.resources (VolumeResourceRequirements)resources represents the minimum resources the volume should have. If RecoverVolumeExpansionFailure feature is enabled users are allowed to specify resource requirements that are lower than previous value but must still be higher than capacity recorded in the status field of the claim. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#resourcesVolumeResourceRequirements describes the storage resource requirements for a volume.resources.limits (map[string]Quantity)Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/resources.requests (map[string]Quantity)Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. Requests cannot exceed Limits. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/volumeName (string)volumeName is the binding reference to the PersistentVolume backing this claim.storageClassName (string)storageClassName is the name of the StorageClass required by the claim. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#class-1volumeMode (string)volumeMode defines what type of volume is required by the claim. Value of Filesystem is implied when not included in claim spec.Beta leveldataSource (TypedLocalObjectReference)dataSource field can be used to specify either: * An existing VolumeSnapshot object (snapshot.storage.k8s.io/VolumeSnapshot) * An existing PVC (PersistentVolumeClaim) If the provisioner or an external controller can support the specified data source, it will create a new volume based on the contents of the specified data source. When the AnyVolumeDataSource feature gate is enabled, dataSource contents will be copied to dataSourceRef, and dataSourceRef contents will be copied to dataSource when dataSourceRef.namespace is not specified. If the namespace is specified, then dataSourceRef will not be copied to dataSource.dataSourceRef (TypedObjectReference)dataSourceRef specifies the object from which to populate the volume with data, if a non-empty volume is desired. This may be any object from a non-empty API group (non core object) or a PersistentVolumeClaim object. When this field is specified, volume binding will only succeed if the type of the specified object matches some installed volume populator or dynamic provisioner. This field will replace the functionality of the dataSource field and as such if both fields are non-empty, they must have the same value. For backwards compatibility, when namespace isn't specified in dataSourceRef, both fields (dataSource and dataSourceRef) will be set to the same value automatically if one of them is empty and the other is non-empty. When namespace is specified in dataSourceRef, dataSource isn't set to the same value and must be empty. There are three important differences between dataSource and dataSourceRef: * While dataSource only allows two specific types of objects, dataSourceRef allows any non-core object, as well as PersistentVolumeClaim objects.While dataSource ignores disallowed values (dropping them), dataSourceRef preserves all values, and generates an error if a disallowed value is specified.While dataSource only allows local objects, dataSourceRef allows objects in any namespaces. (Beta) Using this field requires the AnyVolumeDataSource feature gate to be enabled. (Alpha) Using the namespace field of dataSourceRef requires the CrossNamespaceVolumeDataSource feature gate to be enabled.TypedObjectReference contains enough information to let you locate the typed referenced objectdataSourceRef.kind (string), requiredKind is the type of resource being referenceddataSourceRef.name (string), requiredName is the name of resource being referenceddataSourceRef.apiGroup (string)APIGroup is the group for the resource being referenced. If APIGroup is not specified, the specified Kind must be in the core API group. For any other third-party types, APIGroup is required.dataSourceRef.namespace (string)Namespace is the namespace of resource being referenced Note that when a namespace is specified, a gateway.networking.k8s.io/ReferenceGrant object is required in the referent namespace to allow that namespace's owner to accept the reference. See the ReferenceGrant documentation for details. (Alpha) This field requires the CrossNamespaceVolumeDataSource feature gate to be enabled.volumeAttributesClassName (string)volumeAttributesClassName may be used to set the VolumeAttributesClass used by this claim. If specified, the CSI driver will create or update the volume with the attributes defined in the corresponding VolumeAttributesClass. This has a different purpose than storageClassName, it can be changed after the claim is created. An empty string or nil value indicates that no VolumeAttributesClass will be applied to the claim. If the claim enters an Infeasible error state, this field can be reset to its previous value (including nil) to cancel the modification. If the resource referred to by volumeAttributesClass does not exist, this PersistentVolumeClaim will be set to a Pending state, as reflected by the modifyVolumeStatus field, until such as a resource exists. More info: https://kubernetes.io/docs/concepts/storage/volume-attributes-classes/PersistentVolumeClaimStatusPersistentVolumeClaimStatus is the current status of a persistent volume claim.accessModes ([]string)Atomic: will be replaced during a mergeaccessModes contains the actual access modes the volume backing the PVC has. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#access-modes-1allocatedResourceStatuses (map[string]string)allocatedResourceStatuses stores status of resource being resized for the given PVC. Key names follow standard Kubernetes label syntax. Valid values are either: * Un-prefixed keys: - storage - the capacity of the volume. * Custom resources must use implementation-defined prefixed names such as "example.com/my-custom-resource" Apart from above values - keys that are unprefixed or have kubernetes.io prefix are considered reserved and hence may not be used.ClaimResourceStatus can be in any of following states: - ControllerResizeInProgress: State set when resize controller starts resizing the volume in control-plane. - ControllerResizeFailed: State set when resize has failed in resize controller with a terminal error. - NodeResizePending: State set when resize controller has finished resizing the volume but further resizing of volume is needed on the node. - NodeResizeInProgress: State set when kubelet starts resizing the volume. - NodeResizeFailed: State set when resizing has failed in kubelet with a terminal error. Transient errors don't set NodeResizeFailed. For example: if expanding a PVC for more capacity - this field can be one of the following states: - pvc.status.allocatedResourceStatus['storage'] = "ControllerResizeInProgress" - pvc.status.allocatedResourceStatus['storage'] = "ControllerResizeFailed" - pvc.status.allocatedResourceStatus['storage'] = "NodeResizePending" - pvc.status.allocatedResourceStatus['storage'] = "NodeResizeInProgress" - pvc.status.allocatedResourceStatus['storage'] = "NodeResizeFailed" When this field is not set, it means that no resize operation is in progress for the given PVC.A controller that receives PVC update with previously unknown resourceName or ClaimResourceStatus should ignore the update for the purpose it was designed. For example - a controller that only is responsible for resizing capacity of the volume, should ignore PVC updates that change other valid resources associated with PVC.This is an alpha field and requires enabling RecoverVolumeExpansionFailure feature.allocatedResources (map[string]Quantity)allocatedResources tracks the resources allocated to a PVC including its capacity. Key names follow standard Kubernetes label syntax. Valid values are either: * Un-prefixed keys: - storage - the capacity of the volume. * Custom resources must use implementation-defined prefixed names such as "example.com/my-custom-resource" Apart from above values - keys that are unprefixed or have kubernetes.io prefix are considered reserved and hence may not be used.Capacity reported here may be larger than the actual capacity when a volume expansion operation is requested. For storage quota, the larger value from allocatedResources and PVC.spec.resources is used. If allocatedResources is not set, PVC.spec.resources alone is used for quota calculation. If a volume expansion capacity request is lowered, allocatedResources is only lowered if there are no expansion operations in progress and if the actual volume capacity is equal or lower than the requested capacity.A controller that receives PVC update with previously unknown resourceName should ignore the update for the purpose it was designed. For example - a controller that only is responsible for resizing capacity of the volume, should ignore PVC updates that change other valid resources associated with PVC.This is an alpha field and requires enabling RecoverVolumeExpansionFailure feature.capacity (map[string]Quantity)capacity represents the actual resources of the underlying volume.conditions ([]PersistentVolumeClaimCondition)Patch strategy: merge on key typeMap: unique values on key type will be kept during a mergeconditions is the current Condition of persistent volume claim. If underlying persistent volume is being resized then the Condition will be set to 'Resizing'.PersistentVolumeClaimCondition contains details about state of pvcconditions.status (string), requiredStatus is the status of the condition. Can be True, False, Unknown. More info: https://kubernetes.io/docs/reference/kubernetes-api/config-and-storage-resources/persistent-volume-claim-v1/#:~:text=state%20of%20pvc-,conditions.status,-(string)%2C%20requiredconditions.type (string), requiredType is the type of the condition. More info: https://kubernetes.io/docs/reference/kubernetes-api/config-and-storage-resources/persistent-volume-claim-v1/#:~:text=set%20to%20%27ResizeStarted%27.-,PersistentVolumeClaimCondition,-contains%20details%20aboutconditions.lastProbeTime (Time)lastProbeTime is the time we probed the condition.Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.conditions.lastTransitionTime (Time)lastTransitionTime is the time the condition transitioned from one status to another.Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.conditions.message (string)message is the human-readable message indicating details about last transition.conditions.reason (string)reason is a unique, this should be a short, machine understandable string that gives the reason for condition's last transition. If it reports "Resizing" that means the underlying persistent volume is being resized.currentVolumeAttributesClassName (string)currentVolumeAttributesClassName is the current name of the VolumeAttributesClass the PVC is using. When unset, there is no VolumeAttributeClass applied to this PersistentVolumeClaimmodifyVolumeStatus (ModifyVolumeStatus)ModifyVolumeStatus represents the status object of ControllerModifyVolume operation. When this is unset, there is no ModifyVolume operation being attempted.ModifyVolumeStatus represents the status object of ControllerModifyVolume operationmodifyVolumeStatus.status (string), requiredstatus is the status of the ControllerModifyVolume operation. It can be in any of following states:Pending Pending indicates that the PersistentVolumeClaim cannot be modified due to unmet requirements, such as the specified VolumeAttributesClass not existing.InProgress InProgress indicates that the volume is being modified.Infeasible Infeasible indicates that the request has been rejected as invalid by the CSI driver. To resolve the error, a valid VolumeAttributesClass needs to be specified. Note: New statuses can be added in the future. Consumers should check for unknown statuses and fail appropriately.modifyVolumeStatus.targetVolumeAttributesClassName (string)targetVolumeAttributesClassName is the name of the VolumeAttributesClass the PVC currently being reconciledphase (string)phase represents the current phase of PersistentVolumeClaim.PersistentVolumeClaimListPersistentVolumeClaimList is a list of PersistentVolumeClaim items.apiVersion: v1kind: PersistentVolumeClaimListmetadata (ListMeta)Standard list metadata. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kindsitems ([]PersistentVolumeClaim), requireditems is a list of persistent volume claims. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#persistentvolumeclaimsOperationsget read the specified PersistentVolumeClaimHTTP RequestGET /api/v1/namespaces/{namespace}/persistentvolumeclaims/{name}Parametersname (in path): string, requiredname of the PersistentVolumeClaimnamespace (in path): string, requirednamespacepretty (in query): stringprettyResponse200 (PersistentVolumeClaim): OK401: Unauthorizedget read status of the specified PersistentVolumeClaimHTTP RequestGET /api/v1/namespaces/{namespace}/persistentvolumeclaims/{name}/statusParametersname (in path): string, requiredname of the PersistentVolumeClaimnamespace (in path): string, requirednamespacepretty (in query): stringprettyResponse200 (PersistentVolumeClaim): OK401: Unauthorizedlist list or watch objects of kind PersistentVolumeClaimHTTP RequestGET /api/v1/namespaces/{namespace}/persistentvolumeclaimsParametersnamespace (in path): string, requirednamespaceallowWatchBookmarks (in query): booleanallowWatchBookmarkscontinue (in query): stringcontinuefieldSelector (in query): stringfieldSelectorlabelSelector (in query): stringlabelSelectorlimit (in query): integerlimitpretty (in query): stringprettyresourceVersion (in query): stringresourceVersionresourceVersionMatch (in query): stringresourceVersionMatchsendInitialEvents (in query): booleansendInitialEventstimeoutSeconds (in query): integertimeoutSecondswatch (in query): booleanwatchResponse200 (PersistentVolumeClaimList): OK401: Unauthorizedlist list or watch objects of kind PersistentVolumeClaimHTTP RequestGET /api/v1/persistentvolumeclaimsParametersallowWatchBookmarks (in query): booleanallowWatchBookmarkscontinue (in query): stringcontinuefieldSelector (in query): stringfieldSelectorlabelSelector (in query): stringlabelSelectorlimit (in query): integerlimitpretty (in query): stringprettyresourceVersion (in query): stringresourceVersionresourceVersionMatch (in query): stringresourceVersionMatchsendInitialEvents (in query): booleansendInitialEventstimeoutSeconds (in query): integertimeoutSecondswatch (in query): booleanwatchResponse200 (PersistentVolumeClaimList): OK401: Unauthorizedcreate create a PersistentVolumeClaimHTTP RequestPOST /api/v1/namespaces/{namespace}/persistentvolumeclaimsParametersnamespace (in path): string, requirednamespacebody: PersistentVolumeClaim, requireddryRun (in query): stringdryRunfieldManager (in query): stringfieldManagerfieldValidation (in query): stringfieldValidationpretty (in query): stringprettyResponse200 (PersistentVolumeClaim): OK201 (PersistentVolumeClaim): Created202 (PersistentVolumeClaim): Accepted401: Unauthorizedupdate replace the specified PersistentVolumeClaimHTTP RequestPUT /api/v1/namespaces/{namespace}/persistentvolumeclaims/{name}Parametersname (in path): string, requiredname of the PersistentVolumeClaimnamespace (in path): string, requirednamespacebody: PersistentVolumeClaim, requireddryRun (in query): stringdryRunfieldManager (in query): stringfieldManagerfieldValidation (in query): stringfieldValidationpretty (in query): stringprettyResponse200 (PersistentVolumeClaim): OK201 (PersistentVolumeClaim): Created401: Unauthorizedupdate replace status of the specified PersistentVolumeClaimHTTP RequestPUT /api/v1/namespaces/{namespace}/persistentvolumeclaims/{name}/statusParametersname (in path): string, requiredname of the PersistentVolumeClaimnamespace (in path): string, requirednamespacebody: PersistentVolumeClaim, requireddryRun (in query): stringdryRunfieldManager (in query): stringfieldManagerfieldValidation (in query): stringfieldValidationpretty (in query): stringprettyResponse200 (PersistentVolumeClaim): OK201 (PersistentVolumeClaim): Created401: Unauthorizedpatch partially update the specified PersistentVolumeClaimHTTP RequestPATCH /api/v1/namespaces/{namespace}/persistentvolumeclaims/{name}Parametersname (in path): string, requiredname of the PersistentVolumeClaimnamespace (in path): string, requirednamespacebody: Patch, requireddryRun (in query): stringdryRunfieldManager (in query): stringfieldManagerfieldValidation (in query): stringfieldValidationforce (in query): booleanforcepretty (in query): stringprettyResponse200 (PersistentVolumeClaim): OK201 (PersistentVolumeClaim): Created401: Unauthorizedpatch partially update status of the specified PersistentVolumeClaimHTTP RequestPATCH /api/v1/namespaces/{namespace}/persistentvolumeclaims/{name}/statusParametersname (in path): string, requiredname of the PersistentVolumeClaimnamespace (in path): string, requirednamespacebody: Patch, requireddryRun (in query): stringdryRunfieldManager (in query): stringfieldManagerfieldValidation (in query): stringfieldValidationforce (in query): booleanforcepretty (in query): stringprettyResponse200 (PersistentVolumeClaim): OK201 (PersistentVolumeClaim): Created401: Unauthorizeddelete delete a PersistentVolumeClaimHTTP RequestDELETE /api/v1/namespaces/{namespace}/persistentvolumeclaims/{name}Parametersname (in path): string, requiredname of the PersistentVolumeClaimnamespace (in path): string, requirednamespacebody: DeleteOptionsdryRun (in query): stringdryRungracePeriodSeconds (in query): integergracePeriodSecondsignoreStoreReadErrorWithClusterBreakingPotential (in query): booleanignoreStoreReadErrorWithClusterBreakingPotentialpretty (in query): stringprettypropagationPolicy (in query): stringpropagationPolicyResponse200 (PersistentVolumeClaim): OK202 (PersistentVolumeClaim): Accepted401: Unauthorizeddeletecollection delete collection of PersistentVolumeClaimHTTP RequestDELETE /api/v1/namespaces/{namespace}/persistentvolumeclaimsParametersnamespace (in path): string, requirednamespacebody: DeleteOptionscontinue (in query): stringcontinuedryRun (in query): stringdryRunfieldSelector (in query): stringfieldSelectorgracePeriodSeconds (in query): integergracePeriodSecondsignoreStoreReadErrorWithClusterBreakingPotential (in query): booleanignoreStoreReadErrorWithClusterBreakingPotentiallabelSelector (in query): stringlabelSelectorlimit (in query): integerlimitpretty (in query): stringprettypropagationPolicy (in query): stringpropagationPolicyresourceVersion (in query): stringresourceVersionresourceVersionMatch (in query): stringresourceVersionMatchsendInitialEvents (in query): booleansendInitialEventstimeoutSeconds (in query): integertimeoutSecondsResponse200 (Status): OK401: UnauthorizedThis page is automatically generated.If you plan to report an issue with this page, mention that the page is auto-generated in your issue description. The fix may need to happen elsewhere in the Kubernetes project.


# Bootstrapping clusters with kubeadm

Bootstrapping clusters with kubeadmInstalling kubeadmTroubleshooting kubeadmCreating a cluster with kubeadmCustomizing components with the kubeadm APIOptions for Highly Available TopologyCreating Highly Available Clusters with kubeadmSet up a High Availability etcd Cluster with kubeadmConfiguring each kubelet in your cluster using kubeadmDual-stack support with kubeadm


# Jobs

JobsJobs represent one-off tasks that run to completion and then stop.A Job creates one or more Pods and will continue to retry execution of the Pods until a specified number of them successfully terminate. As pods successfully complete, the Job tracks the successful completions. When a specified number of successful completions is reached, the task (ie, Job) is complete. Deleting a Job will clean up the Pods it created. Suspending a Job will delete its active Pods until the Job is resumed again.A simple case is to create one Job object in order to reliably run one Pod to completion. The Job object will start a new Pod if the first Pod fails or is deleted (for example due to a node hardware failure or a node reboot).You can also use a Job to run multiple Pods in parallel.If you want to run a Job (either a single task, or several in parallel) on a schedule, see CronJob.Running an example JobHere is an example Job config. It computes π to 2000 places and prints it out. It takes around 10s to complete.controllers/job.yaml apiVersion: batch/v1 kind: Job metadata: name: pi spec: template: spec: containers: - name: pi image: perl:5.34.0 command: ["perl", "-Mbignum=bpi", "-wle", "print bpi(2000)"] restartPolicy: Never backoffLimit: 4 You can run the example with this command:kubectl apply -f https://kubernetes.io/examples/controllers/job.yaml The output is similar to this:job.batch/pi created Check on the status of the Job with kubectl: Name: pi Namespace: default Selector: batch.kubernetes.io/controller-uid=c9948307-e56d-4b5d-8302-ae2d7b7da67c Labels: batch.kubernetes.io/controller-uid=c9948307-e56d-4b5d-8302-ae2d7b7da67c batch.kubernetes.io/job-name=pi ... Annotations: batch.kubernetes.io/job-tracking: "" Parallelism: 1 Completions: 1 Start Time: Mon, 02 Dec 2019 15:20:11 +0200 Completed At: Mon, 02 Dec 2019 15:21:16 +0200 Duration: 65s Pods Statuses: 0 Running / 1 Succeeded / 0 Failed Pod Template: Labels: batch.kubernetes.io/controller-uid=c9948307-e56d-4b5d-8302-ae2d7b7da67c batch.kubernetes.io/job-name=pi Containers: pi: Image: perl:5.34.0 Port: <none> Host Port: <none> Command: perl -Mbignum=bpi -wle print bpi(2000) Environment: <none> Mounts: <none> Volumes: <none> Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal SuccessfulCreate 21s job-controller Created pod: pi-xf9p4 Normal Completed 18s job-controller Job completed apiVersion: batch/v1 kind: Job metadata: annotations: batch.kubernetes.io/job-tracking: "" ... creationTimestamp: "2022-11-10T17:53:53Z" generation: 1 labels: batch.kubernetes.io/controller-uid: 863452e6-270d-420e-9b94-53a54146c223 batch.kubernetes.io/job-name: pi name: pi namespace: default resourceVersion: "4751" uid: 204fb678-040b-497f-9266-35ffa8716d14 spec: backoffLimit: 4 completionMode: NonIndexed completions: 1 parallelism: 1 selector: matchLabels: batch.kubernetes.io/controller-uid: 863452e6-270d-420e-9b94-53a54146c223 suspend: false template: metadata: creationTimestamp: null labels: batch.kubernetes.io/controller-uid: 863452e6-270d-420e-9b94-53a54146c223 batch.kubernetes.io/job-name: pi spec: containers: - command: - perl - -Mbignum=bpi - -wle - print bpi(2000) image: perl:5.34.0 imagePullPolicy: IfNotPresent name: pi resources: {} terminationMessagePath: /dev/termination-log terminationMessagePolicy: File dnsPolicy: ClusterFirst restartPolicy: Never schedulerName: default-scheduler securityContext: {} terminationGracePeriodSeconds: 30 status: active: 1 ready: 0 startTime: "2022-11-10T17:53:57Z" uncountedTerminatedPods: {} To view completed Pods of a Job, use kubectl get pods.To list all the Pods that belong to a Job in a machine readable form, you can use a command like this:pods=$(kubectl get pods --selector=batch.kubernetes.io/job-name=pi --output=jsonpath='{.items[*].metadata.name}') echo $pods The output is similar to this:pi-5rwd7 Here, the selector is the same as the selector for the Job. The --output=jsonpath option specifies an expression with the name from each Pod in the returned list.View the standard output of one of the pods:kubectl logs $pods Another way to view the logs of a Job:kubectl logs jobs/pi The output is similar to this:3.1415926535897932384626433832795028841971693993751058209749445923078164062862089986280348253421170679821480865132823066470938446095505822317253594081284811174502841027019385211055596446229489549303819644288109756659334461284756482337867831652712019091456485669234603486104543266482133936072602491412737245870066063155881748815209209628292540917153643678925903600113305305488204665213841469519415116094330572703657595919530921861173819326117931051185480744623799627495673518857527248912279381830119491298336733624406566430860213949463952247371907021798609437027705392171762931767523846748184676694051320005681271452635608277857713427577896091736371787214684409012249534301465495853710507922796892589235420199561121290219608640344181598136297747713099605187072113499999983729780499510597317328160963185950244594553469083026425223082533446850352619311881710100031378387528865875332083814206171776691473035982534904287554687311595628638823537875937519577818577805321712268066130019278766111959092164201989380952572010654858632788659361533818279682303019520353018529689957736225994138912497217752834791315155748572424541506959508295331168617278558890750983817546374649393192550604009277016711390098488240128583616035637076601047101819429555961989467678374494482553797747268471040475346462080466842590694912933136770289891521047521620569660240580381501935112533824300355876402474964732639141992726042699227967823547816360093417216412199245863150302861829745557067498385054945885869269956909272107975093029553211653449872027559602364806654991198818347977535663698074265425278625518184175746728909777727938000816470600161452491921732172147723501414419735685481613611573525521334757418494684385233239073941433345477624168625189835694855620992192221842725502542568876717904946016534668049886272327917860857843838279679766814541009538837863609506800642251252051173929848960841284886269456042419652850222106611863067442786220391949450471237137869609563643719172874677646575739624138908658326459958133904780275901 Writing a Job specAs with all other Kubernetes config, a Job needs apiVersion, kind, and metadata fields.When the control plane creates new Pods for a Job, the .metadata.name of the Job is part of the basis for naming those Pods. The name of a Job must be a valid DNS subdomain value, but this can produce unexpected results for the Pod hostnames. For best compatibility, the name should follow the more restrictive rules for a DNS label. Even when the name is a DNS subdomain, the name must be no longer than 63 characters.A Job also needs a .spec section.Job LabelsJob labels will have batch.kubernetes.io/ prefix for job-name and controller-uid.Pod TemplateThe .spec.template is the only required field of the .spec.The .spec.template is a pod template. It has exactly the same schema as a Pod, except it is nested and does not have an apiVersion or kind.In addition to required fields for a Pod, a pod template in a Job must specify appropriate labels (see pod selector) and an appropriate restart policy.Only a RestartPolicy equal to Never or OnFailure is allowed.Pod selectorThe .spec.selector field is optional. In almost all cases you should not specify it. See section specifying your own pod selector.Parallel execution for JobsThere are three main types of task suitable to run as a Job:Non-parallel Jobsnormally, only one Pod is started, unless the Pod fails.the Job is complete as soon as its Pod terminates successfully.Parallel Jobs with a fixed completion count:specify a non-zero positive value for .spec.completions.the Job represents the overall task, and is complete when there are .spec.completions successful Pods.when using .spec.completionMode="Indexed", each Pod gets a different index in the range 0 to .spec.completions-1.Parallel Jobs with a work queue:do not specify .spec.completions, default to .spec.parallelism.the Pods must coordinate amongst themselves or an external service to determine what each should work on. For example, a Pod might fetch a batch of up to N items from the work queue.each Pod is independently capable of determining whether or not all its peers are done, and thus that the entire Job is done.when any Pod from the Job terminates with success, no new Pods are created.once at least one Pod has terminated with success and all Pods are terminated, then the Job is completed with success.once any Pod has exited with success, no other Pod should still be doing any work for this task or writing any output. They should all be in the process of exiting.For a non-parallel Job, you can leave both .spec.completions and .spec.parallelism unset. When both are unset, both are defaulted to 1.For a fixed completion count Job, you should set .spec.completions to the number of completions needed. You can set .spec.parallelism, or leave it unset and it will default to 1.For a work queue Job, you must leave .spec.completions unset, and set .spec.parallelism to a non-negative integer.For more information about how to make use of the different types of job, see the job patterns section.Controlling parallelismThe requested parallelism (.spec.parallelism) can be set to any non-negative value. If it is unspecified, it defaults to 1. If it is specified as 0, then the Job is effectively paused until it is increased.Actual parallelism (number of pods running at any instant) may be more or less than requested parallelism, for a variety of reasons:For fixed completion count Jobs, the actual number of pods running in parallel will not exceed the number of remaining completions. Higher values of .spec.parallelism are effectively ignored.For work queue Jobs, no new Pods are started after any Pod has succeeded -- remaining Pods are allowed to complete, however.If the Job Controller has not had time to react.If the Job controller failed to create Pods for any reason (lack of ResourceQuota, lack of permission, etc.), then there may be fewer pods than requested.The Job controller may throttle new Pod creation due to excessive previous pod failures in the same Job.When a Pod is gracefully shut down, it takes time to stop.Completion modeFEATURE STATE: Kubernetes v1.24 [stable]Jobs with fixed completion count - that is, jobs that have non null .spec.completions - can have a completion mode that is specified in .spec.completionMode:NonIndexed (default): the Job is considered complete when there have been .spec.completions successfully completed Pods. In other words, each Pod completion is homologous to each other. Note that Jobs that have null .spec.completions are implicitly NonIndexed.Indexed: the Pods of a Job get an associated completion index from 0 to .spec.completions-1. The index is available through four mechanisms:The Pod annotation batch.kubernetes.io/job-completion-index.The Pod label batch.kubernetes.io/job-completion-index (for v1.28 and later). Note the feature gate PodIndexLabel must be enabled to use this label, and it is enabled by default.As part of the Pod hostname, following the pattern $(job-name)-$(index). When you use an Indexed Job in combination with a Service, Pods within the Job can use the deterministic hostnames to address each other via DNS. For more information about how to configure this, see Job with Pod-to-Pod Communication.From the containerized task, in the environment variable JOB_COMPLETION_INDEX.The Job is considered complete when there is one successfully completed Pod for each index. For more information about how to use this mode, see Indexed Job for Parallel Processing with Static Work Assignment.Note:Although rare, more than one Pod could be started for the same index (due to various reasons such as node failures, kubelet restarts, or Pod evictions). In this case, only the first Pod that completes successfully will count towards the completion count and update the status of the Job. The other Pods that are running or completed for the same index will be deleted by the Job controller once they are detected.Handling Pod and container failuresA container in a Pod may fail for a number of reasons, such as because the process in it exited with a non-zero exit code, or the container was killed for exceeding a memory limit, etc. If this happens, and the .spec.template.spec.restartPolicy = "OnFailure", then the Pod stays on the node, but the container is re-run. Therefore, your program needs to handle the case when it is restarted locally, or else specify .spec.template.spec.restartPolicy = "Never". See pod lifecycle for more information on restartPolicy.An entire Pod can also fail, for a number of reasons, such as when the pod is kicked off the node (node is upgraded, rebooted, deleted, etc.), or if a container of the Pod fails and the .spec.template.spec.restartPolicy = "Never". When a Pod fails, then the Job controller starts a new Pod. This means that your application needs to handle the case when it is restarted in a new pod. In particular, it needs to handle temporary files, locks, incomplete output and the like caused by previous runs.By default, each pod failure is counted towards the .spec.backoffLimit limit, see pod backoff failure policy. However, you can customize handling of pod failures by setting the Job's pod failure policy.Additionally, you can choose to count the pod failures independently for each index of an Indexed Job by setting the .spec.backoffLimitPerIndex field (for more information, see backoff limit per index).Note that even if you specify .spec.parallelism = 1 and .spec.completions = 1 and .spec.template.spec.restartPolicy = "Never", the same program may sometimes be started twice.If you do specify .spec.parallelism and .spec.completions both greater than 1, then there may be multiple pods running at once. Therefore, your pods must also be tolerant of concurrency.If you specify the .spec.podFailurePolicy field, the Job controller does not consider a terminating Pod (a pod that has a .metadata.deletionTimestamp field set) as a failure until that Pod is terminal (its .status.phase is Failed or Succeeded). However, the Job controller creates a replacement Pod as soon as the termination becomes apparent. Once the pod terminates, the Job controller evaluates .backoffLimit and .podFailurePolicy for the relevant Job, taking this now-terminated Pod into consideration.If either of these requirements is not satisfied, the Job controller counts a terminating Pod as an immediate failure, even if that Pod later terminates with phase: "Succeeded".Pod backoff failure policyThere are situations where you want to fail a Job after some amount of retries due to a logical error in configuration etc. To do so, set .spec.backoffLimit to specify the number of retries before considering a Job as failed.The .spec.backoffLimit is set by default to 6, unless the backoff limit per index (only Indexed Job) is specified. When .spec.backoffLimitPerIndex is specified, then .spec.backoffLimit defaults to 2147483647 (MaxInt32).Failed Pods associated with the Job are recreated by the Job controller with an exponential back-off delay (10s, 20s, 40s ...) capped at six minutes.The number of retries is calculated in two ways:The number of Pods with .status.phase = "Failed".When using restartPolicy = "OnFailure", the number of retries in all the containers of Pods with .status.phase equal to Pending or Running.If either of the calculations reaches the .spec.backoffLimit, the Job is considered failed.Note:If your job has restartPolicy = "OnFailure", keep in mind that your Pod running the Job will be terminated once the job backoff limit has been reached. This can make debugging the Job's executable more difficult. We suggest setting restartPolicy = "Never" when debugging the Job or using a logging system to ensure output from failed Jobs is not lost inadvertently.Backoff limit per indexFEATURE STATE: Kubernetes v1.33 [stable] (enabled by default: true)When you run an indexed Job, you can choose to handle retries for pod failures independently for each index. To do so, set the .spec.backoffLimitPerIndex to specify the maximal number of pod failures per index.When the per-index backoff limit is exceeded for an index, Kubernetes considers the index as failed and adds it to the .status.failedIndexes field. The succeeded indexes, those with a successfully executed pods, are recorded in the .status.completedIndexes field, regardless of whether you set the backoffLimitPerIndex field.Note that a failing index does not interrupt execution of other indexes. Once all indexes finish for a Job where you specified a backoff limit per index, if at least one of those indexes did fail, the Job controller marks the overall Job as failed, by setting the Failed condition in the status. The Job gets marked as failed even if some, potentially nearly all, of the indexes were processed successfully.You can additionally limit the maximal number of indexes marked failed by setting the .spec.maxFailedIndexes field. When the number of failed indexes exceeds the maxFailedIndexes field, the Job controller triggers termination of all remaining running Pods for that Job. Once all pods are terminated, the entire Job is marked failed by the Job controller, by setting the Failed condition in the Job status.Here is an example manifest for a Job that defines a backoffLimitPerIndex:/controllers/job-backoff-limit-per-index-example.yaml apiVersion: batch/v1 kind: Job metadata: name: job-backoff-limit-per-index-example spec: completions: 10 parallelism: 3 completionMode: Indexed # required for the feature backoffLimitPerIndex: 1 # maximal number of failures per index maxFailedIndexes: 5 # maximal number of failed indexes before terminating the Job execution template: spec: restartPolicy: Never # required for the feature containers: - name: example image: python command: # The jobs fails as there is at least one failed index # (all even indexes fail in here), yet all indexes # are executed as maxFailedIndexes is not exceeded. - python3 - -c - | import os, sys print("Hello world") if int(os.environ.get("JOB_COMPLETION_INDEX")) % 2 == 0: sys.exit(1) In the example above, the Job controller allows for one restart for each of the indexes. When the total number of failed indexes exceeds 5, then the entire Job is terminated.Once the job is finished, the Job status looks as follows:kubectl get -o yaml job job-backoff-limit-per-index-example status: completedIndexes: 1,3,5,7,9 failedIndexes: 0,2,4,6,8 succeeded: 5 # 1 succeeded pod for each of 5 succeeded indexes failed: 10 # 2 failed pods (1 retry) for each of 5 failed indexes conditions: - message: Job has failed indexes reason: FailedIndexes status: "True" type: FailureTarget - message: Job has failed indexes reason: FailedIndexes status: "True" type: Failed The Job controller adds the FailureTarget Job condition to trigger Job termination and cleanup. When all of the Job Pods are terminated, the Job controller adds the Failed condition with the same values for reason and message as the FailureTarget Job condition. For details, see Termination of Job Pods.Additionally, you may want to use the per-index backoff along with a pod failure policy. When using per-index backoff, there is a new FailIndex action available which allows you to avoid unnecessary retries within an index.Pod failure policyFEATURE STATE: Kubernetes v1.31 [stable] (enabled by default: true)A Pod failure policy, defined with the .spec.podFailurePolicy field, enables your cluster to handle Pod failures based on the container exit codes and the Pod conditions.In some situations, you may want to have a better control when handling Pod failures than the control provided by the Pod backoff failure policy, which is based on the Job's .spec.backoffLimit. These are some examples of use cases:To optimize costs of running workloads by avoiding unnecessary Pod restarts, you can terminate a Job as soon as one of its Pods fails with an exit code indicating a software bug.To guarantee that your Job finishes even if there are disruptions, you can ignore Pod failures caused by disruptions (such as preemption, API-initiated eviction or taint-based eviction) so that they don't count towards the .spec.backoffLimit limit of retries.You can configure a Pod failure policy, in the .spec.podFailurePolicy field, to meet the above use cases. This policy can handle Pod failures based on the container exit codes and the Pod conditions.Here is a manifest for a Job that defines a podFailurePolicy:/controllers/job-pod-failure-policy-example.yaml apiVersion: batch/v1 kind: Job metadata: name: job-pod-failure-policy-example spec: completions: 12 parallelism: 3 template: spec: restartPolicy: Never containers: - name: main image: docker.io/library/bash:5 command: ["bash"] # example command simulating a bug which triggers the FailJob action args: - -c - echo "Hello world!" && sleep 5 && exit 42 backoffLimit: 6 podFailurePolicy: rules: - action: FailJob onExitCodes: containerName: main # optional operator: In # one of: In, NotIn values: [42] - action: Ignore # one of: Ignore, FailJob, Count onPodConditions: - type: DisruptionTarget # indicates Pod disruption In the example above, the first rule of the Pod failure policy specifies that the Job should be marked failed if the main container fails with the 42 exit code. The following are the rules for the main container specifically:an exit code of 0 means that the container succeededan exit code of 42 means that the entire Job failedany other exit code represents that the container failed, and hence the entire Pod. The Pod will be re-created if the total number of restarts is below backoffLimit. If the backoffLimit is reached the entire Job failed.Note:Because the Pod template specifies a restartPolicy: Never, the kubelet does not restart the main container in that particular Pod.The second rule of the Pod failure policy, specifying the Ignore action for failed Pods with condition DisruptionTarget excludes Pod disruptions from being counted towards the .spec.backoffLimit limit of retries.Note:If the Job failed, either by the Pod failure policy or Pod backoff failure policy, and the Job is running multiple Pods, Kubernetes terminates all the Pods in that Job that are still Pending or Running.These are some requirements and semantics of the API:if you want to use a .spec.podFailurePolicy field for a Job, you must also define that Job's pod template with .spec.restartPolicy set to Never.the Pod failure policy rules you specify under spec.podFailurePolicy.rules are evaluated in order. Once a rule matches a Pod failure, the remaining rules are ignored. When no rule matches the Pod failure, the default handling applies.you may want to restrict a rule to a specific container by specifying its name inspec.podFailurePolicy.rules[*].onExitCodes.containerName. When not specified the rule applies to all containers. When specified, it should match one the container or initContainer names in the Pod template.you may specify the action taken when a Pod failure policy is matched by spec.podFailurePolicy.rules[*].action. Possible values are:FailJob: use to indicate that the Pod's job should be marked as Failed and all running Pods should be terminated.Ignore: use to indicate that the counter towards the .spec.backoffLimit should not be incremented and a replacement Pod should be created.Count: use to indicate that the Pod should be handled in the default way. The counter towards the .spec.backoffLimit should be incremented.FailIndex: use this action along with backoff limit per index to avoid unnecessary retries within the index of a failed pod.Note:When you use a podFailurePolicy, the job controller only matches Pods in the Failed phase. Pods with a deletion timestamp that are not in a terminal phase (Failed or Succeeded) are considered still terminating. This implies that terminating pods retain a tracking finalizer until they reach a terminal phase. Since Kubernetes 1.27, Kubelet transitions deleted pods to a terminal phase (see: Pod Phase). This ensures that deleted pods have their finalizers removed by the Job controller.Note:Starting with Kubernetes v1.28, when Pod failure policy is used, the Job controller recreates terminating Pods only once these Pods reach the terminal Failed phase. This behavior is similar to podReplacementPolicy: Failed. For more information, see Pod replacement policy.When you use the podFailurePolicy, and the Job fails due to the pod matching the rule with the FailJob action, then the Job controller triggers the Job termination process by adding the FailureTarget condition. For more details, see Job termination and cleanup.Success policyWhen creating an Indexed Job, you can define when a Job can be declared as succeeded using a .spec.successPolicy, based on the pods that succeeded.By default, a Job succeeds when the number of succeeded Pods equals .spec.completions. These are some situations where you might want additional control for declaring a Job succeeded:When running simulations with different parameters, you might not need all the simulations to succeed for the overall Job to be successful.When following a leader-worker pattern, only the success of the leader determines the success or failure of a Job. Examples of this are frameworks like MPI and PyTorch etc.You can configure a success policy, in the .spec.successPolicy field, to meet the above use cases. This policy can handle Job success based on the succeeded pods. After the Job meets the success policy, the job controller terminates the lingering Pods. A success policy is defined by rules. Each rule can take one of the following forms:When you specify the succeededIndexes only, once all indexes specified in the succeededIndexes succeed, the job controller marks the Job as succeeded. The succeededIndexes must be a list of intervals between 0 and .spec.completions-1.When you specify the succeededCount only, once the number of succeeded indexes reaches the succeededCount, the job controller marks the Job as succeeded.When you specify both succeededIndexes and succeededCount, once the number of succeeded indexes from the subset of indexes specified in the succeededIndexes reaches the succeededCount, the job controller marks the Job as succeeded.Note that when you specify multiple rules in the .spec.successPolicy.rules, the job controller evaluates the rules in order. Once the Job meets a rule, the job controller ignores remaining rules.Here is a manifest for a Job with successPolicy:/controllers/job-success-policy.yaml apiVersion: batch/v1 kind: Job metadata: name: job-success spec: parallelism: 10 completions: 10 completionMode: Indexed # Required for the success policy successPolicy: rules: - succeededIndexes: 0,2-3 succeededCount: 1 template: spec: containers: - name: main image: python command: # Provided that at least one of the Pods with 0, 2, and 3 indexes has succeeded, # the overall Job is a success. - python3 - -c - | import os, sys if os.environ.get("JOB_COMPLETION_INDEX") == "2": sys.exit(0) else: sys.exit(1) restartPolicy: Never In the example above, both succeededIndexes and succeededCount have been specified. Therefore, the job controller will mark the Job as succeeded and terminate the lingering Pods when either of the specified indexes, 0, 2, or 3, succeed. The Job that meets the success policy gets the SuccessCriteriaMet condition with a SuccessPolicy reason. After the removal of the lingering Pods is issued, the Job gets the Complete condition.Note that the succeededIndexes is represented as intervals separated by a hyphen. The number are listed in represented by the first and last element of the series, separated by a hyphen.Note:When you specify both a success policy and some terminating policies such as .spec.backoffLimit and .spec.podFailurePolicy, once the Job meets either policy, the job controller respects the terminating policy and ignores the success policy.Job termination and cleanupWhen a Job completes, no more Pods are created, but the Pods are usually not deleted either. Keeping them around allows you to still view the logs of completed pods to check for errors, warnings, or other diagnostic output. The job object also remains after it is completed so that you can view its status. It is up to the user to delete old jobs after noting their status. Delete the job with kubectl (e.g. kubectl delete jobs/pi or kubectl delete -f ./job.yaml). When you delete the job using kubectl, all the pods it created are deleted too.By default, a Job will run uninterrupted unless a Pod fails (restartPolicy=Never) or a Container exits in error (restartPolicy=OnFailure), at which point the Job defers to the .spec.backoffLimit described above. Once .spec.backoffLimit has been reached the Job will be marked as failed and any running Pods will be terminated.Another way to terminate a Job is by setting an active deadline. Do this by setting the .spec.activeDeadlineSeconds field of the Job to a number of seconds. The activeDeadlineSeconds applies to the duration of the job, no matter how many Pods are created. Once a Job reaches activeDeadlineSeconds, all of its running Pods are terminated and the Job status will become type: Failed with reason: DeadlineExceeded.Note that a Job's .spec.activeDeadlineSeconds takes precedence over its .spec.backoffLimit. Therefore, a Job that is retrying one or more failed Pods will not deploy additional Pods once it reaches the time limit specified by activeDeadlineSeconds, even if the backoffLimit is not yet reached.Example:apiVersion: batch/v1 kind: Job metadata: name: pi-with-timeout spec: backoffLimit: 5 activeDeadlineSeconds: 100 template: spec: containers: - name: pi image: perl:5.34.0 command: ["perl", "-Mbignum=bpi", "-wle", "print bpi(2000)"] restartPolicy: Never Note that both the Job spec and the Pod template spec within the Job have an activeDeadlineSeconds field. Ensure that you set this field at the proper level.Keep in mind that the restartPolicy applies to the Pod, and not to the Job itself: there is no automatic Job restart once the Job status is type: Failed. That is, the Job termination mechanisms activated with .spec.activeDeadlineSeconds and .spec.backoffLimit result in a permanent Job failure that requires manual intervention to resolve.Terminal Job conditionsA Job has two possible terminal states, each of which has a corresponding Job condition:Succeeded: Job condition CompleteFailed: Job condition FailedJobs fail for the following reasons:The number of Pod failures exceeded the specified .spec.backoffLimit in the Job specification. For details, see Pod backoff failure policy.The Job runtime exceeded the specified .spec.activeDeadlineSecondsAn indexed Job that used .spec.backoffLimitPerIndex has failed indexes. For details, see Backoff limit per index.The number of failed indexes in the Job exceeded the specified spec.maxFailedIndexes. For details, see Backoff limit per indexA failed Pod matches a rule in .spec.podFailurePolicy that has the FailJob action. For details about how Pod failure policy rules might affect failure evaluation, see Pod failure policy.Jobs succeed for the following reasons:The number of succeeded Pods reached the specified .spec.completionsThe criteria specified in .spec.successPolicy are met. For details, see Success policy.In Kubernetes v1.31 and later the Job controller delays the addition of the terminal conditions,Failed or Complete, until all of the Job Pods are terminated.In Kubernetes v1.30 and earlier, the Job controller added the Complete or the Failed Job terminal conditions as soon as the Job termination process was triggered and all Pod finalizers were removed. However, some Pods would still be running or terminating at the moment that the terminal condition was added.In Kubernetes v1.31 and later, the controller only adds the Job terminal conditions after all of the Pods are terminated. You can control this behavior by using the JobManagedBy and the JobPodReplacementPolicy (both enabled by default) feature gates.Termination of Job podsThe Job controller adds the FailureTarget condition or the SuccessCriteriaMet condition to the Job to trigger Pod termination after a Job meets either the success or failure criteria.Factors like terminationGracePeriodSeconds might increase the amount of time from the moment that the Job controller adds the FailureTarget condition or the SuccessCriteriaMet condition to the moment that all of the Job Pods terminate and the Job controller adds a terminal condition (Failed or Complete).You can use the FailureTarget or the SuccessCriteriaMet condition to evaluate whether the Job has failed or succeeded without having to wait for the controller to add a terminal condition.For example, you might want to decide when to create a replacement Job that replaces a failed Job. If you replace the failed Job when the FailureTarget condition appears, your replacement Job runs sooner, but could result in Pods from the failed and the replacement Job running at the same time, using extra compute resources.Alternatively, if your cluster has limited resource capacity, you could choose to wait until the Failed condition appears on the Job, which would delay your replacement Job but would ensure that you conserve resources by waiting until all of the failed Pods are removed.Clean up finished jobs automaticallyFinished Jobs are usually no longer needed in the system. Keeping them around in the system will put pressure on the API server. If the Jobs are managed directly by a higher level controller, such as CronJobs, the Jobs can be cleaned up by CronJobs based on the specified capacity-based cleanup policy.TTL mechanism for finished JobsFEATURE STATE: Kubernetes v1.23 [stable]Another way to clean up finished Jobs (either Complete or Failed) automatically is to use a TTL mechanism provided by a TTL controller for finished resources, by specifying the .spec.ttlSecondsAfterFinished field of the Job.When the TTL controller cleans up the Job, it will delete the Job cascadingly, i.e. delete its dependent objects, such as Pods, together with the Job. Note that when the Job is deleted, its lifecycle guarantees, such as finalizers, will be honored.For example:apiVersion: batch/v1 kind: Job metadata: name: pi-with-ttl spec: ttlSecondsAfterFinished: 100 template: spec: containers: - name: pi image: perl:5.34.0 command: ["perl", "-Mbignum=bpi", "-wle", "print bpi(2000)"] restartPolicy: Never The Job pi-with-ttl will be eligible to be automatically deleted, 100 seconds after it finishes.If the field is set to 0, the Job will be eligible to be automatically deleted immediately after it finishes. If the field is unset, this Job won't be cleaned up by the TTL controller after it finishes.Note:It is recommended to set ttlSecondsAfterFinished field because unmanaged jobs (Jobs that you created directly, and not indirectly through other workload APIs such as CronJob) have a default deletion policy of orphanDependents causing Pods created by an unmanaged Job to be left around after that Job is fully deleted. Even though the control plane eventually garbage collects the Pods from a deleted Job after they either fail or complete, sometimes those lingering pods may cause cluster performance degradation or in worst case cause the cluster to go offline due to this degradation.You can use LimitRanges and ResourceQuotas to place a cap on the amount of resources that a particular namespace can consume.Job patternsThe Job object can be used to process a set of independent but related work items. These might be emails to be sent, frames to be rendered, files to be transcoded, ranges of keys in a NoSQL database to scan, and so on.In a complex system, there may be multiple different sets of work items. Here we are just considering one set of work items that the user wants to manage together — a batch job.There are several different patterns for parallel computation, each with strengths and weaknesses. The tradeoffs are:One Job object for each work item, versus a single Job object for all work items. One Job per work item creates some overhead for the user and for the system to manage large numbers of Job objects. A single Job for all work items is better for large numbers of items.Number of Pods created equals number of work items, versus each Pod can process multiple work items. When the number of Pods equals the number of work items, the Pods typically requires less modification to existing code and containers. Having each Pod process multiple work items is better for large numbers of items.Several approaches use a work queue. This requires running a queue service, and modifications to the existing program or container to make it use the work queue. Other approaches are easier to adapt to an existing containerised application.When the Job is associated with a headless Service, you can enable the Pods within a Job to communicate with each other to collaborate in a computation.The tradeoffs are summarized here, with columns 2 to 4 corresponding to the above tradeoffs. The pattern names are also links to examples and more detailed description.PatternSingle Job objectFewer pods than work items?Use app unmodified?Queue with Pod Per Work Item✓sometimesQueue with Variable Pod Count✓✓Indexed Job with Static Work Assignment✓✓Job with Pod-to-Pod Communication✓sometimessometimesJob Template Expansion✓When you specify completions with .spec.completions, each Pod created by the Job controller has an identical spec. This means that all pods for a task will have the same command line and the same image, the same volumes, and (almost) the same environment variables. These patterns are different ways to arrange for pods to work on different things.This table shows the required settings for .spec.parallelism and .spec.completions for each of the patterns. Here, W is the number of work items.Pattern.spec.completions.spec.parallelismQueue with Pod Per Work ItemWanyQueue with Variable Pod CountnullanyIndexed Job with Static Work AssignmentWanyJob with Pod-to-Pod CommunicationWWJob Template Expansion1should be 1Advanced usageSuspending a JobFEATURE STATE: Kubernetes v1.24 [stable]When a Job is created, the Job controller will immediately begin creating Pods to satisfy the Job's requirements and will continue to do so until the Job is complete. However, you may want to temporarily suspend a Job's execution and resume it later, or start Jobs in suspended state and have a custom controller decide later when to start them.To suspend a Job, you can update the .spec.suspend field of the Job to true; later, when you want to resume it again, update it to false. Creating a Job with .spec.suspend set to true will create it in the suspended state.When a Job is resumed from suspension, its .status.startTime field will be reset to the current time. This means that the .spec.activeDeadlineSeconds timer will be stopped and reset when a Job is suspended and resumed.When you suspend a Job, any running Pods that don't have a status of Completed will be terminated with a SIGTERM signal. The Pod's graceful termination period will be honored and your Pod must handle this signal in this period. This may involve saving progress for later or undoing changes. Pods terminated this way will not count towards the Job's completions count.An example Job definition in the suspended state can be like so:kubectl get job myjob -o yaml apiVersion: batch/v1 kind: Job metadata: name: myjob spec: suspend: true parallelism: 1 completions: 5 template: spec: ... You can also toggle Job suspension by patching the Job using the command line.Suspend an active Job:kubectl patch job/myjob --type=strategic --patch '{"spec":{"suspend":true}}' Resume a suspended Job:kubectl patch job/myjob --type=strategic --patch '{"spec":{"suspend":false}}' The Job's status can be used to determine if a Job is suspended or has been suspended in the past:kubectl get jobs/myjob -o yaml apiVersion: batch/v1 kind: Job # .metadata and .spec omitted status: conditions: - lastProbeTime: "2021-02-05T13:14:33Z" lastTransitionTime: "2021-02-05T13:14:33Z" status: "True" type: Suspended startTime: "2021-02-05T13:13:48Z" The Job condition of type "Suspended" with status "True" means the Job is suspended; the lastTransitionTime field can be used to determine how long the Job has been suspended for. If the status of that condition is "False", then the Job was previously suspended and is now running. If such a condition does not exist in the Job's status, the Job has never been stopped.Events are also created when the Job is suspended and resumed:kubectl describe jobs/myjob Name: myjob ... Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal SuccessfulCreate 12m job-controller Created pod: myjob-hlrpl Normal SuccessfulDelete 11m job-controller Deleted pod: myjob-hlrpl Normal Suspended 11m job-controller Job suspended Normal SuccessfulCreate 3s job-controller Created pod: myjob-jvb44 Normal Resumed 3s job-controller Job resumed The last four events, particularly the "Suspended" and "Resumed" events, are directly a result of toggling the .spec.suspend field. In the time between these two events, we see that no Pods were created, but Pod creation restarted as soon as the Job was resumed.Mutable Scheduling DirectivesFEATURE STATE: Kubernetes v1.27 [stable]In most cases, a parallel job will want the pods to run with constraints, like all in the same zone, or all either on GPU model x or y but not a mix of both.The suspend field is the first step towards achieving those semantics. Suspend allows a custom queue controller to decide when a job should start; However, once a job is unsuspended, a custom queue controller has no influence on where the pods of a job will actually land.This feature allows updating a Job's scheduling directives before it starts, which gives custom queue controllers the ability to influence pod placement while at the same time offloading actual pod-to-node assignment to kube-scheduler. This is allowed only for suspended Jobs that have never been unsuspended before.The fields in a Job's pod template that can be updated are node affinity, node selector, tolerations, labels, annotations and scheduling gates.Specifying your own Pod selectorNormally, when you create a Job object, you do not specify .spec.selector. The system defaulting logic adds this field when the Job is created. It picks a selector value that will not overlap with any other jobs.However, in some cases, you might need to override this automatically set selector. To do this, you can specify the .spec.selector of the Job.Be very careful when doing this. If you specify a label selector which is not unique to the pods of that Job, and which matches unrelated Pods, then pods of the unrelated job may be deleted, or this Job may count other Pods as completing it, or one or both Jobs may refuse to create Pods or run to completion. If a non-unique selector is chosen, then other controllers (e.g. ReplicationController) and their Pods may behave in unpredictable ways too. Kubernetes will not stop you from making a mistake when specifying .spec.selector.Here is an example of a case when you might want to use this feature.Say Job old is already running. You want existing Pods to keep running, but you want the rest of the Pods it creates to use a different pod template and for the Job to have a new name. You cannot update the Job because these fields are not updatable. Therefore, you delete Job old but leave its pods running, using kubectl delete jobs/old --cascade=orphan. Before deleting it, you make a note of what selector it uses:kubectl get job old -o yaml The output is similar to this:kind: Job metadata: name: old ... spec: selector: matchLabels: batch.kubernetes.io/controller-uid: a8f3d00d-c6d2-11e5-9f87-42010af00002 ... Then you create a new Job with name new and you explicitly specify the same selector. Since the existing Pods have label batch.kubernetes.io/controller-uid=a8f3d00d-c6d2-11e5-9f87-42010af00002, they are controlled by Job new as well.You need to specify manualSelector: true in the new Job since you are not using the selector that the system normally generates for you automatically.kind: Job metadata: name: new ... spec: manualSelector: true selector: matchLabels: batch.kubernetes.io/controller-uid: a8f3d00d-c6d2-11e5-9f87-42010af00002 ... The new Job itself will have a different uid from a8f3d00d-c6d2-11e5-9f87-42010af00002. Setting manualSelector: true tells the system that you know what you are doing and to allow this mismatch.Job tracking with finalizersFEATURE STATE: Kubernetes v1.26 [stable]The control plane keeps track of the Pods that belong to any Job and notices if any such Pod is removed from the API server. To do that, the Job controller creates Pods with the finalizer batch.kubernetes.io/job-tracking. The controller removes the finalizer only after the Pod has been accounted for in the Job status, allowing the Pod to be removed by other controllers or users.Note:See My pod stays terminating if you observe that pods from a Job are stuck with the tracking finalizer.Elastic Indexed JobsFEATURE STATE: Kubernetes v1.31 [stable] (enabled by default: true)You can scale Indexed Jobs up or down by mutating both .spec.parallelism and .spec.completions together such that .spec.parallelism == .spec.completions. When scaling down, Kubernetes removes the Pods with higher indexes.Use cases for elastic Indexed Jobs include batch workloads which require scaling an indexed Job, such as MPI, Horovod, Ray, and PyTorch training jobs.Delayed creation of replacement podsFEATURE STATE: Kubernetes v1.34 [stable] (enabled by default: true)By default, the Job controller recreates Pods as soon they either fail or are terminating (have a deletion timestamp). This means that, at a given time, when some of the Pods are terminating, the number of running Pods for a Job can be greater than parallelism or greater than one Pod per index (if you are using an Indexed Job).You may choose to create replacement Pods only when the terminating Pod is fully terminal (has status.phase: Failed). To do this, set the .spec.podReplacementPolicy: Failed. The default replacement policy depends on whether the Job has a podFailurePolicy set. With no Pod failure policy defined for a Job, omitting the podReplacementPolicy field selects the TerminatingOrFailed replacement policy: the control plane creates replacement Pods immediately upon Pod deletion (as soon as the control plane sees that a Pod for this Job has deletionTimestamp set). For Jobs with a Pod failure policy set, the default podReplacementPolicy is Failed, and no other value is permitted. See Pod failure policy to learn more about Pod failure policies for Jobs.kind: Job metadata: name: new ... spec: podReplacementPolicy: Failed ... Provided your cluster has the feature gate enabled, you can inspect the .status.terminating field of a Job. The value of the field is the number of Pods owned by the Job that are currently terminating.kubectl get jobs/myjob -o yaml apiVersion: batch/v1 kind: Job # .metadata and .spec omitted status: terminating: 3 # three Pods are terminating and have not yet reached the Failed phase Delegation of managing a Job object to external controllerFEATURE STATE: Kubernetes v1.32 [beta] (enabled by default: true)Note:You can only set the managedBy field on Jobs if you enable the JobManagedBy feature gate (enabled by default).This feature allows you to disable the built-in Job controller, for a specific Job, and delegate reconciliation of the Job to an external controller.You indicate the controller that reconciles the Job by setting a custom value for the spec.managedBy field - any value other than kubernetes.io/job-controller. The value of the field is immutable.Note:When using this feature, make sure the controller indicated by the field is installed, otherwise the Job may not be reconciled at all.Note:When developing an external Job controller be aware that your controller needs to operate in a fashion conformant with the definitions of the API spec and status fields of the Job object.Please review these in detail in the Job API. We also recommend that you run the e2e conformance tests for the Job object to verify your implementation.Finally, when developing an external Job controller make sure it does not use the batch.kubernetes.io/job-tracking finalizer, reserved for the built-in controller.Warning:If you are considering to disable the JobManagedBy feature gate, or to downgrade the cluster to a version without the feature gate enabled, check if there are jobs with a custom value of the spec.managedBy field. If there are such jobs, there is a risk that they might be reconciled by two controllers after the operation: the built-in Job controller and the external controller indicated by the field value.AlternativesBare PodsWhen the node that a Pod is running on reboots or fails, the pod is terminated and will not be restarted. However, a Job will create new Pods to replace terminated ones. For this reason, we recommend that you use a Job rather than a bare Pod, even if your application requires only a single Pod.Replication ControllerJobs are complementary to Replication Controllers. A Replication Controller manages Pods which are not expected to terminate (e.g. web servers), and a Job manages Pods that are expected to terminate (e.g. batch tasks).As discussed in Pod Lifecycle, Job is only appropriate for pods with RestartPolicy equal to OnFailure or Never. (Note: If RestartPolicy is not set, the default value is Always.)Single Job starts controller PodAnother pattern is for a single Job to create a Pod which then creates other Pods, acting as a sort of custom controller for those Pods. This allows the most flexibility, but may be somewhat complicated to get started with and offers less integration with Kubernetes.One example of this pattern would be a Job which starts a Pod which runs a script that in turn starts a Spark master controller (see spark example), runs a spark driver, and then cleans up.An advantage of this approach is that the overall process gets the completion guarantee of a Job object, but maintains complete control over what Pods are created and how work is assigned to them.What's nextLearn about Pods.Read about different ways of running Jobs:Coarse Parallel Processing Using a Work QueueFine Parallel Processing Using a Work QueueUse an indexed Job for parallel processing with static work assignmentCreate multiple Jobs based on a template: Parallel Processing using ExpansionsFollow the links within Clean up finished jobs automatically to learn more about how your cluster can clean up completed and / or failed tasks.Job is part of the Kubernetes REST API. Read the Job object definition to understand the API for jobs.Read about CronJob, which you can use to define a series of Jobs that will run based on a schedule, similar to the UNIX tool cron.Practice how to configure handling of retriable and non-retriable pod failures using podFailurePolicy, based on the step-by-step examples.


# kubectl create poddisruptionbudget

kubectl create poddisruptionbudgetSynopsisCreate a pod disruption budget with the specified name, selector, and desired minimum available pods.kubectl create poddisruptionbudget NAME --selector=SELECTOR --min-available=N [--dry-run=server|client|none] Examples # Create a pod disruption budget named my-pdb that will select all pods with the app=rails label # and require at least one of them being available at any point in time kubectl create poddisruptionbudget my-pdb --selector=app=rails --min-available=1 # Create a pod disruption budget named my-pdb that will select all pods with the app=nginx label # and require at least half of the pods selected to be available at any point in time kubectl create pdb my-pdb --selector=app=nginx --min-available=50% Options--allow-missing-template-keys Default: trueIf true, ignore any errors in templates when a field or map key is missing in the template. Only applies to golang and jsonpath output formats.--dry-run string[="unchanged"] Default: "none"Must be "none", "server", or "client". If client strategy, only print the object that would be sent, without sending it. If server strategy, submit server-side request without persisting the resource.--field-manager string Default: "kubectl-create"Name of the manager used to track field ownership.-h, --helphelp for poddisruptionbudget--max-unavailable stringThe maximum number or percentage of unavailable pods this budget requires.--min-available stringThe minimum number or percentage of available pods this budget requires.-o, --output stringOutput format. One of: (json, yaml, name, go-template, go-template-file, template, templatefile, jsonpath, jsonpath-as-json, jsonpath-file).--save-configIf true, the configuration of current object will be saved in its annotation. Otherwise, the annotation will be unchanged. This flag is useful when you want to perform kubectl apply on this object in the future.--selector stringA label selector to use for this budget. Only equality-based selector requirements are supported.--show-managed-fieldsIf true, keep the managedFields when printing objects in JSON or YAML format.--template stringTemplate string or path to template file to use when -o=go-template, -o=go-template-file. The template format is golang templates [http://golang.org/pkg/text/template/#pkg-overview].--validate string[="strict"] Default: "strict"Must be one of: strict (or true), warn, ignore (or false). "true" or "strict" will use a schema to validate the input and fail the request if invalid. It will perform server side validation if ServerSideFieldValidation is enabled on the api-server, but will fall back to less reliable client-side validation if not. "warn" will warn about unknown or duplicate fields without blocking the request if server-side field validation is enabled on the API server, and behave as "ignore" otherwise. "false" or "ignore" will not perform any schema validation, silently dropping any unknown or duplicate fields.Parent Options Inherited--as stringUsername to impersonate for the operation. User could be a regular user or a service account in a namespace.--as-group stringsGroup to impersonate for the operation, this flag can be repeated to specify multiple groups.--as-uid stringUID to impersonate for the operation.--cache-dir string Default: "$HOME/.kube/cache"Default cache directory--certificate-authority stringPath to a cert file for the certificate authority--client-certificate stringPath to a client certificate file for TLS--client-key stringPath to a client key file for TLS--cluster stringThe name of the kubeconfig cluster to use--context stringThe name of the kubeconfig context to use--disable-compressionIf true, opt-out of response compression for all requests to the server--insecure-skip-tls-verifyIf true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure--kubeconfig stringPath to the kubeconfig file to use for CLI requests.--kuberc stringPath to the kuberc file to use for preferences. This can be disabled by exporting KUBECTL_KUBERC=false feature gate or turning off the feature KUBERC=off.--match-server-versionRequire server version to match client version-n, --namespace stringIf present, the namespace scope for this CLI request--password stringPassword for basic authentication to the API server--profile string Default: "none"Name of profile to capture. One of (none|cpu|heap|goroutine|threadcreate|block|mutex)--profile-output string Default: "profile.pprof"Name of the file to write the profile to--request-timeout string Default: "0"The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests.-s, --server stringThe address and port of the Kubernetes API server--storage-driver-buffer-duration duration Default: 1m0sWrites in the storage driver will be buffered for this duration, and committed to the non memory backends as a single transaction--storage-driver-db string Default: "cadvisor"database name--storage-driver-host string Default: "localhost:8086"database host:port--storage-driver-password string Default: "root"database password--storage-driver-secureuse secure connection with database--storage-driver-table string Default: "stats"table name--storage-driver-user string Default: "root"database username--tls-server-name stringServer name to use for server certificate validation. If it is not provided, the hostname used to contact the server is used--token stringBearer token for authentication to the API server--user stringThe name of the kubeconfig user to use--username stringUsername for basic authentication to the API server--version version[=true]--version, --version=raw prints version information and quits; --version=vX.Y.Z... sets the reported version--warnings-as-errorsTreat warnings received from the server as errors and exit with a non-zero exit codeSee Alsokubectl create - Create a resource from a file or from stdinThis page is automatically generated.If you plan to report an issue with this page, mention that the page is auto-generated in your issue description. The fix may need to happen elsewhere in the Kubernetes project.


# Debug Init Containers

Debug Init ContainersThis page shows how to investigate problems related to the execution of Init Containers. The example command lines below refer to the Pod as <pod-name> and the Init Containers as <init-container-1> and <init-container-2>.Before you beginYou need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. It is recommended to run this tutorial on a cluster with at least two nodes that are not acting as control plane hosts. If you do not already have a cluster, you can create one by using minikube or you can use one of these Kubernetes playgrounds:iximiuz LabsKillercodaKodeKloudPlay with KubernetesTo check the version, enter kubectl version.You should be familiar with the basics of Init Containers.You should have Configured an Init Container.Checking the status of Init ContainersDisplay the status of your pod:kubectl get pod <pod-name> For example, a status of Init:1/2 indicates that one of two Init Containers has completed successfully:NAME READY STATUS RESTARTS AGE <pod-name> 0/1 Init:1/2 0 7s See Understanding Pod status for more examples of status values and their meanings.Getting details about Init ContainersView more detailed information about Init Container execution:kubectl describe pod <pod-name> For example, a Pod with two Init Containers might show the following:Init Containers: <init-container-1>: Container ID: ... ... State: Terminated Reason: Completed Exit Code: 0 Started: ... Finished: ... Ready: True Restart Count: 0 ... <init-container-2>: Container ID: ... ... State: Waiting Reason: CrashLoopBackOff Last State: Terminated Reason: Error Exit Code: 1 Started: ... Finished: ... Ready: False Restart Count: 3 ... You can also access the Init Container statuses programmatically by reading the status.initContainerStatuses field on the Pod Spec:kubectl get pod nginx --template '{{.status.initContainerStatuses}}' This command will return the same information as above in raw JSON.Accessing logs from Init ContainersPass the Init Container name along with the Pod name to access its logs.kubectl logs <pod-name> -c <init-container-2> Init Containers that run a shell script print commands as they're executed. For example, you can do this in Bash by running set -x at the beginning of the script.Understanding Pod statusA Pod status beginning with Init: summarizes the status of Init Container execution. The table below describes some example status values that you might see while debugging Init Containers.StatusMeaningInit:N/MThe Pod has M Init Containers, and N have completed so far.Init:ErrorAn Init Container has failed to execute.Init:CrashLoopBackOffAn Init Container has failed repeatedly.PendingThe Pod has not yet begun executing Init Containers.PodInitializing or RunningThe Pod has already finished executing Init Containers.


# Service

ServiceExpose an application running in your cluster behind a single outward-facing endpoint, even when the workload is split across multiple backends.In Kubernetes, a Service is a method for exposing a network application that is running as one or more Pods in your cluster.A key aim of Services in Kubernetes is that you don't need to modify your existing application to use an unfamiliar service discovery mechanism. You can run code in Pods, whether this is a code designed for a cloud-native world, or an older app you've containerized. You use a Service to make that set of Pods available on the network so that clients can interact with it.If you use a Deployment to run your app, that Deployment can create and destroy Pods dynamically. From one moment to the


# kubectl config rename-context

kubectl config rename-contextSynopsisRenames a context from the kubeconfig file.CONTEXT_NAME is the context name that you want to change.NEW_NAME is the new name you want to set.Note: If the context being renamed is the 'current-context', this field will also be updated.kubectl config rename-context CONTEXT_NAME NEW_NAME Examples # Rename the context 'old-name' to 'new-name' in your kubeconfig file kubectl config rename-context old-name new-name Options-h, --helphelp for rename-contextParent Options Inherited--as stringUsername to impersonate for the operation. User could be a regular user or a service account in a namespace.--as-group stringsGroup to impersonate for the operation, this flag can be repeated to specify multiple groups.--as-uid stringUID to impersonate for the operation.--cache-dir string Default: "$HOME/.kube/cache"Default cache directory--certificate-authority stringPath to a cert file for the certificate authority--client-certificate stringPath to a client certificate file for TLS--client-key stringPath to a client key file for TLS--cluster stringThe name of the kubeconfig cluster to use--context stringThe name of the kubeconfig context to use--disable-compressionIf true, opt-out of response compression for all requests to the server--insecure-skip-tls-verifyIf true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure--kubeconfig stringuse a particular kubeconfig file--kuberc stringPath to the kuberc file to use for preferences. This can be disabled by exporting KUBECTL_KUBERC=false feature gate or turning off the feature KUBERC=off.--match-server-versionRequire server version to match client version-n, --namespace stringIf present, the namespace scope for this CLI request--password stringPassword for basic authentication to the API server--profile string Default: "none"Name of profile to capture. One of (none|cpu|heap|goroutine|threadcreate|block|mutex)--profile-output string Default: "profile.pprof"Name of the file to write the profile to--request-timeout string Default: "0"The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests.-s, --server stringThe address and port of the Kubernetes API server--storage-driver-buffer-duration duration Default: 1m0sWrites in the storage driver will be buffered for this duration, and committed to the non memory backends as a single transaction--storage-driver-db string Default: "cadvisor"database name--storage-driver-host string Default: "localhost:8086"database host:port--storage-driver-password string Default: "root"database password--storage-driver-secureuse secure connection with database--storage-driver-table string Default: "stats"table name--storage-driver-user string Default: "root"database username--tls-server-name stringServer name to use for server certificate validation. If it is not provided, the hostname used to contact the server is used--token stringBearer token for authentication to the API server--user stringThe name of the kubeconfig user to use--username stringUsername for basic authentication to the API server--version version[=true]--version, --version=raw prints version information and quits; --version=vX.Y.Z... sets the reported version--warnings-as-errorsTreat warnings received from the server as errors and exit with a non-zero exit codeSee Alsokubectl config - Modify kubeconfig filesThis page is automatically generated.If you plan to report an issue with this page, mention that the page is auto-generated in your issue description. The fix may need to happen elsewhere in the Kubernetes project.