# ValidatingAdmissionPolicy

ValidatingAdmissionPolicyValidatingAdmissionPolicy describes the definition of an admission validation policy that accepts or rejects an object without changing it.apiVersion: admissionregistration.k8s.io/v1import "k8s.io/api/admissionregistration/v1"ValidatingAdmissionPolicyValidatingAdmissionPolicy describes the definition of an admission validation policy that accepts or rejects an object without changing it.apiVersion: admissionregistration.k8s.io/v1kind: ValidatingAdmissionPolicymetadata (ObjectMeta)Standard object metadata; More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata.spec (ValidatingAdmissionPolicySpec)Specification of the desired behavior of the ValidatingAdmissionPolicy.ValidatingAdmissionPolicySpec is the specification of the desired behavior of the AdmissionPolicy.spec.auditAnnotations ([]AuditAnnotation)Atomic: will be replaced during a mergeauditAnnotations contains CEL expressions which are used to produce audit annotations for the audit event of the API request. validations and auditAnnotations may not both be empty; a least one of validations or auditAnnotations is required.AuditAnnotation describes how to produce an audit annotation for an API request.spec.auditAnnotations.key (string), requiredkey specifies the audit annotation key. The audit annotation keys of a ValidatingAdmissionPolicy must be unique. The key must be a qualified name ([A-Za-z0-9][-A-Za-z0-9_.]*) no more than 63 bytes in length.The key is combined with the resource name of the ValidatingAdmissionPolicy to construct an audit annotation key: "{ValidatingAdmissionPolicy name}/{key}".If an admission webhook uses the same resource name as this ValidatingAdmissionPolicy and the same audit annotation key, the annotation key will be identical. In this case, the first annotation written with the key will be included in the audit event and all subsequent annotations with the same key will be discarded.Required.spec.auditAnnotations.valueExpression (string), requiredvalueExpression represents the expression which is evaluated by CEL to produce an audit annotation value. The expression must evaluate to either a string or null value. If the expression evaluates to a string, the audit annotation is included with the string value. If the expression evaluates to null or empty string the audit annotation will be omitted. The valueExpression may be no longer than 5kb in length. If the result of the valueExpression is more than 10kb in length, it will be truncated to 10kb.If multiple ValidatingAdmissionPolicyBinding resources match an API request, then the valueExpression will be evaluated for each binding. All unique values produced by the valueExpressions will be joined together in a comma-separated list.Required.spec.failurePolicy (string)failurePolicy defines how to handle failures for the admission policy. Failures can occur from CEL expression parse errors, type check errors, runtime errors and invalid or mis-configured policy definitions or bindings.A policy is invalid if spec.paramKind refers to a non-existent Kind. A binding is invalid if spec.paramRef.name refers to a non-existent resource.failurePolicy does not define how validations that evaluate to false are handled.When failurePolicy is set to Fail, ValidatingAdmissionPolicyBinding validationActions define how failures are enforced.Allowed values are Ignore or Fail. Defaults to Fail.spec.matchConditions ([]MatchCondition)Patch strategy: merge on key nameMap: unique values on key name will be kept during a mergeMatchConditions is a list of conditions that must be met for a request to be validated. Match conditions filter requests that have already been matched by the rules, namespaceSelector, and objectSelector. An empty list of matchConditions matches all requests. There are a maximum of 64 match conditions allowed.If a parameter object is provided, it can be accessed via the params handle in the same manner as validation expressions.The exact matching logic is (in order):If ANY matchCondition evaluates to FALSE, the policy is skipped.If ALL matchConditions evaluate to TRUE, the policy is evaluated.If any matchCondition evaluates to an error (but none are FALSE):If failurePolicy=Fail, reject the requestIf failurePolicy=Ignore, the policy is skippedMatchCondition represents a condition which must by fulfilled for a request to be sent to a webhook.spec.matchConditions.expression (string), requiredExpression represents the expression which will be evaluated by CEL. Must evaluate to bool. CEL expressions have access to the contents of the AdmissionRequest and Authorizer, organized into CEL variables:'object' - The object from the incoming request. The value is null for DELETE requests. 'oldObject' - The existing object. The value is null for CREATE requests. 'request' - Attributes of the admission request(/pkg/apis/admission/types.go#AdmissionRequest). 'authorizer' - A CEL Authorizer. May be used to perform authorization checks for the principal (user or service account) of the request. See https://pkg.go.dev/k8s.io/apiserver/pkg/cel/library#Authz 'authorizer.requestResource' - A CEL ResourceCheck constructed from the 'authorizer' and configured with the request resource. Documentation on CEL: https://kubernetes.io/docs/reference/using-api/cel/Required.spec.matchConditions.name (string), requiredName is an identifier for this match condition, used for strategic merging of MatchConditions, as well as providing an identifier for logging purposes. A good name should be descriptive of the associated expression. Name must be a qualified name consisting of alphanumeric characters, '-', '' or '.', and must start and end with an alphanumeric character (e.g. 'MyName', or 'my.name', or '123-abc', regex used for validation is '([A-Za-z0-9][-A-Za-z0-9.]*)?[A-Za-z0-9]') with an optional DNS subdomain prefix and '/' (e.g. 'example.com/MyName')Required.spec.matchConstraints (MatchResources)MatchConstraints specifies what resources this policy is designed to validate. The AdmissionPolicy cares about a request if it matches all Constraints. However, in order to prevent clusters from being put into an unstable state that cannot be recovered from via the API ValidatingAdmissionPolicy cannot match ValidatingAdmissionPolicy and ValidatingAdmissionPolicyBinding. Required.MatchResources decides whether to run the admission control policy on an object based on whether it meets the match criteria. The exclude rules take precedence over include rules (if a resource matches both, it is excluded)spec.matchConstraints.excludeResourceRules ([]NamedRuleWithOperations)Atomic: will be replaced during a mergeExcludeResourceRules describes what operations on what resources/subresources the ValidatingAdmissionPolicy should not care about. The exclude rules take precedence over include rules (if a resource matches both, it is excluded)NamedRuleWithOperations is a tuple of Operations and Resources with ResourceNames.spec.matchConstraints.excludeResourceRules.apiGroups ([]string)Atomic: will be replaced during a mergeAPIGroups is the API groups the resources belong to. '' is all groups. If '' is present, the length of the slice must be one. Required.spec.matchConstraints.excludeResourceRules.apiVersions ([]string)Atomic: will be replaced during a mergeAPIVersions is the API versions the resources belong to. '' is all versions. If '' is present, the length of the slice must be one. Required.spec.matchConstraints.excludeResourceRules.operations ([]string)Atomic: will be replaced during a mergeOperations is the operations the admission hook cares about - CREATE, UPDATE, DELETE, CONNECT or * for all of those operations and any future admission operations that are added. If '*' is present, the length of the slice must be one. Required.spec.matchConstraints.excludeResourceRules.resourceNames ([]string)Atomic: will be replaced during a mergeResourceNames is an optional white list of names that the rule applies to. An empty set means that everything is allowed.spec.matchConstraints.excludeResourceRules.resources ([]string)Atomic: will be replaced during a mergeResources is a list of resources this rule applies to.For example: 'pods' means pods. 'pods/log' means the log subresource of pods. '' means all resources, but not subresources. 'pods/' means all subresources of pods. '/scale' means all scale subresources. '/*' means all resources and their subresources.If wildcard is present, the validation rule will ensure resources do not overlap with each other.Depending on the enclosing object, subresources might not be allowed. Required.spec.matchConstraints.excludeResourceRules.scope (string)scope specifies the scope of this rule. Valid values are "Cluster", "Namespaced", and "" "Cluster" means that only cluster-scoped resources will match this rule. Namespace API objects are cluster-scoped. "Namespaced" means that only namespaced resources will match this rule. "" means that there are no scope restrictions. Subresources match the scope of their parent resource. Default is "*".spec.matchConstraints.matchPolicy (string)matchPolicy defines how the "MatchResources" list is used to match incoming requests. Allowed values are "Exact" or "Equivalent".Exact: match a request only if it exactly matches a specified rule. For example, if deployments can be modified via apps/v1, apps/v1beta1, and extensions/v1beta1, but "rules" only included apiGroups:["apps"], apiVersions:["v1"], resources: ["deployments"], a request to apps/v1beta1 or extensions/v1beta1 would not be sent to the ValidatingAdmissionPolicy.Equivalent: match a request if modifies a resource listed in rules, even via another API group or version. For example, if deployments can be modified via apps/v1, apps/v1beta1, and extensions/v1beta1, and "rules" only included apiGroups:["apps"], apiVersions:["v1"], resources: ["deployments"], a request to apps/v1beta1 or extensions/v1beta1 would be converted to apps/v1 and sent to the ValidatingAdmissionPolicy.Defaults to "Equivalent"spec.matchConstraints.namespaceSelector (LabelSelector)NamespaceSelector decides whether to run the admission control policy on an object based on whether the namespace for that object matches the selector. If the object itself is a namespace, the matching is performed on object.metadata.labels. If the object is another cluster scoped resource, it never skips the policy.For example, to run the webhook on any objects whose namespace is not associated with "runlevel" of "0" or "1"; you will set the selector as follows: "namespaceSelector": { "matchExpressions": [ { "key": "runlevel", "operator": "NotIn", "values": [ "0", "1" ] } ] }If instead you want to only run the policy on any objects whose namespace is associated with the "environment" of "prod" or "staging"; you will set the selector as follows: "namespaceSelector": { "matchExpressions": [ { "key": "environment", "operator": "In", "values": [ "prod", "staging" ] } ] }See https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/ for more examples of label selectors.Default to the empty LabelSelector, which matches everything.spec.matchConstraints.objectSelector (LabelSelector)ObjectSelector decides whether to run the validation based on if the object has matching labels. objectSelector is evaluated against both the oldObject and newObject that would be sent to the cel validation, and is considered to match if either object matches the selector. A null object (oldObject in the case of create, or newObject in the case of delete) or an object that cannot have labels (like a DeploymentRollback or a PodProxyOptions object) is not considered to match. Use the object selector only if the webhook is opt-in, because end users may skip the admission webhook by setting the labels. Default to the empty LabelSelector, which matches everything.spec.matchConstraints.resourceRules ([]NamedRuleWithOperations)Atomic: will be replaced during a mergeResourceRules describes what operations on what resources/subresources the ValidatingAdmissionPolicy matches. The policy cares about an operation if it matches any Rule.NamedRuleWithOperations is a tuple of Operations and Resources with ResourceNames.spec.matchConstraints.resourceRules.apiGroups ([]string)Atomic: will be replaced during a mergeAPIGroups is the API groups the resources belong to. '' is all groups. If '' is present, the length of the slice must be one. Required.spec.matchConstraints.resourceRules.apiVersions ([]string)Atomic: will be replaced during a mergeAPIVersions is the API versions the resources belong to. '' is all versions. If '' is present, the length of the slice must be one. Required.spec.matchConstraints.resourceRules.operations ([]string)Atomic: will be replaced during a mergeOperations is the operations the admission hook cares about - CREATE, UPDATE, DELETE, CONNECT or * for all of those operations and any future admission operations that are added. If '*' is present, the length of the slice must be one. Required.spec.matchConstraints.resourceRules.resourceNames ([]string)Atomic: will be replaced during a mergeResourceNames is an optional white list of names that the rule applies to. An empty set means that everything is allowed.spec.matchConstraints.resourceRules.resources ([]string)Atomic: will be replaced during a mergeResources is a list of resources this rule applies to.For example: 'pods' means pods. 'pods/log' means the log subresource of pods. '' means all resources, but not subresources. 'pods/' means all subresources of pods. '/scale' means all scale subresources. '/*' means all resources and their subresources.If wildcard is present, the validation rule will ensure resources do not overlap with each other.Depending on the enclosing object, subresources might not be allowed. Required.spec.matchConstraints.resourceRules.scope (string)scope specifies the scope of this rule. Valid values are "Cluster", "Namespaced", and "" "Cluster" means that only cluster-scoped resources will match this rule. Namespace API objects are cluster-scoped. "Namespaced" means that only namespaced resources will match this rule. "" means that there are no scope restrictions. Subresources match the scope of their parent resource. Default is "*".spec.paramKind (ParamKind)ParamKind specifies the kind of resources used to parameterize this policy. If absent, there are no parameters for this policy and the param CEL variable will not be provided to validation expressions. If ParamKind refers to a non-existent kind, this policy definition is mis-configured and the FailurePolicy is applied. If paramKind is specified but paramRef is unset in ValidatingAdmissionPolicyBinding, the params variable will be null.ParamKind is a tuple of Group Kind and Version.spec.paramKind.apiVersion (string)APIVersion is the API group version the resources belong to. In format of "group/version". Required.spec.paramKind.kind (string)Kind is the API kind the resources belong to. Required.spec.validations ([]Validation)Atomic: will be replaced during a mergeValidations contain CEL expressions which is used to apply the validation. Validations and AuditAnnotations may not both be empty; a minimum of one Validations or AuditAnnotations is required.Validation specifies the CEL expression which is used to apply the validation.spec.validations.expression (string), requiredExpression represents the expression which will be evaluated by CEL. ref: https://github.com/google/cel-spec CEL expressions have access to the contents of the API request/response, organized into CEL variables as well as some other useful variables:'object' - The object from the incoming request. The value is null for DELETE requests. - 'oldObject' - The existing object. The value is null for CREATE requests. - 'request' - Attributes of the API request(ref). - 'params' - Parameter resource referred to by the policy binding being evaluated. Only populated if the policy has a ParamKind. - 'namespaceObject' - The namespace object that the incoming object belongs to. The value is null for cluster-scoped resources. - 'variables' - Map of composited variables, from its name to its lazily evaluated value. For example, a variable named 'foo' can be accessed as 'variables.foo'.'authorizer' - A CEL Authorizer. May be used to perform authorization checks for the principal (user or service account) of the request. See https://pkg.go.dev/k8s.io/apiserver/pkg/cel/library#Authz'authorizer.requestResource' - A CEL ResourceCheck constructed from the 'authorizer' and configured with the request resource.The apiVersion, kind, metadata.name and metadata.generateName are always accessible from the root of the object. No other metadata properties are accessible.Only property names of the form [a-zA-Z_.-/][a-zA-Z0-9_.-/]* are accessible. Accessible property names are escaped according to the following rules when accessed in the expression: - '' escapes to 'underscores' - '.' escapes to 'dot' - '-' escapes to 'dash' - '/' escapes to 'slash' - Property names that exactly match a CEL RESERVED keyword escape to '{keyword}__'. The keywords are: "true", "false", "null", "in", "as", "break", "const", "continue", "else", "for", "function", "if", "import", "let", "loop", "package", "namespace", "return". Examples:Expression accessing a property named "namespace": {"Expression": "object.namespace > 0"}Expression accessing a property named "x-prop": {"Expression": "object.x__dash__prop > 0"}Expression accessing a property named "redact__d": {"Expression": "object.redact__underscores__d > 0"}Equality on arrays with list type of 'set' or 'map' ignores element order, i.e. [1, 2] == [2, 1]. Concatenation on arrays with x-kubernetes-list-type use the semantics of the list type:'set': X + Y performs a union where the array positions of all elements in X are preserved and non-intersecting elements in Y are appended, retaining their partial order.'map': X + Y performs a merge where the array positions of all keys in X are preserved but the values are overwritten by values in Y when the key sets of X and Y intersect. Elements in Y with non-intersecting keys are appended, retaining their partial order. Required.spec.validations.message (string)Message represents the message displayed when validation fails. The message is required if the Expression contains line breaks. The message must not contain line breaks. If unset, the message is "failed rule: {Rule}". e.g. "must be a URL with the host matching spec.host" If the Expression contains line breaks. Message is required. The message must not contain line breaks. If unset, the message is "failed Expression: {Expression}".spec.validations.messageExpression (string)messageExpression declares a CEL expression that evaluates to the validation failure message that is returned when this rule fails. Since messageExpression is used as a failure message, it must evaluate to a string. If both message and messageExpression are present on a validation, then messageExpression will be used if validation fails. If messageExpression results in a runtime error, the runtime error is logged, and the validation failure message is produced as if the messageExpression field were unset. If messageExpression evaluates to an empty string, a string with only spaces, or a string that contains line breaks, then the validation failure message will also be produced as if the messageExpression field were unset, and the fact that messageExpression produced an empty string/string with only spaces/string with line breaks will be logged. messageExpression has access to all the same variables as the expression except for 'authorizer' and 'authorizer.requestResource'. Example: "object.x must be less than max ("+string(params.max)+")"spec.validations.reason (string)Reason represents a machine-readable description of why this validation failed. If this is the first validation in the list to fail, this reason, as well as the corresponding HTTP response code, are used in the HTTP response to the client. The currently supported reasons are: "Unauthorized", "Forbidden", "Invalid", "RequestEntityTooLarge". If not set, StatusReasonInvalid is used in the response to the client.spec.variables ([]Variable)Patch strategy: merge on key nameMap: unique values on key name will be kept during a mergeVariables contain definitions of variables that can be used in composition of other expressions. Each variable is defined as a named CEL expression. The variables defined here will be available under variables in other expressions of the policy except MatchConditions because MatchConditions are evaluated before the rest of the policy.The expression of a variable can refer to other variables defined earlier in the list but not those after. Thus, Variables must be sorted by the order of first appearance and acyclic.Variable is the definition of a variable that is used for composition. A variable is defined as a named expression.spec.variables.expression (string), requiredExpression is the expression that will be evaluated as the value of the variable. The CEL expression has access to the same identifiers as the CEL expressions in Validation.spec.variables.name (string), requiredName is the name of the variable. The name must be a valid CEL identifier and unique among all variables. The variable can be accessed in other expressions through variables For example, if name is "foo", the variable will be available as variables.foostatus (ValidatingAdmissionPolicyStatus)The status of the ValidatingAdmissionPolicy, including warnings that are useful to determine if the policy behaves in the expected way. Populated by the system. Read-only.ValidatingAdmissionPolicyStatus represents the status of an admission validation policy.status.conditions ([]Condition)Map: unique values on key type will be kept during a mergeThe conditions represent the latest available observations of a policy's current state.Condition contains details for one aspect of the current state of this API Resource.status.conditions.lastTransitionTime (Time), requiredlastTransitionTime is the last time the condition transitioned from one status to another. This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable.Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.status.conditions.message (string), requiredmessage is a human readable message indicating details about the transition. This may be an empty string.status.conditions.reason (string), requiredreason contains a programmatic identifier indicating the reason for the condition's last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. The value should be a CamelCase string. This field may not be empty.status.conditions.status (string), requiredstatus of the condition, one of True, False, Unknown.status.conditions.type (string), requiredtype of condition in CamelCase or in foo.example.com/CamelCase.status.conditions.observedGeneration (int64)observedGeneration represents the .metadata.generation that the condition was set based upon. For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date with respect to the current state of the instance.status.observedGeneration (int64)The generation observed by the controller.status.typeChecking (TypeChecking)The results of type checking for each expression. Presence of this field indicates the completion of the type checking.TypeChecking contains results of type checking the expressions in the ValidatingAdmissionPolicystatus.typeChecking.expressionWarnings ([]ExpressionWarning)Atomic: will be replaced during a mergeThe type checking warnings for each expression.ExpressionWarning is a warning information that targets a specific expression.status.typeChecking.expressionWarnings.fieldRef (string), requiredThe path to the field that refers the expression. For example, the reference to the expression of the first item of validations is "spec.validations[0].expression"status.typeChecking.expressionWarnings.warning (string), requiredThe content of type checking information in a human-readable form. Each line of the warning contains the type that the expression is checked against, followed by the type check error from the compiler.ValidatingAdmissionPolicyListValidatingAdmissionPolicyList is a list of ValidatingAdmissionPolicy.items ([]ValidatingAdmissionPolicy), requiredList of ValidatingAdmissionPolicy.apiVersion (string)APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resourceskind (string)Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kindsmetadata (ListMeta)Standard list metadata. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kindsValidatingAdmissionPolicyBindingValidatingAdmissionPolicyBinding binds the ValidatingAdmissionPolicy with paramerized resources. ValidatingAdmissionPolicyBinding and parameter CRDs together define how cluster administrators configure policies for clusters.For a given admission request, each binding will cause its policy to be evaluated N times, where N is 1 for policies/bindings that don't use params, otherwise N is the number of parameters selected by the binding.The CEL expressions of a policy must have a computed CEL cost below the maximum CEL budget. Each evaluation of the policy is given an independent CEL cost budget. Adding/removing policies, bindings, or params can not affect whether a given (policy, binding, param) combination is within its own CEL budget.apiVersion (string)APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resourceskind (string)Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kindsmetadata (ObjectMeta)Standard object metadata; More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata.spec (ValidatingAdmissionPolicyBindingSpec)Specification of the desired behavior of the ValidatingAdmissionPolicyBinding.ValidatingAdmissionPolicyBindingSpec is the specification of the ValidatingAdmissionPolicyBinding.spec.matchResources (MatchResources)MatchResources declares what resources match this binding and will be validated by it. Note that this is intersected with the policy's matchConstraints, so only requests that are matched by the policy can be selected by this. If this is unset, all resources matched by the policy are validated by this binding When resourceRules is unset, it does not constrain resource matching. If a resource is matched by the other fields of this object, it will be validated. Note that this is differs from ValidatingAdmissionPolicy matchConstraints, where resourceRules are required.MatchResources decides whether to run the admission control policy on an object based on whether it meets the match criteria. The exclude rules take precedence over include rules (if a resource matches both, it is excluded)spec.matchResources.excludeResourceRules ([]NamedRuleWithOperations)Atomic: will be replaced during a mergeExcludeResourceRules describes what operations on what resources/subresources the ValidatingAdmissionPolicy should not care about. The exclude rules take precedence over include rules (if a resource matches both, it is excluded)NamedRuleWithOperations is a tuple of Operations and Resources with ResourceNames.spec.matchResources.excludeResourceRules.apiGroups ([]string)Atomic: will be replaced during a mergeAPIGroups is the API groups the resources belong to. '' is all groups. If '' is present, the length of the slice must be one. Required.spec.matchResources.excludeResourceRules.apiVersions ([]string)Atomic: will be replaced during a mergeAPIVersions is the API versions the resources belong to. '' is all versions. If '' is present, the length of the slice must be one. Required.spec.matchResources.excludeResourceRules.operations ([]string)Atomic: will be replaced during a mergeOperations is the operations the admission hook cares about - CREATE, UPDATE, DELETE, CONNECT or * for all of those operations and any future admission operations that are added. If '*' is present, the length of the slice must be one. Required.spec.matchResources.excludeResourceRules.resourceNames ([]string)Atomic: will be replaced during a mergeResourceNames is an optional white list of names that the rule applies to. An empty set means that everything is allowed.spec.matchResources.excludeResourceRules.resources ([]string)Atomic: will be replaced during a mergeResources is a list of resources this rule applies to.For example: 'pods' means pods. 'pods/log' means the log subresource of pods. '' means all resources, but not subresources. 'pods/' means all subresources of pods. '/scale' means all scale subresources. '/*' means all resources and their subresources.If wildcard is present, the validation rule will ensure resources do not overlap with each other.Depending on the enclosing object, subresources might not be allowed. Required.spec.matchResources.excludeResourceRules.scope (string)scope specifies the scope of this rule. Valid values are "Cluster", "Namespaced", and "" "Cluster" means that only cluster-scoped resources will match this rule. Namespace API objects are cluster-scoped. "Namespaced" means that only namespaced resources will match this rule. "" means that there are no scope restrictions. Subresources match the scope of their parent resource. Default is "*".spec.matchResources.matchPolicy (string)matchPolicy defines how the "MatchResources" list is used to match incoming requests. Allowed values are "Exact" or "Equivalent".Exact: match a request only if it exactly matches a specified rule. For example, if deployments can be modified via apps/v1, apps/v1beta1, and extensions/v1beta1, but "rules" only included apiGroups:["apps"], apiVersions:["v1"], resources: ["deployments"], a request to apps/v1beta1 or extensions/v1beta1 would not be sent to the ValidatingAdmissionPolicy.Equivalent: match a request if modifies a resource listed in rules, even via another API group or version. For example, if deployments can be modified via apps/v1, apps/v1beta1, and extensions/v1beta1, and "rules" only included apiGroups:["apps"], apiVersions:["v1"], resources: ["deployments"], a request to apps/v1beta1 or extensions/v1beta1 would be converted to apps/v1 and sent to the ValidatingAdmissionPolicy.Defaults to "Equivalent"spec.matchResources.namespaceSelector (LabelSelector)NamespaceSelector decides whether to run the admission control policy on an object based on whether the namespace for that object matches the selector. If the object itself is a namespace, the matching is performed on object.metadata.labels. If the object is another cluster scoped resource, it never skips the policy.For example, to run the webhook on any objects whose namespace is not associated with "runlevel" of "0" or "1"; you will set the selector as follows: "namespaceSelector": { "matchExpressions": [ { "key": "runlevel", "operator": "NotIn", "values": [ "0", "1" ] } ] }If instead you want to only run the policy on any objects whose namespace is associated with the "environment" of "prod" or "staging"; you will set the selector as follows: "namespaceSelector": { "matchExpressions": [ { "key": "environment", "operator": "In", "values": [ "prod", "staging" ] } ] }See https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/ for more examples of label selectors.Default to the empty LabelSelector, which matches everything.spec.matchResources.objectSelector (LabelSelector)ObjectSelector decides whether to run the validation based on if the object has matching labels. objectSelector is evaluated against both the oldObject and newObject that would be sent to the cel validation, and is considered to match if either object matches the selector. A null object (oldObject in the case of create, or newObject in the case of delete) or an object that cannot have labels (like a DeploymentRollback or a PodProxyOptions object) is not considered to match. Use the object selector only if the webhook is opt-in, because end users may skip the admission webhook by setting the labels. Default to the empty LabelSelector, which matches everything.spec.matchResources.resourceRules ([]NamedRuleWithOperations)Atomic: will be replaced during a mergeResourceRules describes what operations on what resources/subresources the ValidatingAdmissionPolicy matches. The policy cares about an operation if it matches any Rule.NamedRuleWithOperations is a tuple of Operations and Resources with ResourceNames.spec.matchResources.resourceRules.apiGroups ([]string)Atomic: will be replaced during a mergeAPIGroups is the API groups the resources belong to. '' is all groups. If '' is present, the length of the slice must be one. Required.spec.matchResources.resourceRules.apiVersions ([]string)Atomic: will be replaced during a mergeAPIVersions is the API versions the resources belong to. '' is all versions. If '' is present, the length of the slice must be one. Required.spec.matchResources.resourceRules.operations ([]string)Atomic: will be replaced during a mergeOperations is the operations the admission hook cares about - CREATE, UPDATE, DELETE, CONNECT or * for all of those operations and any future admission operations that are added. If '*' is present, the length of the slice must be one. Required.spec.matchResources.resourceRules.resourceNames ([]string)Atomic: will be replaced during a mergeResourceNames is an optional white list of names that the rule applies to. An empty set means that everything is allowed.spec.matchResources.resourceRules.resources ([]string)Atomic: will be replaced during a mergeResources is a list of resources this rule applies to.For example: 'pods' means pods. 'pods/log' means the log subresource of pods. '' means all resources, but not subresources. 'pods/' means all subresources of pods. '/scale' means all scale subresources. '/*' means all resources and their subresources.If wildcard is present, the validation rule will ensure resources do not overlap with each other.Depending on the enclosing object, subresources might not be allowed. Required.spec.matchResources.resourceRules.scope (string)scope specifies the scope of this rule. Valid values are "Cluster", "Namespaced", and "" "Cluster" means that only cluster-scoped resources will match this rule. Namespace API objects are cluster-scoped. "Namespaced" means that only namespaced resources will match this rule. "" means that there are no scope restrictions. Subresources match the scope of their parent resource. Default is "*".spec.paramRef (ParamRef)paramRef specifies the parameter resource used to configure the admission control policy. It should point to a resource of the type specified in ParamKind of the bound ValidatingAdmissionPolicy. If the policy specifies a ParamKind and the resource referred to by ParamRef does not exist, this binding is considered mis-configured and the FailurePolicy of the ValidatingAdmissionPolicy applied. If the policy does not specify a ParamKind then this field is ignored, and the rules are evaluated without a param.ParamRef describes how to locate the params to be used as input to expressions of rules applied by a policy binding.spec.paramRef.name (string)name is the name of the resource being referenced.One of name or selector must be set, but name and selector are mutually exclusive properties. If one is set, the other must be unset.A single parameter used for all admission requests can be configured by setting the name field, leaving selector blank, and setting namespace if paramKind is namespace-scoped.spec.paramRef.namespace (string)namespace is the namespace of the referenced resource. Allows limiting the search for params to a specific namespace. Applies to both name and selector fields.A per-namespace parameter may be used by specifying a namespace-scoped paramKind in the policy and leaving this field empty.If paramKind is cluster-scoped, this field MUST be unset. Setting this field results in a configuration error.If paramKind is namespace-scoped, the namespace of the object being evaluated for admission will be used when this field is left unset. Take care that if this is left empty the binding must not match any cluster-scoped resources, which will result in an error.spec.paramRef.parameterNotFoundAction (string)parameterNotFoundAction controls the behavior of the binding when the resource exists, and name or selector is valid, but there are no parameters matched by the binding. If the value is set to Allow, then no matched parameters will be treated as successful validation by the binding. If set to Deny, then no matched parameters will be subject to the failurePolicy of the policy.Allowed values are Allow or DenyRequiredspec.paramRef.selector (LabelSelector)selector can be used to match multiple param objects based on their labels. Supply selector: {} to match all resources of the ParamKind.If multiple params are found, they are all evaluated with the policy expressions and the results are ANDed together.One of name or selector must be set, but name and selector are mutually exclusive properties. If one is set, the other must be unset.spec.policyName (string)PolicyName references a ValidatingAdmissionPolicy name which the ValidatingAdmissionPolicyBinding binds to. If the referenced resource does not exist, this binding is considered invalid and will be ignored Required.spec.validationActions ([]string)Set: unique values will be kept during a mergevalidationActions declares how Validations of the referenced ValidatingAdmissionPolicy are enforced. If a validation evaluates to false it is always enforced according to these actions.Failures defined by the ValidatingAdmissionPolicy's FailurePolicy are enforced according to these actions only if the FailurePolicy is set to Fail, otherwise the failures are ignored. This includes compilation errors, runtime errors and misconfigurations of the policy.validationActions is declared as a set of action values. Order does not matter. validationActions may not contain duplicates of the same action.The supported actions values are:"Deny" specifies that a validation failure results in a denied request."Warn" specifies that a validation failure is reported to the request client in HTTP Warning headers, with a warning code of 299. Warnings can be sent both for allowed or denied admission responses."Audit" specifies that a validation failure is included in the published audit event for the request. The audit event will contain a validation.policy.admission.k8s.io/validation_failure audit annotation with a value containing the details of the validation failures, formatted as a JSON list of objects, each with the following fields: - message: The validation failure message string - policy: The resource name of the ValidatingAdmissionPolicy - binding: The resource name of the ValidatingAdmissionPolicyBinding - expressionIndex: The index of the failed validations in the ValidatingAdmissionPolicy - validationActions: The enforcement actions enacted for the validation failure Example audit annotation: "validation.policy.admission.k8s.io/validation_failure": "[{\"message\": \"Invalid value\", {\"policy\": \"policy.example.com\", {\"binding\": \"policybinding.example.com\", {\"expressionIndex\": \"1\", {\"validationActions\": [\"Audit\"]}]"Clients should expect to handle additional values by ignoring any values not recognized."Deny" and "Warn" may not be used together since this combination needlessly duplicates the validation failure both in the API response body and the HTTP warning headers.Required.Operationsget read the specified ValidatingAdmissionPolicyHTTP RequestGET /apis/admissionregistration.k8s.io/v1/validatingadmissionpolicies/{name}Parametersname (in path): string, requiredname of the ValidatingAdmissionPolicypretty (in query): stringprettyResponse200 (ValidatingAdmissionPolicy): OK401: Unauthorizedget read status of the specified ValidatingAdmissionPolicyHTTP RequestGET /apis/admissionregistration.k8s.io/v1/validatingadmissionpolicies/{name}/statusParametersname (in path): string, requiredname of the ValidatingAdmissionPolicypretty (in query): stringprettyResponse200 (ValidatingAdmissionPolicy): OK401: Unauthorizedlist list or watch objects of kind ValidatingAdmissionPolicyHTTP RequestGET /apis/admissionregistration.k8s.io/v1/validatingadmissionpoliciesParametersallowWatchBookmarks (in query): booleanallowWatchBookmarkscontinue (in query): stringcontinuefieldSelector (in query): stringfieldSelectorlabelSelector (in query): stringlabelSelectorlimit (in query): integerlimitpretty (in query): stringprettyresourceVersion (in query): stringresourceVersionresourceVersionMatch (in query): stringresourceVersionMatchsendInitialEvents (in query): booleansendInitialEventstimeoutSeconds (in query): integertimeoutSecondswatch (in query): booleanwatchResponse200 (ValidatingAdmissionPolicyList): OK401: Unauthorizedcreate create a ValidatingAdmissionPolicyHTTP RequestPOST /apis/admissionregistration.k8s.io/v1/validatingadmissionpoliciesParametersbody: ValidatingAdmissionPolicy, requireddryRun (in query): stringdryRunfieldManager (in query): stringfieldManagerfieldValidation (in query): stringfieldValidationpretty (in query): stringprettyResponse200 (ValidatingAdmissionPolicy): OK201 (ValidatingAdmissionPolicy): Created202 (ValidatingAdmissionPolicy): Accepted401: Unauthorizedupdate replace the specified ValidatingAdmissionPolicyHTTP RequestPUT /apis/admissionregistration.k8s.io/v1/validatingadmissionpolicies/{name}Parametersname (in path): string, requiredname of the ValidatingAdmissionPolicybody: ValidatingAdmissionPolicy, requireddryRun (in query): stringdryRunfieldManager (in query): stringfieldManagerfieldValidation (in query): stringfieldValidationpretty (in query): stringprettyResponse200 (ValidatingAdmissionPolicy): OK201 (ValidatingAdmissionPolicy): Created401: Unauthorizedupdate replace status of the specified ValidatingAdmissionPolicyHTTP RequestPUT /apis/admissionregistration.k8s.io/v1/validatingadmissionpolicies/{name}/statusParametersname (in path): string, requiredname of the ValidatingAdmissionPolicybody: ValidatingAdmissionPolicy, requireddryRun (in query): stringdryRunfieldManager (in query): stringfieldManagerfieldValidation (in query): stringfieldValidationpretty (in query): stringprettyResponse200 (ValidatingAdmissionPolicy): OK201 (ValidatingAdmissionPolicy): Created401: Unauthorizedpatch partially update the specified ValidatingAdmissionPolicyHTTP RequestPATCH /apis/admissionregistration.k8s.io/v1/validatingadmissionpolicies/{name}Parametersname (in path): string, requiredname of the ValidatingAdmissionPolicybody: Patch, requireddryRun (in query): stringdryRunfieldManager (in query): stringfieldManagerfieldValidation (in query): stringfieldValidationforce (in query): booleanforcepretty (in query): stringprettyResponse200 (ValidatingAdmissionPolicy): OK201 (ValidatingAdmissionPolicy): Created401: Unauthorizedpatch partially update status of the specified ValidatingAdmissionPolicyHTTP RequestPATCH /apis/admissionregistration.k8s.io/v1/validatingadmissionpolicies/{name}/statusParametersname (in path): string, requiredname of the ValidatingAdmissionPolicybody: Patch, requireddryRun (in query): stringdryRunfieldManager (in query): stringfieldManagerfieldValidation (in query): stringfieldValidationforce (in query): booleanforcepretty (in query): stringprettyResponse200 (ValidatingAdmissionPolicy): OK201 (ValidatingAdmissionPolicy): Created401: Unauthorizeddelete delete a ValidatingAdmissionPolicyHTTP RequestDELETE /apis/admissionregistration.k8s.io/v1/validatingadmissionpolicies/{name}Parametersname (in path): string, requiredname of the ValidatingAdmissionPolicybody: DeleteOptionsdryRun (in query): stringdryRungracePeriodSeconds (in query): integergracePeriodSecondsignoreStoreReadErrorWithClusterBreakingPotential (in query): booleanignoreStoreReadErrorWithClusterBreakingPotentialpretty (in query): stringprettypropagationPolicy (in query): stringpropagationPolicyResponse200 (Status): OK202 (Status): Accepted401: Unauthorizeddeletecollection delete collection of ValidatingAdmissionPolicyHTTP RequestDELETE /apis/admissionregistration.k8s.io/v1/validatingadmissionpoliciesParametersbody: DeleteOptionscontinue (in query): stringcontinuedryRun (in query): stringdryRunfieldSelector (in query): stringfieldSelectorgracePeriodSeconds (in query): integergracePeriodSecondsignoreStoreReadErrorWithClusterBreakingPotential (in query): booleanignoreStoreReadErrorWithClusterBreakingPotentiallabelSelector (in query): stringlabelSelectorlimit (in query): integerlimitpretty (in query): stringprettypropagationPolicy (in query): stringpropagationPolicyresourceVersion (in query): stringresourceVersionresourceVersionMatch (in query): stringresourceVersionMatchsendInitialEvents (in query): booleansendInitialEventstimeoutSeconds (in query): integertimeoutSecondsResponse200 (Status): OK401: UnauthorizedThis page is automatically generated.If you plan to report an issue with this page, mention that the page is auto-generated in your issue description. The fix may need to happen elsewhere in the Kubernetes project.


# kubectl

kubectlSynopsiskubectl controls the Kubernetes cluster manager.Find more information in Command line tool (kubectl).kubectl [flags] Options--add-dir-headerIf true, adds the file directory to the header of the log messages--alsologtostderrlog to standard error as well as files--as stringUsername to impersonate for the operation--as-group stringArrayGroup to impersonate for the operation, this flag can be repeated to specify multiple groups.--azure-container-registry-config stringPath to the file containing Azure container registry configuration information.--cache-dir string Default: "$HOME/.kube/cache"Default cache directory--certificate-authority stringPath to a cert file for the certificate authority--client-certificate stringPath to a client certificate file for TLS--client-key stringPath to a client key file for TLS--cloud-provider-gce-l7lb-src-cidrs cidrs Default: 130.211.0.0/22,35.191.0.0/16CIDRs opened in GCE firewall for L7 LB traffic proxy & health checks--cloud-provider-gce-lb-src-cidrs cidrs Default: 130.211.0.0/22,209.85.152.0/22,209.85.204.0/22,35.191.0.0/16CIDRs opened in GCE firewall for L4 LB traffic proxy & health checks--cluster stringThe name of the kubeconfig cluster to use--context stringThe name of the kubeconfig context to use--default-not-ready-toleration-seconds int Default: 300Indicates the tolerationSeconds of the toleration for notReady:NoExecute that is added by default to every pod that does not already have such a toleration.--default-unreachable-toleration-seconds int Default: 300Indicates the tolerationSeconds of the toleration for unreachable:NoExecute that is added by default to every pod that does not already have such a toleration.-h, --helphelp for kubectl--insecure-skip-tls-verifyIf true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure--kubeconfig stringPath to the kubeconfig file to use for CLI requests.--log-backtrace-at traceLocation Default: :0when logging hits line file:N, emit a stack trace--log-dir stringIf non-empty, write log files in this directory--log-file stringIf non-empty, use this log file--log-file-max-size uint Default: 1800Defines the maximum size a log file can grow to. Unit is megabytes. If the value is 0, the maximum file size is unlimited.--log-flush-frequency duration Default: 5sMaximum number of seconds between log flushes--logtostderr Default: truelog to standard error instead of files--match-server-versionRequire server version to match client version-n, --namespace stringIf present, the namespace scope for this CLI request--one-outputIf true, only write logs to their native severity level (vs also writing to each lower severity level)--password stringPassword for basic authentication to the API server--profile string Default: "none"Name of profile to capture. One of (none|cpu|heap|goroutine|threadcreate|block|mutex)--profile-output string Default: "profile.pprof"Name of the file to write the profile to--request-timeout string Default: "0"The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests.-s, --server stringThe address and port of the Kubernetes API server--skip-headersIf true, avoid header prefixes in the log messages--skip-log-headersIf true, avoid headers when opening log files--stderrthreshold severity Default: 2logs at or above this threshold go to stderr--tls-server-name stringServer name to use for server certificate validation. If it is not provided, the hostname used to contact the server is used--token stringBearer token for authentication to the API server--user stringThe name of the kubeconfig user to use--username stringUsername for basic authentication to the API server-v, --v Levelnumber for the log level verbosity--version version[=true]Print version information and quit--vmodule moduleSpeccomma-separated list of pattern=N settings for file-filtered logging--warnings-as-errorsTreat warnings received from the server as errors and exit with a non-zero exit codeEnvironment variablesKUBECONFIGPath to the kubectl configuration ("kubeconfig") file. Default: "$HOME/.kube/config"KUBECTL_COMMAND_HEADERSWhen set to false, turns off extra HTTP headers detailing invoked kubectl command (Kubernetes version v1.22 or later)KUBECTL_EXPLAIN_OPENAPIV3Toggles whether calls to `kubectl explain` use the new OpenAPIv3 data source available. OpenAPIV3 is enabled by default since Kubernetes 1.24.KUBECTL_ENABLE_CMD_SHADOWWhen set to true, external plugins can be used as subcommands for builtin commands if subcommand does not exist. In alpha stage, this feature can only be used for create command(e.g. kubectl create networkpolicy).KUBECTL_PORT_FORWARD_WEBSOCKETSWhen set to true, the kubectl port-forward command will attempt to stream using the websockets protocol. If the upgrade to websockets fails, the commands will fallback to use the current SPDY protocol.KUBECTL_REMOTE_COMMAND_WEBSOCKETSWhen set to true, the kubectl exec, cp, and attach commands will attempt to stream using the websockets protocol. If the upgrade to websockets fails, the commands will fallback to use the current SPDY protocol.KUBECTL_KUBERCWhen set to true, kuberc file is taken into account to define user specific preferences.See Alsokubectl annotate - Update the annotations on a resourcekubectl api-resources - Print the supported API resources on the serverkubectl api-versions - Print the supported API versions on the server, in the form of "group/version"kubectl apply - Apply a configuration to a resource by filename or stdinkubectl attach - Attach to a running containerkubectl auth - Inspect authorizationkubectl autoscale - Auto-scale a Deployment, ReplicaSet, or ReplicationControllerkubectl certificate - Modify certificate resources.kubectl cluster-info - Display cluster infokubectl completion - Output shell completion code for the specified shell (bash or zsh)kubectl config - Modify kubeconfig fileskubectl cordon - Mark node as unschedulablekubectl cp - Copy files and directories to and from containers.kubectl create - Create a resource from a file or from stdin.kubectl debug - Create debugging sessions for troubleshooting workloads and nodeskubectl delete - Delete resources by filenames, stdin, resources and names, or by resources and label selectorkubectl describe - Show details of a specific resource or group of resourceskubectl diff - Diff live version against would-be applied versionkubectl drain - Drain node in preparation for maintenancekubectl edit - Edit a resource on the serverkubectl events - List eventskubectl exec - Execute a command in a containerkubectl explain - Documentation of resourceskubectl expose - Take a replication controller, service, deployment or pod and expose it as a new Kubernetes Servicekubectl get - Display one or many resourceskubectl kustomize - Build a kustomization target from a directory or a remote url.kubectl label - Update the labels on a resourcekubectl logs - Print the logs for a container in a podkubectl options - Print the list of flags inherited by all commandskubectl patch - Update field(s) of a resourcekubectl plugin - Provides utilities for interacting with plugins.kubectl port-forward - Forward one or more local ports to a podkubectl proxy - Run a proxy to the Kubernetes API serverkubectl replace - Replace a resource by filename or stdinkubectl rollout - Manage the rollout of a resourcekubectl run - Run a particular image on the clusterkubectl scale - Set a new size for a Deployment, ReplicaSet or Replication Controllerkubectl set - Set specific features on objectskubectl taint - Update the taints on one or more nodeskubectl top - Display Resource (CPU/Memory/Storage) usage.kubectl uncordon - Mark node as schedulablekubectl version - Print the client and server version informationkubectl wait - Experimental: Wait for a specific condition on one or many resources.


# Enforce Pod Security Standards with Namespace Labels

Enforce Pod Security Standards with Namespace LabelsNamespaces can be labeled to enforce the Pod Security Standards. The three policies privileged, baseline and restricted broadly cover the security spectrum and are implemented by the Pod Security admission controller.Before you beginPod Security Admission was available by default in Kubernetes v1.23, as a beta. From version 1.25 onwards, Pod Security Admission is generally available.To check the version, enter kubectl version.Requiring the baseline Pod Security Standard with namespace labelsThis manifest defines a Namespace my-baseline-namespace that:Blocks any pods that don't satisfy the baseline policy requirements.Generates a user-facing warning and adds an audit annotation to any created pod that does not meet the restricted policy requirements.Pins the versions of the baseline and restricted policies to v1.34.apiVersion: v1 kind: Namespace metadata: name: my-baseline-namespace labels: pod-security.kubernetes.io/enforce: baseline pod-security.kubernetes.io/enforce-version: v1.34 # We are setting these to our _desired_ `enforce` level. pod-security.kubernetes.io/audit: restricted pod-security.kubernetes.io/audit-version: v1.34 pod-security.kubernetes.io/warn: restricted pod-security.kubernetes.io/warn-version: v1.34 Add labels to existing namespaces with kubectl labelNote:When an enforce policy (or version) label is added or changed, the admission plugin will test each pod in the namespace against the new policy. Violations are returned to the user as warnings.It is helpful to apply the --dry-run flag when initially evaluating security profile changes for namespaces. The Pod Security Standard checks will still be run in dry run mode, giving you information about how the new policy would treat existing pods, without actually updating a policy.kubectl label --dry-run=server --overwrite ns --all \ pod-security.kubernetes.io/enforce=baseline Applying to all namespacesIf you're just getting started with the Pod Security Standards, a suitable first step would be to configure all namespaces with audit annotations for a stricter level such as baseline:kubectl label --overwrite ns --all \ pod-security.kubernetes.io/audit=baseline \ pod-security.kubernetes.io/warn=baseline Note that this is not setting an enforce level, so that namespaces that haven't been explicitly evaluated can be distinguished. You can list namespaces without an explicitly set enforce level using this command:kubectl get namespaces --selector='!pod-security.kubernetes.io/enforce' Applying to a single namespaceYou can update a specific namespace as well. This command adds the enforce=restricted policy to my-existing-namespace, pinning the restricted policy version to v1.34.kubectl label --overwrite ns my-existing-namespace \ pod-security.kubernetes.io/enforce=restricted \ pod-security.kubernetes.io/enforce-version=v1.34


# Process ID Limits And Reservations

Process ID Limits And ReservationsFEATURE STATE: Kubernetes v1.20 [stable]Kubernetes allow you to limit the number of process IDs (PIDs) that a Pod can use. You can also reserve a number of allocatable PIDs for each node for use by the operating system and daemons (rather than by Pods).Process IDs (PIDs) are a fundamental resource on nodes. It is trivial to hit the task limit without hitting any other resource limits, which can then cause instability to a host machine.Cluster administrators require mechanisms to ensure that Pods running in the cluster cannot induce PID exhaustion that prevents host daemons (such as the kubelet or kube-proxy, and potentially also the container runtime) from running. In addition, it is important to ensure that PIDs are limited among Pods in order to ensure they have limited impact on other workloads on the same node.Note:On certain Linux installations, the operating system sets the PIDs limit to a low default, such as 32768. Consider raising the value of /proc/sys/kernel/pid_max.You can configure a kubelet to limit the number of PIDs a given Pod can consume. For example, if your node's host OS is set to use a maximum of 262144 PIDs and expect to host less than 250 Pods, one can give each Pod a budget of 1000 PIDs to prevent using up that node's overall number of available PIDs. If the admin wants to overcommit PIDs similar to CPU or memory, they may do so as well with some additional risks. Either way, a single Pod will not be able to bring the whole machine down. This kind of resource limiting helps to prevent simple fork bombs from affecting operation of an entire cluster.Per-Pod PID limiting allows administrators to protect one Pod from another, but does not ensure that all Pods scheduled onto that host are unable to impact the node overall. Per-Pod limiting also does not protect the node agents themselves from PID exhaustion.You can also reserve an amount of PIDs for node overhead, separate from the allocation to Pods. This is similar to how you can reserve CPU, memory, or other resources for use by the operating system and other facilities outside of Pods and their containers.PID limiting is an important sibling to compute resource requests and limits. However, you specify it in a different way: rather than defining a Pod's resource limit in the .spec for a Pod, you configure the limit as a setting on the kubelet. Pod-defined PID limits are not currently supported.Caution:This means that the limit that applies to a Pod may be different depending on where the Pod is scheduled. To make things simple, it's easiest if all Nodes use the same PID resource limits and reservations.Node PID limitsKubernetes allows you to reserve a number of process IDs for the system use. To configure the reservation, use the parameter pid=<number> in the --system-reserved and --kube-reserved command line options to the kubelet. The value you specified declares that the specified number of process IDs will be reserved for the system as a whole and for Kubernetes system daemons respectively.Pod PID limitsKubernetes allows you to limit the number of processes running in a Pod. You specify this limit at the node level, rather than configuring it as a resource limit for a particular Pod. Each Node can have a different PID limit.To configure the limit, you can specify the command line parameter --pod-max-pids to the kubelet, or set PodPidsLimit in the kubelet configuration file.PID based evictionYou can configure kubelet to start terminating a Pod when it is misbehaving and consuming abnormal amount of resources. This feature is called eviction. You can Configure Out of Resource Handling for various eviction signals. Use pid.available eviction signal to configure the threshold for number of PIDs used by Pod. You can set soft and hard eviction policies. However, even with the hard eviction policy, if the number of PIDs growing very fast, node can still get into unstable state by hitting the node PIDs limit. Eviction signal value is calculated periodically and does NOT enforce the limit.PID limiting - per Pod and per Node sets the hard limit. Once the limit is hit, workload will start experiencing failures when trying to get a new PID. It may or may not lead to rescheduling of a Pod, depending on how workload reacts on these failures and how liveness and readiness probes are configured for the Pod. However, if limits were set correctly, you can guarantee that other Pods workload and system processes will not run out of PIDs when one Pod is misbehaving.What's nextRefer to the PID Limiting enhancement document for more information.For historical context, read Process ID Limiting for Stability Improvements in Kubernetes 1.14.Read Managing Resources for Containers.Learn how to Configure Out of Resource Handling.


# kubectl create clusterrolebinding

kubectl create clusterrolebindingSynopsisCreate a cluster role binding for a particular cluster role.kubectl create clusterrolebinding NAME --clusterrole=NAME [--user=username] [--group=groupname] [--serviceaccount=namespace:serviceaccountname] [--dry-run=server|client|none] Examples # Create a cluster role binding for user1, user2, and group1 using the cluster-admin cluster role kubectl create clusterrolebinding cluster-admin --clusterrole=cluster-admin --user=user1 --user=user2 --group=group1 Options--allow-missing-template-keys Default: trueIf true, ignore any errors in templates when a field or map key is missing in the template. Only applies to golang and jsonpath output formats.--clusterrole stringClusterRole this ClusterRoleBinding should reference--dry-run string[="unchanged"] Default: "none"Must be "none", "server", or "client". If client strategy, only print the object that would be sent, without sending it. If server strategy, submit server-side request without persisting the resource.--field-manager string Default: "kubectl-create"Name of the manager used to track field ownership.--group stringsGroups to bind to the clusterrole. The flag can be repeated to add multiple groups.-h, --helphelp for clusterrolebinding-o, --output stringOutput format. One of: (json, yaml, name, go-template, go-template-file, template, templatefile, jsonpath, jsonpath-as-json, jsonpath-file).--save-configIf true, the configuration of current object will be saved in its annotation. Otherwise, the annotation will be unchanged. This flag is useful when you want to perform kubectl apply on this object in the future.--serviceaccount stringsService accounts to bind to the clusterrole, in the format <namespace>:<name>. The flag can be repeated to add multiple service accounts.--show-managed-fieldsIf true, keep the managedFields when printing objects in JSON or YAML format.--template stringTemplate string or path to template file to use when -o=go-template, -o=go-template-file. The template format is golang templates [http://golang.org/pkg/text/template/#pkg-overview].--user stringsUsernames to bind to the clusterrole. The flag can be repeated to add multiple users.--validate string[="strict"] Default: "strict"Must be one of: strict (or true), warn, ignore (or false). "true" or "strict" will use a schema to validate the input and fail the request if invalid. It will perform server side validation if ServerSideFieldValidation is enabled on the api-server, but will fall back to less reliable client-side validation if not. "warn" will warn about unknown or duplicate fields without blocking the request if server-side field validation is enabled on the API server, and behave as "ignore" otherwise. "false" or "ignore" will not perform any schema validation, silently dropping any unknown or duplicate fields.Parent Options Inherited--as stringUsername to impersonate for the operation. User could be a regular user or a service account in a namespace.--as-group stringsGroup to impersonate for the operation, this flag can be repeated to specify multiple groups.--as-uid stringUID to impersonate for the operation.--cache-dir string Default: "$HOME/.kube/cache"Default cache directory--certificate-authority stringPath to a cert file for the certificate authority--client-certificate stringPath to a client certificate file for TLS--client-key stringPath to a client key file for TLS--cluster stringThe name of the kubeconfig cluster to use--context stringThe name of the kubeconfig context to use--disable-compressionIf true, opt-out of response compression for all requests to the server--insecure-skip-tls-verifyIf true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure--kubeconfig stringPath to the kubeconfig file to use for CLI requests.--kuberc stringPath to the kuberc file to use for preferences. This can be disabled by exporting KUBECTL_KUBERC=false feature gate or turning off the feature KUBERC=off.--match-server-versionRequire server version to match client version-n, --namespace stringIf present, the namespace scope for this CLI request--password stringPassword for basic authentication to the API server--profile string Default: "none"Name of profile to capture. One of (none|cpu|heap|goroutine|threadcreate|block|mutex)--profile-output string Default: "profile.pprof"Name of the file to write the profile to--request-timeout string Default: "0"The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests.-s, --server stringThe address and port of the Kubernetes API server--storage-driver-buffer-duration duration Default: 1m0sWrites in the storage driver will be buffered for this duration, and committed to the non memory backends as a single transaction--storage-driver-db string Default: "cadvisor"database name--storage-driver-host string Default: "localhost:8086"database host:port--storage-driver-password string Default: "root"database password--storage-driver-secureuse secure connection with database--storage-driver-table string Default: "stats"table name--storage-driver-user string Default: "root"database username--tls-server-name stringServer name to use for server certificate validation. If it is not provided, the hostname used to contact the server is used--token stringBearer token for authentication to the API server--username stringUsername for basic authentication to the API server--version version[=true]--version, --version=raw prints version information and quits; --version=vX.Y.Z... sets the reported version--warnings-as-errorsTreat warnings received from the server as errors and exit with a non-zero exit codeSee Alsokubectl create - Create a resource from a file or from stdinThis page is automatically generated.If you plan to report an issue with this page, mention that the page is auto-generated in your issue description. The fix may need to happen elsewhere in the Kubernetes project.


# Mixed Version Proxy

Mixed Version ProxyFEATURE STATE: Kubernetes v1.28 [alpha] (enabled by default: false)Kubernetes 1.34 includes an alpha feature that lets an API Server proxy a resource requests to other peer API servers. This is useful when there are multiple API servers running different versions of Kubernetes in one cluster (for example, during a long-lived rollout to a new release of Kubernetes).This enables cluster administrators to configure highly available clusters that can be upgraded more safely, by directing resource requests (made during the upgrade) to the correct kube-apiserver. That proxying prevents users from seeing unexpected 404 Not Found errors that stem from the upgrade process.This mechanism is called the Mixed Version Proxy.Enabling the Mixed Version ProxyEnsure that UnknownVersionInteroperabilityProxy feature gate is enabled when you start the API Server:kube-apiserver \ --feature-gates=UnknownVersionInteroperabilityProxy=true \ # required command line arguments for this feature --peer-ca-file=<path to kube-apiserver CA cert> --proxy-client-cert-file=<path to aggregator proxy cert>, --proxy-client-key-file=<path to aggregator proxy key>, --requestheader-client-ca-file=<path to aggregator CA cert>, # requestheader-allowed-names can be set to blank to allow any Common Name --requestheader-allowed-names=<valid Common Names to verify proxy client cert against>, # optional flags for this feature --peer-advertise-ip=`IP of this kube-apiserver that should be used by peers to proxy requests` --peer-advertise-port=`port of this kube-apiserver that should be used by peers to proxy requests` # …and other flags as usual Proxy transport and authentication between API serversThe source kube-apiserver reuses the existing APIserver client authentication flags --proxy-client-cert-file and --proxy-client-key-file to present its identity that will be verified by its peer (the destination kube-apiserver). The destination API server verifies that peer connection based on the configuration you specify using the --requestheader-client-ca-file command line argument.To authenticate the destination server's serving certs, you must configure a certificate authority bundle by specifying the --peer-ca-file command line argument to the source API server.Configuration for peer API server connectivityTo set the network location of a kube-apiserver that peers will use to proxy requests, use the --peer-advertise-ip and --peer-advertise-port command line arguments to kube-apiserver or specify these fields in the API server configuration file. If these flags are unspecified, peers will use the value from either --advertise-address or --bind-address command line argument to the kube-apiserver. If those too, are unset, the host's default interface is used.Mixed version proxyingWhen you enable mixed version proxying, the aggregation layer loads a special filter that does the following:When a resource request reaches an API server that cannot serve that API (either because it is at a version pre-dating the introduction of the API or the API is turned off on the API server) the API server attempts to send the request to a peer API server that can serve the requested API. It does so by identifying API groups / versions / resources that the local server doesn't recognise, and tries to proxy those requests to a peer API server that is capable of handling the request.If the peer API server fails to respond, the source API server responds with 503 ("Service Unavailable") error.How it works under the hoodWhen an API Server receives a resource request, it first checks which API servers can serve the requested resource. This check happens using the internal StorageVersion API.If the resource is known to the API server that received the request (for example, GET /api/v1/pods/some-pod), the request is handled locally.If there is no internal StorageVersion object found for the requested resource (for example, GET /my-api/v1/my-resource) and the configured APIService specifies proxying to an extension API server, that proxying happens following the usual flow for extension APIs.If a valid internal StorageVersion object is found for the requested resource (for example, GET /batch/v1/jobs) and the API server trying to handle the request (the handling API server) has the batch API disabled, then the handling API server fetches the peer API servers that do serve the relevant API group / version / resource (api/v1/batch in this case) using the information in the fetched StorageVersion object. The handling API server then proxies the request to one of the matching peer kube-apiservers that are aware of the requested resource.If there is no peer known for that API group / version / resource, the handling API server passes the request to its own handler chain which should eventually return a 404 ("Not Found") response.If the handling API server has identified and selected a peer API server, but that peer fails to respond (for reasons such as network connectivity issues, or a data race between the request being received and a controller registering the peer's info into the control plane), then the handling API server responds with a 503 ("Service Unavailable") error.


# Debugging DNS Resolution

Debugging DNS ResolutionThis page provides hints on diagnosing DNS problems.Before you beginYou need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. It is recommended to run this tutorial on a cluster with at least two nodes that are not acting as control plane hosts. If you do not already have a cluster, you can create one by using minikube or you can use one of these Kubernetes playgrounds:iximiuz LabsKillercodaKodeKloudPlay with KubernetesYour cluster must be configured to use the CoreDNS addon or its precursor, kube-dns.Your Kubernetes server must be at or later than version v1.6.To check the version, enter kubectl version.Create a simple Pod to use as a test environmentadmin/dns/dnsutils.yaml apiVersion: v1 kind: Pod metadata: name: dnsutils namespace: default spec: containers: - name: dnsutils image: registry.k8s.io/e2e-test-images/agnhost:2.39 imagePullPolicy: IfNotPresent restartPolicy: Always Note:This example creates a pod in the default namespace. DNS name resolution for services depends on the namespace of the pod. For more information, review DNS for Services and Pods.Use that manifest to create a Pod:kubectl apply -f https://k8s.io/examples/admin/dns/dnsutils.yaml pod/dnsutils created …and verify its status:kubectl get pods dnsutils NAME READY STATUS RESTARTS AGE dnsutils 1/1 Running 0 <some-time> Once that Pod is running, you can exec nslookup in that environment. If you see something like the following, DNS is working correctly.kubectl exec -i -t dnsutils -- nslookup kubernetes.default Server: 10.0.0.10 Address 1: 10.0.0.10 Name: kubernetes.default Address 1: 10.0.0.1 If the nslookup command fails, check the following:Check the local DNS configuration firstTake a look inside the resolv.conf file. (See Customizing DNS Service and Known issues below for more information)kubectl exec -ti dnsutils -- cat /etc/resolv.conf Verify that the search path and name server are set up like the following (note that search path may vary for different cloud providers):search default.svc.cluster.local svc.cluster.local cluster.local google.internal c.gce_project_id.internal nameserver 10.0.0.10 options ndots:5 Errors such as the following indicate a problem with the CoreDNS (or kube-dns) add-on or with associated Services:kubectl exec -i -t dnsutils -- nslookup kubernetes.default Server: 10.0.0.10 Address 1: 10.0.0.10 nslookup: can't resolve 'kubernetes.default' orkubectl exec -i -t dnsutils -- nslookup kubernetes.default Server: 10.0.0.10 Address 1: 10.0.0.10 kube-dns.kube-system.svc.cluster.local nslookup: can't resolve 'kubernetes.default' Check if the DNS pod is runningUse the kubectl get pods command to verify that the DNS pod is running.kubectl get pods --namespace=kube-system -l k8s-app=kube-dns NAME READY STATUS RESTARTS AGE ... coredns-7b96bf9f76-5hsxb 1/1 Running 0 1h coredns-7b96bf9f76-mvmmt 1/1 Running 0 1h ... Note:The value for label k8s-app is kube-dns for both CoreDNS and kube-dns deployments.If you see that no CoreDNS Pod is running or that the Pod has failed/completed, the DNS add-on may not be deployed by default in your current environment and you will have to deploy it manually.Check for errors in the DNS podUse the kubectl logs command to see logs for the DNS containers.For CoreDNS:kubectl logs --namespace=kube-system -l k8s-app=kube-dns Here is an example of a healthy CoreDNS log:.:53 2018/08/15 14:37:17 [INFO] CoreDNS-1.2.2 2018/08/15 14:37:17 [INFO] linux/amd64, go1.10.3, 2e322f6 CoreDNS-1.2.2 linux/amd64, go1.10.3, 2e322f6 2018/08/15 14:37:17 [INFO] plugin/reload: Running configuration MD5 = 24e6c59e83ce706f07bcc82c31b1ea1c See if there are any suspicious or unexpected messages in the logs.Is DNS service up?Verify that the DNS service is up by using the kubectl get service command.kubectl get svc --namespace=kube-system NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE ... kube-dns ClusterIP 10.0.0.10 <none> 53/UDP,53/TCP 1h ... Note:The service name is kube-dns for both CoreDNS and kube-dns deployments.If you have created the Service or in the case it should be created by default but it does not appear, see debugging Services for more information.Are DNS endpoints exposed?You can verify that DNS endpoints are exposed by using the kubectl get endpointslice command.kubectl get endpointslices -l k8s.io/service-name=kube-dns --namespace=kube-system NAME ADDRESSTYPE PORTS ENDPOINTS AGE kube-dns-zxoja IPv4 53 10.180.3.17,10.180.3.17 1h If you do not see the endpoints, see the endpoints section in the debugging Services documentation.For additional Kubernetes DNS examples, see the cluster-dns examples in the Kubernetes GitHub repository.Are DNS queries being received/processed?You can verify if queries are being received by CoreDNS by adding the log plugin to the CoreDNS configuration (aka Corefile). The CoreDNS Corefile is held in a ConfigMap named coredns. To edit it, use the command:kubectl -n kube-system edit configmap coredns Then add log in the Corefile section per the example below:apiVersion: v1 kind: ConfigMap metadata: name: coredns namespace: kube-system data: Corefile: | .:53 { log errors health kubernetes cluster.local in-addr.arpa ip6.arpa { pods insecure upstream fallthrough in-addr.arpa ip6.arpa } prometheus :9153 forward . /etc/resolv.conf cache 30 loop reload loadbalance } After saving the changes, it may take up to minute or two for Kubernetes to propagate these changes to the CoreDNS pods.Next, make some queries and view the logs per the sections above in this document. If CoreDNS pods are receiving the queries, you should see them in the logs.Here is an example of a query in the log:.:53 2018/08/15 14:37:15 [INFO] CoreDNS-1.2.0 2018/08/15 14:37:15 [INFO] linux/amd64, go1.10.3, 2e322f6 CoreDNS-1.2.0 linux/amd64, go1.10.3, 2e322f6 2018/09/07 15:29:04 [INFO] plugin/reload: Running configuration MD5 = 162475cdf272d8aa601e6fe67a6ad42f 2018/09/07 15:29:04 [INFO] Reloading complete 172.17.0.18:41675 - [07/Sep/2018:15:29:11 +0000] 59925 "A IN kubernetes.default.svc.cluster.local. udp 54 false 512" NOERROR qr,aa,rd,ra 106 0.000066649s Does CoreDNS have sufficient permissions?CoreDNS must be able to list service and endpointslice related resources to properly resolve service names.Sample error message:2022-03-18T07:12:15.699431183Z [INFO] 10.96.144.227:52299 - 3686 "A IN serverproxy.contoso.net.cluster.local. udp 52 false 512" SERVFAIL qr,aa,rd 145 0.000091221s First, get the current ClusterRole of system:coredns:kubectl describe clusterrole system:coredns -n kube-system Expected output:PolicyRule: Resources Non-Resource URLs Resource Names Verbs --------- ----------------- -------------- ----- endpoints [] [] [list watch] namespaces [] [] [list watch] pods [] [] [list watch] services [] [] [list watch] endpointslices.discovery.k8s.io [] [] [list watch] If any permissions are missing, edit the ClusterRole to add them:kubectl edit clusterrole system:coredns -n kube-system Example insertion of EndpointSlices permissions:... - apiGroups: - discovery.k8s.io resources: - endpointslices verbs: - list - watch ... Are you in the right namespace for the service?DNS queries that don't specify a namespace are limited to the pod's namespace.If the namespace of the pod and service differ, the DNS query must include the namespace of the service.This query is limited to the pod's namespace:kubectl exec -i -t dnsutils -- nslookup <service-name> This query specifies the namespace:kubectl exec -i -t dnsutils -- nslookup <service-name>.<namespace> To learn more about name resolution, see DNS for Services and Pods.Known issuesSome Linux distributions (e.g. Ubuntu) use a local DNS resolver by default (systemd-resolved). Systemd-resolved moves and replaces /etc/resolv.conf with a stub file that can cause a fatal forwarding loop when resolving names in upstream servers. This can be fixed manually by using kubelet's --resolv-conf flag to point to the correct resolv.conf (With systemd-resolved, this is /run/systemd/resolve/resolv.conf). kubeadm automatically detects systemd-resolved, and adjusts the kubelet flags accordingly.Kubernetes installs do not configure the nodes' resolv.conf files to use the cluster DNS by default, because that process is inherently distribution-specific. This should probably be implemented eventually.Linux's libc (a.k.a. glibc) has a limit for the DNS nameserver records to 3 by default and Kubernetes needs to consume 1 nameserver record. This means that if a local installation already uses 3 nameservers, some of those entries will be lost. To work around this limit, the node can run dnsmasq, which will provide more nameserver entries. You can also use kubelet's --resolv-conf flag.If you are using Alpine version 3.17 or earlier as your base image, DNS may not work properly due to a design issue with Alpine. Until musl version 1.24 didn't include TCP fallback to the DNS stub resolver meaning any DNS call above 512 bytes would fail. Please upgrade your images to Alpine version 3.18 or above.What's nextSee Autoscaling the DNS Service in a Cluster.Read DNS for Services and Pods


# Adding entries to Pod /etc/hosts with HostAliases

Adding entries to Pod /etc/hosts with HostAliasesAdding entries to a Pod's /etc/hosts file provides Pod-level override of hostname resolution when DNS and other options are not applicable. You can add these custom entries with the HostAliases field in PodSpec.The Kubernetes project recommends modifying DNS configuration using the hostAliases field (part of the .spec for a Pod), and not by using an init container or other means to edit /etc/hosts directly. Change made in other ways may be overwritten by the kubelet during Pod creation or restart.Default hosts file contentStart an Nginx Pod which is assigned a Pod IP:kubectl run nginx --image nginx pod/nginx created Examine a Pod IP:kubectl get pods --output=wide NAME READY STATUS RESTARTS AGE IP NODE nginx 1/1 Running 0 13s 10.200.0.4 worker0 The hosts file content would look like this:kubectl exec nginx -- cat /etc/hosts # Kubernetes-managed hosts file. 127.0.0.1 localhost ::1 localhost ip6-localhost ip6-loopback fe00::0 ip6-localnet fe00::0 ip6-mcastprefix fe00::1 ip6-allnodes fe00::2 ip6-allrouters 10.200.0.4 nginx By default, the hosts file only includes IPv4 and IPv6 boilerplates like localhost and its own hostname.Adding additional entries with hostAliasesIn addition to the default boilerplate, you can add additional entries to the hosts file. For example: to resolve foo.local, bar.local to 127.0.0.1 and foo.remote, bar.remote to 10.1.2.3, you can configure HostAliases for a Pod under .spec.hostAliases:service/networking/hostaliases-pod.yaml apiVersion: v1 kind: Pod metadata: name: hostaliases-pod spec: restartPolicy: Never hostAliases: - ip: "127.0.0.1" hostnames: - "foo.local" - "bar.local" - ip: "10.1.2.3" hostnames: - "foo.remote" - "bar.remote" containers: - name: cat-hosts image: busybox:1.28 command: - cat args: - "/etc/hosts" You can start a Pod with that configuration by running:kubectl apply -f https://k8s.io/examples/service/networking/hostaliases-pod.yaml pod/hostaliases-pod created Examine a Pod's details to see its IPv4 address and its status:kubectl get pod --output=wide NAME READY STATUS RESTARTS AGE IP NODE hostaliases-pod 0/1 Completed 0 6s 10.200.0.5 worker0 The hosts file content looks like this:kubectl logs hostaliases-pod # Kubernetes-managed hosts file. 127.0.0.1 localhost ::1 localhost ip6-localhost ip6-loopback fe00::0 ip6-localnet fe00::0 ip6-mcastprefix fe00::1 ip6-allnodes fe00::2 ip6-allrouters 10.200.0.5 hostaliases-pod # Entries added by HostAliases. 127.0.0.1 foo.local bar.local 10.1.2.3 foo.remote bar.remote with the additional entries specified at the bottom.Why does the kubelet manage the hosts file?The kubelet manages the hosts file for each container of the Pod to prevent the container runtime from modifying the file after the containers have already been started. Historically, Kubernetes always used Docker Engine as its container runtime, and Docker Engine would then modify the /etc/hosts file after each container had started.Current Kubernetes can use a variety of container runtimes; even so, the kubelet manages the hosts file within each container so that the outcome is as intended regardless of which container runtime you use.Caution:Avoid making manual changes to the hosts file inside a container.If you make manual changes to the hosts file, those changes are lost when the container exits.


# Kubernetes Object Management

Kubernetes Object ManagementThe kubectl command-line tool supports several different ways to create and manage Kubernetes objects. This document provides an overview of the different approaches. Read the Kubectl book for details of managing objects by Kubectl.Management techniquesWarning:A Kubernetes object should be managed using only one technique. Mixing and matching techniques for the same object results in undefined behavior.Management techniqueOperates onRecommended environmentSupported writersLearning curveImperative commandsLive objectsDevelopment projects1+LowestImperative object configurationIndividual filesProduction projects1ModerateDeclarative object configurationDirectories of filesProduction projects1+HighestImperative commandsWhen using imperative commands, a user operates directly on live objects in a cluster. The user provides operations to the kubectl command as arguments or flags.This is the recommended way to get started or to run a one-off task in a cluster. Because this technique operates directly on live objects, it provides no history of previous configurations.ExamplesRun an instance of the nginx container by creating a Deployment object:kubectl create deployment nginx --image nginx Trade-offsAdvantages compared to object configuration:Commands are expressed as a single action word.Commands require only a single step to make changes to the cluster.Disadvantages compared to object configuration:Commands do not integrate with change review processes.Commands do not provide an audit trail associated with changes.Commands do not provide a source of records except for what is live.Commands do not provide a template for creating new objects.Imperative object configurationIn imperative object configuration, the kubectl command specifies the operation (create, replace, etc.), optional flags and at least one file name. The file specified must contain a full definition of the object in YAML or JSON format.See the API reference for more details on object definitions.Warning:The imperative replace command replaces the existing spec with the newly provided one, dropping all changes to the object missing from the configuration file. This approach should not be used with resource types whose specs are updated independently of the configuration file. Services of type LoadBalancer, for example, have their externalIPs field updated independently from the configuration by the cluster.ExamplesCreate the objects defined in a configuration file:kubectl create -f nginx.yaml Delete the objects defined in two configuration files:kubectl delete -f nginx.yaml -f redis.yaml Update the objects defined in a configuration file by overwriting the live configuration:kubectl replace -f nginx.yaml Trade-offsAdvantages compared to imperative commands:Object configuration can be stored in a source control system such as Git.Object configuration can integrate with processes such as reviewing changes before push and audit trails.Object configuration provides a template for creating new objects.Disadvantages compared to imperative commands:Object configuration requires basic understanding of the object schema.Object configuration requires the additional step of writing a YAML file.Advantages compared to declarative object configuration:Imperative object configuration behavior is simpler and easier to understand.As of Kubernetes version 1.5, imperative object configuration is more mature.Disadvantages compared to declarative object configuration:Imperative object configuration works best on files, not directories.Updates to live objects must be reflected in configuration files, or they will be lost during the next replacement.Declarative object configurationWhen using declarative object configuration, a user operates on object configuration files stored locally, however the user does not define the operations to be taken on the files. Create, update, and delete operations are automatically detected per-object by kubectl. This enables working on directories, where different operations might be needed for different objects.Note:Declarative object configuration retains changes made by other writers, even if the changes are not merged back to the object configuration file. This is possible by using the patch API operation to write only observed differences, instead of using the replace API operation to replace the entire object configuration.ExamplesProcess all object configuration files in the configs directory, and create or patch the live objects. You can first diff to see what changes are going to be made, and then apply:kubectl diff -f configs/ kubectl apply -f configs/ Recursively process directories:kubectl diff -R -f configs/ kubectl apply -R -f configs/ Trade-offsAdvantages compared to imperative object configuration:Changes made directly to live objects are retained, even if they are not merged back into the configuration files.Declarative object configuration has better support for operating on directories and automatically detecting operation types (create, patch, delete) per-object.Disadvantages compared to imperative object configuration:Declarative object configuration is harder to debug and understand results when they are unexpected.Partial updates using diffs create complex merge and patch operations.What's nextManaging Kubernetes Objects Using Imperative CommandsImperative Management of Kubernetes Objects Using Configuration FilesDeclarative Management of Kubernetes Objects Using Configuration FilesDeclarative Management of Kubernetes Objects Using KustomizeKubectl Command ReferenceKubectl BookKubernetes API Reference


# kubectl api-versions

kubectl api-versionsSynopsisPrint the supported API versions on the server, in the form of "group/version".kubectl api-versions Examples # Print the supported API versions kubectl api-versions Options-h, --helphelp for api-versionsParent Options Inherited--as stringUsername to impersonate for the operation. User could be a regular user or a service account in a namespace.--as-group stringsGroup to impersonate for the operation, this flag can be repeated to specify multiple groups.--as-uid stringUID to impersonate for the operation.--cache-dir string Default: "$HOME/.kube/cache"Default cache directory--certificate-authority stringPath to a cert file for the certificate authority--client-certificate stringPath to a client certificate file for TLS--client-key stringPath to a client key file for TLS--cluster stringThe name of the kubeconfig cluster to use--context stringThe name of the kubeconfig context to use--disable-compressionIf true, opt-out of response compression for all requests to the server--insecure-skip-tls-verifyIf true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure--kubeconfig stringPath to the kubeconfig file to use for CLI requests.--kuberc stringPath to the kuberc file to use for preferences. This can be disabled by exporting KUBECTL_KUBERC=false feature gate or turning off the feature KUBERC=off.--match-server-versionRequire server version to match client version-n, --namespace stringIf present, the namespace scope for this CLI request--password stringPassword for basic authentication to the API server--profile string Default: "none"Name of profile to capture. One of (none|cpu|heap|goroutine|threadcreate|block|mutex)--profile-output string Default: "profile.pprof"Name of the file to write the profile to--request-timeout string Default: "0"The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests.-s, --server stringThe address and port of the Kubernetes API server--storage-driver-buffer-duration duration Default: 1m0sWrites in the storage driver will be buffered for this duration, and committed to the non memory backends as a single transaction--storage-driver-db string Default: "cadvisor"database name--storage-driver-host string Default: "localhost:8086"database host:port--storage-driver-password string Default: "root"database password--storage-driver-secureuse secure connection with database--storage-driver-table string Default: "stats"table name--storage-driver-user string Default: "root"database username--tls-server-name stringServer name to use for server certificate validation. If it is not provided, the hostname used to contact the server is used--token stringBearer token for authentication to the API server--user stringThe name of the kubeconfig user to use--username stringUsername for basic authentication to the API server--version version[=true]--version, --version=raw prints version information and quits; --version=vX.Y.Z... sets the reported version--warnings-as-errorsTreat warnings received from the server as errors and exit with a non-zero exit codeSee Alsokubectl - kubectl controls the Kubernetes cluster managerThis page is automatically generated.If you plan to report an issue with this page, mention that the page is auto-generated in your issue description. The fix may need to happen elsewhere in the Kubernetes project.


# Service Internal Traffic Policy

Service Internal Traffic PolicyIf two Pods in your cluster want to communicate, and both Pods are actually running on the same node, use Service Internal Traffic Policy to keep network traffic within that node. Avoiding a round trip via the cluster network can help with reliability, performance (network latency and throughput), or cost.FEATURE STATE: Kubernetes v1.26 [stable]Service Internal Traffic Policy enables internal traffic restrictions to only route internal traffic to endpoints within the node the traffic originated from. The "internal" traffic here refers to traffic originated from Pods in the current cluster. This can help to reduce costs and improve performance.Using Service Internal Traffic PolicyYou can enable the internal-only traffic policy for a Service, by setting its .spec.internalTrafficPolicy to Local. This tells kube-proxy to only use node local endpoints for cluster internal traffic.Note:For pods on nodes with no endpoints for a given Service, the Service behaves as if it has zero endpoints (for Pods on this node) even if the service does have endpoints on other nodes.The following example shows what a Service looks like when you set .spec.internalTrafficPolicy to Local:apiVersion: v1 kind: Service metadata: name: my-service spec: selector: app.kubernetes.io/name: MyApp ports: - protocol: TCP port: 80 targetPort: 9376 internalTrafficPolicy: Local How it worksThe kube-proxy filters the endpoints it routes to based on the spec.internalTrafficPolicy setting. When it's set to Local, only node local endpoints are considered. When it's Cluster (the default), or is not set, Kubernetes considers all endpoints.What's nextRead about Topology Aware RoutingRead about Service External Traffic PolicyFollow the Connecting Applications with Services tutorial