Validating Admission PolicyFEATURE STATE: Kubernetes v1.30 [stable]This page provides an overview of Validating Admission Policy.What is Validating Admission Policy?Validating admission policies offer a declarative, in-process alternative to validating admission webhooks.Validating admission policies use the Common Expression Language (CEL) to declare the validation rules of a policy. Validation admission policies are highly configurable, enabling policy authors to define policies that can be parameterized and scoped to resources as needed by cluster administrators.What Resources Make a PolicyA policy is generally made up of three resources:The ValidatingAdmissionPolicy describes the abstract logic of a policy (think: "this policy makes sure a particular label is set to a particular value").A parameter resource provides information to a ValidatingAdmissionPolicy to make it a concrete statement (think "the owner label must be set to something that ends in .company.com"). A native type such as ConfigMap or a CRD defines the schema of a parameter resource. ValidatingAdmissionPolicy objects specify what Kind they are expecting for their parameter resource.A ValidatingAdmissionPolicyBinding links the above resources together and provides scoping. If you only want to require an owner label to be set for Pods, the binding is where you would specify this restriction.At least a ValidatingAdmissionPolicy and a corresponding ValidatingAdmissionPolicyBinding must be defined for a policy to have an effect.If a ValidatingAdmissionPolicy does not need to be configured via parameters, simply leave spec.paramKind in ValidatingAdmissionPolicy not specified.Getting Started with Validating Admission PolicyValidating Admission Policy is part of the cluster control-plane. You should write and deploy them with great caution. The following describes how to quickly experiment with Validating Admission Policy.Creating a ValidatingAdmissionPolicyThe following is an example of a ValidatingAdmissionPolicy.validatingadmissionpolicy/basic-example-policy.yaml apiVersion: admissionregistration.k8s.io/v1 kind: ValidatingAdmissionPolicy metadata: name: "demo-policy.example.com" spec: failurePolicy: Fail matchConstraints: resourceRules: - apiGroups: ["apps"] apiVersions: ["v1"] operations: ["CREATE", "UPDATE"] resources: ["deployments"] validations: - expression: "object.spec.replicas <= 5"spec.validations contains CEL expressions which use the Common Expression Language (CEL) to validate the request. If an expression evaluates to false, the validation check is enforced according to the spec.failurePolicy field.Note:You can quickly test CEL expressions in CEL Playground.To configure a validating admission policy for use in a cluster, a binding is required. The following is an example of a ValidatingAdmissionPolicyBinding.:validatingadmissionpolicy/basic-example-binding.yaml apiVersion: admissionregistration.k8s.io/v1 kind: ValidatingAdmissionPolicyBinding metadata: name: "demo-binding-test.example.com" spec: policyName: "demo-policy.example.com" validationActions: [Deny] matchResources: namespaceSelector: matchLabels: environment: test When trying to create a deployment with replicas set not satisfying the validation expression, an error will return containing message:ValidatingAdmissionPolicy 'demo-policy.example.com' with binding 'demo-binding-test.example.com' denied request: failed expression: object.spec.replicas <= 5 The above provides a simple example of using ValidatingAdmissionPolicy without a parameter configured.Validation actionsEach ValidatingAdmissionPolicyBinding must specify one or more validationActions to declare how validations of a policy are enforced.The supported validationActions are:Deny: Validation failure results in a denied request.Warn: Validation failure is reported to the request client as a warning.Audit: Validation failure is included in the audit event for the API request.For example, to both warn clients about a validation failure and to audit the validation failures, use:validationActions: [Warn, Audit] Deny and Warn may not be used together since this combination needlessly duplicates the validation failure both in the API response body and the HTTP warning headers.A validation that evaluates to false is always enforced according to these actions. Failures defined by the failurePolicy are enforced according to these actions only if the failurePolicy is set to Fail (or not specified), otherwise the failures are ignored.See Audit Annotations: validation failures for more details about the validation failure audit annotation.Parameter resourcesParameter resources allow a policy configuration to be separate from its definition. A policy can define paramKind, which outlines GVK of the parameter resource, and then a policy binding ties a policy by name (via policyName) to a particular parameter resource via paramRef.If parameter configuration is needed, the following is an example of a ValidatingAdmissionPolicy with parameter configuration.validatingadmissionpolicy/policy-with-param.yaml apiVersion: admissionregistration.k8s.io/v1 kind: ValidatingAdmissionPolicy metadata: name: "replicalimit-policy.example.com" spec: failurePolicy: Fail paramKind: apiVersion: rules.example.com/v1 kind: ReplicaLimit matchConstraints: resourceRules: - apiGroups: ["apps"] apiVersions: ["v1"] operations: ["CREATE", "UPDATE"] resources: ["deployments"] validations: - expression: "object.spec.replicas <= params.maxReplicas" reason: InvalidThe spec.paramKind field of the ValidatingAdmissionPolicy specifies the kind of resources used to parameterize this policy. For this example, it is configured by ReplicaLimit custom resources. Note in this example how the CEL expression references the parameters via the CEL params variable, e.g. params.maxReplicas. spec.matchConstraints specifies what resources this policy is designed to validate. Note that the native types such like ConfigMap could also be used as parameter reference.The spec.validations fields contain CEL expressions. If an expression evaluates to false, the validation check is enforced according to the spec.failurePolicy field.The validating admission policy author is responsible for providing the ReplicaLimit parameter CRD.To configure an validating admission policy for use in a cluster, a binding and parameter resource are created. The following is an example of a ValidatingAdmissionPolicyBinding that uses a cluster-wide param - the same param will be used to validate every resource request that matches the binding:validatingadmissionpolicy/binding-with-param.yaml apiVersion: admissionregistration.k8s.io/v1 kind: ValidatingAdmissionPolicyBinding metadata: name: "replicalimit-binding-test.example.com" spec: policyName: "replicalimit-policy.example.com" validationActions: [Deny] paramRef: name: "replica-limit-test.example.com" namespace: "default" parameterNotFoundAction: Deny matchResources: namespaceSelector: matchLabels: environment: test Notice this binding applies a parameter to the policy for all resources which are in the test environment.The parameter resource could be as following:validatingadmissionpolicy/replicalimit-param.yaml apiVersion: rules.example.com/v1 kind: ReplicaLimit metadata: name: "replica-limit-test.example.com" namespace: "default" maxReplicas: 3 This policy parameter resource limits deployments to a max of 3 replicas.An admission policy may have multiple bindings. To bind all other environments to have a maxReplicas limit of 100, create another ValidatingAdmissionPolicyBinding:validatingadmissionpolicy/binding-with-param-prod.yaml apiVersion: admissionregistration.k8s.io/v1 kind: ValidatingAdmissionPolicyBinding metadata: name: "replicalimit-binding-nontest" spec: policyName: "replicalimit-policy.example.com" validationActions: [Deny] paramRef: name: "replica-limit-prod.example.com" namespace: "default" parameterNotFoundAction: Deny matchResources: namespaceSelector: matchExpressions: - key: environment operator: NotIn values: - test Notice this binding applies a different parameter to resources which are not in the test environment.And have a parameter resource:validatingadmissionpolicy/replicalimit-param-prod.yaml apiVersion: rules.example.com/v1 kind: ReplicaLimit metadata: name: "replica-limit-prod.example.com" maxReplicas: 100For each admission request, the API server evaluates CEL expressions of each (policy, binding, param) combination that match the request. For a request to be admitted it must pass all evaluations.If multiple bindings match the request, the policy will be evaluated for each, and they must all pass evaluation for the policy to be considered passed.If multiple parameters match a single binding, the policy rules will be evaluated for each param, and they too must all pass for the binding to be considered passed. Bindings can have overlapping match criteria. The policy is evaluated for each matching binding-parameter combination. A policy may even be evaluated multiple times if multiple bindings match it, or a single binding that matches multiple parameters.The params object representing a parameter resource will not be set if a parameter resource has not been bound, so for policies requiring a parameter resource, it can be useful to add a check to ensure one has been bound. A parameter resource will not be bound and params will be null if paramKind of the policy, or paramRef of the binding are not specified.For the use cases requiring parameter configuration, we recommend to add a param check in spec.validations[0].expression:- expression: "params != null" message: "params missing but required to bind to this policy" Optional parametersIt can be convenient to be able to have optional parameters as part of a parameter resource, and only validate them if present. CEL provides has(), which checks if the key passed to it exists. CEL also implements Boolean short-circuiting. If the first half of a logical OR evaluates to true, it won’t evaluate the other half (since the result of the entire OR will be true regardless).Combining the two, we can provide a way to validate optional parameters:!has(params.optionalNumber) || (params.optionalNumber >= 5 && params.optionalNumber <= 10)Here, we first check that the optional parameter is present with !has(params.optionalNumber).If optionalNumber hasn’t been defined, then the expression short-circuits since !has(params.optionalNumber) will evaluate to true.If optionalNumber has been defined, then the latter half of the CEL expression will be evaluated, and optionalNumber will be checked to ensure that it contains a value between 5 and 10 inclusive.Per-namespace ParametersAs the author of a ValidatingAdmissionPolicy and its ValidatingAdmissionPolicyBinding, you can choose to specify cluster-wide, or per-namespace parameters. If you specify a namespace for the binding's paramRef, the control plane only searches for parameters in that namespace.However, if namespace is not specified in the ValidatingAdmissionPolicyBinding, the API server can search for relevant parameters in the namespace that a request is against. For example, if you make a request to modify a ConfigMap in the default namespace and there is a relevant ValidatingAdmissionPolicyBinding with no namespace set, then the API server looks for a parameter object in default. This design enables policy configuration that depends on the namespace of the resource being manipulated, for more fine-tuned control.Parameter selectorIn addition to specify a parameter in a binding by name, you may choose instead to specify label selector, such that all resources of the policy's paramKind, and the param's namespace (if applicable) that match the label selector are selected for evaluation. See selector for more information on how label selectors match resources.If multiple parameters are found to meet the condition, the policy's rules are evaluated for each parameter found and the results will be ANDed together.If namespace is provided, only objects of the paramKind in the provided namespace are eligible for selection. Otherwise, when namespace is empty and paramKind is namespace-scoped, the namespace used in the request being admitted will be used.Authorization checksWe introduced the authorization check for parameter resources. User is expected to have read access to the resources referenced by paramKind in ValidatingAdmissionPolicy and paramRef in ValidatingAdmissionPolicyBinding.Note that if a resource in paramKind fails resolving via the restmapper, read access to all resources of groups is required.paramRefThe paramRef field specifies the parameter resource used by the policy. It has the following fields:name: The name of the parameter resource.namespace: The namespace of the parameter resource.selector: A label selector to match multiple parameter resources.parameterNotFoundAction: (Required) Controls the behavior when the specified parameters are not found.Allowed Values:Allow: The absence of matched parameters is treated as a successful validation by the binding.Deny: The absence of matched parameters is subject to the failurePolicy of the policy.One of name or selector must be set, but not both.Note:The parameterNotFoundAction field in paramRef is required. It specifies the action to take when no parameters are found matching the paramRef. If not specified, the policy binding may be considered invalid and will be ignored or could lead to unexpected behavior.Allow: If set to Allow, and no parameters are found, the binding treats the absence of parameters as a successful validation, and the policy is considered to have passed.Deny: If set to Deny, and no parameters are found, the binding enforces the failurePolicy of the policy. If the failurePolicy is Fail, the request is rejected.Make sure to set parameterNotFoundAction according to the desired behavior when parameters are missing.Handling Missing Parameters with parameterNotFoundActionWhen using paramRef with a selector, it's possible that no parameters match the selector. The parameterNotFoundAction field determines how the binding behaves in this scenario.Example:apiVersion: admissionregistration.k8s.io/v1alpha1 kind: ValidatingAdmissionPolicyBinding metadata: name: example-binding spec: policyName: example-policy paramRef: selector: matchLabels: environment: test parameterNotFoundAction: Allow validationActions: - Deny Failure PolicyfailurePolicy defines how mis-configurations and CEL expressions evaluating to error from the admission policy are handled. Allowed values are Ignore or Fail.Ignore means that an error calling the ValidatingAdmissionPolicy is ignored and the API request is allowed to continue.Fail means that an error calling the ValidatingAdmissionPolicy causes the admission to fail and the API request to be rejected.Note that the failurePolicy is defined inside ValidatingAdmissionPolicy:validatingadmissionpolicy/failure-policy-ignore.yaml apiVersion: admissionregistration.k8s.io/v1 kind: ValidatingAdmissionPolicy spec: ... failurePolicy: Ignore # The default is "Fail" validations: - expression: "object.spec.xyz == params.x" Validation Expressionspec.validations[i].expression represents the expression which will be evaluated by CEL. To learn more, see the CEL language specification CEL expressions have access to the contents of the Admission request/response, organized into CEL variables as well as some other useful variables:'object' - The object from the incoming request. The value is null for DELETE requests.'oldObject' - The existing object. The value is null for CREATE requests.'request' - Attributes of the admission request.'params' - Parameter resource referred to by the policy binding being evaluated. The value is null if ParamKind is not specified.namespaceObject - The namespace, as a Kubernetes resource, that the incoming object belongs to. The value is null if the incoming object is cluster-scoped.authorizer - A CEL Authorizer. May be used to perform authorization checks for the principal (authenticated user) of the request. See AuthzSelectors and Authz in the Kubernetes CEL library documentation for more details.authorizer.requestResource - A shortcut for an authorization check configured with the request resource (group, resource, (subresource), namespace, name).In CEL expressions, variables like object and oldObject are strongly-typed. You can access any field in the object's schema, such as object.metadata.labels and fields in spec.For any Kubernetes object, including schemaless Custom Resources, CEL guarantees access to a minimal set of properties: apiVersion, kind, metadata.name, and metadata.generateName.Equality on arrays with list type of 'set' or 'map' ignores element order, i.e. [1, 2] == [2, 1]. Concatenation on arrays with x-kubernetes-list-type use the semantics of the list type:'set': X + Y performs a union where the array positions of all elements in X are preserved and non-intersecting elements in Y are appended, retaining their partial order.'map': X + Y performs a merge where the array positions of all keys in X are preserved but the values are overwritten by values in Y when the key sets of X and Y intersect. Elements in Y with non-intersecting keys are appended, retaining their partial order.Validation expression examplesExpressionPurposeobject.minReplicas <= object.replicas && object.replicas <= object.maxReplicasValidate that the three fields defining replicas are ordered appropriately'Available' in object.stateCountsValidate that an entry with the 'Available' key exists in a map(size(object.list1) == 0) != (size(object.list2) == 0)Validate that one of two lists is non-empty, but not both!('MY_KEY' in object.map1) || object['MY_KEY'].matches('^[a-zA-Z]*$')Validate the value of a map for a specific key, if it is in the mapobject.envars.filter(e, e.name == 'MY_ENV').all(e, e.value.matches('^[a-zA-Z]*$')Validate the 'value' field of a listMap entry where key field 'name' is 'MY_ENV'has(object.expired) && object.created + object.ttl < object.expiredValidate that 'expired' date is after a 'create' date plus a 'ttl' durationobject.health.startsWith('ok')Validate a 'health' string field has the prefix 'ok'object.widgets.exists(w, w.key == 'x' && w.foo < 10)Validate that the 'foo' property of a listMap item with a key 'x' is less than 10type(object) == string ? object == '100%' : object == 1000Validate an int-or-string field for both the int and string casesobject.metadata.name.startsWith(object.prefix)Validate that an object's name has the prefix of another field valueobject.set1.all(e, !(e in object.set2))Validate that two listSets are disjointsize(object.names) == size(object.details) && object.names.all(n, n in object.details)Validate the 'details' map is keyed by the items in the 'names' listSetsize(object.clusters.filter(c, c.name == object.primary)) == 1Validate that the 'primary' property has one and only one occurrence in the 'clusters' listMapRead Supported evaluation on CEL for more information about CEL rules.spec.validation[i].reason represents a machine-readable description of why this validation failed. If this is the first validation in the list to fail, this reason, as well as the corresponding HTTP response code, are used in the HTTP response to the client. The currently supported reasons are: Unauthorized, Forbidden, Invalid, RequestEntityTooLarge. If not set, StatusReasonInvalid is used in the response to the client.Matching requests: matchConditionsYou can define match conditions for a ValidatingAdmissionPolicy if you need fine-grained request filtering. These conditions are useful if you find that match rules, objectSelectors and namespaceSelectors still doesn't provide the filtering you want. Match conditions are CEL expressions. All match conditions must evaluate to true for the resource to be evaluated.Here is an example illustrating a few different uses for match conditions:access/validating-admission-policy-match-conditions.yaml apiVersion: admissionregistration.k8s.io/v1 kind: ValidatingAdmissionPolicy metadata: name: "demo-policy.example.com" spec: failurePolicy: Fail matchConstraints: resourceRules: - apiGroups: ["*"] apiVersions: ["*"] operations: ["CREATE", "UPDATE"] resources: ["*"] matchConditions: - name: 'exclude-leases' # Each match condition must have a unique name expression: '!(request.resource.group == "coordination.k8s.io" && request.resource.resource == "leases")' # Match non-lease resources. - name: 'exclude-kubelet-requests' expression: '!("system:nodes" in request.userInfo.groups)' # Match requests made by non-node users. - name: 'rbac' # Skip RBAC requests. expression: 'request.resource.group != "rbac.authorization.k8s.io"' validations: - expression: "!object.metadata.name.contains('demo') || object.metadata.namespace == 'demo'" Match conditions have access to the same CEL variables as validation expressions.In the event of an error evaluating a match condition the policy is not evaluated. Whether to reject the request is determined as follows:If any match condition evaluated to false (regardless of other errors), the API server skips the policy.Otherwise:for failurePolicy: Fail, reject the request (without evaluating the policy).for failurePolicy: Ignore, proceed with the request but skip the policy.Audit annotationsauditAnnotations may be used to include audit annotations in the audit event of the API request.For example, here is an admission policy with an audit annotation:access/validating-admission-policy-audit-annotation.yaml apiVersion: admissionregistration.k8s.io/v1 kind: ValidatingAdmissionPolicy metadata: name: "demo-policy.example.com" spec: failurePolicy: Fail matchConstraints: resourceRules: - apiGroups: ["apps"] apiVersions: ["v1"] operations: ["CREATE", "UPDATE"] resources: ["deployments"] validations: - expression: "object.spec.replicas > 50" messageExpression: "'Deployment spec.replicas set to ' + string(object.spec.replicas)" auditAnnotations: - key: "high-replica-count" valueExpression: "'Deployment spec.replicas set to ' + string(object.spec.replicas)" When an API request is validated with this admission policy, the resulting audit event will look like:# the audit event recorded { "kind": "Event", "apiVersion": "audit.k8s.io/v1", "annotations": { "demo-policy.example.com/high-replica-count": "Deployment spec.replicas set to 128" # other annotations ... } # other fields ... } In this example the annotation will only be included if the spec.replicas of the Deployment is more than 50, otherwise the CEL expression evaluates to null and the annotation will not be included.Note that audit annotation keys are prefixed by the name of the ValidatingAdmissionPolicy and a /. If another admission controller, such as an admission webhook, uses the exact same audit annotation key, the value of the first admission controller to include the audit annotation will be included in the audit event and all other values will be ignored.Message expressionTo return a more friendly message when the policy rejects a request, we can use a CEL expression to composite a message with spec.validations[i].messageExpression. Similar to the validation expression, a message expression has access to object, oldObject, request, params, and namespaceObject. Unlike validations, message expression must evaluate to a string.For example, to better inform the user of the reason of denial when the policy refers to a parameter, we can have the following validation:access/deployment-replicas-policy.yaml apiVersion: admissionregistration.k8s.io/v1 kind: ValidatingAdmissionPolicy metadata: name: "deploy-replica-policy.example.com" spec: paramKind: apiVersion: rules.example.com/v1 kind: ReplicaLimit matchConstraints: resourceRules: - apiGroups: ["apps"] apiVersions: ["v1"] operations: ["CREATE", "UPDATE"] resources: ["deployments"] validations: - expression: "object.spec.replicas <= params.maxReplicas" messageExpression: "'object.spec.replicas must be no greater than ' + string(params.maxReplicas)" reason: Invalid After creating a params object that limits the replicas to 3 and setting up the binding, when we try to create a deployment with 5 replicas, we will receive the following message.$ kubectl create deploy --image=nginx nginx --replicas=5 error: failed to create deployment: deployments.apps "nginx" is forbidden: ValidatingAdmissionPolicy 'deploy-replica-policy.example.com' with binding 'demo-binding-test.example.com' denied request: object.spec.replicas must be no greater than 3 This is more informative than a static message of "too many replicas".The message expression takes precedence over the static message defined in spec.validations[i].message if both are defined. However, if the message expression fails to evaluate, the static message will be used instead. Additionally, if the message expression evaluates to a multi-line string, the evaluation result will be discarded and the static message will be used if present. Note that static message is validated against multi-line strings.Type checkingWhen a policy definition is created or updated, the validation process parses the expressions it contains and reports any syntax errors, rejecting the definition if any errors are found. Afterward, the referred variables are checked for type errors, including missing fields and type confusion, against the matched types of spec.matchConstraints. The result of type checking can be retrieved from status.typeChecking. The presence of status.typeChecking indicates the completion of type checking, and an empty status.typeChecking means that no errors were detected.For example, given the following policy definition:validatingadmissionpolicy/typechecking.yaml apiVersion: admissionregistration.k8s.io/v1 kind: ValidatingAdmissionPolicy metadata: name: "deploy-replica-policy.example.com" spec: matchConstraints: resourceRules: - apiGroups: ["apps"] apiVersions: ["v1"] operations: ["CREATE", "UPDATE"] resources: ["deployments"] validations: - expression: "object.replicas > 1" # should be "object.spec.replicas > 1" message: "must be replicated" reason: Invalid The status will yield the following information:status: typeChecking: expressionWarnings: - fieldRef: spec.validations[0].expression warning: |- apps/v1, Kind=Deployment: ERROR: <input>:1:7: undefined field 'replicas' | object.replicas > 1 | ......^ If multiple resources are matched in spec.matchConstraints, all of matched resources will be checked against. For example, the following policy definitionvalidatingadmissionpolicy/typechecking-multiple-match.yaml apiVersion: admissionregistration.k8s.io/v1 kind: ValidatingAdmissionPolicy metadata: name: "replica-policy.example.com" spec: matchConstraints: resourceRules: - apiGroups: ["apps"] apiVersions: ["v1"] operations: ["CREATE", "UPDATE"] resources: ["deployments","replicasets"] validations: - expression: "object.replicas > 1" # should be "object.spec.replicas > 1" message: "must be replicated" reason: Invalidwill have multiple types and type checking result of each type in the warning message.status: typeChecking: expressionWarnings: - fieldRef: spec.validations[0].expression warning: |- apps/v1, Kind=Deployment: ERROR: <input>:1:7: undefined field 'replicas' | object.replicas > 1 | ......^ apps/v1, Kind=ReplicaSet: ERROR: <input>:1:7: undefined field 'replicas' | object.replicas > 1 | ......^ Type Checking has the following limitation:No wildcard matching. If spec.matchConstraints.resourceRules contains "*" in any of apiGroups, apiVersions or resources, the types that "*" matches will not be checked.The number of matched types is limited to 10. This is to prevent a policy that manually specifying too many types. to consume excessive computing resources. In the order of ascending group, version, and then resource, 11th combination and beyond are ignored.Type Checking does not affect the policy behavior in any way. Even if the type checking detects errors, the policy will continue to evaluate. If errors do occur during evaluate, the failure policy will decide its outcome.Type Checking does not apply to CRDs, including matched CRD types and reference of paramKind. The support for CRDs will come in future release.Variable compositionIf an expression grows too complicated, or part of the expression is reusable and computationally expensive to evaluate, you can extract some part of the expressions into variables. A variable is a named expression that can be referred later in variables in other expressions.spec: variables: - name: foo expression: "'foo' in object.spec.metadata.labels ? object.spec.metadata.labels['foo'] : 'default'" validations: - expression: variables.foo == 'bar' A variable is lazily evaluated when it is first referred. Any error that occurs during the evaluation will be reported during the evaluation of the referring expression. Both the result and potential error are memorized and count only once towards the runtime cost.The order of variables are important because a variable can refer to other variables that are defined before it. This ordering prevents circular references.The following is a more complex example of enforcing that image repo names match the environment defined in its namespace.access/image-matches-namespace-environment.policy.yaml # This policy enforces that all containers of a deployment has the image repo match the environment label of its namespace. # Except for "exempt" deployments, or any containers that do not belong to the "example.com" organization (e.g. common sidecars). # For example, if the namespace has a label of {"environment": "staging"}, all container images must be either staging.example.com/* # or do not contain "example.com" at all, unless the deployment has {"exempt": "true"} label. apiVersion: admissionregistration.k8s.io/v1 kind: ValidatingAdmissionPolicy metadata: name: "image-matches-namespace-environment.policy.example.com" spec: failurePolicy: Fail matchConstraints: resourceRules: - apiGroups: ["apps"] apiVersions: ["v1"] operations: ["CREATE", "UPDATE"] resources: ["deployments"] variables: - name: environment expression: "'environment' in namespaceObject.metadata.labels ? namespaceObject.metadata.labels['environment'] : 'prod'" - name: exempt expression: "'exempt' in object.metadata.labels && object.metadata.labels['exempt'] == 'true'" - name: containers expression: "object.spec.template.spec.containers" - name: containersToCheck expression: "variables.containers.filter(c, c.image.contains('example.com/'))" validations: - expression: "variables.exempt || variables.containersToCheck.all(c, c.image.startsWith(variables.environment + '.'))" messageExpression: "'only ' + variables.environment + ' images are allowed in namespace ' + namespaceObject.metadata.name"With the policy bound to the namespace default, which is labeled environment: prod, the following attempt to create a deployment would be rejected.kubectl create deploy --image=dev.example.com/nginx invalid The error message is similar to this.error: failed to create deployment: deployments.apps "invalid" is forbidden: ValidatingAdmissionPolicy 'image-matches-namespace-environment.policy.example.com' with binding 'demo-binding-test.example.com' denied request: only prod images are allowed in namespace default API kinds exempt from admission validationThere are certain API kinds that are exempt from admission-time validation checks. For example, you can't create a ValidatingAdmissionPolicy that prevents changes to ValidatingAdmissionPolicyBindings.The list of exempt API kinds is:ValidatingAdmissionPoliciesValidatingAdmissionPolicyBindingsMutatingAdmissionPoliciesMutatingAdmissionPolicyBindingsTokenReviewsLocalSubjectAccessReviewsSelfSubjectAccessReviewsSelfSubjectReviews

Validating Admission Policy


Running Multiple Instances of Your AppObjectivesScale an existing app manually using kubectl.Scaling an applicationYou can create from the start a Deployment with multiple instances using the --replicas parameter for the kubectl create deployment command.Previously we created a Deployment, and then exposed it publicly via a Service. The Deployment created only one Pod for running our application. When traffic increases, we will need to scale the application to keep up with user demand.If you haven't worked through the earlier sections, start from Using minikube to create a cluster.Scaling is accomplished by changing the number of replicas in a Deployment.Note:If you are trying this after the previous section, then you may have deleted the service you created, or have created a Service of type: NodePort. In this section, it is assumed that a service with type: LoadBalancer is created for the kubernetes-bootcamp Deployment.If you have not deleted the Service created in the previous section, first delete that Service and then run the following command to create a new Service with its type set to LoadBalancer:kubectl expose deployment/kubernetes-bootcamp --type="LoadBalancer" --port 8080 Scaling overviewScaling is accomplished by changing the number of replicas in a Deployment.Scaling out a Deployment will ensure new Pods are created and scheduled to Nodes with available resources. Scaling will increase the number of Pods to the new desired state. Kubernetes also supports autoscaling of Pods, but it is outside of the scope of this tutorial. Scaling to zero is also possible, and it will terminate all Pods of the specified Deployment.Running multiple instances of an application will require a way to distribute the traffic to all of them. Services have an integrated load-balancer that will distribute network traffic to all Pods of an exposed Deployment. Services will monitor continuously the running Pods using endpoints, to ensure the traffic is sent only to available Pods.Once you have multiple instances of an application running, you would be able to do Rolling updates without downtime. We'll cover that in the

Running Multiple Instances of Your App


kubectl create secretSynopsisCreate a secret with specified type.A docker-registry type secret is for accessing a container registry.A generic type secret indicate an Opaque secret type.A tls type secret holds TLS certificate and its associated key.kubectl create secret (docker-registry | generic | tls) Options-h, --helphelp for secretParent Options Inherited--as stringUsername to impersonate for the operation. User could be a regular user or a service account in a namespace.--as-group stringsGroup to impersonate for the operation, this flag can be repeated to specify multiple groups.--as-uid stringUID to impersonate for the operation.--cache-dir string Default: "$HOME/.kube/cache"Default cache directory--certificate-authority stringPath to a cert file for the certificate authority--client-certificate stringPath to a client certificate file for TLS--client-key stringPath to a client key file for TLS--cluster stringThe name of the kubeconfig cluster to use--context stringThe name of the kubeconfig context to use--disable-compressionIf true, opt-out of response compression for all requests to the server--insecure-skip-tls-verifyIf true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure--kubeconfig stringPath to the kubeconfig file to use for CLI requests.--kuberc stringPath to the kuberc file to use for preferences. This can be disabled by exporting KUBECTL_KUBERC=false feature gate or turning off the feature KUBERC=off.--match-server-versionRequire server version to match client version-n, --namespace stringIf present, the namespace scope for this CLI request--password stringPassword for basic authentication to the API server--profile string Default: "none"Name of profile to capture. One of (none|cpu|heap|goroutine|threadcreate|block|mutex)--profile-output string Default: "profile.pprof"Name of the file to write the profile to--request-timeout string Default: "0"The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests.-s, --server stringThe address and port of the Kubernetes API server--storage-driver-buffer-duration duration Default: 1m0sWrites in the storage driver will be buffered for this duration, and committed to the non memory backends as a single transaction--storage-driver-db string Default: "cadvisor"database name--storage-driver-host string Default: "localhost:8086"database host:port--storage-driver-password string Default: "root"database password--storage-driver-secureuse secure connection with database--storage-driver-table string Default: "stats"table name--storage-driver-user string Default: "root"database username--tls-server-name stringServer name to use for server certificate validation. If it is not provided, the hostname used to contact the server is used--token stringBearer token for authentication to the API server--user stringThe name of the kubeconfig user to use--username stringUsername for basic authentication to the API server--version version[=true]--version, --version=raw prints version information and quits; --version=vX.Y.Z... sets the reported version--warnings-as-errorsTreat warnings received from the server as errors and exit with a non-zero exit codeSee Alsokubectl create - Create a resource from a file or from stdinkubectl create secret docker-registry - Create a secret for use with a Docker registrykubectl create secret generic - Create a secret from a local file, directory, or literal valuekubectl create secret tls - Create a TLS secretThis page is automatically generated.If you plan to report an issue with this page, mention that the page is auto-generated in your issue description. The fix may need to happen elsewhere in the Kubernetes project.

kubectl create secret


StatusStatus is a return value for calls that don't return other objects.import "k8s.io/apimachinery/pkg/apis/meta/v1"Status is a return value for calls that don't return other objects.apiVersion (string)APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resourcescode (int32)Suggested HTTP return code for this status, 0 if not set.details (StatusDetails)Atomic: will be replaced during a mergeExtended data associated with the reason. Each reason may define its own extended details. This field is optional and the data returned is not guaranteed to conform to any schema except that defined by the reason type.StatusDetails is a set of additional properties that MAY be set by the server to provide additional information about a response. The Reason field of a Status object defines what attributes will be set. Clients must ignore fields that do not match the defined type of each attribute, and should assume that any attribute may be empty, invalid, or under defined.details.causes ([]StatusCause)Atomic: will be replaced during a mergeThe Causes array includes more details associated with the StatusReason failure. Not all StatusReasons may provide detailed causes.StatusCause provides more information about an api.Status failure, including cases when multiple errors are encountered.details.causes.field (string)The field of the resource that has caused this error, as named by its JSON serialization. May include dot and postfix notation for nested attributes. Arrays are zero-indexed. Fields may appear more than once in an array of causes due to fields having multiple errors. Optional.Examples: "name" - the field "name" on the current resource "items[0].name" - the field "name" on the first array entry in "items"details.causes.message (string)A human-readable description of the cause of the error. This field may be presented as-is to a reader.details.causes.reason (string)A machine-readable description of the cause of the error. If this value is empty there is no information available.details.group (string)The group attribute of the resource associated with the status StatusReason.details.kind (string)The kind attribute of the resource associated with the status StatusReason. On some operations may differ from the requested resource Kind. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kindsdetails.name (string)The name attribute of the resource associated with the status StatusReason (when there is a single name which can be described).details.retryAfterSeconds (int32)If specified, the time in seconds before the operation should be retried. Some errors may indicate the client must take an alternate action - for those errors this field may indicate how long to wait before taking the alternate action.details.uid (string)UID of the resource. (when there is a single resource which can be described). More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names#uidskind (string)Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kindsmessage (string)A human-readable description of the status of this operation.metadata (ListMeta)Standard list metadata. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kindsreason (string)A machine-readable description of why this operation is in the "Failure" status. If this value is empty there is no information available. A Reason clarifies an HTTP status code but does not override it.status (string)Status of the operation. One of: "Success" or "Failure". More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-statusThis page is automatically generated.If you plan to report an issue with this page, mention that the page is auto-generated in your issue description. The fix may need to happen elsewhere in the Kubernetes project.

Status


kubectl debugSynopsisDebug cluster resources using interactive debugging containers.'debug' provides automation for common debugging tasks for cluster objects identified by resource and name. Pods will be used by default if no resource is specified.The action taken by 'debug' varies depending on what resource is specified. Supported actions include:Workload: Create a copy of an existing pod with certain attributes changed, for example changing the image tag to a new version.Workload: Add an ephemeral container to an already running pod, for example to add debugging utilities without restarting the pod.Node: Create a new pod that runs in the node's host namespaces and can access the node's filesystem.Note: When a non-root user is configured for the entire target Pod, some capabilities granted by debug profile may not work.kubectl debug (POD | TYPE[[.VERSION].GROUP]/NAME) [ -- COMMAND [args...] ] Examples # Create an interactive debugging session in pod mypod and immediately attach to it. kubectl debug mypod -it --image=busybox # Create an interactive debugging session for the pod in the file pod.yaml and immediately attach to it. # (requires the EphemeralContainers feature to be enabled in the cluster) kubectl debug -f pod.yaml -it --image=busybox # Create a debug container named debugger using a custom automated debugging image. kubectl debug --image=myproj/debug-tools -c debugger mypod # Create a copy of mypod adding a debug container and attach to it kubectl debug mypod -it --image=busybox --copy-to=my-debugger # Create a copy of mypod changing the command of mycontainer kubectl debug mypod -it --copy-to=my-debugger --container=mycontainer -- sh # Create a copy of mypod changing all container images to busybox kubectl debug mypod --copy-to=my-debugger --set-image=*=busybox # Create a copy of mypod adding a debug container and changing container images kubectl debug mypod -it --copy-to=my-debugger --image=debian --set-image=app=app:debug,sidecar=sidecar:debug # Create an interactive debugging session on a node and immediately attach to it. # The container will run in the host namespaces and the host's filesystem will be mounted at /host kubectl debug node/mynode -it --image=busybox Options--arguments-onlyIf specified, everything after -- will be passed to the new container as Args instead of Command.--attachIf true, wait for the container to start running, and then attach as if 'kubectl attach ...' were called. Default false, unless '-i/--stdin' is set, in which case the default is true.-c, --container stringContainer name to use for debug container.--copy-to stringCreate a copy of the target Pod with this name.--custom stringPath to a JSON or YAML file containing a partial container spec to customize built-in debug profiles.--env stringToString Default: []Environment variables to set in the container.-f, --filename stringsidentifying the resource to debug-h, --helphelp for debug--image stringContainer image to use for debug container.--image-pull-policy stringThe image pull policy for the container. If left empty, this value will not be specified by the client and defaulted by the server.--keep-annotationsIf true, keep the original pod annotations.(This flag only works when used with '--copy-to')--keep-init-containers Default: trueRun the init containers for the pod. Defaults to true.(This flag only works when used with '--copy-to')--keep-labelsIf true, keep the original pod labels.(This flag only works when used with '--copy-to')--keep-livenessIf true, keep the original pod liveness probes.(This flag only works when used with '--copy-to')--keep-readinessIf true, keep the original pod readiness probes.(This flag only works when used with '--copy-to')--keep-startupIf true, keep the original startup probes.(This flag only works when used with '--copy-to')--profile string Default: "legacy"Options are "legacy", "general", "baseline", "netadmin", "restricted" or "sysadmin".-q, --quietIf true, suppress informational messages.--replaceWhen used with '--copy-to', delete the original Pod.--same-nodeWhen used with '--copy-to', schedule the copy of target Pod on the same node.--set-image stringToString Default: []When used with '--copy-to', a list of name=image pairs for changing container images, similar to how 'kubectl set image' works.--share-processes Default: trueWhen used with '--copy-to', enable process namespace sharing in the copy.-i, --stdinKeep stdin open on the container(s) in the pod, even if nothing is attached.--target stringWhen using an ephemeral container, target processes in this container name.-t, --ttyAllocate a TTY for the debugging container.Parent Options Inherited--as stringUsername to impersonate for the operation. User could be a regular user or a service account in a namespace.--as-group stringsGroup to impersonate for the operation, this flag can be repeated to specify multiple groups.--as-uid stringUID to impersonate for the operation.--cache-dir string Default: "$HOME/.kube/cache"Default cache directory--certificate-authority stringPath to a cert file for the certificate authority--client-certificate stringPath to a client certificate file for TLS--client-key stringPath to a client key file for TLS--cluster stringThe name of the kubeconfig cluster to use--context stringThe name of the kubeconfig context to use--disable-compressionIf true, opt-out of response compression for all requests to the server--insecure-skip-tls-verifyIf true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure--kubeconfig stringPath to the kubeconfig file to use for CLI requests.--kuberc stringPath to the kuberc file to use for preferences. This can be disabled by exporting KUBECTL_KUBERC=false feature gate or turning off the feature KUBERC=off.--match-server-versionRequire server version to match client version-n, --namespace stringIf present, the namespace scope for this CLI request--password stringPassword for basic authentication to the API server--profile-output string Default: "profile.pprof"Name of the file to write the profile to--request-timeout string Default: "0"The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests.-s, --server stringThe address and port of the Kubernetes API server--storage-driver-buffer-duration duration Default: 1m0sWrites in the storage driver will be buffered for this duration, and committed to the non memory backends as a single transaction--storage-driver-db string Default: "cadvisor"database name--storage-driver-host string Default: "localhost:8086"database host:port--storage-driver-password string Default: "root"database password--storage-driver-secureuse secure connection with database--storage-driver-table string Default: "stats"table name--storage-driver-user string Default: "root"database username--tls-server-name stringServer name to use for server certificate validation. If it is not provided, the hostname used to contact the server is used--token stringBearer token for authentication to the API server--user stringThe name of the kubeconfig user to use--username stringUsername for basic authentication to the API server--version version[=true]--version, --version=raw prints version information and quits; --version=vX.Y.Z... sets the reported version--warnings-as-errorsTreat warnings received from the server as errors and exit with a non-zero exit codeSee Alsokubectl - kubectl controls the Kubernetes cluster managerThis page is automatically generated.If you plan to report an issue with this page, mention that the page is auto-generated in your issue description. The fix may need to happen elsewhere in the Kubernetes project.

kubectl debug


Dynamic Admission ControlIn addition to compiled-in admission plugins, admission plugins can be developed as extensions and run as webhooks configured at runtime. This page describes how to build, configure, use, and monitor admission webhooks.What are admission webhooks?Admission webhooks are HTTP callbacks that receive admission requests and do something with them. You can define two types of admission webhooks, validating admission webhook and mutating admission webhook. Mutating admission webhooks are invoked first, and can modify objects sent to the API server to enforce custom defaults. After all object modifications are complete, and after the incoming object is validated by the API server, validating admission webhooks are invoked and can reject requests to enforce custom policies.Note:Admission webhooks that need to guarantee they see the final state of the object in order to enforce policy should use a validating admission webhook, since objects can be modified after being seen by mutating webhooks.Experimenting with admission webhooksAdmission webhooks are essentially part of the cluster control-plane. You should write and deploy them with great caution. Please read the user guides for instructions if you intend to write/deploy production-grade admission webhooks. In the following, we describe how to quickly experiment with admission webhooks.PrerequisitesEnsure that MutatingAdmissionWebhook and ValidatingAdmissionWebhook admission controllers are enabled. Here is a recommended set of admission controllers to enable in general.Ensure that the admissionregistration.k8s.io/v1 API is enabled.Write an admission webhook serverPlease refer to the implementation of the admission webhook server that is validated in a Kubernetes e2e test. The webhook handles the AdmissionReview request sent by the API servers, and sends back its decision as an AdmissionReview object in the same version it received.See the webhook request section for details on the data sent to webhooks.See the webhook response section for the data expected from webhooks.The example admission webhook server leaves the ClientAuth field empty, which defaults to NoClientCert. This means that the webhook server does not authenticate the identity of the clients, supposedly API servers. If you need mutual TLS or other ways to authenticate the clients, see how to authenticate API servers.Deploy the admission webhook serviceThe webhook server in the e2e test is deployed in the Kubernetes cluster, via the deployment API. The test also creates a service as the front-end of the webhook server. See code.You may also deploy your webhooks outside of the cluster. You will need to update your webhook configurations accordingly.Configure admission webhooks on the flyYou can dynamically configure what resources are subject to what admission webhooks via ValidatingWebhookConfiguration or MutatingWebhookConfiguration.The following is an example ValidatingWebhookConfiguration, a mutating webhook configuration is similar. See the webhook configuration section for details about each config field.apiVersion: admissionregistration.k8s.io/v1 kind: ValidatingWebhookConfiguration metadata: name: "pod-policy.example.com" webhooks: - name: "pod-policy.example.com" rules: - apiGroups: [""] apiVersions: ["v1"] operations: ["CREATE"] resources: ["pods"] scope: "Namespaced" clientConfig: service: namespace: "example-namespace" name: "example-service" caBundle: <CA_BUNDLE> admissionReviewVersions: ["v1"] sideEffects: None timeoutSeconds: 5 Note:You must replace the <CA_BUNDLE> in the above example by a valid CA bundle which is a PEM-encoded (field value is Base64 encoded) CA bundle for validating the webhook's server certificate.The scope field specifies if only cluster-scoped resources ("Cluster") or namespace-scoped resources ("Namespaced") will match this rule. "∗" means that there are no scope restrictions.Note:When using clientConfig.service, the server cert must be valid for <svc_name>.<svc_namespace>.svc.Note:Default timeout for a webhook call is 10 seconds, You can set the timeout and it is encouraged to use a short timeout for webhooks. If the webhook call times out, the request is handled according to the webhook's failure policy.When an API server receives a request that matches one of the rules, the API server sends an admissionReview request to webhook as specified in the clientConfig.After you create the webhook configuration, the system will take a few seconds to honor the new configuration.Authenticate API serversIf your admission webhooks require authentication, you can configure the API servers to use basic auth, bearer token, or a cert to authenticate itself to the webhooks. There are three steps to complete the configuration.When starting the API server, specify the location of the admission control configuration file via the --admission-control-config-file flag.In the admission control configuration file, specify where the MutatingAdmissionWebhook controller and ValidatingAdmissionWebhook controller should read the credentials. The credentials are stored in kubeConfig files (yes, the same schema that's used by kubectl), so the field name is kubeConfigFile. Here is an example admission control configuration file:apiVersion: apiserver.config.k8s.io/v1 kind: AdmissionConfiguration plugins: - name: ValidatingAdmissionWebhook configuration: apiVersion: apiserver.config.k8s.io/v1 kind: WebhookAdmissionConfiguration kubeConfigFile: "<path-to-kubeconfig-file>" - name: MutatingAdmissionWebhook configuration: apiVersion: apiserver.config.k8s.io/v1 kind: WebhookAdmissionConfiguration kubeConfigFile: "<path-to-kubeconfig-file>" # Deprecated in v1.17 in favor of apiserver.config.k8s.io/v1 apiVersion: apiserver.k8s.io/v1alpha1 kind: AdmissionConfiguration plugins: - name: ValidatingAdmissionWebhook configuration: # Deprecated in v1.17 in favor of apiserver.config.k8s.io/v1, kind=WebhookAdmissionConfiguration apiVersion: apiserver.config.k8s.io/v1alpha1 kind: WebhookAdmission kubeConfigFile: "<path-to-kubeconfig-file>" - name: MutatingAdmissionWebhook configuration: # Deprecated in v1.17 in favor of apiserver.config.k8s.io/v1, kind=WebhookAdmissionConfiguration apiVersion: apiserver.config.k8s.io/v1alpha1 kind: WebhookAdmission kubeConfigFile: "<path-to-kubeconfig-file>" For more information about AdmissionConfiguration, see the AdmissionConfiguration (v1) reference. See the webhook configuration section for details about each config field.In the kubeConfig file, provide the credentials:apiVersion: v1 kind: Config users: # name should be set to the DNS name of the service or the host (including port) of the URL the webhook is configured to speak to. # If a non-443 port is used for services, it must be included in the name when configuring 1.16+ API servers. # # For a webhook configured to speak to a service on the default port (443), specify the DNS name of the service: # - name: webhook1.ns1.svc # user: ... # # For a webhook configured to speak to a service on non-default port (e.g. 8443), specify the DNS name and port of the service in 1.16+: # - name: webhook1.ns1.svc:8443 # user: ... # and optionally create a second stanza using only the DNS name of the service for compatibility with 1.15 API servers: # - name: webhook1.ns1.svc # user: ... # # For webhooks configured to speak to a URL, match the host (and port) specified in the webhook's URL. Examples: # A webhook with `url: https://www.example.com`: # - name: www.example.com # user: ... # # A webhook with `url: https://www.example.com:443`: # - name: www.example.com:443 # user: ... # # A webhook with `url: https://www.example.com:8443`: # - name: www.example.com:8443 # user: ... # - name: 'webhook1.ns1.svc' user: client-certificate-data: "<pem encoded certificate>" client-key-data: "<pem encoded key>" # The `name` supports using * to wildcard-match prefixing segments. - name: '*.webhook-company.org' user: password: "<password>" username: "<name>" # '*' is the default match. - name: '*' user: token: "<token>" Of course you need to set up the webhook server to handle these authentication requests.Webhook request and responseRequestWebhooks are sent as POST requests, with Content-Type: application/json, with an AdmissionReview API object in the admission.k8s.io API group serialized to JSON as the body.Webhooks can specify what versions of AdmissionReview objects they accept with the admissionReviewVersions field in their configuration:apiVersion: admissionregistration.k8s.io/v1 kind: ValidatingWebhookConfiguration webhooks: - name: my-webhook.example.com admissionReviewVersions: ["v1", "v1beta1"] admissionReviewVersions is a required field when creating webhook configurations. Webhooks are required to support at least one AdmissionReview version understood by the current and previous API server.API servers send the first AdmissionReview version in the admissionReviewVersions list they support. If none of the versions in the list are supported by the API server, the configuration will not be allowed to be created. If an API server encounters a webhook configuration that was previously created and does not support any of the AdmissionReview versions the API server knows how to send, attempts to call to the webhook will fail and be subject to the failure policy.This example shows the data contained in an AdmissionReview object for a request to update the scale subresource of an apps/v1 Deployment:{ "apiVersion": "admission.k8s.io/v1", "kind": "AdmissionReview", "request": { # Random uid uniquely identifying this admission call "uid": "705ab4f5-6393-11e8-b7cc-42010a800002", # Fully-qualified group/version/kind of the incoming object "kind": { "group": "autoscaling", "version": "v1", "kind": "Scale" }, # Fully-qualified group/version/kind of the resource being modified "resource": { "group": "apps", "version": "v1", "resource": "deployments" }, # Subresource, if the request is to a subresource "subResource": "scale", # Fully-qualified group/version/kind of the incoming object in the original request to the API server # This only differs from `kind` if the webhook specified `matchPolicy: Equivalent` and the original # request to the API server was converted to a version the webhook registered for "requestKind": { "group": "autoscaling", "version": "v1", "kind": "Scale" }, # Fully-qualified group/version/kind of the resource being modified in the original request to the API server # This only differs from `resource` if the webhook specified `matchPolicy: Equivalent` and the original # request to the API server was converted to a version the webhook registered for "requestResource": { "group": "apps", "version": "v1", "resource": "deployments" }, # Subresource, if the request is to a subresource # This only differs from `subResource` if the webhook specified `matchPolicy: Equivalent` and the original # request to the API server was converted to a version the webhook registered for "requestSubResource": "scale", # Name of the resource being modified "name": "my-deployment", # Namespace of the resource being modified, if the resource is namespaced (or is a Namespace object) "namespace": "my-namespace", # operation can be CREATE, UPDATE, DELETE, or CONNECT "operation": "UPDATE", "userInfo": { # Username of the authenticated user making the request to the API server "username": "admin", # UID of the authenticated user making the request to the API server "uid": "014fbff9a07c", # Group memberships of the authenticated user making the request to the API server "groups": [ "system:authenticated", "my-admin-group" ], # Arbitrary extra info associated with the user making the request to the API server # This is populated by the API server authentication layer "extra": { "some-key": [ "some-value1", "some-value2" ] } }, # object is the new object being admitted. It is null for DELETE operations "object": { "apiVersion": "autoscaling/v1", "kind": "Scale" }, # oldObject is the existing object. It is null for CREATE and CONNECT operations "oldObject": { "apiVersion": "autoscaling/v1", "kind": "Scale" }, # options contain the options for the operation being admitted, like meta.k8s.io/v1 CreateOptions, # UpdateOptions, or DeleteOptions. It is null for CONNECT operations "options": { "apiVersion": "meta.k8s.io/v1", "kind": "UpdateOptions" }, # dryRun indicates the API request is running in dry run mode and will not be persisted # Webhooks with side effects should avoid actuating those side effects when dryRun is true "dryRun": false } } ResponseWebhooks respond with a 200 HTTP status code, Content-Type: application/json, and a body containing an AdmissionReview object (in the same version they were sent), with the response stanza populated, serialized to JSON.At a minimum, the response stanza must contain the following fields:uid, copied from the request.uid sent to the webhookallowed, either set to true or falseExample of a minimal response from a webhook to allow a request:{ "apiVersion": "admission.k8s.io/v1", "kind": "AdmissionReview", "response": { "uid": "<value from request.uid>", "allowed": true } } Example of a minimal response from a webhook to forbid a request:{ "apiVersion": "admission.k8s.io/v1", "kind": "AdmissionReview", "response": { "uid": "<value from request.uid>", "allowed": false } } When rejecting a request, the webhook can customize the http code and message returned to the user using the status field. The specified status object is returned to the user. See the API documentation for details about the status type. Example of a response to forbid a request, customizing the HTTP status code and message presented to the user:{ "apiVersion": "admission.k8s.io/v1", "kind": "AdmissionReview", "response": { "uid": "<value from request.uid>", "allowed": false, "status": { "code": 403, "message": "You cannot do this because it is Tuesday and your name starts with A" } } } When allowing a request, a mutating admission webhook may optionally modify the incoming object as well. This is done using the patch and patchType fields in the response. The only currently supported patchType is JSONPatch. See JSON patch documentation for more details. For patchType: JSONPatch, the patch field contains a base64-encoded array of JSON patch operations.As an example, a single patch operation that would set spec.replicas would be [{"op": "add", "path": "/spec/replicas", "value": 3}]Base64-encoded, this would be W3sib3AiOiAiYWRkIiwgInBhdGgiOiAiL3NwZWMvcmVwbGljYXMiLCAidmFsdWUiOiAzfV0=So a webhook response to add that label would be:{ "apiVersion": "admission.k8s.io/v1", "kind": "AdmissionReview", "response": { "uid": "<value from request.uid>", "allowed": true, "patchType": "JSONPatch", "patch": "W3sib3AiOiAiYWRkIiwgInBhdGgiOiAiL3NwZWMvcmVwbGljYXMiLCAidmFsdWUiOiAzfV0=" } } Admission webhooks can optionally return warning messages that are returned to the requesting client in HTTP Warning headers with a warning code of 299. Warnings can be sent with allowed or rejected admission responses.If you're implementing a webhook that returns a warning:Don't include a "Warning:" prefix in the messageUse warning messages to describe problems the client making the API request should correct or be aware ofLimit warnings to 120 characters if possibleCaution:Individual warning messages over 256 characters may be truncated by the API server before being returned to clients. If more than 4096 characters of warning messages are added (from all sources), additional warning messages are ignored.{ "apiVersion": "admission.k8s.io/v1", "kind": "AdmissionReview", "response": { "uid": "<value from request.uid>", "allowed": true, "warnings": [ "duplicate envvar entries specified with name MY_ENV", "memory request less than 4MB specified for container mycontainer, which will not start successfully" ] } } Webhook configurationTo register admission webhooks, create MutatingWebhookConfiguration or ValidatingWebhookConfiguration API objects. The name of a MutatingWebhookConfiguration or a ValidatingWebhookConfiguration object must be a valid DNS subdomain name.Each configuration can contain one or more webhooks. If multiple webhooks are specified in a single configuration, each must be given a unique name. This is required in order to make resulting audit logs and metrics easier to match up to active configurations.Each webhook defines the following things.Matching requests: rulesEach webhook must specify a list of rules used to determine if a request to the API server should be sent to the webhook. Each rule specifies one or more operations, apiGroups, apiVersions, and resources, and a resource scope:operations lists one or more operations to match. Can be "CREATE", "UPDATE", "DELETE", "CONNECT", or "*" to match all.apiGroups lists one or more API groups to match. "" is the core API group. "*" matches all API groups.apiVersions lists one or more API versions to match. "*" matches all API versions.resources lists one or more resources to match."*" matches all resources, but not subresources."*/*" matches all resources and subresources."pods/*" matches all subresources of pods."*/status" matches all status subresources.scope specifies a scope to match. Valid values are "Cluster", "Namespaced", and "*". Subresources match the scope of their parent resource. Default is "*"."Cluster" means that only cluster-scoped resources will match this rule (Namespace API objects are cluster-scoped)."Namespaced" means that only namespaced resources will match this rule."*" means that there are no scope restrictions.If an incoming request matches one of the specified operations, groups, versions, resources, and scope for any of a webhook's rules, the request is sent to the webhook.Here are other examples of rules that could be used to specify which resources should be intercepted.Match CREATE or UPDATE requests to apps/v1 and apps/v1beta1 deployments and replicasets:apiVersion: admissionregistration.k8s.io/v1 kind: ValidatingWebhookConfiguration ... webhooks: - name: my-webhook.example.com rules: - operations: ["CREATE", "UPDATE"] apiGroups: ["apps"] apiVersions: ["v1", "v1beta1"] resources: ["deployments", "replicasets"] scope: "Namespaced" ... Match create requests for all resources (but not subresources) in all API groups and versions:apiVersion: admissionregistration.k8s.io/v1 kind: ValidatingWebhookConfiguration webhooks: - name: my-webhook.example.com rules: - operations: ["CREATE"] apiGroups: ["*"] apiVersions: ["*"] resources: ["*"] scope: "*" Match update requests for all status subresources in all API groups and versions:apiVersion: admissionregistration.k8s.io/v1 kind: ValidatingWebhookConfiguration webhooks: - name: my-webhook.example.com rules: - operations: ["UPDATE"] apiGroups: ["*"] apiVersions: ["*"] resources: ["*/status"] scope: "*" Matching requests: objectSelectorWebhooks may optionally limit which requests are intercepted based on the labels of the objects they would be sent, by specifying an objectSelector. If specified, the objectSelector is evaluated against both the object and oldObject that would be sent to the webhook, and is considered to match if either object matches the selector.A null object (oldObject in the case of create, or newObject in the case of delete), or an object that cannot have labels (like a DeploymentRollback or a PodProxyOptions object) is not considered to match.Use the object selector only if the webhook is opt-in, because end users may skip the admission webhook by setting the labels.This example shows a mutating webhook that would match a CREATE of any resource (but not subresources) with the label foo: bar:apiVersion: admissionregistration.k8s.io/v1 kind: MutatingWebhookConfiguration webhooks: - name: my-webhook.example.com objectSelector: matchLabels: foo: bar rules: - operations: ["CREATE"] apiGroups: ["*"] apiVersions: ["*"] resources: ["*"] scope: "*" See labels concept for more examples of label selectors.Matching requests: namespaceSelectorWebhooks may optionally limit which requests for namespaced resources are intercepted, based on the labels of the containing namespace, by specifying a namespaceSelector.The namespaceSelector decides whether to run the webhook on a request for a namespaced resource (or a Namespace object), based on whether the namespace's labels match the selector. If the object itself is a namespace, the matching is performed on object.metadata.labels. If the object is a cluster scoped resource other than a Namespace, namespaceSelector has no effect.This example shows a mutating webhook that matches a CREATE of any namespaced resource inside a namespace that does not have a "runlevel" label of "0" or "1":apiVersion: admissionregistration.k8s.io/v1 kind: MutatingWebhookConfiguration webhooks: - name: my-webhook.example.com namespaceSelector: matchExpressions: - key: runlevel operator: NotIn values: ["0","1"] rules: - operations: ["CREATE"] apiGroups: ["*"] apiVersions: ["*"] resources: ["*"] scope: "Namespaced" This example shows a validating webhook that matches a CREATE of any namespaced resource inside a namespace that is associated with the "environment" of "prod" or "staging":apiVersion: admissionregistration.k8s.io/v1 kind: ValidatingWebhookConfiguration webhooks: - name: my-webhook.example.com namespaceSelector: matchExpressions: - key: environment operator: In values: ["prod","staging"] rules: - operations: ["CREATE"] apiGroups: ["*"] apiVersions: ["*"] resources: ["*"] scope: "Namespaced" See labels concept for more examples of label selectors.Matching requests: matchPolicyAPI servers can make objects available via multiple API groups or versions.For example, if a webhook only specified a rule for some API groups/versions (like apiGroups:["apps"], apiVersions:["v1","v1beta1"]), and a request was made to modify the resource via another API group/version (like extensions/v1beta1), the request would not be sent to the webhook.The matchPolicy lets a webhook define how its rules are used to match incoming requests. Allowed values are Exact or Equivalent.Exact means a request should be intercepted only if it exactly matches a specified rule.Equivalent means a request should be intercepted if it modifies a resource listed in rules, even via another API group or version.In the example given above, the webhook that only registered for apps/v1 could use matchPolicy:matchPolicy: Exact would mean the extensions/v1beta1 request would not be sent to the webhookmatchPolicy: Equivalent means the extensions/v1beta1 request would be sent to the webhook (with the objects converted to a version the webhook had specified: apps/v1)Specifying Equivalent is recommended, and ensures that webhooks continue to intercept the resources they expect when upgrades enable new versions of the resource in the API server.When a resource stops being served by the API server, it is no longer considered equivalent to other versions of that resource that are still served. For example, extensions/v1beta1 deployments were first deprecated and then removed (in Kubernetes v1.16).Since that removal, a webhook with a apiGroups:["extensions"], apiVersions:["v1beta1"], resources:["deployments"] rule does not intercept deployments created via apps/v1 APIs. For that reason, webhooks should prefer registering for stable versions of resources.This example shows a validating webhook that intercepts modifications to deployments (no matter the API group or version), and is always sent an apps/v1 Deployment object:apiVersion: admissionregistration.k8s.io/v1 kind: ValidatingWebhookConfiguration webhooks: - name: my-webhook.example.com matchPolicy: Equivalent rules: - operations: ["CREATE","UPDATE","DELETE"] apiGroups: ["apps"] apiVersions: ["v1"] resources: ["deployments"] scope: "Namespaced" The matchPolicy for an admission webhooks defaults to Equivalent.Matching requests: matchConditionsFEATURE STATE: Kubernetes v1.30 [stable] (enabled by default: true)You can define match conditions for webhooks if you need fine-grained request filtering. These conditions are useful if you find that match rules, objectSelectors and namespaceSelectors still doesn't provide the filtering you want over when to call out over HTTP. Match conditions are CEL expressions. All match conditions must evaluate to true for the webhook to be called.Here is an example illustrating a few different uses for match conditions:apiVersion: admissionregistration.k8s.io/v1 kind: ValidatingWebhookConfiguration webhooks: - name: my-webhook.example.com matchPolicy: Equivalent rules: - operations: ['CREATE','UPDATE'] apiGroups: ['*'] apiVersions: ['*'] resources: ['*'] failurePolicy: 'Ignore' # Fail-open (optional) sideEffects: None clientConfig: service: namespace: my-namespace name: my-webhook caBundle: '<omitted>' # You can have up to 64 matchConditions per webhook matchConditions: - name: 'exclude-leases' # Each match condition must have a unique name expression: '!(request.resource.group == "coordination.k8s.io" && request.resource.resource == "leases")' # Match non-lease resources. - name: 'exclude-kubelet-requests' expression: '!("system:nodes" in request.userInfo.groups)' # Match requests made by non-node users. - name: 'rbac' # Skip RBAC requests, which are handled by the second webhook. expression: 'request.resource.group != "rbac.authorization.k8s.io"' # This example illustrates the use of the 'authorizer'. The authorization check is more expensive # than a simple expression, so in this example it is scoped to only RBAC requests by using a second # webhook. Both webhooks can be served by the same endpoint. - name: rbac.my-webhook.example.com matchPolicy: Equivalent rules: - operations: ['CREATE','UPDATE'] apiGroups: ['rbac.authorization.k8s.io'] apiVersions: ['*'] resources: ['*'] failurePolicy: 'Fail' # Fail-closed (the default) sideEffects: None clientConfig: service: namespace: my-namespace name: my-webhook caBundle: '<omitted>' # You can have up to 64 matchConditions per webhook matchConditions: - name: 'breakglass' # Skip requests made by users authorized to 'breakglass' on this webhook. # The 'breakglass' API verb does not need to exist outside this check. expression: '!authorizer.group("admissionregistration.k8s.io").resource("validatingwebhookconfigurations").name("my-webhook.example.com").check("breakglass").allowed()' Note:You can define up to 64 elements in the matchConditions field per webhook.Match conditions have access to the following CEL variables:object - The object from the incoming request. The value is null for DELETE requests. The object version may be converted based on the matchPolicy.oldObject - The existing object. The value is null for CREATE requests.request - The request portion of the AdmissionReview, excluding object and oldObject.authorizer - A CEL Authorizer. May be used to perform authorization checks for the principal (authenticated user) of the request. See Authz in the Kubernetes CEL library documentation for more details.authorizer.requestResource - A shortcut for an authorization check configured with the request resource (group, resource, (subresource), namespace, name).For more information on CEL expressions, refer to the Common Expression Language in Kubernetes reference.In the event of an error evaluating a match condition the webhook is never called. Whether to reject the request is determined as follows:If any match condition evaluated to false (regardless of other errors), the API server skips the webhook.Otherwise:for failurePolicy: Fail, reject the request (without calling the webhook).for failurePolicy: Ignore, proceed with the request but skip the webhook.Contacting the webhookOnce the API server has determined a request should be sent to a webhook, it needs to know how to contact the webhook. This is specified in the clientConfig stanza of the webhook configuration.Webhooks can either be called via a URL or a service reference, and can optionally include a custom CA bundle to use to verify the TLS connection.URLurl gives the location of the webhook, in standard URL form (scheme://host:port/path).The host should not refer to a service running in the cluster; use a service reference by specifying the service field instead. The host might be resolved via external DNS in some API servers (e.g., kube-apiserver cannot resolve in-cluster DNS as that would be a layering violation). host may also be an IP address.Please note that using localhost or 127.0.0.1 as a host is risky unless you take great care to run this webhook on all hosts which run an API server which might need to make calls to this webhook. Such installations are likely to be non-portable or not readily run in a new cluster.The scheme must be "https"; the URL must begin with "https://".Attempting to use a user or basic auth (for example user:password@) is not allowed. Fragments (#...) and query parameters (?...) are also not allowed.Here is an example of a mutating webhook configured to call a URL (and expects the TLS certificate to be verified using system trust roots, so does not specify a caBundle):apiVersion: admissionregistration.k8s.io/v1 kind: MutatingWebhookConfiguration webhooks: - name: my-webhook.example.com clientConfig: url: "https://my-webhook.example.com:9443/my-webhook-path" Service referenceThe service stanza inside clientConfig is a reference to the service for this webhook. If the webhook is running within the cluster, then you should use service instead of url. The service namespace and name are required. The port is optional and defaults to 443. The path is optional and defaults to "/".Here is an example of a mutating webhook configured to call a service on port "1234" at the subpath "/my-path", and to verify the TLS connection against the ServerName my-service-name.my-service-namespace.svc using a custom CA bundle:apiVersion: admissionregistration.k8s.io/v1 kind: MutatingWebhookConfiguration webhooks: - name: my-webhook.example.com clientConfig: caBundle: <CA_BUNDLE> service: namespace: my-service-namespace name: my-service-name path: /my-path port: 1234 Note:You must replace the <CA_BUNDLE> in the above example by a valid CA bundle which is a PEM-encoded CA bundle for validating the webhook's server certificate.Side effectsWebhooks typically operate only on the content of the AdmissionReview sent to them. Some webhooks, however, make out-of-band changes as part of processing admission requests.Webhooks that make out-of-band changes ("side effects") must also have a reconciliation mechanism (like a controller) that periodically determines the actual state of the world, and adjusts the out-of-band data modified by the admission webhook to reflect reality. This is because a call to an admission webhook does not guarantee the admitted object will be persisted as is, or at all. Later webhooks can modify the content of the object, a conflict could be encountered while writing to storage, or the server could power off before persisting the object.Additionally, webhooks with side effects must skip those side-effects when dryRun: true admission requests are handled. A webhook must explicitly indicate that it will not have side-effects when run with dryRun, or the dry-run request will not be sent to the webhook and the API request will fail instead.Webhooks indicate whether they have side effects using the sideEffects field in the webhook configuration:None: calling the webhook will have no side effects.NoneOnDryRun: calling the webhook will possibly have side effects, but if a request with dryRun: true is sent to the webhook, the webhook will suppress the side effects (the webhook is dryRun-aware).Here is an example of a validating webhook indicating it has no side effects on dryRun: true requests:apiVersion: admissionregistration.k8s.io/v1 kind: ValidatingWebhookConfiguration webhooks: - name: my-webhook.example.com sideEffects: NoneOnDryRun TimeoutsBecause webhooks add to API request latency, they should evaluate as quickly as possible. timeoutSeconds allows configuring how long the API server should wait for a webhook to respond before treating the call as a failure.If the timeout expires before the webhook responds, the webhook call will be ignored or the API call will be rejected based on the failure policy.The timeout value must be between 1 and 30 seconds.Here is an example of a validating webhook with a custom timeout of 2 seconds:apiVersion: admissionregistration.k8s.io/v1 kind: ValidatingWebhookConfiguration webhooks: - name: my-webhook.example.com timeoutSeconds: 2 The timeout for an admission webhook defaults to 10 seconds.Reinvocation policyA single ordering of mutating admissions plugins (including webhooks) does not work for all cases (see https://issue.k8s.io/64333 as an example). A mutating webhook can add a new sub-structure to the object (like adding a container to a pod), and other mutating plugins which have already run may have opinions on those new structures (like setting an imagePullPolicy on all containers).To allow mutating admission plugins to observe changes made by other plugins, built-in mutating admission plugins are re-run if a mutating webhook modifies an object, and mutating webhooks can specify a reinvocationPolicy to control whether they are reinvoked as well.reinvocationPolicy may be set to Never or IfNeeded. It defaults to Never.Never: the webhook must not be called more than once in a single admission evaluation.IfNeeded: the webhook may be called again as part of the admission evaluation if the object being admitted is modified by other admission plugins after the initial webhook call.The important elements to note are:The number of additional invocations is not guaranteed to be exactly one.If additional invocations result in further modifications to the object, webhooks are not guaranteed to be invoked again.Webhooks that use this option may be reordered to minimize the number of additional invocations.To validate an object after all mutations are guaranteed complete, use a validating admission webhook instead (recommended for webhooks with side-effects).Here is an example of a mutating webhook opting into being re-invoked if later admission plugins modify the object:apiVersion: admissionregistration.k8s.io/v1 kind: MutatingWebhookConfiguration webhooks: - name: my-webhook.example.com reinvocationPolicy: IfNeeded Mutating webhooks must be idempotent, able to successfully process an object they have already admitted and potentially modified. This is true for all mutating admission webhooks, since any change they can make in an object could already exist in the user-provided object, but it is essential for webhooks that opt into reinvocation.Failure policyfailurePolicy defines how unrecognized errors and timeout errors from the admission webhook are handled. Allowed values are Ignore or Fail.Ignore means that an error calling the webhook is ignored and the API request is allowed to continue.Fail means that an error calling the webhook causes the admission to fail and the API request to be rejected.Here is a mutating webhook configured to reject an API request if errors are encountered calling the admission webhook:apiVersion: admissionregistration.k8s.io/v1 kind: MutatingWebhookConfiguration webhooks: - name: my-webhook.example.com failurePolicy: Fail The default failurePolicy for an admission webhooks is Fail.Monitoring admission webhooksThe API server provides ways to monitor admission webhook behaviors. These monitoring mechanisms help cluster admins to answer questions like:Which mutating webhook mutated the object in a API request?What change did the mutating webhook applied to the object?Which webhooks are frequently rejecting API requests? What's the reason for a rejection?Mutating webhook auditing annotationsSometimes it's useful to know which mutating webhook mutated the object in a API request, and what change did the webhook apply.The Kubernetes API server performs auditing on each mutating webhook invocation. Each invocation generates an auditing annotation capturing if a request object is mutated by the invocation, and optionally generates an annotation capturing the applied patch from the webhook admission response. The annotations are set in the audit event for given request on given stage of its execution, which is then pre-processed according to a certain policy and written to a backend.The audit level of a event determines which annotations get recorded:At Metadata audit level or higher, an annotation with key mutation.webhook.admission.k8s.io/round_{round idx}_index_{order idx} gets logged with JSON payload indicating a webhook gets invoked for given request and whether it mutated the object or not.For example, the following annotation gets recorded for a webhook being reinvoked. The webhook is ordered the third in the mutating webhook chain, and didn't mutated the request object during the invocation.# the audit event recorded { "kind": "Event", "apiVersion": "audit.k8s.io/v1", "annotations": { "mutation.webhook.admission.k8s.io/round_1_index_2": "{\"configuration\":\"my-mutating-webhook-configuration.example.com\",\"webhook\":\"my-webhook.example.com\",\"mutated\": false}" # other annotations ... } # other fields ... } # the annotation value deserialized { "configuration": "my-mutating-webhook-configuration.example.com", "webhook": "my-webhook.example.com", "mutated": false } The following annotation gets recorded for a webhook being invoked in the first round. The webhook is ordered the first in the mutating webhook chain, and mutated the request object during the invocation.# the audit event recorded { "kind": "Event", "apiVersion": "audit.k8s.io/v1", "annotations": { "mutation.webhook.admission.k8s.io/round_0_index_0": "{\"configuration\":\"my-mutating-webhook-configuration.example.com\",\"webhook\":\"my-webhook-always-mutate.example.com\",\"mutated\": true}" # other annotations ... } # other fields ... } # the annotation value deserialized { "configuration": "my-mutating-webhook-configuration.example.com", "webhook": "my-webhook-always-mutate.example.com", "mutated": true } At Request audit level or higher, an annotation with key patch.webhook.admission.k8s.io/round_{round idx}_index_{order idx} gets logged with JSON payload indicating a webhook gets invoked for given request and what patch gets applied to the request object.For example, the following annotation gets recorded for a webhook being reinvoked. The webhook is ordered the fourth in the mutating webhook chain, and responded with a JSON patch which got applied to the request object.# the audit event recorded { "kind": "Event", "apiVersion": "audit.k8s.io/v1", "annotations": { "patch.webhook.admission.k8s.io/round_1_index_3": "{\"configuration\":\"my-other-mutating-webhook-configuration.example.com\",\"webhook\":\"my-webhook-always-mutate.example.com\",\"patch\":[{\"op\":\"add\",\"path\":\"/data/mutation-stage\",\"value\":\"yes\"}],\"patchType\":\"JSONPatch\"}" # other annotations ... } # other fields ... } # the annotation value deserialized { "configuration": "my-other-mutating-webhook-configuration.example.com", "webhook": "my-webhook-always-mutate.example.com", "patchType": "JSONPatch", "patch": [ { "op": "add", "path": "/data/mutation-stage", "value": "yes" } ] } Admission webhook metricsThe API server exposes Prometheus metrics from the /metrics endpoint, which can be used for monitoring and diagnosing API server status. The following metrics record status related to admission webhooks.API server admission webhook rejection countSometimes it's useful to know which admission webhooks are frequently rejecting API requests, and the reason for a rejection.The API server exposes a Prometheus counter metric recording admission webhook rejections. The metrics are labelled to identify the causes of webhook rejection(s):name: the name of the webhook that rejected a request.operation: the operation type of the request, can be one of CREATE, UPDATE, DELETE and CONNECT.type: the admission webhook type, can be one of admit and validating.error_type: identifies if an error occurred during the webhook invocation that caused the rejection. Its value can be one of:calling_webhook_error: unrecognized errors or timeout errors from the admission webhook happened and the webhook's Failure policy is set to Fail.no_error: no error occurred. The webhook rejected the request with allowed: false in the admission response. The metrics label rejection_code records the .status.code set in the admission response.apiserver_internal_error: an API server internal error happened.rejection_code: the HTTP status code set in the admission response when a webhook rejected a request.Example of the rejection count metrics:# HELP apiserver_admission_webhook_rejection_count [ALPHA] Admission webhook rejection count, identified by name and broken out for each admission type (validating or admit) and operation. Additional labels specify an error type (calling_webhook_error or apiserver_internal_error if an error occurred; no_error otherwise) and optionally a non-zero rejection code if the webhook rejects the request with an HTTP status code (honored by the apiserver when the code is greater or equal to 400). Codes greater than 600 are truncated to 600, to keep the metrics cardinality bounded. # TYPE apiserver_admission_webhook_rejection_count counter apiserver_admission_webhook_rejection_count{error_type="calling_webhook_error",name="always-timeout-webhook.example.com",operation="CREATE",rejection_code="0",type="validating"} 1 apiserver_admission_webhook_rejection_count{error_type="calling_webhook_error",name="invalid-admission-response-webhook.example.com",operation="CREATE",rejection_code="0",type="validating"} 1 apiserver_admission_webhook_rejection_count{error_type="no_error",name="deny-unwanted-configmap-data.example.com",operation="CREATE",rejection_code="400",type="validating"} 13 Best practices and warningsFor recommendations and considerations when writing mutating admission webhooks, see Admission Webhooks Good Practices.

Dynamic Admission Control