# Seccomp and Kubernetes

Seccomp and KubernetesSeccomp stands for secure computing mode and has been a feature of the Linux kernel since version 2.6.12. It can be used to sandbox the privileges of a process, restricting the calls it is able to make from userspace into the kernel. Kubernetes lets you automatically apply seccomp profiles loaded onto a node to your Pods and containers.Seccomp fieldsFEATURE STATE: Kubernetes v1.19 [stable]There are four ways to specify a seccomp profile for a pod:for the whole Pod using spec.securityContext.seccompProfilefor a single container using spec.containers[*].securityContext.seccompProfilefor an (restartable / sidecar) init container using spec.initContainers[*].securityContext.seccompProfilefor an ephemeral container using spec.ephemeralContainers[*].securityContext.seccompProfilepods/security/seccomp/fields.yaml apiVersion: v1 kind: Pod metadata: name: pod spec: securityContext: seccompProfile: type: Unconfined ephemeralContainers: - name: ephemeral-container image: debian securityContext: seccompProfile: type: RuntimeDefault initContainers: - name: init-container image: debian securityContext: seccompProfile: type: RuntimeDefault containers: - name: container image: docker.io/library/debian:stable securityContext: seccompProfile: type: Localhost localhostProfile: my-profile.json The Pod in the example above runs as Unconfined, while the ephemeral-container and init-container specifically defines RuntimeDefault. If the ephemeral or init container would not have set the securityContext.seccompProfile field explicitly, then the value would be inherited from the Pod. The same applies to the container, which runs a Localhost profile my-profile.json.Generally speaking, fields from (ephemeral) containers have a higher priority than the Pod level value, while containers which do not set the seccomp field inherit the profile from the Pod.Note:It is not possible to apply a seccomp profile to a Pod or container running with privileged: true set in the container's securityContext. Privileged containers always run as Unconfined.The following values are possible for the seccompProfile.type:UnconfinedThe workload runs without any seccomp restrictions.RuntimeDefaultA default seccomp profile defined by the container runtime is applied. The default profiles aim to provide a strong set of security defaults while preserving the functionality of the workload. It is possible that the default profiles differ between container runtimes and their release versions, for example when comparing those from CRI-O and containerd.LocalhostThe localhostProfile will be applied, which has to be available on the node disk (on Linux it's /var/lib/kubelet/seccomp). The availability of the seccomp profile is verified by the container runtime on container creation. If the profile does not exist, then the container creation will fail with a CreateContainerError.Localhost profilesSeccomp profiles are JSON files following the scheme defined by the OCI runtime specification. A profile basically defines actions based on matched syscalls, but also allows to pass specific values as arguments to syscalls. For example:{ "defaultAction": "SCMP_ACT_ERRNO", "defaultErrnoRet": 38, "syscalls": [ { "names": [ "adjtimex", "alarm", "bind", "waitid", "waitpid", "write", "writev" ], "action": "SCMP_ACT_ALLOW" } ] } The defaultAction in the profile above is defined as SCMP_ACT_ERRNO and will return as fallback to the actions defined in syscalls. The error is defined as code 38 via the defaultErrnoRet field.The following actions are generally possible:SCMP_ACT_ERRNOReturn the specified error code.SCMP_ACT_ALLOWAllow the syscall to be executed.SCMP_ACT_KILL_PROCESSKill the process.SCMP_ACT_KILL_THREAD and SCMP_ACT_KILLKill only the thread.SCMP_ACT_TRAPThrow a SIGSYS signal.SCMP_ACT_NOTIFY and SECCOMP_RET_USER_NOTIF.Notify the user space.SCMP_ACT_TRACENotify a tracing process with the specified value.SCMP_ACT_LOGAllow the syscall to be executed after the action has been logged to syslog or auditd.Some actions like SCMP_ACT_NOTIFY or SECCOMP_RET_USER_NOTIF may be not supported depending on the container runtime, OCI runtime or Linux kernel version being used. There may be also further limitations, for example that SCMP_ACT_NOTIFY cannot be used as defaultAction or for certain syscalls like write. All those limitations are defined by either the OCI runtime (runc, crun) or libseccomp.The syscalls JSON array contains a list of objects referencing syscalls by their respective names. For example, the action SCMP_ACT_ALLOW can be used to create a whitelist of allowed syscalls as outlined in the example above. It would also be possible to define another list using the action SCMP_ACT_ERRNO but a different return (errnoRet) value.It is also possible to specify the arguments (args) passed to certain syscalls. More information about those advanced use cases can be found in the OCI runtime spec and the Seccomp Linux kernel documentation.Further readingRestrict a Container's Syscalls with seccompPod Security Standards


# Kubernetes API Concepts

Kubernetes API ConceptsThe Kubernetes API is a resource-based (RESTful) programmatic interface provided via HTTP. It supports retrieving, creating, updating, and deleting primary resources via the standard HTTP verbs (POST, PUT, PATCH, DELETE, GET).For some resources, the API includes additional subresources that allow fine-grained authorization (such as separate views for Pod details and log retrievals), and can accept and serve those resources in different representations for convenience or efficiency.Kubernetes supports efficient change notifications on resources via watches:in the Kubernetes API, watch is a verb that is used to track changes to an object in Kubernetes as a stream. It is used for the efficient detection of changes.Kubernetes also provides consistent list operations so that API clients can effectively cache, track, and synchronize the state of resources.You can view the API reference online, or read on to learn about the API in general.Kubernetes API terminologyKubernetes generally leverages common RESTful terminology to describe the API concepts:A resource type is the name used in the URL (pods, namespaces, services)All resource types have a concrete representation (their object schema) which is called a kindA list of instances of a resource type is known as a collectionA single instance of a resource type is called a resource, and also usually represents an objectFor some resource types, the API includes one or more sub-resources, which are represented as URI paths below the resourceMost Kubernetes API resource types are objects – they represent a concrete instance of a concept on the cluster, like a pod or namespace. A smaller number of API resource types are virtual in that they often represent operations on objects, rather than objects, such as a permission check (use a POST with a JSON-encoded body of SubjectAccessReview to the subjectaccessreviews resource), or the eviction sub-resource of a Pod (used to trigger API-initiated eviction).Object namesAll objects you can create via the API have a unique object name to allow idempotent creation and retrieval, except that virtual resource types may not have unique names if they are not retrievable, or do not rely on idempotency. Within a namespace, only one object of a given kind can have a given name at a time. However, if you delete the object, you can make a new object with the same name. Some objects are not namespaced (for example: Nodes), and so their names must be unique across the whole cluster.API verbsAlmost all object resource types support the standard HTTP verbs - GET, POST, PUT, PATCH, and DELETE. Kubernetes also uses its own verbs, which are often written in lowercase to distinguish them from HTTP verbs.Kubernetes uses the term list to describe the action of returning a collection of resources, to distinguish it from retrieving a single resource which is usually called a get. If you sent an HTTP GET request with the ?watch query parameter, Kubernetes calls this a watch and not a get (see Efficient detection of changes for more details).For PUT requests, Kubernetes internally classifies these as either create or update based on the state of the existing object. An update is different from a patch; the HTTP verb for a patch is PATCH.Resource URIsAll resource types are either scoped by the cluster (/apis/GROUP/VERSION/*) or to a namespace (/apis/GROUP/VERSION/namespaces/NAMESPACE/*). A namespace-scoped resource type will be deleted when its namespace is deleted and access to that resource type is controlled by authorization checks on the namespace scope.Note: core resources use /api instead of /apis and omit the GROUP path segment.Examples:/api/v1/namespaces/api/v1/pods/api/v1/namespaces/my-namespace/pods/apis/apps/v1/deployments/apis/apps/v1/namespaces/my-namespace/deployments/apis/apps/v1/namespaces/my-namespace/deployments/my-deploymentYou can also access collections of resources (for example: listing all Nodes). The following paths are used to retrieve collections and resources:Cluster-scoped resources:GET /apis/GROUP/VERSION/RESOURCETYPE - return the collection of resources of the resource typeGET /apis/GROUP/VERSION/RESOURCETYPE/NAME - return the resource with NAME under the resource typeNamespace-scoped resources:GET /apis/GROUP/VERSION/RESOURCETYPE - return the collection of all instances of the resource type across all namespacesGET /apis/GROUP/VERSION/namespaces/NAMESPACE/RESOURCETYPE - return collection of all instances of the resource type in NAMESPACEGET /apis/GROUP/VERSION/namespaces/NAMESPACE/RESOURCETYPE/NAME - return the instance of the resource type with NAME in NAMESPACESince a namespace is a cluster-scoped resource type, you can retrieve the list (“collection”) of all namespaces with GET /api/v1/namespaces and details about a particular namespace with GET /api/v1/namespaces/NAME.Cluster-scoped subresource: GET /apis/GROUP/VERSION/RESOURCETYPE/NAME/SUBRESOURCENamespace-scoped subresource: GET /apis/GROUP/VERSION/namespaces/NAMESPACE/RESOURCETYPE/NAME/SUBRESOURCEThe verbs supported for each subresource will differ depending on the object - see the API reference for more information. It is not possible to access sub-resources across multiple resources - generally a new virtual resource type would be used if that becomes necessary.HTTP media typesOver HTTP, Kubernetes supports JSON, YAML, CBOR and Protobuf wire encodings.By default, Kubernetes returns objects in JSON serialization, using the application/json media type. Although JSON is the default, clients may request a response in YAML, or use the more efficient binary Protobuf representation for better performance at scale.The Kubernetes API implements standard HTTP content type negotiation: passing an Accept header with a GET call will request that the server tries to return a response in your preferred media type. If you want to send an object in Protobuf to the server for a PUT or POST request, you must set the Content-Type request header appropriately.If you request an available media type, the API server returns a response with a suitable Content-Type; if none of the media types you request are supported, the API server returns a 406 Not acceptable error message. All built-in resource types support the application/json media type.Chunked encoding of collectionsFor JSON and Protobuf encoding, Kubernetes implements custom encoders that write item, by item. The feature doesn't change the output, but allows API server to avoid loading whole LIST response into memory. Using other types of encoding (including pretty representation of JSON) should be avoided for large collections of resources (>100MB) as it can have negative performance impact.JSON resource encodingThe Kubernetes API defaults to using JSON for encoding HTTP message bodies.For example:List all of the pods on a cluster, without specifying a preferred formatGET /api/v1/pods 200 OK Content-Type: application/json … JSON encoded collection of Pods (PodList object) Create a pod by sending JSON to the server, requesting a JSON response.POST /api/v1/namespaces/test/pods Content-Type: application/json Accept: application/json … JSON encoded Pod object 200 OK Content-Type: application/json { "kind": "Pod", "apiVersion": "v1", … } YAML resource encodingKubernetes also supports the application/yaml media type for both requests and responses. YAML can be used for defining Kubernetes manifests and API interactions.For example:List all of the pods on a cluster in YAML formatGET /api/v1/pods Accept: application/yaml 200 OK Content-Type: application/yaml … YAML encoded collection of Pods (PodList object) Create a pod by sending YAML-encoded data to the server, requesting a YAML response:POST /api/v1/namespaces/test/pods Content-Type: application/yaml Accept: application/yaml … YAML encoded Pod object 200 OK Content-Type: application/yaml apiVersion: v1 kind: Pod metadata: name: my-pod … Kubernetes Protobuf encodingKubernetes uses an envelope wrapper to encode Protobuf responses. That wrapper starts with a 4 byte magic number to help identify content in disk or in etcd as Protobuf (as opposed to JSON). The 4 byte magic number data is followed by a Protobuf encoded wrapper message, which describes the encoding and type of the underlying object. Within the Protobuf wrapper message, the inner object data is recorded using the raw field of Unknown (see the IDL for more detail).For example:List all of the pods on a cluster in Protobuf format.GET /api/v1/pods Accept: application/vnd.kubernetes.protobuf 200 OK Content-Type: application/vnd.kubernetes.protobuf … binary encoded collection of Pods (PodList object) Create a pod by sending Protobuf encoded data to the server, but request a response in JSON.POST /api/v1/namespaces/test/pods Content-Type: application/vnd.kubernetes.protobuf Accept: application/json … binary encoded Pod object 200 OK Content-Type: application/json { "kind": "Pod", "apiVersion": "v1", ... } You can use both techniques together and use Kubernetes' Protobuf encoding to interact with any API that supports it, for both reads and writes. Only some API resource types are compatible with Protobuf.The wrapper format is:A four byte magic number prefix: Bytes 0-3: "k8s\x00" [0x6b, 0x38, 0x73, 0x00] An encoded Protobuf message with the following IDL: message Unknown { // typeMeta should have the string values for "kind" and "apiVersion" as set on the JSON object optional TypeMeta typeMeta = 1; // raw will hold the complete serialized object in protobuf. See the protobuf definitions in the client libraries for a given kind. optional bytes raw = 2; // contentEncoding is encoding used for the raw data. Unspecified means no encoding. optional string contentEncoding = 3; // contentType is the serialization method used to serialize 'raw'. Unspecified means application/vnd.kubernetes.protobuf and is usually // omitted. optional string contentType = 4; } message TypeMeta { // apiVersion is the group/version for this type optional string apiVersion = 1; // kind is the name of the object schema. A protobuf definition should exist for this object. optional string kind = 2; } Note:Clients that receive a response in application/vnd.kubernetes.protobuf that does not match the expected prefix should reject the response, as future versions may need to alter the serialization format in an incompatible way and will do so by changing the prefix.Compatibility with Kubernetes ProtobufNot all API resource types support Kubernetes' Protobuf encoding; specifically, Protobuf isn't available for resources that are defined as CustomResourceDefinitions or are served via the aggregation layer.As a client, if you might need to work with extension types you should specify multiple content types in the request Accept header to support fallback to JSON. For example:Accept: application/vnd.kubernetes.protobuf, application/json CBOR resource encodingFEATURE STATE: Kubernetes v1.32 [alpha] (enabled by default: false)With the CBORServingAndStorage feature gate enabled, request and response bodies for all built-in resource types and all resources defined by a CustomResourceDefinition may be encoded to the CBOR binary data format. CBOR is also supported at the aggregation layer if it is enabled in individual aggregated API servers.Clients should indicate the IANA media type application/cbor in the Content-Type HTTP request header when the request body contains a single CBOR encoded data item, and in the Accept HTTP request header when prepared to accept a CBOR encoded data item in the response. API servers will use application/cbor in the Content-Type HTTP response header when the response body contains a CBOR-encoded object.If an API server encodes its response to a watch request using CBOR, the response body will be a CBOR Sequence and the Content-Type HTTP response header will use the IANA media type application/cbor-seq. Each entry of the sequence (if any) is a single CBOR-encoded watch event.In addition to the existing application/apply-patch+yaml media type for YAML-encoded server-side apply configurations, API servers that enable CBOR will accept the application/apply-patch+cbor media type for CBOR-encoded server-side apply configurations. There is no supported CBOR equivalent for application/json-patch+json or application/merge-patch+json, or application/strategic-merge-patch+json.Efficient detection of changesThe Kubernetes API allows clients to make an initial request for an object or a collection, and then to track changes since that initial request: a watch. Clients can send a list or a get and then make a follow-up watch request.To make this change tracking possible, every Kubernetes object has a resourceVersion field representing the version of that resource as stored in the underlying persistence layer. When retrieving a collection of resources (either namespace or cluster scoped), the response from the API server contains a resourceVersion value. The client can use that resourceVersion to initiate a watch against the API server.When you send a watch request, the API server responds with a stream of changes. These changes itemize the outcome of operations (such as create, delete, and update) that occurred after the resourceVersion you specified as a parameter to the watch request. The overall watch mechanism allows a client to fetch the current state and then subscribe to subsequent changes, without missing any events.If a client watch is disconnected then that client can start a new watch from the last returned resourceVersion; the client could also perform a fresh get / list request and begin again. See Resource Version Semantics for more detail.For example:List all of the pods in a given namespace.GET /api/v1/namespaces/test/pods --- 200 OK Content-Type: application/json { "kind": "PodList", "apiVersion": "v1", "metadata": {"resourceVersion":"10245"}, "items": [...] } Starting from resource version 10245, receive notifications of any API operations (such as create, delete, patch or update) that affect Pods in the test namespace. Each change notification is a JSON document. The HTTP response body (served as application/json) consists a series of JSON documents.GET /api/v1/namespaces/test/pods?watch=1&resourceVersion=10245 --- 200 OK Transfer-Encoding: chunked Content-Type: application/json { "type": "ADDED", "object": {"kind": "Pod", "apiVersion": "v1", "metadata": {"resourceVersion": "10596", ...}, ...} } { "type": "MODIFIED", "object": {"kind": "Pod", "apiVersion": "v1", "metadata": {"resourceVersion": "11020", ...}, ...} } ... A given Kubernetes server will only preserve a historical record of changes for a limited time. Clusters using etcd 3 preserve changes in the last 5 minutes by default. When the requested watch operations fail because the historical version of that resource is not available, clients must handle the case by recognizing the status code 410 Gone, clearing their local cache, performing a new get or list operation, and starting the watch from the resourceVersion that was returned.For subscribing to collections, Kubernetes client libraries typically offer some form of standard tool for this list-then-watch logic. (In the Go client library, this is called a Reflector and is located in the k8s.io/client-go/tools/cache package.)Watch bookmarksTo mitigate the impact of short history window, the Kubernetes API provides a watch event named BOOKMARK. It is a special kind of event to mark that all changes up to a given resourceVersion the client is requesting have already been sent. The document representing the BOOKMARK event is of the type requested by the request, but only includes a .metadata.resourceVersion field. For example:GET /api/v1/namespaces/test/pods?watch=1&resourceVersion=10245&allowWatchBookmarks=true --- 200 OK Transfer-Encoding: chunked Content-Type: application/json { "type": "ADDED", "object": {"kind": "Pod", "apiVersion": "v1", "metadata": {"resourceVersion": "10596", ...}, ...} } ... { "type": "BOOKMARK", "object": {"kind": "Pod", "apiVersion": "v1", "metadata": {"resourceVersion": "12746"} } } As a client, you can request BOOKMARK events by setting the allowWatchBookmarks=true query parameter to a watch request, but you shouldn't assume bookmarks are returned at any specific interval, nor can clients assume that the API server will send any BOOKMARK event even when requested.Streaming listsFEATURE STATE: Kubernetes v1.34 [beta] (enabled by default: true)On large clusters, retrieving the collection of some resource types may result in a significant increase of resource usage (primarily RAM) on the control plane. To alleviate the impact and simplify the user experience of the list + watch pattern, Kubernetes v1.32 promotes to beta the feature that allows requesting the initial state (previously requested via the list request) as part of the watch request.On the client-side the initial state can be requested by specifying sendInitialEvents=true as query string parameter in a watch request. If set, the API server starts the watch stream with synthetic init events (of type ADDED) to build the whole state of all existing objects followed by a BOOKMARK event (if requested via allowWatchBookmarks=true option). The bookmark event includes the resource version to which is synced. After sending the bookmark event, the API server continues as for any other watch request.When you set sendInitialEvents=true in the query string, Kubernetes also requires that you set resourceVersionMatch to NotOlderThan value. If you provided resourceVersion in the query string without providing a value or don't provide it at all, this is interpreted as a request for consistent read; the bookmark event is sent when the state is synced at least to the moment of a consistent read from when the request started to be processed. If you specify resourceVersion (in the query string), the bookmark event is sent when the state is synced at least to the provided resource version.ExampleAn example: you want to watch a collection of Pods. For that collection, the current resource version is 10245 and there are two pods: foo and bar. Then sending the following request (explicitly requesting consistent read by setting empty resource version using resourceVersion=) could result in the following sequence of events:GET /api/v1/namespaces/test/pods?watch=1&sendInitialEvents=true&allowWatchBookmarks=true&resourceVersion=&resourceVersionMatch=NotOlderThan --- 200 OK Transfer-Encoding: chunked Content-Type: application/json { "type": "ADDED", "object": {"kind": "Pod", "apiVersion": "v1", "metadata": {"resourceVersion": "8467", "name": "foo"}, ...} } { "type": "ADDED", "object": {"kind": "Pod", "apiVersion": "v1", "metadata": {"resourceVersion": "5726", "name": "bar"}, ...} } { "type": "BOOKMARK", "object": {"kind": "Pod", "apiVersion": "v1", "metadata": {"resourceVersion": "10245"} } } ... <followed by regular watch stream starting from resourceVersion="10245"> Response compressionFEATURE STATE: Kubernetes v1.16 [beta] (enabled by default: true)APIResponseCompression is an option that allows the API server to compress the responses for get and list requests, reducing the network bandwidth and improving the performance of large-scale clusters. It is enabled by default since Kubernetes 1.16 and it can be disabled by including APIResponseCompression=false in the --feature-gates flag on the API server.API response compression can significantly reduce the size of the response, especially for large resources or collections. For example, a list request for pods can return hundreds of kilobytes or even megabytes of data, depending on the number of pods and their attributes. By compressing the response, the network bandwidth can be saved and the latency can be reduced.To verify if APIResponseCompression is working, you can send a get or list request to the API server with an Accept-Encoding header, and check the response size and headers. For example:GET /api/v1/pods Accept-Encoding: gzip --- 200 OK Content-Type: application/json content-encoding: gzip ... The content-encoding header indicates that the response is compressed with gzip.Retrieving large results sets in chunksFEATURE STATE: Kubernetes v1.29 [stable] (enabled by default: true)On large clusters, retrieving the collection of some resource types may result in very large responses that can impact the server and client. For instance, a cluster may have tens of thousands of Pods, each of which is equivalent to roughly 2 KiB of encoded JSON. Retrieving all pods across all namespaces may result in a very large response (10-20MB) and consume a large amount of server resources.The Kubernetes API server supports the ability to break a single large collection request into many smaller chunks while preserving the consistency of the total request. Each chunk can be returned sequentially which reduces both the total size of the request and allows user-oriented clients to display results incrementally to improve responsiveness.You can request that the API server handles a list by serving single collection using pages (which Kubernetes calls chunks). To retrieve a single collection in chunks, two query parameters limit and continue are supported on requests against collections, and a response field continue is returned from all list operations in the collection's metadata field. A client should specify the maximum results they wish to receive in each chunk with limit and the server will return up to limit resources in the result and include a continue value if there are more resources in the collection.As an API client, you can then pass this continue value to the API server on the


# Secret

SecretSecret holds secret data of a certain type.apiVersion: v1import "k8s.io/api/core/v1"SecretSecret holds secret data of a certain type. The total bytes of the values in the Data field must be less than MaxSecretSize bytes.apiVersion: v1kind: Secretmetadata (ObjectMeta)Standard object's metadata. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadatadata (map[string][]byte)Data contains the secret data. Each key must consist of alphanumeric characters, '-', '_' or '.'. The serialized form of the secret data is a base64 encoded string, representing the arbitrary (possibly non-string) data value here. Described in https://tools.ietf.org/html/rfc4648#section-4immutable (boolean)Immutable, if set to true, ensures that data stored in the Secret cannot be updated (only object metadata can be modified). If not set to true, the field can be modified at any time. Defaulted to nil.stringData (map[string]string)stringData allows specifying non-binary secret data in string form. It is provided as a write-only input field for convenience. All keys and values are merged into the data field on write, overwriting any existing values. The stringData field is never output when reading from the API.type (string)Used to facilitate programmatic handling of secret data. More info: https://kubernetes.io/docs/concepts/configuration/secret/#secret-typesSecretListSecretList is a list of Secret.apiVersion: v1kind: SecretListmetadata (ListMeta)Standard list metadata. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kindsitems ([]Secret), requiredItems is a list of secret objects. More info: https://kubernetes.io/docs/concepts/configuration/secretOperationsget read the specified SecretHTTP RequestGET /api/v1/namespaces/{namespace}/secrets/{name}Parametersname (in path): string, requiredname of the Secretnamespace (in path): string, requirednamespacepretty (in query): stringprettyResponse200 (Secret): OK401: Unauthorizedlist list or watch objects of kind SecretHTTP RequestGET /api/v1/namespaces/{namespace}/secretsParametersnamespace (in path): string, requirednamespaceallowWatchBookmarks (in query): booleanallowWatchBookmarkscontinue (in query): stringcontinuefieldSelector (in query): stringfieldSelectorlabelSelector (in query): stringlabelSelectorlimit (in query): integerlimitpretty (in query): stringprettyresourceVersion (in query): stringresourceVersionresourceVersionMatch (in query): stringresourceVersionMatchsendInitialEvents (in query): booleansendInitialEventstimeoutSeconds (in query): integertimeoutSecondswatch (in query): booleanwatchResponse200 (SecretList): OK401: Unauthorizedlist list or watch objects of kind SecretHTTP RequestGET /api/v1/secretsParametersallowWatchBookmarks (in query): booleanallowWatchBookmarkscontinue (in query): stringcontinuefieldSelector (in query): stringfieldSelectorlabelSelector (in query): stringlabelSelectorlimit (in query): integerlimitpretty (in query): stringprettyresourceVersion (in query): stringresourceVersionresourceVersionMatch (in query): stringresourceVersionMatchsendInitialEvents (in query): booleansendInitialEventstimeoutSeconds (in query): integertimeoutSecondswatch (in query): booleanwatchResponse200 (SecretList): OK401: Unauthorizedcreate create a SecretHTTP RequestPOST /api/v1/namespaces/{namespace}/secretsParametersnamespace (in path): string, requirednamespacebody: Secret, requireddryRun (in query): stringdryRunfieldManager (in query): stringfieldManagerfieldValidation (in query): stringfieldValidationpretty (in query): stringprettyResponse200 (Secret): OK201 (Secret): Created202 (Secret): Accepted401: Unauthorizedupdate replace the specified SecretHTTP RequestPUT /api/v1/namespaces/{namespace}/secrets/{name}Parametersname (in path): string, requiredname of the Secretnamespace (in path): string, requirednamespacebody: Secret, requireddryRun (in query): stringdryRunfieldManager (in query): stringfieldManagerfieldValidation (in query): stringfieldValidationpretty (in query): stringprettyResponse200 (Secret): OK201 (Secret): Created401: Unauthorizedpatch partially update the specified SecretHTTP RequestPATCH /api/v1/namespaces/{namespace}/secrets/{name}Parametersname (in path): string, requiredname of the Secretnamespace (in path): string, requirednamespacebody: Patch, requireddryRun (in query): stringdryRunfieldManager (in query): stringfieldManagerfieldValidation (in query): stringfieldValidationforce (in query): booleanforcepretty (in query): stringprettyResponse200 (Secret): OK201 (Secret): Created401: Unauthorizeddelete delete a SecretHTTP RequestDELETE /api/v1/namespaces/{namespace}/secrets/{name}Parametersname (in path): string, requiredname of the Secretnamespace (in path): string, requirednamespacebody: DeleteOptionsdryRun (in query): stringdryRungracePeriodSeconds (in query): integergracePeriodSecondsignoreStoreReadErrorWithClusterBreakingPotential (in query): booleanignoreStoreReadErrorWithClusterBreakingPotentialpretty (in query): stringprettypropagationPolicy (in query): stringpropagationPolicyResponse200 (Status): OK202 (Status): Accepted401: Unauthorizeddeletecollection delete collection of SecretHTTP RequestDELETE /api/v1/namespaces/{namespace}/secretsParametersnamespace (in path): string, requirednamespacebody: DeleteOptionscontinue (in query): stringcontinuedryRun (in query): stringdryRunfieldSelector (in query): stringfieldSelectorgracePeriodSeconds (in query): integergracePeriodSecondsignoreStoreReadErrorWithClusterBreakingPotential (in query): booleanignoreStoreReadErrorWithClusterBreakingPotentiallabelSelector (in query): stringlabelSelectorlimit (in query): integerlimitpretty (in query): stringprettypropagationPolicy (in query): stringpropagationPolicyresourceVersion (in query): stringresourceVersionresourceVersionMatch (in query): stringresourceVersionMatchsendInitialEvents (in query): booleansendInitialEventstimeoutSeconds (in query): integertimeoutSecondsResponse200 (Status): OK401: UnauthorizedThis page is automatically generated.If you plan to report an issue with this page, mention that the page is auto-generated in your issue description. The fix may need to happen elsewhere in the Kubernetes project.


# Managing Kubernetes Objects Using Imperative Commands

Managing Kubernetes Objects Using Imperative CommandsKubernetes objects can quickly be created, updated, and deleted directly using imperative commands built into the kubectl command-line tool. This document explains how those commands are organized and how to use them to manage live objects.Before you beginInstall kubectl.You need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. It is recommended to run this tutorial on a cluster with at least two nodes that are not acting as control plane hosts. If you do not already have a cluster, you can create one by using minikube or you can use one of these Kubernetes playgrounds:iximiuz LabsKillercodaKodeKloudPlay with KubernetesTo check the version, enter kubectl version.Trade-offsThe kubectl tool supports three kinds of object management:Imperative commandsImperative object configurationDeclarative object configurationSee Kubernetes Object Management for a discussion of the advantages and disadvantage of each kind of object management.How to create objectsThe kubectl tool supports verb-driven commands for creating some of the most common object types. The commands are named to be recognizable to users unfamiliar with the Kubernetes object types.run: Create a new Pod to run a Container.expose: Create a new Service object to load balance traffic across Pods.autoscale: Create a new Autoscaler object to automatically horizontally scale a controller, such as a Deployment.The kubectl tool also supports creation commands driven by object type. These commands support more object types and are more explicit about their intent, but require users to know the type of objects they intend to create.create <objecttype> [<subtype>] <instancename>Some objects types have subtypes that you can specify in the create command. For example, the Service object has several subtypes including ClusterIP, LoadBalancer, and NodePort. Here's an example that creates a Service with subtype NodePort:kubectl create service nodeport <myservicename> In the preceding example, the create service nodeport command is called a subcommand of the create service command.You can use the -h flag to find the arguments and flags supported by a subcommand:kubectl create service nodeport -h How to update objectsThe kubectl command supports verb-driven commands for some common update operations. These commands are named to enable users unfamiliar with Kubernetes objects to perform updates without knowing the specific fields that must be set:scale: Horizontally scale a controller to add or remove Pods by updating the replica count of the controller.annotate: Add or remove an annotation from an object.label: Add or remove a label from an object.The kubectl command also supports update commands driven by an aspect of the object. Setting this aspect may set different fields for different object types:set <field>: Set an aspect of an object.Note:In Kubernetes version 1.5, not every verb-driven command has an associated aspect-driven command.The kubectl tool supports these additional ways to update a live object directly, however they require a better understanding of the Kubernetes object schema.edit: Directly edit the raw configuration of a live object by opening its configuration in an editor.patch: Directly modify specific fields of a live object by using a patch string. For more details on patch strings, see the patch section in API Conventions.How to delete objectsYou can use the delete command to delete an object from a cluster:delete <type>/<name>Note:You can use kubectl delete for both imperative commands and imperative object configuration. The difference is in the arguments passed to the command. To use kubectl delete as an imperative command, pass the object to be deleted as an argument. Here's an example that passes a Deployment object named nginx:kubectl delete deployment/nginx How to view an objectThere are several commands for printing information about an object:get: Prints basic information about matching objects. Use get -h to see a list of options.describe: Prints aggregated detailed information about matching objects.logs: Prints the stdout and stderr for a container running in a Pod.Using set commands to modify objects before creationThere are some object fields that don't have a flag you can use in a create command. In some of those cases, you can use a combination of set and create to specify a value for the field before object creation. This is done by piping the output of the create command to the set command, and then back to the create command. Here's an example:kubectl create service clusterip my-svc --clusterip="None" -o yaml --dry-run=client | kubectl set selector --local -f - 'environment=qa' -o yaml | kubectl create -f - The kubectl create service -o yaml --dry-run=client command creates the configuration for the Service, but prints it to stdout as YAML instead of sending it to the Kubernetes API server.The kubectl set selector --local -f - -o yaml command reads the configuration from stdin, and writes the updated configuration to stdout as YAML.The kubectl create -f - command creates the object using the configuration provided via stdin.Using --edit to modify objects before creationYou can use kubectl create --edit to make arbitrary changes to an object before it is created. Here's an example:kubectl create service clusterip my-svc --clusterip="None" -o yaml --dry-run=client > /tmp/srv.yaml kubectl create --edit -f /tmp/srv.yaml The kubectl create service command creates the configuration for the Service and saves it to /tmp/srv.yaml.The kubectl create --edit command opens the configuration file for editing before it creates the object.What's nextImperative Management of Kubernetes Objects Using Configuration FilesDeclarative Management of Kubernetes Objects Using Configuration FilesKubectl Command ReferenceKubernetes API Reference


# kubectl config view

kubectl config viewSynopsisDisplay merged kubeconfig settings or a specified kubeconfig file.You can use --output jsonpath={...} to extract specific values using a jsonpath expression.kubectl config view [flags] Examples # Show merged kubeconfig settings kubectl config view # Show merged kubeconfig settings, raw certificate data, and exposed secrets kubectl config view --raw # Get the password for the e2e user kubectl config view -o jsonpath='{.users[?(@.name == "e2e")].user.password}' Options--allow-missing-template-keys Default: trueIf true, ignore any errors in templates when a field or map key is missing in the template. Only applies to golang and jsonpath output formats.--flattenFlatten the resulting kubeconfig file into self-contained output (useful for creating portable kubeconfig files)-h, --helphelp for view--merge tristate[=true] Default: trueMerge the full hierarchy of kubeconfig files--minifyRemove all information not used by current-context from the output-o, --output string Default: "yaml"Output format. One of: (json, yaml, name, go-template, go-template-file, template, templatefile, jsonpath, jsonpath-as-json, jsonpath-file).--rawDisplay raw byte data and sensitive data--show-managed-fieldsIf true, keep the managedFields when printing objects in JSON or YAML format.--template stringTemplate string or path to template file to use when -o=go-template, -o=go-template-file. The template format is golang templates [http://golang.org/pkg/text/template/#pkg-overview].Parent Options Inherited--as stringUsername to impersonate for the operation. User could be a regular user or a service account in a namespace.--as-group stringsGroup to impersonate for the operation, this flag can be repeated to specify multiple groups.--as-uid stringUID to impersonate for the operation.--cache-dir string Default: "$HOME/.kube/cache"Default cache directory--certificate-authority stringPath to a cert file for the certificate authority--client-certificate stringPath to a client certificate file for TLS--client-key stringPath to a client key file for TLS--cluster stringThe name of the kubeconfig cluster to use--context stringThe name of the kubeconfig context to use--disable-compressionIf true, opt-out of response compression for all requests to the server--insecure-skip-tls-verifyIf true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure--kubeconfig stringuse a particular kubeconfig file--kuberc stringPath to the kuberc file to use for preferences. This can be disabled by exporting KUBECTL_KUBERC=false feature gate or turning off the feature KUBERC=off.--match-server-versionRequire server version to match client version-n, --namespace stringIf present, the namespace scope for this CLI request--password stringPassword for basic authentication to the API server--profile string Default: "none"Name of profile to capture. One of (none|cpu|heap|goroutine|threadcreate|block|mutex)--profile-output string Default: "profile.pprof"Name of the file to write the profile to--request-timeout string Default: "0"The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests.-s, --server stringThe address and port of the Kubernetes API server--storage-driver-buffer-duration duration Default: 1m0sWrites in the storage driver will be buffered for this duration, and committed to the non memory backends as a single transaction--storage-driver-db string Default: "cadvisor"database name--storage-driver-host string Default: "localhost:8086"database host:port--storage-driver-password string Default: "root"database password--storage-driver-secureuse secure connection with database--storage-driver-table string Default: "stats"table name--storage-driver-user string Default: "root"database username--tls-server-name stringServer name to use for server certificate validation. If it is not provided, the hostname used to contact the server is used--token stringBearer token for authentication to the API server--user stringThe name of the kubeconfig user to use--username stringUsername for basic authentication to the API server--version version[=true]--version, --version=raw prints version information and quits; --version=vX.Y.Z... sets the reported version--warnings-as-errorsTreat warnings received from the server as errors and exit with a non-zero exit codeSee Alsokubectl config - Modify kubeconfig filesThis page is automatically generated.If you plan to report an issue with this page, mention that the page is auto-generated in your issue description. The fix may need to happen elsewhere in the Kubernetes project.


# Developing Cloud Controller Manager

Developing Cloud Controller ManagerFEATURE STATE: Kubernetes v1.11 [beta]The cloud-controller-manager is a Kubernetes control plane component that embeds cloud-specific control logic. The cloud controller manager lets you link your cluster into your cloud provider's API, and separates out the components that interact with that cloud platform from components that only interact with your cluster.By decoupling the interoperability logic between Kubernetes and the underlying cloud infrastructure, the cloud-controller-manager component enables cloud providers to release features at a different pace compared to the main Kubernetes project.BackgroundSince cloud providers develop and release at a different pace compared to the Kubernetes project, abstracting the provider-specific code to the cloud-controller-manager binary allows cloud vendors to evolve independently from the core Kubernetes code.The Kubernetes project provides skeleton cloud-controller-manager code with Go interfaces to allow you (or your cloud provider) to plug in your own implementations. This means that a cloud provider can implement a cloud-controller-manager by importing packages from Kubernetes core; each cloudprovider will register their own code by calling cloudprovider.RegisterCloudProvider to update a global variable of available cloud providers.DevelopingOut of treeTo build an out-of-tree cloud-controller-manager for your cloud:Create a go package with an implementation that satisfies cloudprovider.Interface.Use main.go in cloud-controller-manager from Kubernetes core as a template for your main.go. As mentioned above, the only difference should be the cloud package that will be imported.Import your cloud package in main.go, ensure your package has an init block to run cloudprovider.RegisterCloudProvider.Many cloud providers publish their controller manager code as open source. If you are creating a new cloud-controller-manager from scratch, you could take an existing out-of-tree cloud controller manager as your starting point.In treeFor in-tree cloud providers, you can run the in-tree cloud controller manager as a DaemonSet in your cluster. See Cloud Controller Manager Administration for more details.


# kubectl cluster-info dump

kubectl cluster-info dumpSynopsisDump cluster information out suitable for debugging and diagnosing cluster problems. By default, dumps everything to stdout. You can optionally specify a directory with --output-directory. If you specify a directory, Kubernetes will build a set of files in that directory. By default, only dumps things in the current namespace and 'kube-system' namespace, but you can switch to a different namespace with the --namespaces flag, or specify --all-namespaces to dump all namespaces.The command also dumps the logs of all of the pods in the cluster; these logs are dumped into different directories based on namespace and pod name.kubectl cluster-info dump [flags] Examples # Dump current cluster state to stdout kubectl cluster-info dump # Dump current cluster state to /path/to/cluster-state kubectl cluster-info dump --output-directory=/path/to/cluster-state # Dump all namespaces to stdout kubectl cluster-info dump --all-namespaces # Dump a set of namespaces to /path/to/cluster-state kubectl cluster-info dump --namespaces default,kube-system --output-directory=/path/to/cluster-state Options-A, --all-namespacesIf true, dump all namespaces. If true, --namespaces is ignored.--allow-missing-template-keys Default: trueIf true, ignore any errors in templates when a field or map key is missing in the template. Only applies to golang and jsonpath output formats.-h, --helphelp for dump--namespaces stringsA comma separated list of namespaces to dump.-o, --output string Default: "json"Output format. One of: (json, yaml, name, go-template, go-template-file, template, templatefile, jsonpath, jsonpath-as-json, jsonpath-file).--output-directory stringWhere to output the files. If empty or '-' uses stdout, otherwise creates a directory hierarchy in that directory--pod-running-timeout duration Default: 20sThe length of time (like 5s, 2m, or 3h, higher than zero) to wait until at least one pod is running--show-managed-fieldsIf true, keep the managedFields when printing objects in JSON or YAML format.--template stringTemplate string or path to template file to use when -o=go-template, -o=go-template-file. The template format is golang templates [http://golang.org/pkg/text/template/#pkg-overview].Parent Options Inherited--as stringUsername to impersonate for the operation. User could be a regular user or a service account in a namespace.--as-group stringsGroup to impersonate for the operation, this flag can be repeated to specify multiple groups.--as-uid stringUID to impersonate for the operation.--cache-dir string Default: "$HOME/.kube/cache"Default cache directory--certificate-authority stringPath to a cert file for the certificate authority--client-certificate stringPath to a client certificate file for TLS--client-key stringPath to a client key file for TLS--cluster stringThe name of the kubeconfig cluster to use--context stringThe name of the kubeconfig context to use--disable-compressionIf true, opt-out of response compression for all requests to the server--insecure-skip-tls-verifyIf true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure--kubeconfig stringPath to the kubeconfig file to use for CLI requests.--kuberc stringPath to the kuberc file to use for preferences. This can be disabled by exporting KUBECTL_KUBERC=false feature gate or turning off the feature KUBERC=off.--match-server-versionRequire server version to match client version-n, --namespace stringIf present, the namespace scope for this CLI request--password stringPassword for basic authentication to the API server--profile string Default: "none"Name of profile to capture. One of (none|cpu|heap|goroutine|threadcreate|block|mutex)--profile-output string Default: "profile.pprof"Name of the file to write the profile to--request-timeout string Default: "0"The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests.-s, --server stringThe address and port of the Kubernetes API server--storage-driver-buffer-duration duration Default: 1m0sWrites in the storage driver will be buffered for this duration, and committed to the non memory backends as a single transaction--storage-driver-db string Default: "cadvisor"database name--storage-driver-host string Default: "localhost:8086"database host:port--storage-driver-password string Default: "root"database password--storage-driver-secureuse secure connection with database--storage-driver-table string Default: "stats"table name--storage-driver-user string Default: "root"database username--tls-server-name stringServer name to use for server certificate validation. If it is not provided, the hostname used to contact the server is used--token stringBearer token for authentication to the API server--user stringThe name of the kubeconfig user to use--username stringUsername for basic authentication to the API server--version version[=true]--version, --version=raw prints version information and quits; --version=vX.Y.Z... sets the reported version--warnings-as-errorsTreat warnings received from the server as errors and exit with a non-zero exit codeSee Alsokubectl cluster-info - Display cluster informationThis page is automatically generated.If you plan to report an issue with this page, mention that the page is auto-generated in your issue description. The fix may need to happen elsewhere in the Kubernetes project.


# kubectl run

kubectl runSynopsisCreate and run a particular image in a pod.kubectl run NAME --image=image [--env="key=value"] [--port=port] [--dry-run=server|client] [--overrides=inline-json] [--command] -- [COMMAND] [args...] Examples # Start a nginx pod kubectl run nginx --image=nginx # Start a hazelcast pod and let the container expose port 5701 kubectl run hazelcast --image=hazelcast/hazelcast --port=5701 # Start a hazelcast pod and set environment variables "DNS_DOMAIN=cluster" and "POD_NAMESPACE=default" in the container kubectl run hazelcast --image=hazelcast/hazelcast --env="DNS_DOMAIN=cluster" --env="POD_NAMESPACE=default" # Start a hazelcast pod and set labels "app=hazelcast" and "env=prod" in the container kubectl run hazelcast --image=hazelcast/hazelcast --labels="app=hazelcast,env=prod" # Dry run; print the corresponding API objects without creating them kubectl run nginx --image=nginx --dry-run=client # Start a nginx pod, but overload the spec with a partial set of values parsed from JSON kubectl run nginx --image=nginx --overrides='{ "apiVersion": "v1", "spec": { ... } }' # Start a busybox pod and keep it in the foreground, don't restart it if it exits kubectl run -i -t busybox --image=busybox --restart=Never # Start the nginx pod using the default command, but use custom arguments (arg1 .. argN) for that command kubectl run nginx --image=nginx -- <arg1> <arg2> ... <argN> # Start the nginx pod using a different command and custom arguments kubectl run nginx --image=nginx --command -- <cmd> <arg1> ... <argN> Options--allow-missing-template-keys Default: trueIf true, ignore any errors in templates when a field or map key is missing in the template. Only applies to golang and jsonpath output formats.--annotations stringsAnnotations to apply to the pod.--attachIf true, wait for the Pod to start running, and then attach to the Pod as if 'kubectl attach ...' were called. Default false, unless '-i/--stdin' is set, in which case the default is true. With '--restart=Never' the exit code of the container process is returned.--cascade string[="background"] Default: "background"Must be "background", "orphan", or "foreground". Selects the deletion cascading strategy for the dependents (e.g. Pods created by a ReplicationController). Defaults to background.--commandIf true and extra arguments are present, use them as the 'command' field in the container, rather than the 'args' field which is the default.--dry-run string[="unchanged"] Default: "none"Must be "none", "server", or "client". If client strategy, only print the object that would be sent, without sending it. If server strategy, submit server-side request without persisting the resource.--env stringsEnvironment variables to set in the container.--expose --portIf true, create a ClusterIP service associated with the pod. Requires --port.--field-manager string Default: "kubectl-run"Name of the manager used to track field ownership.-f, --filename stringsto use to replace the resource.--forceIf true, immediately remove resources from API and bypass graceful deletion. Note that immediate deletion of some resources may result in inconsistency or data loss and requires confirmation.--grace-period int Default: -1Period of time in seconds given to the resource to terminate gracefully. Ignored if negative. Set to 1 for immediate shutdown. Can only be set to 0 when --force is true (force deletion).-h, --helphelp for run--image stringThe image for the container to run.--image-pull-policy stringThe image pull policy for the container. If left empty, this value will not be specified by the client and defaulted by the server.-k, --kustomize stringProcess a kustomization directory. This flag can't be used together with -f or -R.-l, --labels stringComma separated labels to apply to the pod. Will override previous values.--leave-stdin-openIf the pod is started in interactive mode or with stdin, leave stdin open after the first attach completes. By default, stdin will be closed after the first attach completes.-o, --output stringOutput format. One of: (json, yaml, name, go-template, go-template-file, template, templatefile, jsonpath, jsonpath-as-json, jsonpath-file).--override-type string Default: "merge"The method used to override the generated object: json, merge, or strategic.--overrides stringAn inline JSON override for the generated object. If this is non-empty, it is used to override the generated object. Requires that the object supply a valid apiVersion field.--pod-running-timeout duration Default: 1m0sThe length of time (like 5s, 2m, or 3h, higher than zero) to wait until at least one pod is running--port stringThe port that this container exposes.--privilegedIf true, run the container in privileged mode.-q, --quietIf true, suppress prompt messages.-R, --recursiveProcess the directory used in -f, --filename recursively. Useful when you want to manage related manifests organized within the same directory.--restart string Default: "Always"The restart policy for this Pod. Legal values [Always, OnFailure, Never].--rmIf true, delete the pod after it exits. Only valid when attaching to the container, e.g. with '--attach' or with '-i/--stdin'.--save-configIf true, the configuration of current object will be saved in its annotation. Otherwise, the annotation will be unchanged. This flag is useful when you want to perform kubectl apply on this object in the future.--show-managed-fieldsIf true, keep the managedFields when printing objects in JSON or YAML format.-i, --stdinKeep stdin open on the container in the pod, even if nothing is attached.--template stringTemplate string or path to template file to use when -o=go-template, -o=go-template-file. The template format is golang templates [http://golang.org/pkg/text/template/#pkg-overview].--timeout durationThe length of time to wait before giving up on a delete, zero means determine a timeout from the size of the object-t, --ttyAllocate a TTY for the container in the pod.--waitIf true, wait for resources to be gone before returning. This waits for finalizers.Parent Options Inherited--as stringUsername to impersonate for the operation. User could be a regular user or a service account in a namespace.--as-group stringsGroup to impersonate for the operation, this flag can be repeated to specify multiple groups.--as-uid stringUID to impersonate for the operation.--cache-dir string Default: "$HOME/.kube/cache"Default cache directory--certificate-authority stringPath to a cert file for the certificate authority--client-certificate stringPath to a client certificate file for TLS--client-key stringPath to a client key file for TLS--cluster stringThe name of the kubeconfig cluster to use--context stringThe name of the kubeconfig context to use--disable-compressionIf true, opt-out of response compression for all requests to the server--insecure-skip-tls-verifyIf true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure--kubeconfig stringPath to the kubeconfig file to use for CLI requests.--kuberc stringPath to the kuberc file to use for preferences. This can be disabled by exporting KUBECTL_KUBERC=false feature gate or turning off the feature KUBERC=off.--match-server-versionRequire server version to match client version-n, --namespace stringIf present, the namespace scope for this CLI request--password stringPassword for basic authentication to the API server--profile string Default: "none"Name of profile to capture. One of (none|cpu|heap|goroutine|threadcreate|block|mutex)--profile-output string Default: "profile.pprof"Name of the file to write the profile to--request-timeout string Default: "0"The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests.-s, --server stringThe address and port of the Kubernetes API server--storage-driver-buffer-duration duration Default: 1m0sWrites in the storage driver will be buffered for this duration, and committed to the non memory backends as a single transaction--storage-driver-db string Default: "cadvisor"database name--storage-driver-host string Default: "localhost:8086"database host:port--storage-driver-password string Default: "root"database password--storage-driver-secureuse secure connection with database--storage-driver-table string Default: "stats"table name--storage-driver-user string Default: "root"database username--tls-server-name stringServer name to use for server certificate validation. If it is not provided, the hostname used to contact the server is used--token stringBearer token for authentication to the API server--user stringThe name of the kubeconfig user to use--username stringUsername for basic authentication to the API server--version version[=true]--version, --version=raw prints version information and quits; --version=vX.Y.Z... sets the reported version--warnings-as-errorsTreat warnings received from the server as errors and exit with a non-zero exit codeSee Alsokubectl - kubectl controls the Kubernetes cluster managerThis page is automatically generated.If you plan to report an issue with this page, mention that the page is auto-generated in your issue description. The fix may need to happen elsewhere in the Kubernetes project.


# Glossary

GlossaryThis glossary is intended to be a comprehensive, standardized list of Kubernetes terminology. It includes technical terms that are specific to Kubernetes, as well as more general terms that provide useful context.Filter terms according to their tags.The inner components of Kubernetes.Related to Kubernetes open-source development.A resource type that Kubernetes supports by default.Supported customizations of Kubernetes.Relevant for a first-time user of Kubernetes.How Kubernetes components talk to each other (and to programs outside the cluster).Starting and maintaining Kubernetes.Keeping Kubernetes applications safe and secure.How Kubernetes applications handle persistent data.Software that makes Kubernetes easier or better to use.Represents a common type of Kubernetes user.Applications running on Kubernetes.Architecture Community Core Object Extension Fundamental Networking Operation Security Storage Tool User Type Workload Select all Deselect allClick on the [+] indicators below to get a longer explanation for any particular term.Add-onsResources that extend the functionality of Kubernetes.[+]Installing addons explains more about using add-ons with your cluster, and lists some popular add-ons.Admission ControllerA piece of code that intercepts requests to the Kubernetes API server prior to persistence of the object.[+]Admission controllers are configurable for the Kubernetes API server and may be "validating", "mutating", or both. Any admission controller may reject the request. Mutating controllers may modify the objects they admit; validating controllers may not.Admission controllers in the Kubernetes documentationAffinityIn Kubernetes, affinity is a set of rules that give hints to the scheduler about where to place pods.[+]There are two kinds of affinity:node affinitypod-to-pod affinityThe rules are defined using the Kubernetes labels, and selectors specified in pods, and they can be either required or preferred, depending on how strictly you want the scheduler to enforce them.Aggregation LayerThe aggregation layer lets you install additional Kubernetes-style APIs in your cluster.[+]When you've configured the Kubernetes API Server to support additional APIs, you can add APIService objects to "claim" a URL path in the Kubernetes API.AnnotationA key-value pair that is used to attach arbitrary non-identifying metadata to objects.[+]The metadata in an annotation can be small or large, structured or unstructured, and can include characters not permitted by labels. Clients such as tools and libraries can retrieve this metadata.API GroupA set of related paths in Kubernetes API.[+]You can enable or disable each API group by changing the configuration of your API server. You can also disable or enable paths to specific resources. An API group makes it easier to extend the Kubernetes API. The API group is specified in a REST path and in the apiVersion field of a serialized object.Read API Group for more information.API resourceAlso known as: ResourceAn entity in the Kubernetes type system, corresponding to an endpoint on the Kubernetes API. A resource typically represents an object. Some resources represent an operation on other objects, such as a permission check.[+]Each resource represents an HTTP endpoint (URI) on the Kubernetes API server, defining the schema for the objects or operations on that resource.API serverAlso known as: kube-apiserverThe API server is a component of the Kubernetes control plane that exposes the Kubernetes API. The API server is the front end for the Kubernetes control plane.[+]The main implementation of a Kubernetes API server is kube-apiserver. kube-apiserver is designed to scale horizontally—that is, it scales by deploying more instances. You can run several instances of kube-apiserver and balance traffic between those instances.API-initiated evictionAPI-initiated eviction is the process by which you use the Eviction API to create an Eviction object that triggers graceful pod termination.[+]You can request eviction either by directly calling the Eviction API using a client of the kube-apiserver, like the kubectl drain command. When an Eviction object is created, the API server terminates the Pod.API-initiated evictions respect your configured PodDisruptionBudgets and terminationGracePeriodSeconds.API-initiated eviction is not the same as node-pressure eviction.See API-initiated eviction for more information.App ContainerApplication containers (or app containers) are the containers in a pod that are started after any init containers have completed.[+]An init container lets you separate initialization details that are important for the overall workload, and that don't need to keep running once the application container has started. If a pod doesn't have any init containers configured, all the containers in that pod are app containers.Application ArchitectA person responsible for the high-level design of an application.[+]An architect ensures that an app's implementation allows it to interact with its surrounding components in a scalable, maintainable way. Surrounding components include databases, logging infrastructure, and other microservices.Application DeveloperA person who writes an application that runs in a Kubernetes cluster.[+]An application developer focuses on one part of an application. The scale of their focus may vary significantly in size.ApplicationsThe layer where various containerized applications run. [+]The layer where various containerized applications run.ApproverA person who can review and approve Kubernetes code contributions.[+]While code review is focused on code quality and correctness, approval is focused on the holistic acceptance of a contribution. Holistic acceptance includes backwards/forwards compatibility, adhering to API and flag conventions, subtle performance and correctness issues, interactions with other parts of the system, and others. Approver status is scoped to a part of the codebase. Approvers were previously referred to as maintainers.cAdvisorcAdvisor (Container Advisor) provides container users an understanding of the resource usage and performance characteristics of their running containers.[+]It is a running daemon that collects, aggregates, processes, and exports information about running containers. Specifically, for each container it keeps resource isolation parameters, historical resource usage, histograms of complete historical resource usage and network statistics. This data is exported by container and machine-wide.CertificateA cryptographically secure file used to validate access to the Kubernetes cluster.[+]Certificates enable applications within a Kubernetes cluster to access the Kubernetes API securely. Certificates validate that clients are allowed to access the API.cgroup (control group)A group of Linux processes with optional resource isolation, accounting and limits.[+]cgroup is a Linux kernel feature that limits, accounts for, and isolates the resource usage (CPU, memory, disk I/O, network) for a collection of processes.CIDRCIDR (Classless Inter-Domain Routing) is a notation for describing blocks of IP addresses and is used heavily in various networking configurations.[+]In the context of Kubernetes, each Node is assigned a range of IP addresses through the start address and a subnet mask using CIDR. This allows Nodes to assign each Pod a unique IP address. Although originally a concept for IPv4, CIDR has also been expanded to include IPv6.CLA (Contributor License Agreement)Terms under which a contributor grants a license to an open source project for their contributions.[+]CLAs help resolve legal disputes involving contributed material and intellectual property (IP).Cloud Controller ManagerA Kubernetes control plane component that embeds cloud-specific control logic. The cloud controller manager lets you link your cluster into your cloud provider's API, and separates out the components that interact with that cloud platform from components that only interact with your cluster.[+]By decoupling the interoperability logic between Kubernetes and the underlying cloud infrastructure, the cloud-controller-manager component enables cloud providers to release features at a different pace compared to the main Kubernetes project.Cloud Native Computing Foundation (CNCF)The Cloud Native Computing Foundation (CNCF) builds sustainable ecosystems and fosters a community around projects that orchestrate containers as part of a microservices architecture.Kubernetes is a CNCF project.[+]The CNCF is a sub-foundation of the Linux Foundation. Its mission is to make cloud native computing ubiquitous.Cloud ProviderAlso known as: Cloud Service ProviderA business or other organization that offers a cloud computing platform.[+]Cloud providers, sometimes called Cloud Service Providers (CSPs), offer cloud computing platforms or services.Many cloud providers offer managed infrastructure (also called Infrastructure as a Service or IaaS). With managed infrastructure the cloud provider is responsible for servers, storage, and networking while you manage layers on top of that such as running a Kubernetes cluster.You can also find Kubernetes as a managed service; sometimes called Platform as a Service, or PaaS. With managed Kubernetes, your cloud provider is responsible for the Kubernetes control plane as well as the nodes and the infrastructure they rely on: networking, storage, and possibly other elements such as load balancers.ClusterA set of worker machines, called nodes, that run containerized applications. Every cluster has at least one worker node.[+]The worker node(s) host the Pods that are the components of the application workload. The control plane manages the worker nodes and the Pods in the cluster. In production environments, the control plane usually runs across multiple computers and a cluster usually runs multiple nodes, providing fault-tolerance and high availability.Cluster ArchitectA person who designs infrastructure that involves one or more Kubernetes clusters.[+]Cluster architects are concerned with best practices for distributed systems, for example: high availability and security.Cluster InfrastructureThe infrastructure layer provides and maintains VMs, networking, security groups and others. [+]The infrastructure layer provides and maintains VMs, networking, security groups and others.Cluster OperationsThe work involved in managing a Kubernetes cluster: managing day-to-day operations, and co-ordinating upgrades.[+]Examples of cluster operations work include: deploying new Nodes to scale the cluster; performing software upgrades; implementing security controls; adding or removing storage; configuring cluster networking; managing cluster-wide observability; and responding to events.Cluster OperatorA person who configures, controls, and monitors clusters.[+]Their primary responsibility is keeping a cluster up and running, which may involve periodic maintenance activities or upgrades.Note:Cluster operators are different from the Operator pattern that extends the Kubernetes API.Code ContributorA person who develops and contributes code to the Kubernetes open source codebase.[+]They are also an active community member who participates in one or more Special Interest Groups (SIGs).Common Expression LanguageAlso known as: CELA general-purpose expression language that's designed to be fast, portable, and safe to execute.[+]In Kubernetes, CEL can be used to run queries and perform fine-grained filtering. For example, you can use CEL expressions with dynamic admission control to filter for specific fields in requests, and with dynamic resource allocation (DRA) to select resources based on specific attributes.ConfigMapAn API object used to store non-confidential data in key-value pairs. Pods can consume ConfigMaps as environment variables, command-line arguments, or as configuration files in a volume.[+]A ConfigMap allows you to decouple environment-specific configuration from your container images, so that your applications are easily portable.ContainerA lightweight and portable executable image that contains software and all of its dependencies.[+]Containers decouple applications from underlying host infrastructure to make deployment easier in different cloud or OS environments, and for easier scaling. The applications that run inside containers are called containerized applications. The process of bundling these applications and their dependencies into a container image is called containerization.Container Environment VariablesContainer environment variables are name=value pairs that provide useful information into containers running in a pod[+]Container environment variables provide information that is required by the running containerized applications along with information about important related details to the containers. For example, file system details, information about the container itself, and other cluster resources such as service endpoints.Container Lifecycle HooksThe lifecycle hooks expose events in the Container management lifecycle and let the user run code when the events occur.[+]Two hooks are exposed to Containers: PostStart which executes immediately after a container is created and PreStop which is blocking and is called immediately before a container is terminated.Container network interface (CNI)Container network interface (CNI) plugins are a type of Network plugin that adheres to the appc/CNI specification.[+]For information on Kubernetes and CNI, see Network Plugins.Container RuntimeA fundamental component that empowers Kubernetes to run containers effectively. It is responsible for managing the execution and lifecycle of containers within the Kubernetes environment.[+]Kubernetes supports container runtimes such as containerd, CRI-O, and any other implementation of the Kubernetes CRI (Container Runtime Interface).Container Runtime Interface (CRI)The main protocol for the communication between the kubelet and Container Runtime.[+]The Kubernetes Container Runtime Interface (CRI) defines the main gRPC protocol for the communication between the node components kubelet and container runtime.Container Storage Interface (CSI)The Container Storage Interface (CSI) defines a standard interface to expose storage systems to containers.[+]CSI allows vendors to create custom storage plugins for Kubernetes without adding them to the Kubernetes repository (out-of-tree plugins). To use a CSI driver from a storage provider, you must first deploy it to your cluster. You will then be able to create a Storage Class that uses that CSI driver.CSI in the Kubernetes documentationList of available CSI driverscontainerdA container runtime with an emphasis on simplicity, robustness and portability[+]containerd is a container runtime that runs as a daemon on Linux or Windows. containerd takes care of fetching and storing container images, executing containers, providing network access, and more.ContributorSomeone who donates code, documentation, or their time to help the Kubernetes project or community.[+]Contributions include pull requests (PRs), issues,


# Ephemeral Volumes

Ephemeral VolumesThis document describes ephemeral volumes in Kubernetes. Familiarity with volumes is suggested, in particular PersistentVolumeClaim and PersistentVolume.Some applications need additional storage but don't care whether that data is stored persistently across restarts. For example, caching services are often limited by memory size and can move infrequently used data into storage that is slower than memory with little impact on overall performance.Other applications expect some read-only input data to be present in files, like configuration data or secret keys.Ephemeral volumes are designed for these use cases. Because volumes follow the Pod's lifetime and get created and deleted along with the Pod, Pods can be stopped and restarted without being limited to where some persistent volume is available.Ephemeral volumes are specified inline in the Pod spec, which simplifies application deployment and management.Types of ephemeral volumesKubernetes supports several different kinds of ephemeral volumes for different purposes:emptyDir: empty at Pod startup, with storage coming locally from the kubelet base directory (usually the root disk) or RAMconfigMap, downwardAPI, secret: inject different kinds of Kubernetes data into a Podimage: allows mounting container image files or artifacts, directly to a Pod.CSI ephemeral volumes: similar to the previous volume kinds, but provided by special CSI drivers which specifically support this featuregeneric ephemeral volumes, which can be provided by all storage drivers that also support persistent volumesemptyDir, configMap, downwardAPI, secret are provided as local ephemeral storage. They are managed by kubelet on each node.CSI ephemeral volumes must be provided by third-party CSI storage drivers.Generic ephemeral volumes can be provided by third-party CSI storage drivers, but also by any other storage driver that supports dynamic provisioning. Some CSI drivers are written specifically for CSI ephemeral volumes and do not support dynamic provisioning: those then cannot be used for generic ephemeral volumes.The advantage of using third-party drivers is that they can offer functionality that Kubernetes itself does not support, for example storage with different performance characteristics than the disk that is managed by kubelet, or injecting different data.CSI ephemeral volumesFEATURE STATE: Kubernetes v1.25 [stable]Note:CSI ephemeral volumes are only supported by a subset of CSI drivers. The Kubernetes CSI Drivers list shows which drivers support ephemeral volumes.Conceptually, CSI ephemeral volumes are similar to configMap, downwardAPI and secret volume types: the storage is managed locally on each node and is created together with other local resources after a Pod has been scheduled onto a node. Kubernetes has no concept of rescheduling Pods anymore at this stage. Volume creation has to be unlikely to fail, otherwise Pod startup gets stuck. In particular, storage capacity aware Pod scheduling is not supported for these volumes. They are currently also not covered by the storage resource usage limits of a Pod, because that is something that kubelet can only enforce for storage that it manages itself.Here's an example manifest for a Pod that uses CSI ephemeral storage:kind: Pod apiVersion: v1 metadata: name: my-csi-app spec: containers: - name: my-frontend image: busybox:1.28 volumeMounts: - mountPath: "/data" name: my-csi-inline-vol command: [ "sleep", "1000000" ] volumes: - name: my-csi-inline-vol csi: driver: inline.storage.kubernetes.io volumeAttributes: foo: bar The volumeAttributes determine what volume is prepared by the driver. These attributes are specific to each driver and not standardized. See the documentation of each CSI driver for further instructions.CSI driver restrictionsCSI ephemeral volumes allow users to provide volumeAttributes directly to the CSI driver as part of the Pod spec. A CSI driver allowing volumeAttributes that are typically restricted to administrators is NOT suitable for use in an inline ephemeral volume. For example, parameters that are normally defined in the StorageClass should not be exposed to users through the use of inline ephemeral volumes.Cluster administrators who need to restrict the CSI drivers that are allowed to be used as inline volumes within a Pod spec may do so by:Removing Ephemeral from volumeLifecycleModes in the CSIDriver spec, which prevents the driver from being used as an inline ephemeral volume.Using an admission webhook to restrict how this driver is used.Generic ephemeral volumesFEATURE STATE: Kubernetes v1.23 [stable]Generic ephemeral volumes are similar to emptyDir volumes in the sense that they provide a per-pod directory for scratch data that is usually empty after provisioning. But they may also have additional features:Storage can be local or network-attached.Volumes can have a fixed size that Pods are not able to exceed.Volumes may have some initial data, depending on the driver and parameters.Typical operations on volumes are supported assuming that the driver supports them, including snapshotting, cloning, resizing, and storage capacity tracking.Example:kind: Pod apiVersion: v1 metadata: name: my-app spec: containers: - name: my-frontend image: busybox:1.28 volumeMounts: - mountPath: "/scratch" name: scratch-volume command: [ "sleep", "1000000" ] volumes: - name: scratch-volume ephemeral: volumeClaimTemplate: metadata: labels: type: my-frontend-volume spec: accessModes: [ "ReadWriteOnce" ] storageClassName: "scratch-storage-class" resources: requests: storage: 1Gi Lifecycle and PersistentVolumeClaimThe key design idea is that the parameters for a volume claim are allowed inside a volume source of the Pod. Labels, annotations and the whole set of fields for a PersistentVolumeClaim are supported. When such a Pod gets created, the ephemeral volume controller then creates an actual PersistentVolumeClaim object in the same namespace as the Pod and ensures that the PersistentVolumeClaim gets deleted when the Pod gets deleted.That triggers volume binding and/or provisioning, either immediately if the StorageClass uses immediate volume binding or when the Pod is tentatively scheduled onto a node (WaitForFirstConsumer volume binding mode). The latter is recommended for generic ephemeral volumes because then the scheduler is free to choose a suitable node for the Pod. With immediate binding, the scheduler is forced to select a node that has access to the volume once it is available.In terms of resource ownership, a Pod that has generic ephemeral storage is the owner of the PersistentVolumeClaim(s) that provide that ephemeral storage. When the Pod is deleted, the Kubernetes garbage collector deletes the PVC, which then usually triggers deletion of the volume because the default reclaim policy of storage classes is to delete volumes. You can create quasi-ephemeral local storage using a StorageClass with a reclaim policy of retain: the storage outlives the Pod, and in this case you need to ensure that volume clean up happens separately.While these PVCs exist, they can be used like any other PVC. In particular, they can be referenced as data source in volume cloning or snapshotting. The PVC object also holds the current status of the volume.PersistentVolumeClaim namingNaming of the automatically created PVCs is deterministic: the name is a combination of the Pod name and volume name, with a hyphen (-) in the middle. In the example above, the PVC name will be my-app-scratch-volume. This deterministic naming makes it easier to interact with the PVC because one does not have to search for it once the Pod name and volume name are known.The deterministic naming also introduces a potential conflict between different Pods (a Pod "pod-a" with volume "scratch" and another Pod with name "pod" and volume "a-scratch" both end up with the same PVC name "pod-a-scratch") and between Pods and manually created PVCs.Such conflicts are detected: a PVC is only used for an ephemeral volume if it was created for the Pod. This check is based on the ownership relationship. An existing PVC is not overwritten or modified. But this does not resolve the conflict because without the right PVC, the Pod cannot start.Caution:Take care when naming Pods and volumes inside the same namespace, so that these conflicts can't occur.SecurityUsing generic ephemeral volumes allows users to create PVCs indirectly if they can create Pods, even if they do not have permission to create PVCs directly. Cluster administrators must be aware of this. If this does not fit their security model, they should use an admission webhook that rejects objects like Pods that have a generic ephemeral volume.The normal namespace quota for PVCs still applies, so even if users are allowed to use this new mechanism, they cannot use it to circumvent other policies.What's nextEphemeral volumes managed by kubeletSee local ephemeral storage.CSI ephemeral volumesFor more information on the design, see the Ephemeral Inline CSI volumes KEP.For more information on further development of this feature, see the enhancement tracking issue #596.Generic ephemeral volumesFor more information on the design, see the Generic ephemeral inline volumes KEP.