Link Search Menu Expand Document

Insecure Functionality Exposed in Kubernetes

Kubernetes has many components and functionalities that need to be carefully configured in order to maintain the security of the cluster.

API Server

The API server is the pivotal point where the cluster’s security configuration is applied. All the other components are able to interact directly with the cluster’s state store via its REST API.

For this reason, client access to the Kubernetes API should be authenticated, authorized, and encrypted.

The insecure HTTP port (set with --insecure-port and --insecure-bind-address) is intended for local testing; it should never be exposed other than to localhost.

Kubelet

Kubelet is an agent running on every node that should be configured correctly to prevent unauthorized access from the pods.

In prior versions of Kubernetes (prior to 1.7), Kubelet had read-write access to the Node and Pod APIs, allowing, if compromised, numerous resources to be taken control of. More recent versions introduced the Node Authorizer which limits Kubelet to access only resources that are related specifically to the node on which Kubelet runs.

REST APIs are exposed on ports 10250 (read/write) and 10255 (read-only) ports.

Read-write port

Unrestricted access to the read-write port could result in a take over of the cluster, as it’s possible to run commands inside the pod’s containers or start new resources. This can’t be fully disabled because it is needed for collecting metrics and other important functions, so the access should be carefully controlled.

Kubelet should be configured to enforce authentication of all clients. The methods that can be enabled are X.509 client certificates, or requests with Authorization headers containing authentication tokens.

In the case of X.509 client certificates, the contents of a CA bundle need to be made available to Kubelet (--client-ca-file, authentication.x509.clientCAFile).

By default, when receiving anonymous requests Kubelet passes them on for authorization, rather than rejecting them.

The authorization setting (authorization.mode, --authorization-mode) can operate in one of two modes: AlwaysAllow (default) or Webhook. The first option disables the authentication, while the Webhook value performs further checks to determine whether the subject is allowed to make the request or not.

To disable anonymous access and send 401 Unauthorized responses to unauthenticated requests, the anonymous settings (authentication.anonymous.enabled, --anonymous-auth) must be set to false.

Read-only port

The read-only access could be also abused to disclose sensitive information regarding the cluster.

For this reason, the read-only port should be disabled by removing the option, or by setting the option (readOnlyPort, --read-only-port ) to 0.

Etcd

Kubernetes uses etcd key-value store for storing cluster state and configurations. For this reason, getting write access to the etcd backend for the APIs is equivalent to granting cluster-admin access, and read access can be used to escalate fairly quickly. It listens on TCP port 2379 for client and TCP port 2380 for server-to-server communications.

In versions prior to etcd 2.1, authentication did not exist; however, currently etcd supports strong credentials such as mutual authentication via client certificates. This should always be used by creating a new key pair for every entity in the cluster that needs to access the etcd API.

Namespaces

Namespaces allow for the creation of logical partitions, enforce separation of resources, as well as limit the scope of the permissions. Namespaces do not enforce any network segmentation, but are used to isolate resource groups into logically named groups.

Network

By default, no network segmentation is in place, and every pod can communicate with every other pod.

To address this problem, use a network plugin that firmly enforces network policies, or implement full separation of Kubernetes clusters to ensure that attackers can’t exploit lax network policies to jump across network segments.

Services

Since Kubernetes pods are mortal and continuously receive new IP addresses, Services pinpoint an IP address to connect to when referring to a pod or a set of pods.

Services can be of various types:

  • ClusterIP: Exposes the Service on an IP only reachable from within the cluster. This is the default ServiceType.

  • NodePort: Exposes the Service at a static port, reachable from outside the cluster.

  • LoadBalancer: Exposes the Service externally using a cloud provider’s load balancer.

  • ExternalName: Maps the Service to the contents of a CNAME record

Kubernetes clusters usually run a DNS add-on to manage the internal names and creates a set of DNS records for each one, meaning all pods should automatically resolve Services by their DNS name.

References

Kubernetes - Accessing the API

Kubernetes - Namespaces

Kubernetes - Securing a Cluster