Identity and Access Management¶
Identity and Access Management (IAM) is an AWS service that performs two essential functions: Authentication and Authorization. Authentication involves the verification of a identity whereas authorization governs the actions that can be performed by AWS resources. Within AWS, a resource can be another AWS service, e.g. EC2, or an AWS principal such as an IAM User or Role. The rules governing the actions that a resource is allowed to perform are expressed as IAM policies.
Controlling Access to EKS Clusters¶
The Kubernetes project supports a variety of different strategies to authenticate requests to the kube-apiserver service, e.g. Bearer Tokens, X.509 certificates, OIDC, etc. EKS currently has native support for webhook token authentication, service account tokens, and as of February 21, 2021, OIDC authentication.
The webhook authentication strategy calls a webhook that verifies bearer tokens. On EKS, these bearer tokens are generated by the AWS CLI or the aws-iam-authenticator client when you run kubectl commands. As you execute commands, the token is passed to the kube-apiserver which forwards it to the authentication webhook. If the request is well-formed, the webhook calls a pre-signed URL embedded in the token's body. This URL validates the request's signature and returns information about the user, e.g. the user's account, Arn, and UserId to the kube-apiserver.
To manually generate a authentication token, type the following command in a terminal window:
You can also get a token programmatically. Below is an example written in Go:
package main
import (
"fmt"
"log"
"sigs.k8s.io/aws-iam-authenticator/pkg/token"
)
func main() {
g, _ := token.NewGenerator(false, false)
tk, err := g.Get("<cluster_name>")
if err != nil {
log.Fatal(err)
}
fmt.Println(tk)
}
The output should resemble this:
{
"kind": "ExecCredential",
"apiVersion": "client.authentication.k8s.io/v1alpha1",
"spec": {},
"status": {
"expirationTimestamp": "2020-02-19T16:08:27Z",
"token": "k8s-aws-v1.aHR0cHM6Ly9zdHMuYW1hem9uYXdzLmNvbS8_QWN0aW9uPUdldENhbGxlcklkZW50aXR5JlZlcnNpb249MjAxMS0wNi0xNSZYLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFKTkdSSUxLTlNSQzJXNVFBJTJGMjAyMDAyMTklMkZ1cy1lYXN0LTElMkZzdHMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDIwMDIxOVQxNTU0MjdaJlgtQW16LUV4cGlyZXM9NjAmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JTNCeC1rOHMtYXdzLWlkJlgtQW16LVNpZ25hdHVyZT0yMjBmOGYzNTg1ZTMyMGRkYjVlNjgzYTVjOWE0MDUzMDFhZDc2NTQ2ZjI0ZjI4MTExZmRhZDA5Y2Y2NDhhMzkz"
}
}
k8s-aws-v1. followed by a base64 encoded string. The string, when decoded, should resemble this:
https://sts.amazonaws.com/?Action=GetCallerIdentity&Version=2011-06-15&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAJPFRILKNSRC2W5QA%2F20200219%2Fus-east-1%2Fsts%2Faws4_request&X-Amz-Date=20200219T155427Z&X-Amz-Expires=60&X-Amz-SignedHeaders=host%3Bx-k8s-aws-id&X-Amz-Signature=220f8f3285e320ddb5e683a5c9a405301ad76546f24f28111fdad09cf648a393
The token has a time to live (TTL) of 15 minutes after which a new token will need to be generated. This is handled automatically when you use a client like kubectl, however, if you're using the Kubernetes dashboard, you will need to generate a new token and re-authenticate each time the token expires.
Once the user's identity has been authenticated by the AWS IAM service, the kube-apiserver reads the aws-auth ConfigMap in the kube-system Namespace to determine the RBAC group to associate with the user. The aws-auth ConfigMap is used to create a static mapping between IAM principals, i.e. IAM Users and Roles, and Kubernetes RBAC groups. RBAC groups can be referenced in Kubernetes RoleBindings or ClusterRoleBindings. They are similar to IAM Roles in that they define a set of actions (verbs) that can be performed against a collection of Kubernetes resources (objects).
Recommendations¶
Don't use a service account token for authentication¶
A service account token is a long-lived, static credential. If it is compromised, lost, or stolen, an attacker may be able to perform all the actions associated with that token until the service account is deleted. At times, you may need to grant an exception for applications that have to consume the Kubernetes API from outside the cluster, e.g. a CI/CD pipeline application. If such applications run on AWS infrastructure, like EC2 instances, consider using an instance profile and mapping that to a Kubernetes RBAC role in the aws-auth ConfigMap instead.
Employ least privileged access to AWS Resources¶
An IAM User does not need to be assigned privileges to AWS resources to access the Kubernetes API. If you need to grant an IAM user access to an EKS cluster, create an entry in the aws-auth ConfigMap for that user that maps to a specific Kubernetes RBAC group.
Use IAM Roles when multiple users need identical access to the cluster¶
Rather than creating an entry for each individual IAM User in the aws-auth ConfigMap, allow those users to assume an IAM Role and map that role to a Kubernetes RBAC group. This will be easier to maintain, especially as the number of users that require access grows.
Attention
When accessing the EKS cluster with the IAM entity mapped by aws-auth ConfigMap, the username described in aws-auth ConfigMap is recorded in the user field of the Kubernetes audit log. If you're using an IAM role, the actual users who assume that role aren't recorded and can't be audited.
When assigning K8s RBAC permissions to an IAM role using mapRoles in aws-auth ConfigMap, you should include {{SessionName}} in your username. That way, the audit log will record the session name so you can track who the actual user assume this role along with the CloudTrail log.
- rolearn: arn:aws:iam::XXXXXXXXXXXX:role/testRole
username: testRole:{{SessionName}}
groups:
- system:masters
In Kubernetes 1.20 and above, this change is no longer required, since user.extra.sessionName.0 was added to the Kubernetes audit log.
Employ least privileged access when creating RoleBindings and ClusterRoleBindings¶
Like the earlier point about granting access to AWS Resources, RoleBindings and ClusterRoleBindings should only include the set of permissions necessary to perform a specific function. Avoid using ["*"] in your Roles and ClusterRoles unless it's absolutely necessary. If you're unsure what permissions to assign, consider using a tool like audit2rbac to automatically generate Roles and binding based on the observed API calls in the Kubernetes Audit Log.
Make the EKS Cluster Endpoint private¶
By default when you provision an EKS cluster, the API cluster endpoint is set to public, i.e. it can be accessed from the Internet. Despite being accessible from the Internet, the endpoint is still considered secure because it requires all API requests to be authenticated by IAM and then authorized by Kubernetes RBAC. That said, if your corporate security policy mandates that you restrict access to the API from the Internet or prevents you from routing traffic outside the cluster VPC, you can:
- Configure the EKS cluster endpoint to be private. See Modifying Cluster Endpoint Access for further information on this topic.
- Leave the cluster endpoint public and specify which CIDR blocks can communicate with the cluster endpoint. The blocks are effectively a whitelisted set of public IP addresses that are allowed to access the cluster endpoint.
- Configure public access with a set of whitelisted CIDR blocks and set private endpoint access to enabled. This will allow public access from a specific range of public IPs while forcing all network traffic between the kubelets (workers) and the Kubernetes API through the cross-account ENIs that get provisioned into the cluster VPC when the control plane is provisioned.
Create the cluster with a dedicated IAM role¶
When you create an Amazon EKS cluster, the IAM entity user or role, such as a federated user that creates the cluster, is automatically granted system:masters permissions in the cluster's RBAC configuration. This access cannot be removed and is not managed through the aws-auth ConfigMap. Therefore it is a good idea to create the cluster with a dedicated IAM role and regularly audit who can assume this role. This role should not be used to perform routine actions on the cluster, and instead additional users should be granted access to the cluster through the aws-auth ConfigMap for this purpose. After the aws-auth ConfigMap is configured, the role can be deleted and only recreated in an emergency / break glass scenario where the aws-auth ConfigMap is corrupted and the cluster is otherwise inaccessible. This can be particularly useful in production clusters which do not usually have direct user access configured.
Use tools to make changes to the aws-auth ConfigMap¶
An improperly formatted aws-auth ConfigMap may cause you to lose access to the cluster. If you need to make changes to the ConfigMap, use a tool.
eksctl
The eksctl CLI includes a command for adding identity mappings to the aws-auth
ConfigMap.
View CLI Help:
Make an IAM Role a Cluster Admin:
eksctl create iamidentitymapping --cluster <clusterName> --region=<region> --arn arn:aws:iam::123456:role/testing --group system:masters --username admin
For more information, review eksctl docs
aws-auth by keikoproj
aws-auth by keikoproj includes both a cli and a go library.
Download and view help CLI help:
Alternatively, install aws-auth with the krew plugin manager for kubectl.
Review the aws-auth docs on GitHub for more information, including the go library.
The aws-iam-authenticator project includes a CLI for updating the ConfigMap.
Download a release on GitHub.
Add cluster permissions to an IAM Role:
./aws-iam-authenticator add role --rolearn arn:aws:iam::185309785115:role/lil-dev-role-cluster --username lil-dev-user --groups system:masters --kubeconfig ~/.kube/config
Regularly audit access to the cluster¶
Who requires access is likely to change over time. Plan to periodically audit the aws-auth ConfigMap to see who has been granted access and the rights they've been assigned. You can also use open source tooling like kubectl-who-can, or rbac-lookup to examine the roles bound to a particular service account, user, or group. We'll explore this topic further when we get to the section on auditing. Additional ideas can be found in this article from NCC Group.
Alternative Approaches to Authentication and Access Management¶
While IAM is the preferred way to authenticate users who need access to an EKS cluster, it is possible to use an OIDC identity provider such as GitHub using an authentication proxy and Kubernetes impersonation. Posts for two such solutions have been published on the AWS Open Source blog:
- Authenticating to EKS Using GitHub Credentials with Teleport
- Consistent OIDC authentication across multiple EKS clusters using kube-oidc-proxy
Attention
EKS natively supports OIDC authentication without using a proxy. For further information, please read the launch blog, Introducing OIDC identity provider authentication for Amazon EKS. For an example showing how to configure EKS with Dex, a popular open source OIDC provider with connectors for a variety of different authention methods, see Using Dex & dex-k8s-authenticator to authenticate to Amazon EKS. As described in the blogs, the username/group of users authenticated by an OIDC provider will appear in the Kubernetes audit log.
You can also use AWS SSO to federate AWS with an external identity provider, e.g. Azure AD. If you decide to use this, the AWS CLI v2.0 includes an option to create a named profile that makes it easy to associate an SSO session with your current CLI session and assume an IAM role. Know that you must assume a role prior to running kubectl as the IAM role is used to determine the user's Kubernetes RBAC group.
Additional Resources¶
rbac.dev A list of additional resources, including blogs and tools, for Kubernetes RBAC
Pods Identities¶
Certain applications that run within a Kubernetes cluster need permission to call the Kubernetes API to function properly. For example, the AWS Load Balancer Controller needs to be able to list a Service's Endpoints. The controller also needs to be able to invoke AWS APIs to provision and configure an ALB. In this section we will explore the best practices for assigning rights and privileges to Pods.
Kubernetes Service Accounts¶
A service account is a special type of object that allows you to assign a Kubernetes RBAC role to a pod. A default service account is created automatically for each Namespace within a cluster. When you deploy a pod into a Namespace without referencing a specific service account, the default service account for that Namespace will automatically get assigned to the Pod and the Secret, i.e. the service account (JWT) token for that service account, will get mounted to the pod as a volume at /var/run/secrets/kubernetes.io/serviceaccount. Decoding the service account token in that directory will reveal the following metadata:
{
"iss": "kubernetes/serviceaccount",
"kubernetes.io/serviceaccount/namespace": "default",
"kubernetes.io/serviceaccount/secret.name": "default-token-5pv4z",
"kubernetes.io/serviceaccount/service-account.name": "default",
"kubernetes.io/serviceaccount/service-account.uid": "3b36ddb5-438c-11ea-9438-063a49b60fba",
"sub": "system:serviceaccount:default:default"
}
The default service account has the following permissions to the Kubernetes API.
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
annotations:
rbac.authorization.kubernetes.io/autoupdate: "true"
creationTimestamp: "2020-01-30T18:13:25Z"
labels:
kubernetes.io/bootstrapping: rbac-defaults
name: system:discovery
resourceVersion: "43"
selfLink: /apis/rbac.authorization.k8s.io/v1/clusterroles/system%3Adiscovery
uid: 350d2ab8-438c-11ea-9438-063a49b60fba
rules:
- nonResourceURLs:
- /api
- /api/*
- /apis
- /apis/*
- /healthz
- /openapi
- /openapi/*
- /version
- /version/
verbs:
- get
When an application running within a Pod calls the Kubernetes APIs, the Pod needs to be assigned a service account that explicitly grants it permission to call those APIs. Similar to guidelines for user access, the Role or ClusterRole bound to a service account should be restricted to the API resources and methods that the application needs to function and nothing else. To use a non-default service account simply set the spec.serviceAccountName field of a Pod to the name of the service account you wish to use. For additional information about creating service accounts, see https://kubernetes.io/docs/reference/access-authn-authz/rbac/#service-account-permissions.
Note
Prior to Kubernetes 1.24, Kubernetes would automatically create a secret for each a service account. This secret was mounted to the pod at /var/run/secrets/kubernetes.io/serviceaccount and would be used by the pod to authenticate to the Kubernetes API server. In Kubernetes 1.24, a service account token is dynamically generated when the pod runs and is only valid for an hour by default. A secret for the service account will not be created. If you have an application that runs outside the cluster that needs to authenticate to the Kubernetes API, e.g. Jenkins, you will need to create a secret of type kubernetes.io/service-account-token along with an annotation that references the service account such as metadata.annotations.kubernetes.io/service-account.name: <SERVICE_ACCOUNT_NAME>. Secrets created in this way do not expire.
IAM Roles for Service Accounts (IRSA)¶
IRSA is a feature that allows you to assign an IAM role to a Kubernetes service account. It works by leveraging a Kubernetes feature known as Service Account Token Volume Projection. When Pods are configured with a Service Account that references an IAM Role, the Kubernetes API server will call the public OIDC discovery endpoint for the cluster on startup. The endpoint cryptographically signs the OIDC token issued by Kubernetes and the resulting token mounted as a volume. This signed token allows the Pod to call the AWS APIs associated IAM role. When an AWS API is invoked, the AWS SDKs calls sts:AssumeRoleWithWebIdentity. After validating the token's signature, IAM exchanges the Kubernetes issued token for a temporary AWS role credential.
Decoding the (JWT) token for IRSA will produce output similar to the example you see below:
{
"aud": [
"sts.amazonaws.com"
],
"exp": 1582306514,
"iat": 1582220114,
"iss": "https://oidc.eks.us-west-2.amazonaws.com/id/D43CF17C27A865933144EA99A26FB128",
"kubernetes.io": {
"namespace": "default",
"pod": {
"name": "alpine-57b5664646-rf966",
"uid": "5a20f883-5407-11ea-a85c-0e62b7a4a436"
},
"serviceaccount": {
"name": "s3-read-only",
"uid": "a720ba5c-5406-11ea-9438-063a49b60fba"
}
},
"nbf": 1582220114,
"sub": "system:serviceaccount:default:s3-read-only"
}
{
"AssumedRoleUser": {
"AssumedRoleId": "AROA36C6WWEJULFUYMPB6:abc",
"Arn": "arn:aws:sts::123456789012:assumed-role/eksctl-winterfell-addon-iamserviceaccount-de-Role1-1D61LT75JH3MB/abc"
},
"Audience": "sts.amazonaws.com",
"Provider": "arn:aws:iam::123456789012:oidc-provider/oidc.eks.us-west-2.amazonaws.com/id/D43CF17C27A865933144EA99A26FB128",
"SubjectFromWebIdentityToken": "system:serviceaccount:default:s3-read-only",
"Credentials": {
"SecretAccessKey": "ORJ+8Adk+wW+nU8FETq7+mOqeA8Z6jlPihnV8hX1",
"SessionToken": "FwoGZXIvYXdzEGMaDMLxAZkuLpmSwYXShiL9A1S0X87VBC1mHCrRe/pB2oes+l1eXxUYnPJyC9ayOoXMvqXQsomq0xs6OqZ3vaa5Iw1HIyA4Cv1suLaOCoU3hNvOIJ6C94H1vU0siQYk7DIq9Av5RZe+uE2FnOctNBvYLd3i0IZo1ajjc00yRK3v24VRq9nQpoPLuqyH2jzlhCEjXuPScPbi5KEVs9fNcOTtgzbVf7IG2gNiwNs5aCpN4Bv/Zv2A6zp5xGz9cWj2f0aD9v66vX4bexOs5t/YYhwuwAvkkJPSIGvxja0xRThnceHyFHKtj0H+bi/PWAtlI8YJcDX69cM30JAHDdQH+ltm/4scFptW1hlvMaP+WReCAaCrsHrAT+yka7ttw5YlUyvZ8EPog+j6fwHlxmrXM9h1BqdikomyJU00gm1++FJelfP+1zAwcyrxCnbRl3ARFrAt8hIlrT6Vyu8WvWtLxcI8KcLcJQb/LgkW+sCTGlYcY8z3zkigJMbYn07ewTL5Ss7LazTJJa758I7PZan/v3xQHd5DEc5WBneiV3iOznDFgup0VAMkIviVjVCkszaPSVEdK2NU7jtrh6Jfm7bU/3P6ZG+CkyDLIa8MBn9KPXeJd/y+jTk5Ii+fIwO/+mDpGNUribg6TPxhzZ8b/XdZO1kS1gVgqjXyVC+M+BRBh6C4H21w/eMzjCtDIpoxt5rGKL6Nu/IFMipoC4fgx6LIIHwtGYMG7SWQi7OsMAkiwZRg0n68/RqWgLzBt/4pfjSRYuk=",
"Expiration": "2020-02-20T18:49:50Z",
"AccessKeyId": "ASIA36C6WWEJUMHA3L7Z"
}
}
A mutating webhook that runs as part of the EKS control plane injects the AWS Role ARN and the path to a web identity token file into the Pod as environment variables. These values can also be supplied manually.
AWS_ROLE_ARN=arn:aws:iam::AWS_ACCOUNT_ID:role/IAM_ROLE_NAME
AWS_WEB_IDENTITY_TOKEN_FILE=/var/run/secrets/eks.amazonaws.com/serviceaccount/token
The kubelet will automatically rotate the projected token when it is older than 80% of its total TTL, or after 24 hours. The AWS SDKs are responsible for reloading the token when it rotates. For further information about IRSA, see https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts-technical-overview.html.
Recommendations¶
Update the aws-node daemonset to use IRSA¶
At present, the aws-node daemonset is configured to use a role assigned to the EC2 instances to assign IPs to pods. This role includes several AWS managed policies, e.g. AmazonEKS_CNI_Policy and EC2ContainerRegistryReadOnly that effectively allow all pods running on a node to attach/detach ENIs, assign/unassign IP addresses, or pull images from ECR. Since this presents a risk to your cluster, it is recommended that you update the aws-node daemonset to use IRSA. A script for doing this can be found in the repository for this guide.
Restrict access to the instance profile assigned to the worker node¶
When you use IRSA, it updates the credential chain of the pod to use the IRSA token, however, the pod can still inherit the rights of the instance profile assigned to the worker node. When using IRSA, it is strongly recommended that you block access instance metadata to minimize the blast radius of a breach.
Caution
Blocking access to instance metadata will prevent pods that do not use IRSA from inheriting the role assigned to the worker node.
You can block access to instance metadata by requiring the instance to use IMDSv2 only and updating the hop count to 1 as in the example below. You can also include these settings in the node group's launch template. Do not disable instance metadata as this will prevent components like the node termination handler and other things that rely on instance metadata from working properly.
aws ec2 modify-instance-metadata-options --instance-id <value> --http-tokens required --http-put-response-hop-limit 1
If you are using Terraform to create launch templates for use with Managed Node Groups, add the metadata block to configure the hop count as seen in this code snippet:
resource "aws_launch_template" "foo" {
name = "foo"
...
metadata_options {
http_endpoint = "enabled"
http_tokens = "required"
http_put_response_hop_limit = 1
instance_metadata_tags = "enabled"
}
...
You can also block a pod's access to EC2 metadata by manipulating iptables on the node. For further information about this method, see Limiting access to the instance metadata service.
If you have an application that is using an older version of the AWS SDK that doesn't support IRSA, and you want the pod to inherit the role assigned to the instance, consider using Kubernetes network policies to selectively allow access EC2 metadata.
Start with a policy that blocks access to the metadata service from all Pods:
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: deny-metadata-access
namespace: example
spec:
podSelector: {}
policyTypes:
- Egress
egress:
- to:
- ipBlock:
cidr: 0.0.0.0/0
except:
- 169.254.169.254/32
Then allow access from select pods by adding following policy, modifying the PodSelector as appropriate.
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: allow-metadata-access
namespace: example
spec:
podSelector:
matchLabels:
app: myapp
policyTypes:
- Egress
egress:
- to:
- ipBlock:
cidr: 169.254.169.254/32
Scope the IAM Role trust policy for IRSA to the service account name¶
The trust policy can be scoped to a Namespace or a specific service account within a Namespace. When using IRSA it's best to make the role trust policy as explicit as possible by including the service account name. This will effectively prevent other Pods within the same Namespace from assuming the role. The CLI eksctl will do this automatically when you use it to create service accounts/IAM roles. See https://eksctl.io/usage/iamserviceaccounts/ for further information.
When your application needs access to IMDS, use IMDSv2 and increase the hop limit on EC2 instances to 2¶
IMDSv2 requires you use a PUT request to get a session token. The initial PUT request has to include a TTL for the session token. Newer versions of the AWS SDKs will handle this and the renewal of said token automatically. It's also important to be aware that the default hop limit on EC2 instances is intentionally set to 1 to prevent IP forwarding. As a consequence, Pods that request a session token that are run on EC2 instances may eventually time out and fallback to using the IMDSv1 data flow. EKS adds support IMDSv2 by enabling both v1 and v2 and changing the hop limit to 2 on nodes provisioned by eksctl or with the official CloudFormation templates.
Disable auto-mounting of service account tokens¶
If your application doesn't need to call the Kubernetes API set the automountServiceAccountToken attribute to false in the PodSpec for your application or patch the default service account in each namespace so that it's no longer mounted to pods automatically. For example:
Use dedicated service accounts for each application¶
Each application should have its own dedicated service account. This applies to service accounts for the Kubernetes API as well as IRSA.
Attention
If you employ a blue/green approach to cluster upgrades instead of performing an in-place cluster upgrade, you will need to update the trust policy of each of the IRSA IAM roles with the OIDC endpoint of the new cluster. A blue/green cluster upgrade is where you create a cluster running a newer version of Kubernetes alongside the old cluster and use a load balancer or a service mesh to seamlessly shift traffic from services running on the old cluster to the new cluster.
Run the application as a non-root user¶
Containers run as root by default. While this allows them to read the web identity token file, running a container as root is not considered a best practice. As an alternative, consider adding the spec.securityContext.runAsUser attribute to the PodSpec. The value of runAsUser is arbitrary value.
In the following example, all processes within the Pod will run under the user ID specified in the runAsUser field.
apiVersion: v1
kind: Pod
metadata:
name: security-context-demo
spec:
securityContext:
runAsUser: 1000
runAsGroup: 3000
containers:
- name: sec-ctx-demo
image: busybox
command: [ "sh", "-c", "sleep 1h" ]
When you run a container as a non-root user, it prevents the container from reading the IRSA service account token because the token is assigned 0600 [root] permissions by default. If you update the securityContext for your container to include fsgroup=65534 [Nobody] it will allow the container to read the token.
In Kubernetes 1.19 and above, this change is no longer required.
Grant least privileged access to applications¶
Action Hero is a utility that you can run alongside your application to identify the AWS API calls and corresponding IAM permissions your application needs to function properly. It is similar to IAM Access Advisor in that it helps you gradually limit the scope of IAM roles assigned to applications. Consult the documentation on granting least privileged access to AWS resources for further information.
Review and revoke unnecessary anonymous access¶
Ideally anonymous access should be disabled for all API actions. Anonymous access is granted by creating a RoleBinding or ClusterRoleBinding for the Kubernetes built-in user system:anonymous. You can use the rbac-lookup tool to identify permissions that system:anonymous user has on your cluster:
./rbac-lookup | grep -P 'system:(anonymous)|(unauthenticated)'
system:anonymous cluster-wide ClusterRole/system:discovery
system:unauthenticated cluster-wide ClusterRole/system:discovery
system:unauthenticated cluster-wide ClusterRole/system:public-info-viewer
Any role or ClusterRole other than system:public-info-viewer should not be bound to system:anonymous user or system:unauthenticated group.
There may be some legitimate reasons to enable anonymous access on specific APIs. If this is the case for your cluster ensure that only those specific APIs are accessible by anonymous user and exposing those APIs without authentication doesn’t make your cluster vulnerable.
Prior to Kubernetes/EKS Version 1.14, system:unauthenticated group was associated to system:discovery and system:basic-user ClusterRoles by default. Note that even if you have updated your cluster to version 1.14 or higher, these permissions may still be enabled on your cluster, since cluster updates do not revoke these permissions. To check which ClusterRoles have "system:unauthenticated" except system:public-info-viewer you can run the following command (requires jq util):
kubectl get ClusterRoleBinding -o json | jq -r '.items[] | select(.subjects[]?.name =="system:unauthenticated") | select(.metadata.name != "system:public-info-viewer") | .metadata.name'
And "system:unauthenticated" can be removed from all the roles except "system:public-info-viewer" using:
kubectl get ClusterRoleBinding -o json | jq -r '.items[] | select(.subjects[]?.name =="system:unauthenticated") | select(.metadata.name != "system:public-info-viewer") | del(.subjects[] | select(.name =="system:unauthenticated"))' | kubectl apply -f -
Alternatively, you can check and remove it manually by kubectl describe and kubectl edit. To check if system:unauthenticated group has system:discovery permissions on your cluster run the following command:
kubectl describe clusterrolebindings system:discovery
Name: system:discovery
Labels: kubernetes.io/bootstrapping=rbac-defaults
Annotations: rbac.authorization.kubernetes.io/autoupdate: true
Role:
Kind: ClusterRole
Name: system:discovery
Subjects:
Kind Name Namespace
---- ---- ---------
Group system:authenticated
Group system:unauthenticated
To check if system:unauthenticated group has system:basic-user permission on your cluster run the following command:
kubectl describe clusterrolebindings system:basic-user
Name: system:basic-user
Labels: kubernetes.io/bootstrapping=rbac-defaults
Annotations: rbac.authorization.kubernetes.io/autoupdate: true
Role:
Kind: ClusterRole
Name: system:basic-user
Subjects:
Kind Name Namespace
---- ---- ---------
Group system:authenticated
Group system:unauthenticated
If system:unauthenticated group is bound to system:discovery and/or system:basic-user ClusterRoles on your cluster, you should disassociate these roles from system:unauthenticated group. Edit system:discovery ClusterRoleBinding using the following command:
The above command will open the current definition of system:discovery ClusterRoleBinding in an editor as shown below:# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will be
# reopened with the relevant failures.
#
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
annotations:
rbac.authorization.kubernetes.io/autoupdate: "true"
creationTimestamp: "2021-06-17T20:50:49Z"
labels:
kubernetes.io/bootstrapping: rbac-defaults
name: system:discovery
resourceVersion: "24502985"
selfLink: /apis/rbac.authorization.k8s.io/v1/clusterrolebindings/system%3Adiscovery
uid: b7936268-5043-431a-a0e1-171a423abeb6
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: system:discovery
subjects:
- apiGroup: rbac.authorization.k8s.io
kind: Group
name: system:authenticated
- apiGroup: rbac.authorization.k8s.io
kind: Group
name: system:unauthenticated
Repeat the same steps for system:basic-user ClusterRoleBinding.
Alternative approaches¶
While IRSA is the preferred way to assign an AWS "identity" to a pod, it requires that you include recent version of the AWS SDKs in your application. For a complete listing of the SDKs that currently support IRSA, see https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts-minimum-sdk.html. If you have an application that you can't immediately update with a IRSA-compatible SDK, there are several community-built solutions available for assigning IAM roles to Kubernetes pods, including kube2iam and kiam. Although AWS doesn't endorse or condone the use of these solutions, they are frequently used by the community at large to achieve similar results as IRSA.