Ten simple steps to secure your Amazon EKS

Securing your managed Kubernetes services, especially with services like Amazon Elastic Kubernetes Service (EKS), demands a vigilant and strategic approach. The world of containers and microservices continuously brings along a fresh set of threats and security challenges. Kubernetes breaks down applications into smaller bits called microservices, neatly packed into containers and managed by Kubernetes itself. Sounds great, but each of these elements – applications, containers, and the orchestrator – is a potential target for cyber threats. A hit on any of them could mean data breaches or service disruptions. Making things trickier, Kubernetes environments are constantly changing. This dynamic nature calls for a systematic security approach, one that covers the whole lifecycle of your application. Let’s explore the essentials of securing your EKS clusters and keeping your digital assets intact.

What are the best practices to implement EKS securely?

Let’s examine key areas that demand close attention to ensure the effectiveness of security controls for EKS clusters and workloads.

• Identity Access Management: AWS Identity and Access Management (IAM) is a cornerstone service, pivotal for authentication and authorization. It offers a robust suite of features. Grant shared access to your account securely, implement granular permissions tailored to specific resources, and ensure applications on Amazon EC2 have credentials handled with precision. IAM also supports multi-factor authentication for heightened security, identity federation for temporary access, and is PCI DSS compliant. With seamless integration across AWS services, it provides a comprehensive and cost-effective solution for access management. (read more)
• Multi-tenancy: Kubernetes operates as a single-tenant orchestrator, where the control plane is shared across all cluster tenants. To establish a semblance of multi-tenancy, utilize Kubernetes objects like Namespaces and Role-based access controls (RBAC) for logical isolation. Additionally, employ Quotas and Limit Ranges to manage each tenant's consumption of cluster resources effectively. This approach is crucial as an attacker gaining access to a cluster host could compromise sensitive information and manipulate node attributes, posing significant risks.
• Logging and auditing: Implementing robust logging and monitoring practices heightens the security of operations. Audit logs, seamlessly sent to Amazon CloudWatch Logs, serve multiple purposes—enabling root cause analysis, attributing changes to specific users, and detecting anomalies. Utilizing CloudWatch Log Insights, set up alarms to automatically flag increased 403 Forbidden and 401 Unauthorized responses, providing proactive security alerts. With CloudTrail, effortlessly trace AWS API calls to IAM principals, ensuring comprehensive visibility and early detection of unusual activity. This proactive approach is essential for maintaining the integrity and security of your EKS environment.
• Network security: Apply stringent rules for network traffic control using tools like Network Policies and Security Groups. Encryption in transit is fortified through mechanisms like Service Mesh, Container Network Interfaces (CNIs), Ingress Controllers, and Load Balancers. To get started, follow the Principle of Least Privilege, creating a default deny policy and incrementally adding rules. Monitor enforcement with network policy editor, audit logs, and automated testing, ensuring continuous compliance and adapting to evolving application needs. Use Open Policy Agent (OPA) to guarantee Network Policy existence for secure onboarding of application pods.
• Data encryption: AWS offers native storage options like EBS, EFS, and FSx for Lustre, all supporting encryption at rest using service-managed or customer master keys (CMK). Consider encryption at rest a best practice, and when in doubt, encrypt your data. Configure KMS for CMK rotation, rotating keys annually and preserving old keys for decryption. Leverage EFS access points for shared datasets with diverse file permissions. Secure sensitive information like API keys using Kubernetes secrets, ensuring encrypted storage in EBS volumes for etcd nodes on EKS. Activate audit logging on EKS for enhanced security monitoring, creating CloudWatch metrics filters and alarms for secret usage alerts.
• Runtime security: Runtime security actively shields containers from malicious activities using Linux kernel mechanisms like Linux capabilities, seccomp, AppArmor, or SELinux. Amazon GuardDuty is a recommended tool for runtime monitoring, offering threat detection with simplified configuration. Optionally, explore third-party solutions, especially if managing Seccomp and AppArmor profiles seems complex. Evaluate adding/removing Linux capabilities before implementing Seccomp for enhanced control. This comprehensive approach guarantees robust protection and threat identification in your EKS runtime.
• Infrastructure security: Securing your container images is vital, but safeguarding the infrastructure running them is equally critical for a robust Amazon EKS setup. Optimize your OS with choices like Flatcar Linux or use the EKS optimized AMI for Kubernetes worker nodes. Keep the OS updated, treating infrastructure as immutable for enhanced security. Automate worker node replacement to minimize human oversight and utilize tools like kube-bench to verify compliance and minimize access to worker nodes with SSM Session Manager. Deploy workers in private subnets for reduced exposure. Leverage Amazon Inspector to assess hosts for vulnerabilities and deviations from best practices.
• Regulatory compliance: Compliance is a shared responsibility between AWS and its consumers, where AWS handles "security of the cloud," and users manage "security in the cloud." In the case of EKS, AWS takes charge of the managed Kubernetes control plane, while users are responsible for areas like IAM, pod security, runtime security, and network security. A recommended practice is employing policy-as-code tools in pipelines to identify violations before deployment, enhancing security measures.
• Incident response and forensics: Reacting swiftly to incidents is crucial in minimizing damage from breaches. A robust alerting system is essential for early detection. In an incident, deciding whether to destroy and replace or isolate and inspect the affected container is vital. If isolation is chosen, follow these steps:
  • Identify the offending Pod and worker node by workload or service account name.
  • Identify Pods with compromised images or worker nodes, considering malware, known bad images, or exploited CVEs.
  • Isolate the Pod using a Network Policy with deny all traffic rules.
  • Revoke temporary security credentials if assigned.
  • Cordon the worker node to prevent new pod scheduling.
  • Enable termination protection on the impacted worker node.
  • Practice security game days, simulating real-world attacks with tools like Kubesploit.
  • Run periodic penetration tests against your cluster for vulnerability discovery.

This comprehensive incident response plan ensures effective handling and mitigation in case of security breaches.

• Image security: Ensuring image security is crucial for defense against potential attacks. Key recommendations:
  • Create minimal images and remove extraneous binaries to reduce the attack surface while inspecting Dockerhub images with tools like Dive for layer details.
  • Employ multi-stage builds for minimal images and enhanced security in order to minimize the final image size to reduce vulnerabilities.
  • Generate Software Bill of Materials (SBOMs) to inventory software artifacts in container images and address concerns about visibility, provenance verification, trustworthiness, and dependency trust.
  • Scan container images regularly while leveraging scanning options.
  • Use attestations to validate artifact integrity and trust in the software supply chain and create signed statements to assert the truth about artifacts.
  • Implement IAM policies to restrict ECR repository access based on teams. Utilize ECR namespaces for grouping and organizing repositories.
  • Configure private endpoints for ECR to access registries privately. Implement endpoint policies to limit API access to ECR repositories and prevent unauthorized access and potential data exfiltration.
  • Set lifecycle policies for ECR repositories to manage image expiration. Remove stale images to prevent accidental deployment of outdated software.
  • Create curated images for different application stacks and automate CI/CD pipelines to compile and store artifacts securely.
  • Add the USER directive to Dockerfiles to run containers as non-root users.
  • Use linting tools like dockerfile_lint to enforce Dockerfile guidelines. Integrate linting into CI pipelines for automated validation.
  • Minimize the attack surface by creating minimal images from scratch.
  • Use immutable tags to prevent overwriting images with malicious versions. Facilitate unique identification.
  • Sign container images, SBOMs, pipeline runs, and vulnerability reports. Leverage tools like AWS Signer or Sigstore Cosign for cryptographic verification.
  • Admit deployments only when security metadata complies with defined policies.

Adhering to these recommendations fortifies container images, mitigates risks, and strengthens the overall security posture of Kubernetes clusters.

Secure your AWS EKS today

System security has always been a leading factor for organizational success, and securing systems has never been as important. Opcito’s experts understand the ins and outs of securing EKS environments and have helped multiple organizations with the right approach and tools. Write to contact@opcito.com to begin your AWS EKS security journey today.