Creating Kubernetes Clusters on AWS using KOPS
Why Kubernetes?
Kubernetes is an open-source system for automating containerized application deployment, scaling, and management. Kubernetes has emerged as the go-to solution for all your distributed architecture queries. It groups containers that make up an application into logical units for easy management and discovery. But how do you actually provision and deploy a production-ready Kubernetes cluster while ensuring high availability?
The answer is KOPS.
Why KOPS?
KOPS offers a one-stop solution for deploying Kubernetes clusters with Amazon Web Services. It is an open-source tool designed to make installing secure, highly available clusters easy and automatable. In this blog, we will focus on launching clusters in private topology.
Prerequisite
- Linux hands-on.
- AWS Account with following service access neccessary permissions.
If you don't have an AWS account visit https://aws.amazon.com/ to create one.
- AWS Access key and Secret key of AWS account.
Step 1 | Install KOPS 1.5.1
We need to install the CLI tool kops to spin up the Kubernetes cluster.
Download KOPS latest release here.
$ wget https://github.com/kubernetes/kops/releases/download/1.5.1/kops-linux-amd64
To change Permissions, use the following command:
$ chmod a+x kops-linux-amd64
Move KOPS to /usr/local/bin
$ sudo mv kops-linux-amd64 /usr/local/bin/kops
Confirm KOPS installation,
$ kops
Your output is:
kops is Kubernetes ops.
It allows you to create, destroy, upgrade, and maintain clusters.
Usage:
kops [command]
Available Commands
completion Output shell completion code for the given shell (bash)
create Create a resource by filename or stdin
delete delete clusters
describe describe objects
edit Edit resource
export export clusters/kubecfg
get list or get objects
import import clusters
replace Replace a resource by filename or stdin rolling-update rolling update
clusters
secrets Manage secrets & keys
toolbox Misc infrequently used commands
update update clusters
upgrade upgrade clusters
validate Validate Cluster
version Print the client version information
Flags:
--alsologtostderr log to standard error as well as files
--config string config file (default is $HOME/.kops.yaml)
-h, --help help for kops
--log_backtrace_at traceLocation when logging hits line file:N, emit a stack trace
(default :0)
--log_dir string If non-empty, write log files in this directory
--logtostderr log to standard error instead of files (default false)
--name string Name of cluster
--state string Location of state storage
--stderrthreshold severity logs at or above this threshold go to stderr (default2)
-v, --v Level log level for V logs
--vmodule moduleSpec comma-separated list of pattern=N settings for
file-filtered logging
Use "kops [command] --help" for more information about a command.
Step 2 | Install Kubectl
To download Kubectl,
$ wget https://storage.googleapis.com/kubernetes-release/release/v1.5.2/bin/linux/amd64/ kubectl
Change Permissions,
$ chmod a+x kubectl
Move kubectl to /usr/local/bin,
$ sudo mv kubectl /usr/local/bin/kubectl
Confirm Kubectl installation,
$ kubectl
kubectl controls the Kubernetes cluster manager.
Find more information at
Basic Commands (Beginner):
create Create a resource by filename or stdin
expose Take a replication controller, service, deployment or pod and expose it as a
new Kubernetes Service
run Run a particular image on the cluster
set Set specific features on objects
For Intermediate:
get Display one or many resources
explain Documentation of resources
edit Edit a resource on the server
delete Delete resources by filenames, stdin, resources and names, or by resources
and label selector
Deploy Commands:
rollout Manage a deployment rollout
rolling-update Perform a rolling update of the given ReplicationController
scale Set a new size for a Deployment, ReplicaSet, Replication
Controller, or Job
autoscale Auto-scale a Deployment, ReplicaSet, or ReplicationController
Cluster Management Commands:
certificate Modify certificate resources.
cluster-info Display cluster info
top Display Resource (CPU/Memory/Storage) usage
cordon Mark node as unschedulable
uncordon Mark node as schedulable
drain Drain node in preparation for maintenance
taint Update the taints on one or more nodes
Troubleshooting and Debugging Commands:
describe Show details of a specific resource or group of resources
logs Print the logs for a container in a pod
attach Attach to a running container
exec Execute a command in a container
port-forward Forward one or more local ports to a pod
proxy Run a proxy to the Kubernetes API server
cp Copy files and directories to and from containers.
Advanced Commands:
apply Apply a configuration to a resource by filename or stdin
patch Update field(s) of a resource using strategic merge patch
replace Replace a resource by filename or stdin
convert Convert config files between different API versions
Settings Commands:
label Update the labels on a resource
annotate Update the annotations on a resource
completion Output shell completion code for the given shell (bash or zsh)
Other Commands:
api-versions Print the supported API versions on the server, in the form of
"group/version"
config Modify kubeconfig files
help Help about any command
version Print the client and server version information
Use "kubectl <command> --help" for more information about a given command.
Use "kubectl options" for a list of global command-line options (applies to all commands).
Step 3 | Setting up the AWS environment
In order to do this, we need first to configure AWS with Secret Key and Access Key. The Secret Key and Access Key have all the necessary permissions, as mentioned in `prerequisite'. You can also create a separate user for KOPS with this script.
Configure AWS,
$ aws configure
Provide the right Secret Key, Access Key, and Region.
If you are using a script [Which is more recommended],
$ curl -O https://raw.githubusercontent.com/kubernetes/kops/master/hack/new-iam-user.sh $ sh new-iam-user.sh <group> <user> $ aws iam list-users
In response, you’ll get a secret key and an access key. Note down the secret key and access key for this user.
Step 4 | Create Hosted Zone for Cluster
For this, we require a hosted zone associated with Route 53, which must be publicly resolvable. (However, KOPS also allows to use private DNS, which is more tricky.)
We are creating Hosted Zone using Testikod.in,
$ aws route53 create-hosted-zone --name testikod.in --caller-reference 2017-02-24-11:12 --hosted-zone-config Comment="Hosted Zone for KOPS"
Step 5 | Create State Store
KOPS internally uses Terraform. So we required an external state store to store a cluster's states. We are using Amazon S3 for storing state.
$ aws s3api create-bucket --bucket testikod-in-state-store --region us-west-2
Step 6 | Creating Cluster
Setup environment variable for STATE STORE and cluster name NAME:
$ export NAME=cluster.testikod.in $ export KOPS_STATE_STORE="s3://testikod-in-state-store"
Forming custom KOPS command to use private topology,
Parameters : --cloud aws : We are launching cluster in AWS. --zones us-west-2a,us-west-2b,us-west-2c : This describes availability zones for nodes. [Note that this availability zones should be available for AWS account. You can check it through, aws ec2 describe-availability-zones --region us-west-1 ] --node-count 5 : The number kubernetes nodes. --node-size t2.medium : Size of kubernetes nodes. --dns-zone testikod.in : Hosted zone which we created earlier. --master-size m3.medium : Size of a Kubernetes master node. --master-zones us-west-2a,us-west-2b,us-west-2c : This will tell kops to spread masters across those availability zones. Which will give High Availability to KOPS cluster. --topology private : We define that we want to use a private network topology with kops. --networking calico : We tell kops to use Calico for our overlay network.Overlay networks are required for this configuration. -- bastion : Add this flag to tell kops to create a bastion server so you can SSH into the cluster.
KOPS Command,
kops create cluster \
--cloud aws \
--node-count 5 \
--node-size t2.medium \
--master-size m3.medium \
--zones us-west-2a,us-west-2b,us-west-2c \
--master-zones us-west-2a,us-west-2b,us-west-2c \
--dns-zone testikod.in \
--topology private \
--networking calico \
--bastion \
${NAME}
Kops will set the default to ~/.ssh/id_rsa.pub for backend access. You can override this with --ssh-public-key /path/to/key.pub
Cluster configuration
Above command will create a blueprint for the cluster. To edit configurations,
$ kops edit cluster ${NAME}
Apply the changes,
$ kops update cluster ${NAME} --yes
Cluster creation will take up to 10 minutes.
Step 7 | Verify cluster and Start using Cluster
Check your APIs are working with Kubectl,
$ kubectl get nodes
It should provide you the number of nodes in response.
To access admin via Bastion host,
$ exec ssh-agent bash $ ssh-add ~/.ssh/id_rsa $ ssh -A admin@bastion.cluster.testikod.in $ ssh admin@<master_private_ip>
Conclusion:
All things considered, K8S is among the best available container orchestration tools today. It provides a solid platform to provision and deploy clusters. It offers many exciting features and also allows a wide range of customization.
