Skip to main content

Delete a cluster

To delete a cluster you must first export the following:

$ export AWS_PROFILE=moj-cp
$ export KOPS_STATE_STORE=s3://cloud-platform-kops-state

Start from the root directory of a working copy of the infrastructure repo.

Delete the cluster using the script

There is a destroy-cluster.rb script which you can use to delete your cluster.

Read the script before using it. Deleting a cluster is something you should be very cautious about, and ensure you know exactly what you’re doing.

The script is entirely non-interactive, and will not prompt you to confirm anything. It just destroys things.

First, run make tools-shell

The delete cluster script must always be run in a container. This ensures that the environment of the script is fully controlled, and you don’t run into problems such as the kubernetes context being changed in another window, or extra environment variables causing unwanted effects.

Then invoke the script like this:

./destroy-cluster.rb --name [short cluster name] --yes

Run without --yes to do a dry run, and see what commands would be executed.

You can get more information using:

./destroy-cluster.rb --help

If any steps fail:

  • Fix the underlying problem
  • Edit the script to comment out any sections of the ClusterDeleter.run function which you no longer need to run
  • Re-run the script

Delete the cluster using concourse fly commands

In case you prefer concourse pipeline to destroy the cluster, these are the steps to follow, to delete the cluster using “concourse fly commands”

First, cd to the working copy of the concourse pipelines repo. Make below two changes to the create-test-destroy.yaml file.

  • In the create-test-destroy pipeline definition, literally add CLUSTER_NAME var, under params as below: params: CLUSTER_NAME: ((CLUSTER_NAME)) We will specify the CLUSTER_NAME value using –var NAME=VALUE in the later step, when we run the fly command to set_pipeline.

  • Comment out this line here in destroy-cluster job.

  args:
    # export $(cat keyval/keyval.properties | grep CLUSTER_NAME )

commenting out this will not set the CLUSTER_NAME provided by the create-cluster-run-tests job.

Run the below commands updating the <cluster-name-to-be-deleted>. The first fly command will apply the changes made for the create-test-destroy.yaml file and set the CLUSTER_NAME static var, the second command will trigger the destroy-cluster job using the CLUSTER_NAME provided.

fly -t manager sp -p create-test-destroy -c create-test-destroy.yaml -v CLUSTER_NAME=<cluster-name-to-be-deleted>
fly -t manager trigger-job -j create-test-destroy/destroy-cluster

Note: After the destroy-cluster job completed sucessfully, run the bootstrap pipleine to discard the changes made to create-test-destroy.yaml file.

fly -t manager trigger-job -j bootstrap/bootstrap-pipelines

Delete the cluster manually

In case you prefer not to use the destroy-cluster.rb script, these are the individual steps to follow, to delete the cluster “by hand”

First, set the kubectl context for the cluster you are deleting. The easiest way to do this is with kops. e.g:

$ kops export kubecfg david-test1.cloud-platform.service.justice.gov.uk

You should see this output:

kops has set your kubectl context to david-test1.cloud-platform.service.justice.gov.uk

Then, from the components directory, run these commands to destroy all cluster components, and delete the terraform workspace.

$ cd terraform/cloud-platform-components
$ terraform init
$ terraform workspace select <clusterName e.g. cloud-platform-test-3>
$ terraform destroy -target helm_release.prometheus_operator # delete first, to avoid conflicts later
$ terraform destroy
$ terraform workspace select default
$ terraform workspace delete <clusterName>

Then run the following kops command (this will not delete the cluster).

Append it with --yes to confirm deletion.

$ kops delete cluster --name <clusterName>.cloud-platform.service.justice.gov.uk

Change directories and perform the following to destroy the cluster essentials, and delete the terraform workspace.

$ cd ../cloud-platform
$ terraform init
$ terraform workspace select <clusterName e.g. cloud-platform-test-3>
$ terraform destroy
$ terraform workspace select default
$ terraform workspace delete <clusterName>

Change directories and perform the following to destroy the cluster VPC, and delete the terraform workspace.

$ cd ../cloud-platform-network
$ terraform init
$ terraform workspace select <clusterName e.g. cloud-platform-test-3>
$ terraform destroy
$ terraform workspace select default
$ terraform workspace delete <clusterName>

Delete an EKS cluster

The steps can be found here - Delete an EKS Cluster

This page was last reviewed on 23 June 2021. It needs to be reviewed again on 23 September 2021 by the page owner #cloud-platform .
This page was set to be reviewed before 23 September 2021 by the page owner #cloud-platform. This might mean the content is out of date.