Skip to main content

Moving components modules into core

In order to migrate a terraform module from components into core there are a number of steps that need to be done in order to make sure the process runs smoothly. From a high level the terraform module needs to be moved and then tested in a test cluster.

The aim is to make sure that when a terraform plan is run against both core and compoents after the MTM tool is ran there should be no terraform changes.

There is a pipeline in Concourse that accepts $CLUSTER_NAME, $MODULE_NAME and $RESOURCE setting these and running the pipeline will migrate that module/resource on the cluster specified. Resources can be passed as a comma seperated string. Getting a list of resources to migrate is a manual process that requires you to read through the terraform files in components, some can be found by tracing back dependencies others should be obvious via there name. The name format for resources follow $resource_type.$resource name so resource "kubernetes_storage_class" "storageclass" becomes kubernetes_storage_class.storageclass, you can double check this by running terraform state list.

If you need to upgrade the mtm cli you can update the $MTM_VERSION parameter in the pipeline.

The exact steps should follow:

  1. Notify #cloud-platform that you are pausing concourse pipelines
  2. Pause bootstrap, live-2, manager and live pipelines.
  3. Raise PR with migrated module
  4. Log in to Concourse using fly -t moj-cp login -c
  5. Ensure your in the cloud-platform-terraform-concourse directory.
  6. Update pipelines with module and/or resources name and cluster name: fly -t moj-cp set-pipeline --pipeline migrate-module --config migrate-module.yaml -v cluster_name=cp-0306-0730 -v module=tigera_calico -v resources=kubectl_manifest.calico_crds,http.calico_crds
  7. Run migrate-module pipeline from the Concourse UI.
  8. Unpause pipeline environment specific pipeline (e.g. live-2)
  9. Re-run PR plan - the plan should show “No changes. Your infrastructure matches the configuration.”
  10. Re-run step 5-9 with the next environment (e.g. manager)
  11. Get approval for the PR and merge.
  12. Un-pause live-2 and check for a clean apply
  13. Repeat 9. with manager and live
  14. Update #cloud-platform informing the team of the completed migration and that pipelines are unpaused.

Disaster Recovery

If for some reason something goes wrong then you can use the backed up state file to overwrite the remote.

Alternatively all our tf state objects are versioned in S3 so you can restore a previous known working version from the AWS console.

This page was last reviewed on 3 April 2024. It needs to be reviewed again on 3 October 2024 by the page owner #cloud-platform .
This page was set to be reviewed before 3 October 2024 by the page owner #cloud-platform. This might mean the content is out of date.