Civo recently made object storage generally available, which is exciting for two reasons.

The first is object storage. What’s not to love?

The second reason is the Civo Terraform provider. If you have used Terraform to provision infrastructure on Civo, you might have noticed no native way to store your Terraform state in the platform.

This tutorial teaches you how to use Civo’s object storage as a backend for Terraform state files. However, before we jump into the guide, let’s take a quick look at what Terraform state is and why it matters.

Terraform state

To keep track of the current state of your infrastructure, Terraform maintains a state. This is typically a JSON file that contains information about resources currently deployed and their dependencies. Without this, Terraform wouldn’t be able to compare the current of your infrastructure with the desired state.

Challenges of managing Terraform state locally

By default, Terraform stores its state file in the root directory where your project was initialized in a file called terraform.tfstate. This is great if you're trying to quickly spin up a few resources for testing or creating a small project. However, this approach quickly brings a new set of challenges as the size and complexity of your project grows.

One major challenge with local state files is keeping them in sync when multiple people work on the same project.

A typical way to solve this is to check it in version control with the rest of your Terraform code. This is largely discouraged because state files often contain sensitive data stored in plaintext. You can refer to this section of the Terraform documentation for more information on the types of sensitive data that can end up in your state file.

Another case where local state management falls short is the event where you lose your state file. Depending on your state file, you can reinitialize Terraform to keep your resources in sync, which is a great way to cause state drift. This section of the Terraform docs goes into more detail on managing state drift.

Remote State Management

Storing your state remotely means Terraform writes state information to a remote data store at the time of writing. Terraform supports Postgres, S3-compatible object stores, Consul, and Kubernetes, to mention a few.

Storing state in a central location solves the problem of keeping state in sync across teams while also addressing the issue of accidentally leaking sensitive data. Remotely storing your state would also enable you to take as many backups as possible in the unfortunate event of data store deletion or data loss.

Now that we understand what Terraform state management is and why storing your state files remotely is important, let's implement this using Civo’s object storage.

Prerequisites

This tutorial assumes some familiarity with Terraform. In addition, you will also need the following:

Create Object Store

Generating Object Store Credentials

We’ll begin by generating your first object store credentials before creating your first object store. Using the Civo CLI:

civo objectstore credentials create tf-experiments

This should be visible through your dashboard.

Civo dashboard

Click on “show detail” and copy your Access Key ID and Secret Key, as we’ll be needing these later.

Creating an Object store

Next, we would create an object storage bucket and associate the credentials we generated earlier with this object store.

Create a file called main.tf and follow along with the code below:

`civo objectstore create tf-store --owner-access-key tf-experiments --wait`

This would create an object store and associate the credentials tf-experiments with the object store.

Refer to the Object Store documentation for more information on object store and credentials management.

Provisioning infrastructure

Now we have an object store, the next step is to write some Terraform! In this guide, I will create a Kubernetes cluster using the Civo Terraform provider.

Create a file called main.tf and follow along with the code below:

terraform {
  required_providers {
    civo = {
      source  = "civo/civo"
      version = "1.0.26"
    }
  }
  backend "s3" {
    endpoint                    = "<https://objectstore.lon1.civo.com>"
    bucket                      = "tf-store"
    key                         = "terraform.tfstate"
    region                      = "LON1"
    skip_region_validation      = true
    skip_credentials_validation = true
    skip_metadata_api_check     = true
    force_path_style            = true
    access_key = "OBJECT-STORE-ACCESS-KEY-ID"
    secret_key = "YOUR-SECRET-KEY"
  }
}

provider "civo" {
  # Configuration options
  token  = "YOUR-CIVO-API-TOKEN"
  region = "LON1"
}

# Query xsmall instance size
data "civo_size" "xsmall" {
  filter {
    key    = "type"
    values = ["kubernetes"]
  }

  sort {
    key       = "ram"
    direction = "asc"
  }
}

# Create a firewall
resource "civo_firewall" "my-firewall" {
  name = "my-firewall"
}

# Create a firewall rule
resource "civo_firewall_rule" "kubernetes" {
  firewall_id = civo_firewall.my-firewall.id
  protocol    = "tcp"
  start_port  = "6443"
  end_port    = "6443"
  cidr        = ["0.0.0.0/0"]
  direction   = "ingress"
  label       = "kubernetes-api-server"
  action      = "allow"
}

# Create a cluster
resource "civo_kubernetes_cluster" "my-cluster" {
  name         = "my-terraform-cluster"
  applications = "Redis"
  firewall_id  = civo_firewall.my-firewall.id
  pools {
    label      = "front-end" // Optional
    size       = element(data.civo_size.xsmall.sizes, 0).name
    node_count = 3
  }
}

Configuring Terraform backend

The important part of this configuration is the backend block:

  • endpoint tells Terraform the address of the object store, which can be retrieved by running civo objectstore info tf-store
  • bucket is the name of the object store we created earlier
  • key allows you to specify what name Terraform should store the state file under
  • skip_* disables AWS s3 specific checks.

To apply the changes, run the following commands:

# initalize the terraform provider
terraform init
# preview the infrastructure 
terraform plan 
# apply the changes 
terraform apply --auto-approve

Once your cluster has been provisioned, head over to your dashboard and select the object storage bucket you just created, you should see the state file for the cluster you just created.

Civo object store bucket that was created in this tutorial

Summary

State management is a crucial part of what Terraform does as it enables it to track your resources and their dependencies, ensuring that resources are always in their desired state.

Through this tutorial, we discussed some of the challenges associated with storing your state files locally and how to leverage Civo’s S3-compatible storage to manage your state files remotely.

Further resources

If you are interested in other options that Terraform supports, check out this section on the documentation here. Finally, the Hashicorp blog has an excellent post on detecting and managing state drift. If you would like to know more about state drift, click here.