Terraform State Management

Introduction✍️

Imagine that you and a teammate are working on creating an infrastructure using Terraform. You both start by creating a .tf file, which contains the configuration details of your resources, such as the instance type, size, and number of instances.

Terraform generates a .tfstate file which contains the current state of the infrastructure. This state file includes information such as the running state of your resources, their IP addresses, and storage.

Problem⁉️

When multiple developers are working on the same infrastructure project.

For example, your teammate decides to add another instance to the infrastructure but forgets to tell you. When they run the command to apply the changes, their .tfstate file will show two instances running, while your .tfstate file still shows only one.

This creates a conflict because both state files are different, even though you're working on the same project. If this conflict goes unnoticed, it could lead to serious issues with your infrastructure.

Solution‼️

This is where State Management comes in.

What is State Management?

In Terraform, state management refers to the process of managing and tracking the current state of your infrastructure. To avoid conflicts, you need a proper state management strategy.

By using a remote backend or Terraform Cloud, you can manage your state files in a centralized location and work collaboratively with your team. This ensures consistency and prevents conflicts in the configuration of your infrastructure.

Why not GIT?

GIT is a version control system and you're thinking now, to avoid the conflicts of working with multiple developers, we can use GIT because it keeps track of each and every change.
But we cannot, because, the infrastructure state files have a lot of information about the objects we've created e.g. access keys, secret values, and private IP addresses of instances.

If these details get exposed, it could put your infrastructure and data at risk.
Git is only designed to handle source code changes, not the infrastructure state.
Because infrastructure state files can become quite large and complex depending on the size and scale of your project.
Git is not designed to handle these types of large binary files, and can quickly become bloated, slow and difficult to work with as a result.

Remote Backend

So, you know how when you're working on a project with your team, you need to have a central location to store and share your configuration data so everyone's on the same page? And we just saw above that GIT cannot be the option.
That's where remote backends in Terraform come in handy!
Instead of storing your Terraform state file on your local machine, remote backends allow you to store it in a cloud storage system like Amazon S3 or Microsoft Azure.

State Locking -

Now, state locking is a cool feature in Terraform that helps to prevent any accidents from happening.
When two or more people are working on the same file at the same time, this can cause data loss, incorrect updates or inconsistencies with the state of infrastructure.
To prevent this from happening, Terraform has a state-locking feature, which ensures that only one person is changing the file at a time.
When one team member runs the terraform apply command, Terraform requests a lock on the state file to make sure that no one else can modify it at the same time.
Once the changes are made, the lock is released so others can make changes.

Other methods of Storing Terraform State file

We saw a method above to store tfstate file. There are a few different ways to store Terraform state files. Here are a few examples:

1. Locally: One way to store the state file is on your local machine. This is the simplest method but is not recommended for production use cases. You can simply specify the path of the state file in your Terraform code and Terraform will generate the state file locally.

terraform {
  backend "local" {
    path = "relative/path/to/terraform.tfstate"
  }
}

2. S3 Bucket: Another option is to store the state file in an S3 bucket. This is a great choice when working in production because the S3 bucket can be accessed securely and can handle multiple users accessing it concurrently.

This comes under remote backend storage.

terraform {
  backend "s3" {
    bucket = "my-terraform-state-bucket"
    key    = "terraform.tfstate"
    region = "us-west-2"
  }
}

3. HashiCorp Consul: It is possible to store the state file in Consul, a service discovery and orchestration platform. This method is best suited for small teams and projects. And HashiCorp consul is one of the.

terraform {
  backend "consul" {
    address = "consul.example.com:8500"
    path    = "/terraform/state"
  }
}

4. Azure Blob Storage: Azure Blob Storage can also be used to store the state file. This is a good option if you are using Azure for your infrastructure.

terraform {
  backend "azurerm" {
    storage_account_name = "mystorageaccount"
    container_name       = "myterraformstate"
    key                  = "terraform.tfstate"
  }
}

As you can see, there are many different options and you can choose the one that best fits your needs and preferences.

Creating Remote backend storage for `tfstate` files

Now let's jump right into the hands-on part!

Again, First thing first, create an EC2 instance manually from the AWS dashboard with the free tier architecture to perform further procedures on it.
Copy the public IP address of the created instance id👇

See☝️
Run this command to connect the instance to your local system.
```
  $ ssh -i /my-key-pair ubuntu@<ip-add>
```
Give the correct path of your generated key and provide the correct IP add.
Make sure you've Terraform installed in it by running,
```
  $ terraform --version
```
And if not, then install it from here.

Also, check aws-cli is installed and configured.

 $ sudo apt install aws cli
 $ aws configure

Now firstly, we will look how .tf created .tfstate file automatically in the local system if not specified.

Create a directory remote_infra

 $ mkdir remote_infra
 $ cd remote_infra

Now create separate files for providers, resources and main.tf file

 provider "aws" {
      region = "us-east-1"
 }

providers.tf file☝️

 resource "aws_s3_bucket" "my_s3_bucket_2" {
     bucket = "terraweek-demo-state-bucket-123"
 }

 resource "aws_dynamodb_table" "my_dynamo_table" {
     name = "terraweek-demo-state-table"
     billing_mode = "PAY_PER_REQUEST"
     hash_key = "LockID"
     attribute {
     name = "LockID"
     type = "S"
     }
 }

resource.tf file☝️

 terraform {
     required_providers {
     aws = {
     source = "hashicorp/aws"
     version = "4.66.1"
 }
 }
 }

terraform.tf file☝️

Now run,
```
 $ terraform init
```
Run terraform plan and terraform apply command to make the changes. And when now you see, a tfstate file is created in the same directory
```
$ ls
```
Now make another directory in which we will make a remote backend so that tfstate file will be created on the AWS S3 bucket.
```
$ mkdir remote_demo
```

Here, create the required files.

terraform {

required_providers {
    aws = {
    source = "hashicorp/aws"
    version = "4.66.1"
    }
 }

backend "s3" {
    bucket = "terraweek-demo-state-bucket-123"
    key = "terraform.tfstate"
    region = "us-east-1"
    dynamodb_table = "terraweek-demo-state-table"
}
}

terraform.tf file☝️

the bucket is where the tfstate file is going to store, and

dynamodb_table: Specifies the name of the DynamoDB table to use for state locking.

provider "aws" {
    region = "us-east-1"
}

resource "aws_instance" "my_ec2_instance" {
    count = 3
    ami = "ami-04a0ae173da5807d3"
    instance_type = "t2.micro"
    tags = {
    Name = "terraweek-instance"
}
}

In the above main.tf file we're creating three aws-instance.☝️

Now run,
```
$ terraform init
```
You can see the success message!☝️
Now run,
```
$ terraform plan
```
You can see what we are going to do with the infrastructure.☝️
Now finally run,
```
$ terraform apply
```
The three instances have been created now.☝️
Now run the ls command and let see tfstate file is created or not?

See, there is no file called .tfstate locally.☝️ Let's check the AWS console now.
Go to S3 buckets and click on the newly created bucket.👇

Our .tfstate file is here!☝️
Go to the DynamoDB table section, you can see the table is created.👇
By running terraform state list command, you can see the listed state