HomeCasesBlogsContact

Terraform and for_each

In this article I explain the use of ‘for_each’ in Terraform through examples. I explain how it handles different data types and ways to resolve common errors.

When I started working with Terraform from an developer background I had some issues with the inner workings of for_each. Here are some examples and the background why Terraform works this way.

Terraform and for_each error: List of strings

I would be a millionaire if I had a dime for every time I saw this error:

The given “for_each” argument value is unsuitable: the “for_each” argument must be a map, or set of strings, and you have provided a value of type list of │ string.

Let’s look at the following sample code:

data "aws_ami" "ubuntu" {
  most_recent = true
  filter {
    name   = "name"
    values = ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-*"]
  }
  filter {
    name   = "virtualization-type"
    values = ["hvm"]
  }
  owners = ["099720109477"] # Canonical
}
variable "test" {
  type    = list(string)
  default = ["b", "a", "c", "c"]
}
resource "aws_instance" "web" {
  for_each = var.test
  ami           = data.aws_ami.ubuntu.id
  instance_type = "t3.micro"
  tags = {
    Name = "HelloWorld-${each.value}"
  }
}

So why is this? You would then go about changing the var.test to toset(var.test) and it would magically start working!

To gain a better understanding we need to dig a bit deeper in how Terraform tracks its resources in the state file. The state file makes sure Terraform knows the state of your environment and is able to manage changes to it.

Let’s first remove the for_each:

resource "aws_instance" "web" {
  ami           = data.aws_ami.ubuntu.id
  instance_type = "t3.micro"

  tags = {
    Name = "HelloWorld"
  }
}

And look at the plan output:

We can see Terraform will create a resource named aws_instance.web. If we apply we can see this is also how it is stored in the state file:

If we have a list in Terraform of strings, like we have here:

["b", "a", "c", "c"]

You can view this as an array with elements of a certain type and this array is indexed by integers (0,1 and 2).

If we would allow for_each with integer keys we would run into issues when the order of the elements changes.

Next to that lists in Terraform can have the same elements. So in my example you can see we have two times the “c” and this would cause problems for the state file as well.

So if we look what toset() does, it discards the ordering and removes any duplicate elements from the list. So after we run toset on our variable list, it becomes:

["a", "b", "c"]

Now we can see that Terraform tracks the instances through an index_key and the error is solved:

Terraform and for_each errors: List of objects

Now you would think, I got toset in my toolbox so I can solve all my for_each problems. Unfortunately this is not the case. If we make our example a bit more complex and try looping a list of objects.

First let us try the way of changing the test variable to reflect this:

variable "test" {
  type    = list(object)
  default = [
    {
      name = "x",
      value = "y"
    },  
    {
      name  = "x",
      value = "y"
    },
    {
      name = "z",
      value = "d"
    }    
  ]
}

Now we run into the error:

Error 1: Invalid type specification

The object type constructor requires one argument specifying the attribute types and values as a map.

This is because Terraform needs you to specify the form of the object in the list. So if we change the variable to be:

variable "test" {
  type = list(object({ name = string, value = string }))
  default = [
    {
      name  = "x",
      value = "y"
    },
    {
      name  = "x",
      value = "y"
    },
    {
      name  = "z",
      value = "d"
    }
  ]
}

So we are making progress! On to the next error:

Error 2: Invalid for_each set argument

The given “for_each” argument value is unsuitable: “for_each” supports maps and sets of strings, but you have provided a set containing type object.

This is because the for_each block only supports maps / sets of strings. It does not support a list of objects.

So we can’t use toset here to solve our problems. However we can use the for function to build our own compatible set:

resource "aws_instance" "web" {
  for_each   = {
    for index, testValue in var.test: testValue.name => testValue
  }
  ami           = data.aws_ami.ubuntu.id
  instance_type = "t3.micro"
  tags = {
    Name = "HelloWorld-${each.key}-${each.value}"
  }
}

So what is going on here? Let’s split the code:

  • for index, testValue in var.test – This loops over var.test and puts the value into testValue as a local variable for that loop
  • testvalue.name => testValue – The first part is the key and the second part is the value

If you are reading along and you remember our array, now we are using the key name to create the resources. We see we have two “x” in that variable array. So we run into the following error:

Error 3: Duplicate object key

Two different items produced the key “x” in this ‘for’ expression. If duplicates are expected, use the ellipsis (…) after the value expression to enable │ grouping by key.

If we deduplicate the name:

variable "test" {
  type = list(object({ name = string, value = string }))
  default = [
    {
      name  = "x",
      value = "y"
    },
    {
      name  = "xx",
      value = "y"
    },
    {
      name  = "z",
      value = "d"
    }
  ]
}

The code works and uses the keys for the resources x, xx, and d. This means in your objects you always have to have a unique value to create the resources with. There are ways around it but they are not reliable and can cause issues later on (ie. using the index of the object but that changes when the order of the list changes).

The great thing about using the for expression is that it is very powerful and you can use it to filter elements as well.

The most complex error: For_each over derived results

The most complex error to resolve is the following:

The “for_each” set includes values derived from resource attributes that cannot be determined until apply, and so Terraform cannot determine the full set of keys that will identify the instances of this resource.

When working with unknown values in for_each, it’s better to use a map value where the keys are defined statically in your configuration and where only the values contain apply-time results.

Alternatively, you could use the -target planning option to first apply only the resources that the for_each value depends on, and then apply a second time to fully converge.

You might run into this error when all your code worked just fine, but now rerunning the code with an empty environment yields you this error. The message is a bit confusing as well. I will explain what is going on with some sample code:

module "vpc" {
  source = "terraform-aws-modules/vpc/aws"
  name = "my-vpc"
  cidr = "10.0.0.0/16"
  azs             = ["eu-west-1a", "eu-west-1b", "eu-west-1c"]
  private_subnets = ["10.0.1.0/24", "10.0.2.0/24", "10.0.3.0/24"]
  public_subnets  = ["10.0.101.0/24", "10.0.102.0/24", "10.0.103.0/24"]
  enable_nat_gateway = false
  enable_vpn_gateway = false
}

resource "aws_vpc_endpoint" "s3" {
  vpc_id            = module.vpc.vpc_id
  vpc_endpoint_type = "Gateway"
  service_name      = "com.amazonaws.eu-west-1.s3"
}

resource "aws_vpc_endpoint_route_table_association" "s3_public" {
  for_each        = toset(module.vpc.public_route_table_ids)
  vpc_endpoint_id = aws_vpc_endpoint.s3.id
  route_table_id  = each.value
}

resource "aws_vpc_endpoint_route_table_association" "s3_private" {
  for_each        = toset(module.vpc.private_route_table_ids)
  vpc_endpoint_id = aws_vpc_endpoint.s3.id
  route_table_id  = each.value
}

You can never apply this code and will run into the following error:

The “for_each” set includes values derived from resource attributes that cannot be determined until apply, and so Terraform cannot determine the full set of keys that will identify the instances of this resource.

This is because our usage of the toset function as well. As explained above you can see that the toset() call will yield something like:

["rtb-1234567", "rtb-2156115"]

So we want to use the Route Table ID as a key in our object. However, this route table ID will only be known after Terraform has applied the changes and AWS has given it a Route Table ID. So Terraform can never reserve a spot in its state file for this resource. That is why Terraform blocks the apply. If the route table ID’s are already known (ie. because the changes to the VPC have already been applied in a previous step) the apply will work just fine.

How to fix this depends a bit on your situation, but if we can give Terraform a fixed key to reserve the resource in the state file we should be able to apply. Since in this case the route tables are built by the VPC module we can decide on a keying scheme. So the keys are always available even before the apply completes. For this we need to look at the routing table setup that this VPC configuration generates:

We can see in our example that we have a single route table for the public part and three route tables for every availability zone for the private part of the network. This means we can opt for the following keying schema. This keying schema depends just on the azs parameter that we pass into the VPC module:

For the public part we can opt for this for loop:

resource "aws_vpc_endpoint_route_table_association" "s3_public" {
  for_each        = {
    for index, routeTableId in module.vpc.public_route_table_ids : "public" => routeTableId
  } 
  vpc_endpoint_id = aws_vpc_endpoint.s3.id
  route_table_id  = each.value
}

We can see that we would get the following resource in our statefile:

For the private part we can rely on the availability zone:

resource "aws_vpc_endpoint_route_table_association" "s3_private" {
  for_each        = {
    for index, routeTableId in module.vpc.private_route_table_ids : "private-${module.vpc.azs[index]}" => routeTableId
  }
  vpc_endpoint_id = aws_vpc_endpoint.s3.id
  route_table_id  = each.value
}

We can see it will create three of these VPC endpoints now:

Note that it is a bad idea to work with integer indexes and include them in the key. Because the order and count could change and this can cause issues later on.

In summary, for this problem you need to make sure your for_each keys are clear to Terraform, before the apply actually happens.

I hope the for_each loop in Terraform is a bit clearer for you now! But if you are really tired from working with Terraform, best to let ElasticScale handle this frustrating work for you! I am easily accessible for questions. Contact me without obligation for a free 60-minute call.