From Terraform to Pulumi

Introduction

In the past month, we started to experiment with Pulumi. We previously worked a lot with Terraform and Terragrunt setting up and managing AWS infrastructures and initially started working with Pulumi because it is an open-source tool that uses popular, general purpose languages. The fact that you can write IaC code in your preferred language opens up many possibilities for customization.

Our goal was to create a modularized Pulumi code to start with. One from where we can easily copy modules to other more complex projects. If you want to check it out you can see the repo at: https://github.com/codefactoryhu/pulumi-starter-kit

What problems can Pulumi solve for us?

The reason we loved Terragrunt is that it allowed us to manage multiple environments. In Pulumi it’s even more elegantly done with stacks, where you manage the differences between environments in the stack’s config file so you don’t have to create copies of the same infrastructure definitions.

$ pulumi stack ls
NAME  LAST UPDATE     RESOURCE COUNT  URL
dev   4 days ago      0               https://app.pulumi.com/user/aws-starter-kit/dev
prod  3 days ago      0               https://app.pulumi.com/user/aws-starter-kit/prod
test  16 seconds ago  22              https://app.pulumi.com/user/aws-starter-kit/test

Because Pulumi allows the usage of general purpose languages there wasn’t a strict project structure we had to follow. This can be a little overwhelming at first, but it can also be very handy. For example, it makes it a lot easier to create more than one of the same resources (like three identical S3 buckets) with looping, which can be a headache when you are using Terragrunt with Terraform modules. In contrast to Terraform which switched to the Business Source License model lately, Pulumi is fully open-source with Apache 2.0 license. Pulumi also comes with a variety of interesting new features like Pulumi AI for generating Pulumi code from prompts.

How hard is it to migrate from Terraform to Pulumi?

Building up a bigger project block by block is a very time-consuming task. In our situation we already had our AWS infrastructure written in Terraform. Luckily Pulumi offers many ways to make the migration less painful. Firstly the Pulumi Converter which has an online version but you can also run it with the pulumi convert command. Of course, it won’t be able to structurize your project, that is still something you have to do, but it can be a great help in making the process faster. For us that was harder to use, since our original project was in Terragrunt.

With Pulumi you can also import existing infrastructure. The pulumi import command generates a Pulumi code from any resource in the language you choose and includes it in the state. It can save you some hours searching for different attributes in the documentation. In the end, we ended up combining these tools with Pulumi AI to make the migration faster, refactoring at every step.

It was also very easy to keep the logic of our existing pipelines for testing and deploying Terragrunt code to Pulumi project as they work very similarly. Pulumi also provides guides and resources to help with continuous integration/continuous delivery. They are available for Github workflows, Gitlab, Jenkins and TeamCity just to name a few.

How did we structure our project?

Another thing we liked in Terragrunt was using the terragrunt.hcl configuration file. It came really handy, because it allowed us to define any variable we used in the project. With that, we could create a single file, where we could see and change the attributes of the created AWS resources. Whether we wanted to upgrade a database to the next available version or add bigger ec2 instances to an EKS cluster this was the only file we needed to change. It made the project clean and manageable even for someone who isn’t familiar with Terraform.

That is why we were very happy to see, that this is also something possible in Pulumi. It provides configuration options through cli and also yaml configuration files. Variables can be set globally or specifically on each stack representing an environment. Here is a part of our Pulumi.dev.yaml configuration file:

# GLOBAL
  aws-starter-kit:project: aws-starter-kit
  aws-starter-kit:env: dev
  aws-starter-kit:availabilityZone:
    - eu-west-2a
    - eu-west-2b
    - eu-west-2c

  aws:defaultTags:
    tags:
      project: aws-starter-kit
      env: dev
      version: "v1.2.26"

  # VPC
  aws-starter-kit:vpc:
    name: pulumi-vpc
    cidr: 10.0.0.0/16
    instanceTenancy: default
    numberOfAvailabilityZones: 3
    enableDnsHostnames: true
    enableDnsSupport: true
    enableFlowLogs: true

Conclusions

Pulumi is a constantly evolving tool that is definitely worth trying out. It solves many of the issues that we faced using Terraform due to it being a lot more flexible and allowing you to write your own code logic. Are you interesed in using IaC solutions for you own company or in need of partner certified in the field of cloud solutions? If you are, please don’t hesitate to contact us.