I've written before on the value of developers being involved in the deployment and infrastructure of the applications they build. In my article Blending Dev with Ops Using Cloud Computing I touched on the history of operations and how for so long it's been this kind of black box where developers toss their code over the wall to the IT/Infrastructure team and pay no attention to how it deploys.
That was all fine and dandy but I didn't give a lot of concrete ways in that article that can help start blending dev and ops. Today I want to look at one of the tools that we can use to do that, namely Terraform.
Deployment Comes First
One of the things I find most interesting when I talk with product teams that are hell-bent on delivering a product to market is how little thought they give to deploying it. They spend all this time thinking about all the tiny little details about how their app is going to crush it on the market and all the features they need and how much time it's going to take to get to an MVP. But in the end, they're like a dog chasing a car, once they get it they don't know what to do.
It turns out that to crush it with your market, you have to make the product available to the market.
One of the things I've learned straddling a role where I play both the developer and DevOps person is the importance of thinking about HOW you're going to deploy something as much as you think about WHAT that thing is you're going to deploy.
What kind of architecture is it? What domains do we want it to use? What type of availability will it need? Is it available to everyone or just your internal team? Where's the data live? Do we need a database?
On and on the questions go. Sometimes they're clearly DevOps related, other times they blur that line. It makes sense why this gets kicked down the road to the end. These are hard questions to answer when you're still trying to nail down the main product and figure out what it will be in the wild.
What I can tell you is that if you're asking these types of questions once you have a working MVP in your hand, you're already behind and it's' going to take some catching up to bring the infrastructure up to match.
Enter Terraform
All that's a rather long segue into the actual point of this post but what's important about starting with deployment first is that it wasn't as much of a conversation to be had before tools like Terraform.
When deployments were just scripts that downloaded code onto a box in a server rack and ran the start-up script, there was no reason to discuss the deployment first. Ops would just process that just said, "give me the code and instructions on how to run it and I'll get it going". The same thing goes when you just used AWS, GCP, or Azure console to manually set up networks and load balancers and all the things your app needed to run in the cloud.
Then along comes infrastructures as code (IaC) like Terraform and suddenly you can ideate on the deployments at the same time as development. You can set up pipelines in which early versions of your app are being deployed and tested automatically with code pushes. As you grow, you can start making version-controlled tweaks to the infrastructure and learn how the app works in the real world.
Now I know that not all of these things are specific and unique to Terraform but after using Terraform for around 5 years I've compiled a list of 10 reasons why you should set it up from the start on every project moving forward.
The only exception to that might be if you plan to deploy to an out-of-the-box provider like Heroku or something that does the actual provisioning based on a simple config file but even then, Terraform can do that and you'd probably be better off with it rather than without.
So without further ado, here's why you should use Terraform on your next project!
10 Reasons to start using Terraform today
1. It provides insight into the actual infrastructure
As we talk about the ways that we can make sure to bring developers into the realm of deployment, there is no better way than to put the configuration for the infrastructure right there alongside the code.
Now obviously, I understand there's some reason why this may be inadvisable. Sometimes you don't want the wandering dev to start making changes to your servers and running up your AWS bill or screwing up your network. You certainly would need to have good code review practices to do something like this but the beautiful thing about Terraform is that you can isolate certain aspects of your infrastructure that may have more significant effects.
For me, I like to keep a repo with all the main infrastructure such as networking, DNS, etc. where it's unlikely to be accidentally changed even by someone that knows what they're doing.
Having the infrastructure code right alongside the application code is one of the best ways to push the teams to start blending and ensure that everyone knows that their actions affect both the application code as well as the infrastructure.
2. It provides the safety of version control
This is a big one and one that's pretty obvious but seriously though, being able to stick infrastructure files in a repo and have their version controlled so that you can see what's been fiddled with and where you need to roll things back is huge.
As someone that started my DevOps career on Elastic Beanstalk just clicking buttons until something worked, I can tell you, there were so many times I wished I'd had a repo somewhere that told me what the heck I'd done to get it to work. Or heck, even where my infrastructure was when I went looking for it months later to change or fix something.
3. It allows multiple team members to easily manage infrastructure
Again this may be something you don't want to happen too easily which is why good code review and protected branches are crucial. I have yet to have an issue where someone changed the Terraform code without meaning to or in a destructive way.
Instead what I've found is that team members often can make small edits like adding a new environment variable or copying a module config for a new piece of functionality when otherwise they'd be blocked until someone else did it for them.
4. It makes provisioning multiple environments super easy
I think this is hands down one of the best features of Terraform. With Terraform workspaces you can easily create multiple environments (e.g. test, development, production) with just a few simple commands. You can spin them up temporarily or persist them. They all have their isolated state that can be the same or different and when you don't want one around anymore you simply run the destroy command.
Never has it been easier to automate the provisioning and de-provisioning of multiple environments in your infrastructure.
5. It keeps a complete backup of your infrastructure state
Out-of-the-box Terraform stores this state on your local machine which is great for getting started but the real power comes when you connect it to a remote backend. Something like S3 gives you the ability to keep the state in a centralized place no matter where you're deploying from (local machine or CI/CD tool).
Adding on S3 bucket versioning means if anything ever goes wrong, you can always roll back your state to a previous version. Now I wouldn't make it a habit of doing this but I did have a Terraform version update go wrong and needed to roll back the state with a little bit of manual tweaking and man was it a lifesaver.
6. It has SOOO many helper functions
Not only is Terraform an awesome tool for just configuring and provisioning infrastructure, but it also comes with so many built-in helpers to do things like creating common reusable variables, loading in a bunch of different file types, creating conditional code, and dynamic string interpolation. It is truly a programming language with its own set of utilities.
What's awesome about this is that not only are you able to configure your resources, you can make them super dynamic and reusable and apply different resources based on any number of actions.
7. It allows you to provision in one or more cloud providers
I can't say this is one I've personally made use of nor am I itching to start managing infrastructure across multiple cloud providers but with Terraform you can configure as many providers as you want so if you want to run your database in Azure, your backend in AWS, and your front end in GCP, then Terraform has your back.
Please don't do that.
This is huge for systems that have legacy infrastructure but are maybe trying to migrate or are needed to integrate with resources across providers like pulling in a data source from one provider and provisioning to another.
8. It allows you to create reusable infrastructure modules
Next to the whole multi-environment thing I mentioned above, I think this is my next favorite feature of Terraform. Terraform has what are called modules that behave essentially like their own isolated chunks of infrastructure that can be reused in the same project or across multiple projects. This is huge! Instead of having to re-write all the same configurations or manually remember which buttons you clicked in the console, you simply copy a module instance and pass in the updated variables.
This has been huge for me when I've used AWS Lambdas for a lot of different parts of an app. Each time I need a new Lambda, I just copy the same configuration for a module, add the updated variables and apply the changes. I never have to think about all the little pieces or remember what needs to be configured each time.
9. It protects you from provisioning conflicting infrastructure
Coupled with a lock table like Dynamo DB, Terraform adds even more power to what I mentioned above with a remote back end. You now have the ability for a Terraform run on a workspace to lock out any other runs on the same workspace. This means you'll never accidentally have two Terraform runs fighting over who got there first (providing you name your resources properly).
10. It allows you to reference already deployed resources
Remember above when I mentioned keeping some Terraform configurations in another repo to prevent someone from editing something super impactful or having resources that are shared across multiple apps like a network?
Well, yet another cool thing about Terraform is its ability to fetch data from your cloud provider. This is awesome because rather than having to hard code in a VPC id or some other value, you can simply fetch it by name and access all the same attributes as if it were actually in your configuration. I've used this a handful of times to reference non-terraform or global resources in my provider.
11. It grows your understanding of underlying resources
Ok I know I said 10 but I cheated and threw in a bonus reason so you're welcome!
Those of you new to Terraform looking at the config files are probably thinking, "Whaaa!? How does this confusing stuff make it easier to understand!". It's true, Terraform takes some getting used to just like any programming language does. But working in Terraform has taught me more about what I'm provisioning than I ever would have learned using the console.
Yes, the console is super easy to click around and add resources but what's not clear in the console is how much happens under the hood that you don't have insight into. Things like IAM roles that get created automatically or policies that are attached are all things you have to do directly with Terraform.
At first, it's a huge pain because you're like "Why can't AWS just do this for me!?". But man did it improve my understanding of how what resources are needed to do certain things and what configuration options I have for each resource.
Some Disclaimers
I want to clarify a couple of things since I took a few liberties with some terminology and explanations.
Terraform vs. ...
I think it's important to note that some of the things I mentioned as wins for Terraform aren't entirely or even a little bit exclusive to Terraform. My point was not to say necessarily that Terraform is the only tool that does X, Y, or Z. Only that it's the one I'm advocating for and while others can perform the same or similar functions I will choose Terraform every time. I also haven't used other tools like CloudFormation, Ansible, or one of the many others in-depth enough to compare them all.
Provisioning vs Deploying
To some extent, I kind of used these words interchangeably throughout the post and I want to make it clear that they're not the same thing.
To be clear, provisioning is strictly referring to the act of creating infrastructure in one or more providers. That is different from the act of deploying code to those resources that were provisioned.
Terraform is primarily a provisioning tool although you can do some deployment-like things with it. As soon as I figured that out and stopped trying to use Terraform to deploy my code it made life a lot easier.
For all you veteran IT/DevOps people out there, you might be going "Duh, of course it's just provisioning" but that's one of the cool things about Terraform. It blurs the lines between the two and it's both awesome and something you have to wrap your head around.
Wrapping Up
I hope this post has given you some motivation to dive in and learn more about Terraform and how it works. It's one of the tools I've learned over the years that I now use on every project and have no regrets. The points above are just a few reasons why it's an irreplaceable tool in a DevOps tool belt and one of the best ways to start sharing insight between development and deployment.
If you aren't using an infrastructure-as-code tool for your workflow now, you should be and if you're trying to figure out which one that should be, I think you should pick Terraform.
If you need help getting Terraform set up or need someone to help you build out your Terraform configurations, head on over to my services page and fill out the contact form so we can discuss your needs and how I can help get your project off the ground.