Easy Collaboration with Terraform Cloud

Our team at Rubrik uses Terraform extensively to manage our infrastructure as code. This means that our infrastructure configurations are version controlled and resources are provisioned in an automated fashion through CI/CD workflows. Because it’s a customer-zero environment, we’re constantly evaluating new tools to find better ways to manage and scale the environment. This led us to trying out Terraform Cloud. 

Easy collaboration is the name of the game with Terraform Cloud. It offers team-oriented remote execution and is consumed as a SaaS platform. In this post, I’ll cover remote state management, cost estimation, and collaboration with Terraform Cloud.

Remote State Management

State files capture the existing state of provisioned infrastructure for a specific workspace. State files are stored on the local machine by default. This becomes unwieldy when the rest of the team is involved. 

Remote state management is a design consideration with which we’ve extensively experimented. My colleague, Chris Wahl, has written about using Amazon S3 to store state, which is how we have historically managed state. This resembles the following:

terraform {
  backend "s3" {
    bucket = "technicloud-bucket-tfstate"
    key    = "dev/terraform.tfstate"
    region = "us-east-1"
  }
}

Using Terraform Cloud to manage remote state resembles the following:

terraform {
  backend "remote" {
    hostname     = "app.terraform.io"
    organization = "technicloud"

    workspaces {
      name = "scaling-compute"
    }
  }
}

With Terraform Cloud, the state file is abstracted from the user; it exists but is secured and managed by the platform. This allows for granular access control, versioning, and backup so that I’m able review previous points in time. While Amazon S3 provides these same features, it requires quite a bit more effort to do so. For example, remote state management with Terraform Cloud provides integrated locking, eliminating the need to spin up a DynamoDB table.

Terraform Cloud enables teams to easily collaborate asynchronously by using the platform as remote state file storage.

Cost Estimation

A very cool feature that stood out was the cost estimation, which displayed an approximate monthly cost with each workflow run. This is particularly beneficial to me because we use Terraform to deploy resources across all three major cloud service providers. Holistic billing management across multiple clouds has long plagued me:

https://platform.twitter.com/widgets.js

This standard interface provides a valuable way for our team to analyze, report on, and visualize cloud spend across cloud providers.

While this alone does not give a complete picture of our monthly bill, it certainly helps us be mindful of cost when testing and building demos. We are regularly building demos to showcase our product’s cloud functionality; this process consists of design time spent architecting a solution and then usually a lot of prototyping to get the demo perfect. The prototyping phase consists of deploying and destroying resources numerous times, which can quickly rack up a big bill when not paying attention to cost.

However, the Terraform Cloud Cost Estimation API provides a lot of granular data that can be pulled into our central billing dashboard. This helps us be mindful of monthly costs to operate our cloud environment. Using this data, we made the decision to use demo leases of 4 hours to help minimize costs for demo; after 4 hours, the resources are stopped. This helps us keep central IT off our backs 🙂

Team Collaboration

Terraform Cloud offers a number of collaboration features to help teams easily work together. Our team prioritizes making our code as reusable as possible; we regularly write modules that fit our design specifications and use cases. The Private Module Registry allows us to easily share the different use case modules that we’ve built. 

There’s also multi-tenancy with the ability to create and manage multiple teams and organizations and enforcing Role Based Access Control (RBAC) across the different workspaces. Moreover, you can manage Terraform Cloud configurations using Terraform.

Here’s an example of using the Terraform Cloud provider to create an organization, workspace, team, and permissions:

# Create the Terraform Cloud Organization
resource "tfe_organization" "technicloud" {
 name  = "technicloud"
 email = "rebecca@technicloud.com"
}
 
# Create the Technicloud Workspace
resource "tfe_workspace" "technicloud-wordpress" {
 name         = "technicloud-wordpress"
 organization = tfe_organization.technicloud.id
}
 
# Add Web Dev Team
resource "tfe_team" "web-dev" {
 name = "technicloud-web-dev"
 organization = tfe_organization.technicloud.id
}
 
# Add User to Web Dev Team
resource "tfe_team_member" "user1" {
 team_id  = tfe_team.web-dev.id
 username = "rfitzhugh"
}
 
resource "tfe_team_access" "test" {
 access       = "plan"
 team_id      = tfe_team.web-dev.id
 workspace_id = tfe_workspace.technicloud-wordpress.id
}

So basically…

You can find the above code sample on GitHub.

Summary

In this post I reviewed a handful of compelling Terraform Cloud features. This includes remote state management, cost estimation, and collaboration features. Consider using Terraform Cloud for state storage and collaboration (especially the Private Module Registry), it’s free for small teams (up to 5)! Since we do not yet use Sentinel, I did not get a chance to test out Sentinel policies with Terraform Cloud but hope to implement it soon. 

If you have any questions, please reach out to me on Twitter.

Ravello Ends its Silence

On 23 May, I attended the Oracle Blogger Day at HQ in Redwood City. Ravello, an Oracle family member, has been quiet ever since acquisition. After an excellent day spent deep-diving into the product, I think it is safe to say that Ravello will be silent no more.

If you are not familiar with Ravello, it is an overlay cloud service that allows you to take any VMware based multi-VM application and run it in the cloud. This can be done without any change to workload, storage, or the network configuration. Ravello allows you to seamlessly deploy your existing VMware or KVM based data center workloads on Oracle Public Cloud, AWS, or Google Cloud as-is.

 

Screen Shot 2017-06-04 at 6.54.52 PM
On-premises to cloud migration

Most organizations face challenges when moving an enterprise application to the cloud. This image outlines some of the considerations and difficulties.

Screen Shot 2017-06-04 at 6.59.50 PM
Challenges when moving enterprise application to the cloud

Ravello creates a software layer that abstracts the differences between on-premises and cloud infrastructure (networking, storage, virtualization).

The heart and soul of Ravello is HVX. HVX is a virtualization engine designed specifically for nesting. This is what allows vExperts to easily run ESXi hosts in their cloud service without issue. HVX is designed to run on already virtualized hardware using binary translation with direct execution. It exposes VMware or KVM virtual devices, which is why no changes to the VM are required.

Screen Shot 2017-06-04 at 6.17.49 PM
HVX technology overview

But most impressively, it allows you to run any VM in any cloud. Or so they say.

Currently in Ravello, you can add and/or remove NICs, do simple IP filtering, but not much more. One of the items that was demoed by the team was enhanced network editing. The upcoming new networking capabilities include a visual network topology that allows for the creation of switches, configuration of ports, subnets, VLANs, etc.

Screen Shot 2017-06-04 at 6.18.12 PM
Ravello network overlay

Considering how rudimentary networking has been with Ravello, this enhancement will be warmly welcomed once released.

The storage overlay abstracts underlying cloud storage, exposing block devices to the guests. Ravello uses an image caching, copy on write file system.

Screen Shot 2017-06-04 at 6.18.27 PM
Ravello storage overlay

Transparent RAID0 is used, as needed, for large disks.

At this point, you may be wondering where Oracle is going with Ravello. Imagine deep integration with Oracle Cloud — a true lift and shift to the cloud with even heavy enterprise workloads. Think:

  • Ravello on Oracle’s Bare Metal Cloud Service (BMCS)
  • Leverage virtualization with hardware assist
  • Integration of BMCS and Ravello networking

So, where does this fit for businesses?

  • If Oracle and Ravello can really deliver on a “lift and shift” type migration without need for redesign or reconfigure then this will ease transition into the cloud.
  • Creation of identical cloud resources matching that of on-premises for security or penetration testing, update testing, etc.
  • Identical but isolated environments, such as for educational purposes.

Ultimately, I am glad to see that Ravello is still around and that its use cases are growing. Being integrated with Oracle Cloud has propelled towards becoming a more evolved platform. A big issues that I’ve seen as a consulting architect is the difficult and complex migration of legacy applications from on-premises to the cloud.

All in all, as far as Oracle Cloud and Ravello have come, there is still a lot of work to be done. I’m looking forward to what future releases bring.

 

Thank you to Oracle, Ravello, and the Tech Reckoning crew (John, Kat, Amy) for inviting me out to the inaugural blogger day. 

ZeroStack Aims to be a ‘Self Driving Cloud’

ZeroStack is a turnkey solution that provides a private (on premises) cloud or a hybrid solution with AWS integration. I had the opportunity to hear more about this company during Tech Field Day (TFD) 13 earlier this month.

The company’s aim is to be an intelligent “hands off” cloud platform that essentially becomes self driving. According to ZeroStack CEO Ajay Gulati, there are seven layers of a self driving cloud:

  1. Automated cloud deployment & configuration
  2. Integration with other systems: clouds storage, virtualized environments and IT systems
  3. One click, template driven application deployment
  4. Real time alerts, events, and stats
  5. Self monitoring & self healing control plane
  6. Batch analysis for longer term decisions
  7. Automated zero touch upgrades

You can find more information about about what it means to be a ‘self driving cloud’ in the following video.

Currently there are three different ways to acquire ZeroStack:

  • Z-Block Cloud Appliance – this provides a turnkey hyperconverged appliance that deliver a “cloud-in-a-box.”
  • Partner hardware – currently there are validated models of Dell, HPE, SuperMicro, and Cisco UCS hardware that may be acquired.
  • BYOH – bring your own hardware! This allows you to deploy ZeroStack on your choice of supported models of hardware.

To see a demo of a ZeroStack deployment, check out the following video.

I was impressed that ZeroStack already had a partnership with AWS and is able to seamlessly integrate allowing workload deployment both on-premises and in AWS. You can read more about their hybrid cloud offering here (https://www.zerostack.com/use-cases/hybrid-cloud/).

Another thing that I liked was their clean, easy to read and use interface. You can watch a demo of a Hadoop deployment in the follow video and see the interface for yourself.

A point of concern for me is the lack of prioritization of VMs for high availability (HA). There did not seem to be a way to prioritize which VMs should come up first in the event of failure. Another manageability issue is that is seems HA is configure on a per-VM basis…at least that was the impression that I got from the demos. I could see this as a configuration and management nightmare in an enterprise deployment.

All in all I found ZeroStack to be quite interesting and it is a company that I will keep in eye on in the future.