OpenTofu CI/CD Component

🚧 NOTE 🚧

The src/gitlab-tofu.sh script is still merely a copy from gitlab-terraform. Therefore, lots of things in this script and in the templates are still Terraform-related and haven't been changed to their OpenTofu equivalents. Have a look at the Migrating from the Terraform CI/CD templates section when migrating from Terraform CI/CD templates.

This project is home to the OpenTofu CI/CD component and it's related assets, like the gitlab-tofu wrapper script and OCI images containing that script together with an OpenTofu version.

Read more:

• CI/CD components
• Development guide for GitLab CI/CD components
• CI/CD Catalog

Note: Please make sure to use a released version of this CI/CD component. You find all releases on the Releases Overview Page.

♻️ Migrating from the Terraform CI/CD templates? Check this out.

• OpenTofu CI/CD Component
  • Usage
    • Opinionated Templates
    • Job Templates
    • Inputs
    • Available OpenTofu Versions
    • Variables
    • Install additional tools
    • Best Practices
      • Lockfile Handling
  • Releases & Versioning
    • Component Versions
    • Image Versions
  • Usage on self-managed
  • Migrating from the Terraform CI/CD templates
    • OpenTofu component inputs vs. Terraform template variables
  • Can I use this component with Terraform?
  • Contributing

Usage

include:
  - component: $CI_SERVER_FQDN/$CI_PROJECT_PATH/full-pipeline@<VERSION>
    inputs:
      # The version must currently be specified explicitly as an input,
      # to find the correctly associated images. # This can be removed
      # once https://gitlab.com/gitlab-org/gitlab/-/issues/438275 is solved.
      version: <VERSION>
      opentofu_version: <OPENTOFU_VERSION>

stages: [validate, test, build, deploy, cleanup]

---

# ... or without the destroy jobs:
include:
  - component: $CI_SERVER_FQDN/$CI_PROJECT_PATH/validate-plan-apply@<VERSION>
    inputs:
      # The version must currently be specified explicitly as an input,
      # to find the correctly associated images. # This can be removed
      # once https://gitlab.com/gitlab-org/gitlab/-/issues/438275 is solved.
      version: <VERSION>
      opentofu_version: <OPENTOFU_VERSION>

stages: [validate, build, deploy]

A concrete example may look like this:

# Using version `0.10.0`:
include:
  - component: $CI_SERVER_FQDN/$CI_PROJECT_PATH/full-pipeline@0.10.0
    inputs:
      # The version must currently be specified explicitly as an input,
      # to find the correctly associated images. # This can be removed
      # once https://gitlab.com/gitlab-org/gitlab/-/issues/438275 is solved.
      version: 0.10.0
      opentofu_version: 1.6.1

stages: [validate, test, build, deploy, cleanup]

---

# ... in case you absolutely know what you are doing and are
# aware that this may introduce breaking changes, you may use the latest release:
include:
  - component: $CI_SERVER_FQDN/$CI_PROJECT_PATH/full-pipeline@~latest
    inputs:
      # The version must currently be specified explicitly as an input,
      # to find the correctly associated images. # This can be removed
      # once https://gitlab.com/gitlab-org/gitlab/-/issues/438275 is solved.
      version: latest
      opentofu_version: 1.6.1

stages: [validate, test, build, deploy, cleanup]

Or import all jobs as hidden templates ready to be extended:

include:
  - component: $CI_SERVER_FQDN/$CI_PROJECT_PATH/job-templates@<VERSION>
    inputs:
      # The version must currently be specified explicitly as an input,
      # to find the correctly associated images. # This can be removed
      # once https://gitlab.com/gitlab-org/gitlab/-/issues/438275 is solved.
      version: <VERSION>
      opentofu_version: <OPENTOFU_VERSION>

stages: [...]

fmt:
  extends: [.opentofu:fmt]

...

Opinionated Templates

This component repository also provides some templates that may often be used, for example one that only runs validation (fmt and validate), plan and an apply, but no destructive actions.

• validate-plan
• validate-plan-apply

Job Templates

Instead of including the full-pipeline or another opinionated template, it's also possible to include individual jobs and compose your own pipeline, for example, to just run the fmt job you can do:

include:
  - component: $CI_SERVER_FQDN/$CI_PROJECT_PATH/fmt@<VERSION>
    inputs:
      # The version must currently be specified explicitly as an input,
      # to find the correctly associated images. # This can be removed
      # once https://gitlab.com/gitlab-org/gitlab/-/issues/438275 is solved.
      version: <VERSION>
      opentofu_version: 1.6.1
      root_dir: tofu/

Or you can also include the job-templates template, that will include all available OpenTofu jobs as hidden job templates prefixed with .opentofu:. Those are especially useful when you want to minimize your includes and you want to extend the jobs:

include:
  - component: $CI_SERVER_FQDN/$CI_PROJECT_PATH/job-templates@<VERSION>
    inputs:
      # The version must currently be specified explicitly as an input,
      # to find the correctly associated images. # This can be removed
      # once https://gitlab.com/gitlab-org/gitlab/-/issues/438275 is solved.
      version: <VERSION>
      opentofu_version: 1.6.1

plan:
  extends: [.opentofu:plan]
  parallel:
    matrix:
      - TF_ROOT: test/
      - TF_ROOT: prod/

Have a look at the full-pipeline for how it's constructed.

The following job components exist:

• fmt
• validate
• test
• plan
• apply
• destroy
• delete-state
• custom-command

Have a look at the individual template spec to learn about the available inputs.

Inputs

Available OpenTofu Versions

The following OpenTofu versions are available with this component via the opentofu_version input:

• 1.7.1
• 1.7.0
• 1.7.0-alpha1
• 1.6.2
• 1.6.1
• 1.6.0

Variables

(🚧 This section is work in progress)

Have a look at the src/gitlab-tofu.sh script and how the TF_-prefixed variables are being used. You may set them according to your needs.

Install additional tools

The gitlab-opentofu container image deliberately comes with minimal tooling to keep the image size small and be the least common denominator for our users.

However, it is sometimes necessary to install additional tools. To do that you can overwrite the included jobs with a before_script entry. The gitlab-opentofu image uses alpine as its base image and therefore apk can be used to install the tools. For example to install jq:

include:
  - component: $CI_SERVER_FQDN/$CI_PROJECT_PATH/validate-plan@<VERSION>
    inputs:
      version: <VERSION>
      opentofu_version: 1.6.1

plan:
  before_script:
    - apk add jq

Best Practices

This section is a collection of some best practices. Feel free to contribute more that generally apply. If a best practice really becomes the de-facto standard we may make it the default behavior if possible.

Lockfile Handling

If you commit the Lockfile (.terraform.lock.hcl) to your repository we recommend setting either the TF_INIT_FLAGS (handled by this component) or TF_CLI_ARGS_init (handled by OpenTofu directly) to -lockfile=readonly to prevent any changes to the lockfile during the pipeline job and with that ensuring that OpenTofu really uses the locked dependencies.

Releases & Versioning

This project currently releases tagged commits. An overview of releases can be found on the Releases page and a Changelog can be found here.

Each release is accessible in the CI/CD Catalog.

Component Versions

The component release versions follow Semantic Versioning 2.0.0.

Image Versions

This project releases multiple OCI image variants that can be used with the component. The intention is that the images used in a component have the same version and or not mixed. Due to the limitations described in https://gitlab.com/gitlab-org/gitlab/-/issues/438275 it's currently required to provide the component version in the component include field and as the version input. Check out the Usage section for examples.

Each component release deploys the following images:

• $CI_REGISTRY/$CI_PROJECT_PATH/gitlab-opentofu:<VERSION>-opentofu<OPENTOFU_VERSION>
• $CI_REGISTRY/$CI_PROJECT_PATH/gitlab-opentofu:<VERSION>-opentofu
  • Includes the latest stable OpenTofu version at the time of releasing the component
• $CI_REGISTRY/$CI_PROJECT_PATH/gitlab-opentofu:<VERSION>
  • Includes the latest stable OpenTofu version at the time of releasing the component

In the above examples <VERSION> references the component version and <OPENTOFU_VERSION> an OpenTofu release, from here.

Note: unfortunately, these image versions are not SemVer compatible, because - indicates a prerelease (which they are not in this case). However, we cannot use the alternative + which would indicate build metadata as we'd like. See https://github.com/distribution/distribution/issues/1201

Usage on self-managed

GitLab CI/CD components are not yet distributed and available on self-managed GitLab instances. (see details here). It's also not possible to just include CI/CD components across instance, thus an include like - component: gitlab.com/components/opentofu/full-pipeline@~latest won't work from a self-managed instance. However, you could mirror this project from GitLab.com onto any self-managed instance using a repository pull mirror.

If the component is being mirrored to another path than components/opentofu, then you also need to change that path in the include:component and additionally provide the correct image_registry_base input.

See also the official GitLab documentation for it here.

If you want to save runner resources you may disable the unit and integration tests by setting the SKIP_TESTS CI/CD variable to true.

Migrating from the Terraform CI/CD templates

When migrating from the GitLab Terraform CI/CD templates you can use the following migration rules:

• Used Terraform.gitlab-ci.yml -> Migrate to validate-plan-apply.
• Used Terraform/Base.gitlab-ci.yml -> Migrate to job-templates.
  • Migrate the .terraform: job prefix to .opentofu:.
• Used the kics-iac-sast job -> Additionally include the Jobs/SAST-IaC.latest.gitlab-ci.yml template.
• Migrate the following job names:
  • build -> plan
  • deploy -> apply
• Migrate the TF_ROOT variable to the root_dir input.
  • Although the TF_ROOT variable is still used and maybe overwritten after the import on individual jobs.
• Migrate the TF_STATE_NAME variable to the state_name input.
  • Although the TF_STATE_NAME variable is still used and maybe overwritten after the import on individual jobs.
• Migrate the TF_AUTO_DEPLOY variable to the auto_apply input.
• Used other variables -> Use the same variables with this component.

The same rules apply for the latest templates. We also recommend to check out the Usage section for more details about the available templates and inputs.

OpenTofu component inputs vs. Terraform template variables

This OpenTofu CI/CD component makes use of inputs whereas the Terraform CI/CD templates used variables. We recommend that you use the inputs with the OpenTofu component where available and required. However, if needed you may overwrite the jobs and set the variables you like.

Can I use this component with Terraform?

Probably. Although, we don't officially support it or maintain any compatibility layer where necessary.

The OpenTofu CI/CD component job mainly interface with the gitlab-tofu script that is distributed with the gitlab-opentofu container image used as the base image for the jobs. This base image also contains the tofu binary.

If you'd want to use Terraform instead you may provide your own container image that contains at least a script called gitlab-tofu so that it's compatible with the component jobs. Everything else in the job can be custom, like replacing tofu with terraform.

You may provide the image_registry_base input to any of the component includes, pointing to the container registry URI hosting the container image. The container image name can be configured in the image_name input. The image has be versioned so that it is compatible with the image versioning of this project.

Contributing

See the CONTRIBUTING.md guide.