Ditch those static CI credentials! The beauty of dynamic cloud credentials for your pipelines using OIDC

2022-Jun-21 • by David Norton

If you're a developer working with softwarer deployed in the cloud, you are probably familiar with this scenario: you need to set up a deployment pipeline, and you need a way to authenticate to the cloud (let's say AWS) from the pipeline.

You have a few options:

  1. Run your CI workers on AWS with an instance profile.
    1. Do you want to give the same access to each pipeline? Or to builds on every branch on that pipeline? Probably not... so we would need to run multiple workers that can each assume different IAM roles, and limit various jobs to run on the correct workers.
  2. Configure your CI system with access keys for an AWS IAM user, and allow certain pipelines to use those AWS credentials.
    1. Similarly, we want to ensure different pipelines have different permissions, so we should create multiple sets of credentials and grant them access from different pipelines.
    2. These credentials don't expire by default, so you should set up a credential-rotation process to ensure we don't have years-old credentials hanging around.
  3. Obviously, we would never embed credentials in the source control repository.
  4. Also, we don't want to rely on credentials living openly on the filesystem or other local network location.
  5. Utilize your CI system's capability as an OIDC Identity Provider to authenticate using dynamic, short-lived credentials.
    1. Guess what! This is what we're going to demo here.

OpenID Connect (OIDC) is a standard authentication mechanism based on the OAuth 2.0 protocol. It's used across the internet for user authentication (e.g. the Sign in with Google or Sign in with Facebook buttons you see everywhere), and is now being used more and more for workload authentication.

The tech stack

Today we're going to demonstrate OIDC authentication with the following technologies:

  1. GitLab CI for the pipeline
    1. You could substitute GitHub Actions here, or any other CI tool that acts as an identity provider (IdP) -- examples include Azure Pipelines or CircleCI
  2. Amazon Web Services as the authentication target
    1. You can also use these practices to authenticate to Google Cloud Platform, Azure, or any other service that supports OIDC authentication.
  3. AWS CLI to do the deployment
    1. You could use Terraform, Boto, or any other tool that supports the default AWS credential chain.
  4. Terraform to configure everything
    1. You could use your preferred cloud configuration tool such as CloudFormation, CDK, ClickOps (gross), or script it via the AWS CLI.

Step 1: Add GitLab as an OIDC provider in AWS

We first have to configure AWS with an OIDC provider so we can later use it in IAM roles. We'll use Terraform for all of the AWS configuration.

These values work for GitLab.com -- if you host your own instance, or use another CI tool, you'd need to use slightly different configuration.

locals {
  gitlab_url = "https://gitlab.com"
}

data "tls_certificate" "gitlab" {
  url = local.gitlab_url
}

resource "aws_iam_openid_connect_provider" "gitlab" {
  thumbprint_list = [data.tls_certificate.gitlab.certificates.0.sha1_fingerprint]

  client_id_list = [local.gitlab_url]
  url            = local.gitlab_url
}

Step 2: Create IAM role

Now that we've got GitLab configured as a trusted OIDC provider, we need to allow certain builds to assume an IAM role. This is done by creating an IAM role and specifying a trust policy (or "Assume Role Policy"), specifying who (or what) can assume the role.

This assume role policy says that builds running on the main branch of the platformers/website project on GitLab.com can assume this role.

locals {
  gitlab_group = "platformers"
  gitlab_project = "website"
  gitlab_branch = "main"
}

resource "aws_iam_role" "gitlab_website_ci" {
  name               = "gitlab_website_ci"
  assume_role_policy = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Federated": "${aws_iam_openid_connect_provider.gitlab.arn}"
      },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "gitlab.com:sub": "project_path:${local.gitlab_group}/${local.gitlab_project}:ref_type:branch:ref:${local.gitlab_branch}"
        }
      }
    }
  ]
}
EOF
}

We then attach policy to the role -- allowing anyone (or anything) that assumes the role to do what it needs to do. This policy allows some basic S3 operations on a particular bucket

locals {
  bucket_name = "my-bucket"
}

resource "aws_iam_role_policy" "gitlab_website_ci" {
  name   = "s3"
  role   = aws_iam_role.gitlab_website_ci.name
  policy = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:PutObject",
        "s3:DeleteObject"
      ],
      "Resource": "arn:aws:s3:::${local.bucket_name}/*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "s3:ListBucket"
      ],
      "Resource": "arn:aws:s3:::${local.bucket_name}"
    }
  ]
}
EOF
}

Step 3: Assume the role via GitLab CI

This GitLab CI uses two standard environment variables that are used in nearly every AWS client configuration. AWS_ROLE_ARN defines the role to be assumed, and AWS_WEB_IDENTITY_TOKEN_FILE defines a file that will be used to read the token.

The before_script takes the token provided by GitLab, and places it in the file for the AWS CLI to pick up. 

image: amazon/aws-cli:2.7.9

variables:
   AWS_ROLE_ARN: arn:aws:iam::1111111111111:role/gitlab_website_ci
   AWS_WEB_IDENTITY_TOKEN_FILE: /tmp/web-identity-token

before_script:
  - echo "$CI_JOB_JWT_V2" > $AWS_WEB_IDENTITY_TOKEN_FILE

# ... whatever you do to build or test your application ...

publish:
  stage: deploy
  script:
  # just some debug logging to help ensure you have the right identity...
  - aws sts get-caller-identity
  - aws s3 sync public/ s3://my-bucket/
  rules:
    - if: $CI_COMMIT_REF_NAME == $CI_DEFAULT_BRANCH

If you replace the role ARN and bucket name with your own values, you should see the build pass (on the main branch)!

Conclusion

This tutorial showed specifically how to authenticate from GitLab CI builds to AWS, and more generally should have made clear how this pattern could be used for CI tool that acts as an OIDC identity provider, and any cloud provider that allows authentication via OIDC.

This eliminates a static set of credentials that need to be rotated and could be compromised. It also adheres to the principle of least privilege -- avoiding granting all of your CI builds "God mode" by using a broad IAM role across all CI workers.

Shameless plug: If you need help with public cloud, containers, or CI/CD -- Platformers may be able to help. Consider reaching out today!