Managing Environment Variables and Secrets in CI/CD Pipelines

A Leaked Key Is Usually a Pipeline Problem

The 2022 Travis CI incident is the clearest cautionary tale in this space: for years, environment variables, including secrets, were printed into logs of public repository builds, exposing tokens for thousands of organisations. Nobody wrote echo $AWS_SECRET_ACCESS_KEY. The platform leaked the values by default, and most teams never noticed.

That is the uncomfortable truth about secrets in continuous integration. The credential rarely leaks because someone was reckless. It leaks because of a default setting, a transformed value that defeated log masking, or a long-lived key that should never have existed in the first place. Get the configuration right and most of these failures disappear.

This article covers how to handle environment variables and secrets across a CI/CD pipeline: the distinction between config and secrets, how to scope and mask values, why OIDC beats static keys, and the specific mistakes that put credentials in your build logs.

Config and Secrets Are Not the Same Thing

The single most useful distinction is between non-sensitive configuration and actual secrets. They look identical in a KEY=value list, but they demand completely different handling.

The Twelve-Factor App config guidance ↗ argues that all config should live in the environment, separate from code. That is correct, but in a CI/CD context you have to split the environment into two tiers.

TypeExamplesStorageVisible in logs?
Plain configBuild target, region name, feature flags, log levelPlain CI variables, committed config filesFine to print
SecretsAPI tokens, database passwords, signing keys, cloud credentialsEncrypted secret store or external managerNever print, always masked

The mistake teams make is treating everything as a secret (which makes pipelines opaque and hard to debug) or treating everything as config (which leaks credentials). Classify each value once, store it in the right tier, and the rest of your hardening follows naturally.

Where Secrets Should Actually Live

A secret has a source of truth and a delivery path. The source of truth should never be the repository or a workflow file. Ranked from weakest to strongest:

Secret Storage, Weakest to Strongest Hard-coded in repo or workflow file (never) Plain CI variable (unencrypted, readable) Encrypted CI secret store, scoped External secrets manager (Vault, cloud KMS) OIDC short-lived tokens (no stored secret)

The encrypted CI secret store is the baseline every team should reach. The GitHub Actions secrets documentation ↗ describes how values are encrypted at rest and only decrypted into the runner at job execution. GitLab CI offers protected and masked variables; CircleCI offers contexts. All three give you encryption, masking, and access scoping.

For broader patterns beyond CI specifically, our guide on secrets management for developers covers the wider tooling landscape, and environment variables done right covers how to structure config in the application itself.

Scope Secrets as Tightly as the Platform Allows

A secret that every job in every branch can read is a secret with a large blast radius. Scope aggressively.

  • By environment. A staging deploy key should never be readable by a production job, and vice versa. Most platforms support environment-scoped secrets so a deploy token only exists for the job that targets that environment.
  • By branch or protection rule. Production credentials should only be available on protected branches. GitLab’s “protected” flag and GitHub’s environment protection rules both enforce this.
  • By job, not by pipeline. Inject a secret into the single step that needs it rather than exporting it for the whole workflow. The fewer steps that can see a value, the fewer places it can leak.

The principle is least privilege applied to your pipeline. If you are already thinking about this for your application code, our article on the developer’s guide to secure coding practices covers the same mindset at the code level.

Masking Is Not Foolproof

Every serious CI platform masks known secret values in log output, replacing them with ***. This is genuinely useful and you should rely on it, but understand its limits, because they are exactly where leaks happen.

Masking works by string matching. The platform knows the literal value of the secret and redacts that exact string from the logs. The moment the value is transformed, the match breaks:

# Masked correctly: the literal value is matched
echo "$API_TOKEN"          # prints ***

# Leaks: base64 encoding produces a string the masker does not recognise
echo "$API_TOKEN" | base64 # prints the encoded secret in clear

# Leaks: shell tracing prints every expanded command
set -x
curl -H "Authorization: Bearer $API_TOKEN" https://api.example.com
# the trace line shows the expanded header with the token

Three rules keep masking effective:

  1. Never enable shell command tracing (set -x) in jobs that handle secrets. It prints expanded commands, including the secret values, before masking can catch reformatted output.
  2. Do not transform secrets in the shell. Base64, JSON wrapping, and string concatenation all produce values the masker will not redact.
  3. Set tools to quiet mode. Many CLIs echo their configuration, including credentials passed as flags. Prefer passing secrets via environment variables or stdin, not command-line arguments, which can also appear in process listings.

Stop Storing Long-Lived Cloud Keys

The largest single source of CI credential leaks is the long-lived static cloud access key. It is created once, copied into a secret store, and then lives forever: in the store, in any log that ever captured it, in the memory of whoever set it up. Rotation is a manual chore that everyone postpones.

OpenID Connect removes the stored secret entirely. Instead of holding a permanent key, the pipeline proves its identity to the cloud provider and receives a short-lived token scoped to that specific repository and workflow.

OIDC: No Stored Secret CI runner presents identity token Cloud provider validates trust policy Short-lived token expires in minutes No permanent access key is stored anywhere. The token rotates on every run. Trust is scoped to a specific repository, branch, and workflow.

GitHub Actions, GitLab CI, and most managed platforms support OIDC against AWS, GCP, and Azure. The setup cost is a one-time trust policy on the cloud side. After that, every deploy job authenticates with a token that expires before it could plausibly be abused, and there is no key left to rotate, log, or steal.

If OIDC is not available for a given integration, treat the static secret as a liability with a clock on it: scope it, monitor its use, and rotate it on a fixed schedule.

Inject at Build Versus Inject at Deploy

A subtle but important decision is when a secret enters the pipeline. Baking secrets into a build artefact is almost always wrong.

If you embed a production API key into a Docker image or a compiled bundle during the build stage, that secret is now permanently inside an artefact that may be cached, pushed to a registry, or shared across environments. Anyone who pulls the image can extract it.

The pattern that works is build once, inject at deploy. Build a single environment-agnostic artefact, then supply environment-specific secrets at deploy or runtime through the orchestrator’s secret mechanism. This aligns with how artefact promotion works in deployment strategies like blue-green and canary releases, where the same artefact moves between environments and only the injected configuration changes.

The same logic applies to container builds. Use build-time secret mounts that do not persist in the final image rather than ARG or ENV instructions that bake values into a layer. Our guide on Docker for developers covers the layering mechanics in more detail.

Mind the Fork

Workflows triggered by pull requests from forked repositories are a special hazard. A contributor you have never met can submit a PR, and if your pipeline runs their code with access to your secrets, they can simply add a step that exfiltrates them.

Most platforms default to withholding secrets from fork-triggered runs for this reason. Do not override that default casually. Keep secret-dependent steps, such as deploys and integration tests against real services, off the code path that executes untrusted contributions. Run untrusted code in a sandboxed job with no credentials, and gate any privileged work behind a maintainer approval or a trusted trigger.

A Practical Hardening Checklist

Work through this in order. The early items are cheap and prevent the most common leaks.

PriorityActionPrevents
1Classify every value as config or secretSecrets stored as plain variables
2Move all secrets into the encrypted storePlaintext credentials in repo or variables
3Disable set -x and quiet noisy toolsSecrets printed via tracing or tool output
4Scope secrets by environment and branchStaging access reaching production keys
5Replace static cloud keys with OIDCLong-lived credentials leaking or going stale
6Inject secrets at deploy, not buildCredentials baked into shipped artefacts
7Block secret access on fork-triggered runsUntrusted PR code exfiltrating secrets
8Rotate remaining static secrets on a scheduleForgotten credentials with unbounded lifetime

For the wider pipeline context these controls sit inside, see our guide on how to build a CI/CD pipeline that actually works.

Treat Secrets as Pipeline Infrastructure

The teams that avoid credential leaks are not the ones with the most diligent engineers. They are the ones who configured the pipeline so that leaking a secret takes deliberate effort rather than happening by default. Classify your values, lean on OIDC to eliminate stored keys, scope what remains, and keep transformed secrets out of your logs.

Start with the encrypted store and set -x, because those two changes prevent the majority of real-world leaks. Then work toward OIDC, which removes the problem at its root by removing the secret itself.

Frequently asked questions

Should I store secrets as environment variables in CI?

Environment variables are an acceptable transport for secrets inside a CI job, but they should never be the source of truth. Inject them at runtime from your CI platform's encrypted secret store or an external secrets manager such as Vault or a cloud provider's secret service. Never commit secrets to the repository, hard-code them in workflow files, or paste them into plain configuration variables, because those are stored unencrypted and are visible to anyone with read access.

Why does my secret still appear in the CI logs?

CI platforms mask known secret values in log output, but masking only works when the platform knows the exact string. If a secret is transformed (base64 encoded, concatenated, or printed by a tool that reformats it), the masking pattern no longer matches and the value leaks. Avoid echoing secrets, set tools to quiet mode, and never run shell with command tracing enabled when secrets are in scope.

What is OIDC and why is it better than long-lived cloud keys?

OpenID Connect lets your CI pipeline request a short-lived token from a cloud provider by proving its identity, rather than storing a permanent access key. The token expires in minutes, is scoped to a specific repository and workflow, and never sits in your secret store. This removes the single largest source of credential leaks: long-lived static keys that get copied, logged, or forgotten.

How often should I rotate CI secrets?

Rotate static secrets on a schedule that matches their blast radius. Production database credentials and cloud keys warrant rotation every 30 to 90 days at minimum, and immediately after any team member with access leaves. The better answer is to eliminate long-lived secrets where possible by using OIDC and short-lived tokens, which rotate automatically on every run and remove rotation as a manual chore.

Can pull requests from forks access my repository secrets?

By default, most CI platforms do not expose secrets to workflows triggered by pull requests from forked repositories, precisely because a malicious contributor could otherwise add a step that prints them. Be very careful with any trigger that runs untrusted code with access to secrets. Use dedicated triggers for trusted workflows and keep secret-dependent steps off the path that runs fork-submitted code.

Enjoyed this article? Get more developer tips straight to your inbox.

Comments

Join the conversation. Share your experience or ask a question below.

0/1000

No comments yet. Be the first to share your thoughts.