Shift Left, Honestly: Security Controls in a CI/CD Pipeline

“Shift left” started as a scheduling observation: defects found late cost more to fix than defects found early. It was true when it was said about testing and it remains true about security. Somewhere between the original insight and the current vendor market, it turned into a purchasing instruction — buy scanners, wire them into the pipeline, watch the dashboard go green.

That version fails in a predictable way. A team adds five scanners to pull request CI. Build time goes from four minutes to nineteen. The tools produce a combined few thousand findings against a codebase nobody has ever scanned. Nothing is triaged because nothing is prioritised. Within two sprints someone adds continue-on-error: true to unblock a release, and within two months every gate is advisory. The tooling is still installed. The security posture is unchanged and the pipeline is slower.

Shifting left works when you treat it as a latency problem, not a coverage problem. The goal is not to run every possible check as early as possible; it is to give a developer the shortest possible loop between writing a defect and learning about it, with a signal strong enough to justify stopping. This article maps each control class to the stage where it earns its cost, and is deliberately blunt about which ones should ever block a merge.

What “Left” Actually Buys You

The value of moving a check earlier is not that the finding is cheaper to fix in some abstract accounting sense. It is that the developer still has context. Someone who wrote a SQL string concatenation eleven minutes ago can fix it in thirty seconds. The same finding surfaced eleven weeks later in a quarterly report goes into a backlog, gets assigned to whoever inherited the service, and requires an hour of archaeology before a line changes.

That framing gives you a decision rule for every proposed gate. A check belongs earlier if it is fast, deterministic, and produces findings the author can act on without external context. A check belongs later if it needs a running application, cross-service state, or a human risk decision.

A security gate has exactly one job: to stop a change that a developer would agree should be stopped. Every finding that fails that test is not a control, it is a tax — and taxes get evaded.

Stage Zero: The Developer Machine

Pre-commit hooks are the cheapest security control that exists, and the only one that prevents a class of problem rather than detecting it. The obvious candidate is secret detection, because a credential that reaches a remote branch must be treated as compromised and rotated even if you force-push it away. Catching it before the commit object exists is qualitatively different from catching it in CI.

Keep the hook set small and fast. Anything over roughly two seconds gets bypassed with --no-verify, and once developers learn that flag they use it habitually.

# .pre-commit-config.yaml
repos:
  - repo: https://github.com/gitleaks/gitleaks
    rev: v8.21.2
    hooks:
      - id: gitleaks

  - repo: https://github.com/bridgecrewio/checkov
    rev: 3.2.257
    hooks:
      - id: checkov
        args: ["--framework", "terraform", "--compact", "--quiet"]
        files: ^infra/.*\.tf$

  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v5.0.0
    hooks:
      - id: detect-private-key
      - id: check-merge-conflict

Two things make this stick. First, install it via a repository bootstrap script so nobody has to remember. Second, run the identical checks server-side in CI, because local hooks are advisory by nature — a developer can skip them, and the CI run is what makes the control real. Local hooks are for speed; CI is for enforcement.

Pre-Merge: What Belongs in Pull Request CI

This is the contested stage. Everything wants to live here and not everything should.

Secret scanning belongs here unconditionally and should block. Scan the full diff range of the pull request, not just the tip commit, because secrets get introduced and then rewritten within a branch. Findings are near-deterministic when you scan for high-entropy strings matched against known provider formats — an AWS access key ID, a Slack token, a private key header. Verified-secret modes, which test whether a candidate credential actually authenticates, cut false positives close to zero and are worth enabling.

SAST belongs here, but scoped to changed files by default and to high-confidence rule sets only. Running a full-repository deep taint analysis on every pull request is how you get nineteen-minute builds. Run the incremental, diff-aware mode in PR CI and reserve the full scan for a nightly or weekly job whose findings go to a backlog rather than a merge gate.

Software composition analysis belongs here and needs the most careful gate design. The naive policy — fail on any high or critical CVE in any dependency — is unworkable, because a large fraction of reported vulnerabilities are in code paths your application never reaches, and many have no fix available at the time of the alert. The two filters that make SCA gates tolerable are reachability and fixability. Fail the build when a vulnerability is both fixable and in a directly imported package; report everything else without blocking.

IaC scanning belongs here and should block on a small, curated rule set: public network exposure, missing encryption, wildcard IAM, disabled logging. Terraform, CloudFormation and Kubernetes manifests are declarative, so findings are precise and remediation is usually a single attribute. Resist enabling the full default policy pack, which will flag hundreds of stylistic and low-severity items on first run.

DAST does not belong here. It needs a deployed, reachable application and it is slow. Its home is a post-deploy job against a staging environment.

Build Time: Images, SBOMs and Provenance

Once a change merges, the build stage produces the artifact that actually reaches production. Three things belong here.

Image scanning catches what SCA misses: vulnerabilities in the operating system packages of your base image. This is often the largest single source of critical findings, and it is also the easiest to fix, because the remediation is usually rebuilding on a current base image or switching to a minimal or distroless variant. A gate that fails on fixable critical OS package CVEs is reasonable precisely because remediation does not require code changes.

SBOM generation should be unconditional. Generate it at build time, from the built artifact rather than from the manifest files, and store it alongside the image. The value shows up on the day a new vulnerability lands in a widely used library and someone asks which of your two hundred services ship it. Answering that from stored SBOMs takes minutes; answering it by rebuilding and rescanning everything takes days.

Signing and provenance close the loop. A signature proves the artifact came from your pipeline; a provenance attestation records which source commit, which builder, and which parameters produced it. Keyless signing using workload identity removes the key management burden that killed earlier signing efforts.

name: build-and-attest
on:
  push:
    branches: [main]

permissions:
  contents: read

jobs:
  build:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      packages: write
      id-token: write        # required for keyless signing
    steps:
      - uses: actions/checkout@v4

      - uses: docker/build-push-action@v6
        id: build
        with:
          push: true
          tags: ghcr.io/acme/api:${{ github.sha }}

      - name: Generate SBOM from the built image
        run: |
          syft "ghcr.io/acme/api@${{ steps.build.outputs.digest }}" \
            -o spdx-json=sbom.spdx.json

      - name: Fail only on fixable critical OS package CVEs
        run: |
          grype "sbom:sbom.spdx.json" \
            --fail-on critical \
            --only-fixed

      - uses: sigstore/cosign-installer@v3

      - name: Sign the image digest, keyless
        run: cosign sign --yes "ghcr.io/acme/api@${{ steps.build.outputs.digest }}"

      - name: Attach the SBOM as a signed attestation
        run: |
          cosign attest --yes \
            --predicate sbom.spdx.json \
            --type spdxjson \
            "ghcr.io/acme/api@${{ steps.build.outputs.digest }}"

Note that everything references the image by digest, never by tag. Tags are mutable; a signature over a tag proves nothing. This is the single most common implementation error in artifact signing.

Signing is only half the control. The other half is enforcement at the deployment boundary — an admission controller or deployment policy that refuses unsigned images. Without that, you have produced signatures nobody verifies.

Post-Deploy: DAST and Runtime Feedback

DAST earns its place against a deployed staging environment, ideally driven by your API schema rather than a blind crawl. Schema-driven scanning against an OpenAPI document reaches authenticated endpoints and understands parameter types, which raises the hit rate substantially over spidering.

Treat DAST output as ticket-generating, never merge-blocking. Runs are long, results vary between executions, and a flaky gate on the critical path to production will be disabled within a month.

Runtime signals close the loop in the other direction. Production telemetry — which packages are actually loaded, which endpoints receive traffic, which service accounts are used — is the highest-fidelity input you have for prioritising everything upstream. A critical vulnerability in a library that is present in the image but never loaded at runtime is a genuinely lower priority than a medium one in a request-handling path. Feeding that back into SCA triage is what separates mature programmes from noisy ones.

Tuning Gates So Developers Do Not Route Around Them

False-positive fatigue is not a tooling problem you can buy your way out of. It is a policy problem, and it has a small number of workable answers.

Start every new tool in report-only mode. Run it for two to four weeks, triage the findings, tune the rules, and only then turn on blocking. Enabling a scanner in blocking mode on day one against an existing codebase guarantees a backlog nobody will clear.

Apply a baseline. Every mature scanner supports suppressing pre-existing findings so the gate applies only to newly introduced ones. This is the difference between “fix eight hundred things before you can ship” and “do not make it worse.” The existing findings still need a remediation plan, but that plan belongs on a roadmap, not in a merge gate.

Gate on a narrow, defensible rule set. For each blocking rule, ask whether you would personally defend stopping a Friday afternoon release over it. If not, downgrade it to reporting. A gate of twenty high-confidence rules that developers respect beats two hundred that they resent.

Make suppression legitimate, visible and expiring. Developers will suppress findings. If you do not provide a sanctioned path, they will comment out the step. Provide inline suppression that requires a justification string, surface all active suppressions in a review dashboard, and expire them after a fixed period so they get revisited.

Measure the gate itself. Track false-positive rate per rule, mean time from finding to fix, and how often the gate is bypassed. A rule with a high suppression rate is a broken rule. Retire it rather than defending it.

Keep pull request CI under ten minutes total. This is a security requirement, not a developer-experience nicety. Slow pipelines drive batching, batching produces large pull requests, and large pull requests get reviewed badly. Parallelise scan jobs, cache dependency databases, and push anything slow to a nightly schedule.

Ownership and the Part Tooling Cannot Do

The pipeline enforces decisions; it does not make them. Threat modelling at design time still catches the class of problem no scanner detects — missing authorisation checks between tenants, a workflow that lets a user escalate their own role, a webhook without signature verification. Those are logic flaws, and every scanner on the market is structurally blind to them.

The organisational pattern that works is embedding: a named security contact per team who reviews design documents, owns the tuning of that team’s gates, and has an escalation path into the central security function. Central teams own the platform, the baseline policy, and the exception process. Product teams own their findings. Security findings that sit in a central queue with no owning engineer do not get fixed.

Key Takeaways

• Place each control where its feedback is fastest and its signal is cleanest. Secrets and IaC belong at pre-commit, diff-scoped SAST and SCA at pull request, image scanning and signing at build, DAST after deploy.
• Only block merges on checks that are fast, deterministic, and produce findings the author can fix immediately. Everything else reports.
• Baseline existing findings so gates apply to new code only, then work the backlog on a roadmap rather than in the merge path.
• Sign and attest artifacts by digest, never by tag, and enforce verification at the deployment boundary or the signatures mean nothing.
• Treat false-positive rate and bypass frequency as first-class metrics. A rule with a high suppression rate is a broken rule, not a discipline problem.
• Keep pull request CI under ten minutes. Slow pipelines produce big batches, and big batches get reviewed badly.