CI/CD Pipeline Controls for AI-Generated Code

CI/CD pipelines were built on a quiet assumption: a human wrote the code, made a series of deliberate tradeoffs, and can be pulled into a Slack thread to explain why. AI-generated code breaks that assumption at the root, and this piece is about what has to change in the pipeline itself to account for it.

The model didn't choose anything. It has no memory of the last commit it wrote, no sense of what environment the code will run in, and nobody to hold accountable when it goes wrong. Speed makes this worse before it makes it better: developers using AI assistance commit code noticeably faster than those who don't, which means volume now outpaces whatever manual review process used to catch problems on the way out the door.

The flaw rate isn't a bug that better models will fix, either. Veracode's testing across more than a hundred large language models found the rate at which models introduce security flaws holding roughly steady across GPT-4, GPT-5, Claude, and Gemini. Newer, more capable models are not writing meaningfully safer code; they're writing more of it, faster. Worth sitting with that for a second, because it cuts against the instinct that this is a temporary growing pain we'll model our way out of.

Here's the part that should bother you more than the raw flaw rate: when a model is given a choice between a secure implementation and an insecure one, it picks the insecure path at close to the same frequency it introduces flaws in general. That's a systematic lean toward whatever's easier to generate, convenience over safety, baked into how these models predict the next token. Add in Sonatype's research showing AI coding tools recommend nonexistent or incorrect packages at a rate that isn't trivial, and you get a second failure mode stacked on the first: bad dependencies entering the pipeline with less scrutiny than a human-written import would ever get, because AI-generated code tends to get accepted fast.

Put the three together, volume, per-unit flaw rate, and acceptance speed, and you get risk accumulating faster than any human review gate can drain it. Conventional pipelines were built to catch the occasional human mistake. That design assumption doesn't hold when the generator produces flawed output at scale, by design, every single time you run it.

Where current pipelines are exposed when AI code moves through them

Start with the pipeline itself, not the application code, because that's where a lot of the actual damage happens. A large share of secrets that scanners find in repositories trace back to CI/CD infrastructure directly, not to application logic. The pipeline is an attack surface in its own right, with its own credentials, its own logs, its own blast radius.

The tj-actions/changed-files compromise in March 2025 is the case study everyone in this space should know cold. Attackers altered version tags on a widely-used GitHub Action to point at a malicious commit, which then dumped CI/CD secrets straight into build logs. CISA tracked it as CVE-2025-30066. It hit tens of thousands of repositories before anyone caught it. The lesson isn't subtle: the action runners and third-party tools sitting inside your pipeline are just as attackable as the code moving through them, maybe more so, because nobody's running SAST on a GitHub Action.

Then there's the open-source supply itself. Sonatype identified hundreds of thousands of new malicious open-source packages in a single year. When an AI model hallucinates a dependency name, or confuses a real package with a similarly-named one, there's a real chance it's pointing straight at one of those. That's the exact mechanism attackers now register malicious packages to exploit, betting that an AI tool will recommend the name before a human catches it.

Self-hosted runners add another layer of exposure that doesn't get enough attention. A meaningful share of enterprises run non-ephemeral self-hosted runners with weaker configurations than they'd tolerate anywhere else, and that opens the door to lateral movement across repositories and cloud environments. Now put AI-generated code, executing arbitrary logic, on that same runner. The risk isn't additive; it compounds.

Here's the part that should worry platform teams specifically: alert fatigue isn't a tooling gap, it's a structural one. When a large share of security findings go uninvestigated because nobody can tell what they mean or who owns them, bolting on more scanners doesn't help. It just produces more alerts nobody looks at. A substantial majority of organizations already admit to shipping code with known vulnerabilities, and per Wiz's State of Code Security Report, most organizations have cloud credentials sitting exposed in public repositories right now. Those credentials reach production directly. A pipeline that passes code without scanning for secrets is an open path to production with a few extra steps in between.

The security gate model that AI code volume actually requires

Here's the shift that actually matters: pipelines need to move from "scan and report" to "scan and block." A finding that gets filed as a ticket for someone to review next sprint doesn't help when the underlying problem is volume outpacing review capacity in the first place. The gate has to stop the deploy, not just note that something looked wrong.

Three control layers need to sit at every commit, not just at the release branch. Static Application Security Testing catches vulnerable patterns in the generated code before it executes anywhere. Software Composition Analysis validates every dependency the model recommended, checking for fabricated package names, known malicious packages, and version pins that don't match the lockfile. Container and image scanning catches what got baked into the build artifact before that artifact reaches any runtime at all.

Why every commit? Because by the time a vulnerability reaches a release gate, given how fast AI-generated code gets accepted and how much of it there is, it may already be sitting in dozens of downstream builds. Shift-left thinking, catching problems earlier in the cycle, is necessary here. But it isn't sufficient on its own; scanning without blocking still lets a high proportion of flawed code through, because scanning alone doesn't stop anything, it just tells you about it later.

Severity thresholds have to carry real teeth. Critical findings auto-block, full stop. High findings route to mandatory triage before merge, not after. The gate needs to function as an actual gate, not an advisory note that a busy engineer can dismiss under deadline pressure.

Three governance frameworks map cleanly onto this structure, and it's worth knowing them by name. NIST's Secure Software Development Framework (SP 800-218) addresses process-level controls. SLSA (Supply-chain Levels for Software Artifacts) addresses build integrity and provenance. OWASP's CI/CD Top 10 addresses the pipeline-specific attack patterns (things like poisoned pipeline execution and dependency confusion) that generic application security frameworks don't cover well. Each hits a different layer, and none of them alone covers the full surface.

A Pipeline Bill of Materials, or PBOM, closes a gap that none of the individual scanners catch: it tracks provenance across commit, build, deployment, and runtime, so when something goes wrong you can trace it back to the exact AI-generated dependency or code path that introduced it. Without that lineage, you're debugging blind.

None of this should be fully automated, though. High-risk actions (production deploys, infrastructure changes, new dependency additions) need explicit human sign-off, and that requirement has to be encoded into the pipeline itself. Leaving it to convention, to "someone should probably look at this," is exactly how it gets skipped the first time a deadline gets tight.

Dependency validation as the highest-leverage control for AI-specific risk

If you can only fix one thing first, fix this one. A human developer who pulls in a package made a deliberate choice, however flawed. An AI model can hallucinate a plausible-sounding package name that happens to map to something malicious, or to nothing at all, and it will say it with exactly the same confidence either way.

Sonatype's research on this is the number that should reorganize your priorities: AI coding tools recommend nonexistent or incorrect packages at a rate that isn't negligible, and because AI-generated code tends to get accepted quickly, those recommendations land in the dependency graph before anyone checks them. That raises an important question: what does SCA actually need to do differently here, compared to how it's traditionally been run against human-written code?

A few things, concretely. Package existence has to be verified before installation, full stop; anything that can't be confirmed against a known-good registry gets rejected outright. Version pins need to be explicit and validated against a lockfile, since AI models frequently recommend dependencies with loose or missing pins. Every dependency should get cross-referenced against known malicious package lists; with hundreds of thousands of new malicious packages showing up in open-source registries in a single year, per Sonatype's data, this isn't a theoretical threat anymore. And it can't stop at direct dependencies. Flag every newly introduced transitive dependency for review, because that's exactly where attackers count on nobody looking.

The pipeline's own configuration deserves the same scrutiny as the application code running through it. The tj-actions incident proved attackers go after the actions and runners themselves, not just the code being built. If an AI model generates a workflow file, a Dockerfile, or a build script, that artifact needs the same review discipline as anything else it produces. One concrete practice worth adopting immediately: pin every third-party GitHub Action to a full commit SHA, never a mutable version tag, and audit that list on a set schedule rather than whenever someone happens to remember.

Secrets scanning belongs in this same conversation, even though it's technically a separate control. AI models sometimes emit secrets inline (API keys, tokens, credentials), particularly when they've been trained on code that contained them in the first place. A scanner running on every single commit catches that before it becomes part of the repository's permanent history, which matters because removing a secret from history after the fact is a much harder problem than catching it before it lands.

Why execution isolation is a pipeline control, not just a runtime concern

Most teams think about isolation as a production problem: containers, Kubernetes namespaces, the usual runtime boundaries. But AI-generated code runs inside the pipeline long before it gets anywhere near production, in test runners, build steps, linting scripts, evaluation harnesses. That's an execution surface people tend to overlook entirely.

If untrusted AI-generated code runs directly on a shared CI runner, it has access to whatever that runner has access to: secrets, environment variables, network interfaces. That's the exact attack surface the tj-actions compromise exploited, and it's sitting there every time a build step runs code nobody on the team actually wrote or reviewed line by line.

Standard Docker containers share the host kernel, and the general consensus among people who work on this seriously is that a permissive container isn't enough isolation for untrusted, LLM-generated code. A compromised container can attempt to escape to the host. That's not a hypothetical either; it's the reason microVM-based isolation (Firecracker and similar approaches) has become the standard for anything running code the system itself didn't author. Each execution environment gets its own kernel and hardware-enforced boundaries, so a compromised guest can't touch the host or any neighboring sandbox.

What does that look like in practice? Each AI-generated code evaluation or test run fires inside its own ephemeral sandbox, never on a shared runner. That sandbox has no access to the host filesystem, no visibility into other sandboxes, and no path to production credentials. Network egress gets restricted by default: the Shai-Hulud supply-chain campaigns in late 2025 compromised npm packages specifically targeting AI agent workflows. Without egress restrictions, a single malicious package can exfiltrate whatever secrets the runner had access to. The sandbox gets destroyed after use. Nothing persists to the next run, which means nothing an attacker plants can wait around for the next execution.

Startup latency isn't a minor detail here. It's the whole ballgame. If spinning up an isolated environment takes ten or fifteen seconds, teams under deadline pressure will find a way to skip it, quietly, the first time it's inconvenient. Some purpose-built infrastructure in this space offers sub-90ms cold starts, Docker-native compatibility, and full isolation between environments, built specifically for running code the team didn't write, at the speed a CI pipeline actually demands. When isolation is fast enough that skipping it isn't even a meaningful time save, it stops being a tradeoff anyone has to make.

One more thing worth flagging: teams need to know what their sandbox is actually doing under the hood. A black-box execution environment creates an audit gap, and compliance frameworks like SOC 2, HIPAA, and GDPR don't leave much room for "we're not entirely sure what happened in there."

Observability and auditability as structural requirements, not instrumentation afterthoughts

The alert-fatigue problem from earlier deserves a second look here, because the fix is more context attached to each finding, not more scanners. Volume was never the goal. Actionable signal was.

What does that mean specifically for AI-generated code? Attribution comes first: which lines were AI-generated, which model produced them, and which prompt triggered the generation. Without that, a security finding has nowhere to go, because there's no clear owner to route it to. Lineage comes next, and this is where the PBOM from earlier pays off directly, linking commit to build to deploy to runtime so that a vulnerability discovered in production can be traced back to the exact line and the exact generation event that introduced it. Execution logs from every sandboxed run matter more than people expect: what did the code actually do once it ran? What network calls did it attempt? What files did it touch that it had no obvious reason to touch?

Auditability isn't optional once you're in a regulated space. Sensitive data, build artifacts, test logs, deployment configurations, needs to be redacted or masked before it ever gets processed by an AI model inside the pipeline. That's the baseline requirement for teams in healthcare or finance.

There's an explainability piece too, and it's easy to underrate. AI agents operating inside a CI/CD pipeline need to produce output a human reviewer can actually interpret and verify. A pipeline that auto-generates code and auto-merges it without leaving a human-readable trail behind isn't a compliant system, whatever else it might be doing well.

All of this ties back to the gate model directly: when a gate fires, the finding needs enough context (what the code does, what it touches, who owns it) that the on-call engineer can act within minutes instead of spending an hour figuring out what they're looking at. Four concerns run through all of this: data security, auditability, human-in-the-loop control, and explainability. They form one coherent design requirement, and a pipeline running AI-generated code in any production or regulated context has to treat them that way.

Translating these controls into a practical baseline for teams building AI pipelines today

Here's the reframe that matters most, and it's worth stating plainly: this is the baseline architecture for any pipeline that runs AI-generated code at all, not hardening bolted onto a pipeline that already works fine. Treating it as optional polish is how teams end up retrofitting controls under pressure, usually right after an incident.

On every commit: SAST with auto-block on critical findings, SCA with dependency existence validation and cross-referencing against known malicious packages, secrets scanning to catch AI-emitted credentials before they enter repository history, and every third-party action pinned to a full commit SHA rather than a mutable tag.

On every build: container image scanning before pushing anything to a registry, and a PBOM generated and stored so every artifact carries a traceable lineage record from that point forward.

On every test and evaluation run: AI-generated code executes inside an isolated sandbox, never on a shared runner, with network egress restricted and filesystem access scoped tightly to the project directory. Capture and retain execution logs, because you'll want them later even if nothing looks wrong today.

On every production deploy: human sign-off required for high-risk changes, infrastructure edits, new dependencies, anything touching privilege escalation, and SLSA provenance attestation to verify the artifact wasn't tampered with somewhere between build and deploy.

Compliance doesn't bend around any of this. SOC 2, HIPAA, and GDPR apply to whatever runs in production, regardless of who or what wrote it. Teams that treat compliance as a concern for the production gate, rather than something baked into pipeline design from the start, will find themselves rebuilding under pressure later, usually at the worst possible time.

This also isn't a problem any one team solves alone. Security, platform engineering, and whoever's building the AI features all need to agree on where the gates actually live and who owns each finding when it fires. The pipeline itself becomes the record of that agreement, whether anyone wrote it down elsewhere or not.

For teams running AI agents or AI-generated code through CI today, the execution layer benefits most from a sandboxed environment purpose-built for untrusted code, fast enough that isolation never becomes the thing people skip under deadline pressure, and compatible with the Docker-based tooling most teams already have in place. The rest of the baseline, the gates, the dependency checks, the observability layer, still has to get built by the team itself. Even so, the isolation problem, at least, doesn't need to be solved from scratch by every team that runs into it.