Detecting OpenAPI Contract Drift in CI

You committed specs/api.yaml months ago and everyone treats it as gospel — but the deployed provider has quietly shipped three field renames since, and your mocks were generated from a spec revision that no longer matches either. Nothing failed, because nothing was comparing the documents. This page shows how to make continuous integration diff your OpenAPI specs and fail the build the instant a breaking change slips in.

Why this happens

An OpenAPI document is only as trustworthy as the discipline that keeps it current, and three forces pull it out of sync:

  • The spec evolves in a pull request. A developer edits api.yaml to remove a field or add a required parameter. Reviewers reading a 400-line YAML diff rarely notice that the change breaks every existing consumer.
  • The provider evolves without the spec. Backend code ships a response change; the hand-maintained spec is updated late or never. The repository now documents an API that no longer exists.
  • The mocks freeze in time. Fixtures generated from an older spec revision keep passing local tests, so the drift is invisible until an integration environment hits the real provider.

The first two are spec-to-spec comparisons a machine can make cheaply. Tools like oasdiff (and the older openapi-diff) parse two OpenAPI documents, compute a structural diff, and classify each change by severity. Wiring that into CI turns a silent divergence into a red build. This is the breaking-change edge of the broader contract testing and drift detection loop; pair it with fixture validation so both the spec’s evolution and your mocks are policed.

Solution

1. Try oasdiff locally

oasdiff is a single Go binary with no runtime dependencies. Run it against two revisions of your spec to see the classification:

# Install (macOS/Linux)
brew install oasdiff
# or: go install github.com/oasdiff/oasdiff@latest

# Human-readable summary of every change between two specs
oasdiff diff specs/api-v1.yaml specs/api-v2.yaml --format text

# Only the breaking changes, with severity
oasdiff breaking specs/api-v1.yaml specs/api-v2.yaml

A representative breaking report looks like this:

1 breaking changes: 1 error, 0 warning

error at specs/api-v2.yaml

    in API GET /api/v1/users/{id}
        removed the response property 'email' for status '200'

The command exits non-zero when it finds changes at or above the severity you gate on, which is what makes it CI-friendly.

2. Gate on a severity threshold

oasdiff breaking grades changes as ERR (breaks existing clients), WARN (risky but tolerable), or INFO (additive). Use --fail-on to decide what turns the build red:

# Fail only on definite breaking changes (recommended default)
oasdiff breaking specs/base.yaml specs/head.yaml --fail-on ERR

# Stricter: also fail on warnings (e.g. a newly-added optional response header)
oasdiff breaking specs/base.yaml specs/head.yaml --fail-on WARN

To keep known, deliberate exceptions from blocking forever, maintain a warn-ignore file rather than loosening the global threshold:

# .oasdiff-ignore  — one breaking-change id or message substring per line
# response property 'legacyId' removed — retired in the 2026-Q1 deprecation window

oasdiff breaking specs/base.yaml specs/head.yaml \
  --fail-on ERR \
  --warn-ignore .oasdiff-ignore

3. Compare the pull request spec against the base branch

The most valuable check for authors: does this change break the contract relative to what is already on main? Because the base spec lives in git history, no external service is needed.

#!/usr/bin/env bash
# scripts/check-spec-drift.sh
set -euo pipefail

BASE_REF="${1:-origin/main}"
SPEC_PATH="specs/api.yaml"

# Extract the base version of the spec from git into a temp file.
BASE_SPEC="$(mktemp --suffix=.yaml)"
git show "${BASE_REF}:${SPEC_PATH}" > "${BASE_SPEC}"

echo "Comparing ${SPEC_PATH} against ${BASE_REF}..."
oasdiff breaking "${BASE_SPEC}" "${SPEC_PATH}" --fail-on ERR
echo "No breaking changes relative to ${BASE_REF}."

4. Optionally diff the committed spec against the live provider

To catch the provider drifting away from the repo, fetch the document the deployed service actually serves (most frameworks expose it at /openapi.json) and diff the committed spec against it:

#!/usr/bin/env bash
# scripts/check-provider-drift.sh
set -euo pipefail

PROVIDER_URL="${PROVIDER_OPENAPI_URL:?set PROVIDER_OPENAPI_URL}"
LIVE_SPEC="$(mktemp --suffix=.json)"

curl -fsSL "${PROVIDER_URL}" -o "${LIVE_SPEC}"

echo "Comparing committed spec against live provider at ${PROVIDER_URL}..."
oasdiff breaking specs/api.yaml "${LIVE_SPEC}" --fail-on ERR
echo "Committed spec matches the deployed provider."

5. Wire it into GitHub Actions

# .github/workflows/spec-drift.yml
name: Spec drift

on:
  pull_request:
    branches: [main]

jobs:
  spec-drift:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0            # needed to read the base-branch spec

      - name: Install oasdiff
        run: |
          curl -fsSL https://raw.githubusercontent.com/oasdiff/oasdiff/main/install.sh | sh
          oasdiff --version

      - name: Breaking changes vs. base branch
        run: bash scripts/check-spec-drift.sh "origin/${{ github.base_ref }}"

      - name: Drift vs. deployed provider
        if: ${{ env.PROVIDER_OPENAPI_URL != '' }}
        env:
          PROVIDER_OPENAPI_URL: ${{ vars.STAGING_OPENAPI_URL }}
        run: bash scripts/check-provider-drift.sh

      - name: Post summary
        if: always()
        run: echo "Spec-drift check finished — review step logs for details."

Mark the spec-drift job as a required status check in branch protection. Without that, the workflow only reports drift; making it required is what turns a red result into a blocked merge. This runs alongside the mock stack described in running mock servers in CI pipelines, so a single pull-request pipeline exercises both the spec and the mocks it drives.

Verification

Run the drift script against a deliberately broken copy of the spec and confirm it exits non-zero:

cp specs/api.yaml /tmp/head.yaml
# Remove a response property to simulate a breaking edit
python3 -c "import yaml;d=yaml.safe_load(open('/tmp/head.yaml'));d['paths']['/api/v1/users/{id}']['get']['responses']['200']['content']['application/json']['schema']['properties'].pop('email');yaml.safe_dump(d,open('/tmp/head.yaml','w'))"

oasdiff breaking specs/api.yaml /tmp/head.yaml --fail-on ERR
echo "exit code: $?"
# Expected: a reported breaking change and "exit code: 1"

Gotchas and edge cases

  • $ref resolution across files. If your spec is split across multiple files with external $ref pointers, bundle it into a single document first (oasdiff accepts a root file and follows local refs, but some external-file layouts need redocly bundle or swagger-cli bundle beforehand). Diffing an unbundled fragment produces misleading “removed” results because the referenced schema is simply not visible.

  • OpenAPI 3.0 vs 3.1 false positives. Comparing a 3.0 spec against a 3.1 spec surfaces spurious breaking changes from the nullable keyword being replaced by type: [..., "null"]. Normalise both documents to the same OpenAPI version before diffing, or upgrade the base spec so both sides speak 3.1. This is the same nullable mismatch that trips up validating mock responses against OpenAPI with ajv.

  • A rename is a removal plus an addition. oasdiff cannot know that created_at becoming createdAt is “the same field renamed” — it sees a removed property and a new one, and correctly flags the removal as breaking. If the rename is intentional, run a deprecation window: keep both fields, migrate consumers, then remove the old one. Consumer-side proof that nothing still depends on the old shape comes from consumer-driven contract testing with Pact.

FAQ

What counts as a breaking change in an OpenAPI diff?

Anything that could break an existing client: removing an endpoint, removing or retyping a response field, adding a required request parameter, narrowing an enum, or tightening a format. oasdiff classifies these as ERR. Additive changes like a new optional field or a new endpoint are non-breaking and reported at WARN or INFO, which is why the recommended gate is --fail-on ERR.

Should I diff against the deployed provider or the base branch spec?

Both, for different reasons. Diffing the pull request spec against the base branch catches an author making a breaking edit. Diffing the committed spec against the document the live provider actually serves catches the provider drifting away from what the repo claims. The first guards the spec’s evolution; the second guards its truthfulness.

← Back to Contract Testing & Drift Detection