Contract Testing & Drift Detection

This page covers keeping three artefacts in agreement — your OpenAPI specification, the mocks your local and CI tests run against, and the real provider — and catching the moment any one of them drifts from the others before it breaks integration.

Prerequisites

  • An OpenAPI 3.x or JSON Schema document that describes your API responses (the canonical contract)
  • Node.js 20+ and npm available locally and in CI
  • ajv and ajv-formats installed: npm i -D ajv ajv-formats
  • A running mock server whose responses you want to police — an MSW handler setup for browser and Node tests, or a WireMock standalone instance for language-agnostic stubs
  • Fixtures produced by schema-driven data generation rather than hand-authored blobs, so validation has a single source of truth
  • @pact-foundation/pact installed for the consumer/provider phase: npm i -D @pact-foundation/pact

Why drift happens

The OpenAPI spec, the mocks, and the provider are edited by different people, in different repositories, on different schedules. A backend engineer renames created_at to createdAt; the provider ships it; the spec is updated a sprint later; the mocks are never touched at all. Nothing compares the three until an integration test — or a customer — hits the gap. That silent divergence is contract drift, and it is invisible precisely because each artefact looks internally consistent.

Contract testing closes the loop by making the three artefacts continuously check each other. The diagram below shows the detection loop this page builds: the spec generates and validates mocks, the consumer’s expectations are recorded and replayed against the provider, and a CI gate fails the build the moment any edge of the triangle stops agreeing.

Contract Drift Detection Loop An OpenAPI spec generates mocks and is validated against them with ajv. A consumer records expectations into a Pact file that is replayed against the real provider. A CI gate compares all three artefacts and fails the build when any pair disagrees. OpenAPI Spec canonical contract Mocks / Fixtures MSW · WireMock Real Provider deployed service generate + ajv validate oasdiff breaking-change Pact: record → replay CI gate fails the build on any mismatch

The rest of this page builds that loop in three phases: validate mocks against the spec with ajv, verify the provider against consumer expectations with Pact, then wire both into a CI gate that blocks the merge.

Phase 1 — Export OpenAPI to JSON Schema and validate fixtures with ajv

The first edge to lock down is the cheapest: prove that every mock response obeys the shapes your spec documents. OpenAPI response schemas are almost JSON Schema — they use the same keyword vocabulary but wrap each schema under paths → method → responses → status → content → media-type → schema, and they may use $ref pointers into components/schemas. The helper below loads the spec, resolves those references, and compiles a validator per operation.

// contract/extract-schemas.ts
import { readFileSync } from 'node:fs';
import { parse } from 'yaml';

export interface ResponseSchema {
  operationId: string;
  method: string;
  path: string;
  status: string;
  schema: unknown;
}

/** Load an OpenAPI 3.x document (YAML or JSON) into a plain object. */
export function loadSpec(specPath: string): Record<string, any> {
  const raw = readFileSync(specPath, 'utf8');
  return specPath.endsWith('.json') ? JSON.parse(raw) : parse(raw);
}

/**
 * Walk every path/method/response and pull out the application/json schema.
 * Returns one entry per (operation, status) pair so fixtures can be matched
 * to the exact response they claim to represent.
 */
export function extractResponseSchemas(spec: Record<string, any>): ResponseSchema[] {
  const out: ResponseSchema[] = [];
  const paths = spec.paths ?? {};

  for (const [path, methods] of Object.entries<Record<string, any>>(paths)) {
    for (const [method, op] of Object.entries<any>(methods)) {
      if (!['get', 'post', 'put', 'patch', 'delete'].includes(method)) continue;
      const responses = op.responses ?? {};
      for (const [status, response] of Object.entries<any>(responses)) {
        const schema = response?.content?.['application/json']?.schema;
        if (!schema) continue;
        out.push({
          operationId: op.operationId ?? `${method.toUpperCase()} ${path}`,
          method,
          path,
          status,
          schema,
        });
      }
    }
  }
  return out;
}

Now compile each schema with ajv and validate the matching fixture. The key detail is registering components/schemas so that $ref: "#/components/schemas/User" resolves without manual dereferencing.

// contract/validate-fixtures.ts
import Ajv, { type ValidateFunction } from 'ajv';
import addFormats from 'ajv-formats';
import { readFileSync, readdirSync } from 'node:fs';
import { join } from 'node:path';
import { loadSpec, extractResponseSchemas } from './extract-schemas';

const SPEC_PATH = process.env.OPENAPI_SPEC ?? './specs/api.yaml';
const FIXTURE_DIR = process.env.FIXTURE_DIR ?? './mocks/generated';

function buildValidators(): Map<string, ValidateFunction> {
  const spec = loadSpec(SPEC_PATH);

  const ajv = new Ajv({
    strict: false,        // OpenAPI adds keywords ajv does not know (nullable, example)
    allErrors: true,      // report every violation, not just the first
  });
  addFormats(ajv);

  // Register component schemas so internal $ref pointers resolve.
  for (const [name, schema] of Object.entries<any>(spec.components?.schemas ?? {})) {
    ajv.addSchema(schema, `#/components/schemas/${name}`);
  }

  const validators = new Map<string, ValidateFunction>();
  for (const entry of extractResponseSchemas(spec)) {
    const key = `${entry.method.toUpperCase()} ${entry.path} ${entry.status}`;
    validators.set(key, ajv.compile(entry.schema as object));
  }
  return validators;
}

/**
 * Fixtures are named "<method>__<path-with-slashes-as-dashes>__<status>.json",
 * e.g. "get__api-v1-users__200.json". Decode the name back into a lookup key.
 */
function keyFromFixtureName(fileName: string): string {
  const base = fileName.replace(/\.json$/, '');
  const [method, path, status] = base.split('__');
  return `${method.toUpperCase()} /${path.replace(/-/g, '/')} ${status}`;
}

function main(): void {
  const validators = buildValidators();
  const files = readdirSync(FIXTURE_DIR).filter((f) => f.endsWith('.json'));

  let failures = 0;
  for (const file of files) {
    const key = keyFromFixtureName(file);
    const validate = validators.get(key);
    if (!validate) {
      console.warn(`⚠ no schema for fixture ${file} (looked up "${key}")`);
      continue;
    }
    const data = JSON.parse(readFileSync(join(FIXTURE_DIR, file), 'utf8'));
    if (!validate(data)) {
      failures++;
      console.error(`✗ ${file} violates ${key}`);
      for (const err of validate.errors ?? []) {
        console.error(`    ${err.instancePath || '/'} ${err.message}`);
      }
    } else {
      console.log(`✓ ${file}`);
    }
  }

  if (failures > 0) {
    console.error(`\n${failures} fixture(s) drifted from the spec.`);
    process.exit(1);
  }
  console.log('\nAll fixtures match the OpenAPI contract.');
}

main();

Expose it as a script so local runs and CI share one command:

{
  "scripts": {
    "contract:fixtures": "tsx contract/validate-fixtures.ts"
  }
}

This phase policing is inexpensive and runs in milliseconds, so wire it into a pre-commit hook as well. It guarantees mock hygiene — but note what it cannot prove: that the deployed provider still returns the shapes the spec documents. That guarantee needs Phase 2. For a deeper focus on this fixture-versus-spec check in isolation, see validating mock responses against OpenAPI.

Phase 2 — Consumer/provider contract verification with Pact

Schema validation asks “do my mocks obey the documented shapes?” Pact asks a sharper question: “does the provider still return exactly what my consumer depends on?” The workflow is consumer-driven — the consumer writes a test describing the interactions it needs, Pact records those interactions into a pact file, and the provider replays that file against a real (or freshly booted) instance.

The consumer test below stands up a Pact mock provider, declares one interaction, exercises the real client code against it, and writes a pact file on success.

// contract/consumer/users.consumer.test.ts
import { PactV3, MatchersV3 } from '@pact-foundation/pact';
import { resolve } from 'node:path';
import { describe, it, expect } from 'vitest';
import { fetchUser } from '../../src/clients/users';

const { like, string, integer } = MatchersV3;

const provider = new PactV3({
  consumer: 'web-frontend',
  provider: 'users-api',
  dir: resolve(process.cwd(), 'pacts'),
});

describe('users API contract', () => {
  it('returns a user by id', async () => {
    provider
      .given('a user with id 42 exists')
      .uponReceiving('a request for user 42')
      .withRequest({
        method: 'GET',
        path: '/api/v1/users/42',
        headers: { Accept: 'application/json' },
      })
      .willRespondWith({
        status: 200,
        headers: { 'Content-Type': 'application/json' },
        // Matchers assert shape/type, not exact values, so the provider is
        // free to return real data as long as the contract holds.
        body: like({
          id: integer(42),
          username: string('ada.lovelace'),
          email: string('[email protected]'),
          role: string('editor'),
        }),
      });

    await provider.executeTest(async (mockServer) => {
      const user = await fetchUser(mockServer.url, 42);
      expect(user.id).toBe(42);
      expect(user.role).toBeDefined();
    });
  });
});

The client under test is ordinary application code — no Pact awareness:

// src/clients/users.ts
export interface User {
  id: number;
  username: string;
  email: string;
  role: string;
}

export async function fetchUser(baseUrl: string, id: number): Promise<User> {
  const res = await fetch(`${baseUrl}/api/v1/users/${id}`, {
    headers: { Accept: 'application/json' },
  });
  if (!res.ok) throw new Error(`user ${id}: HTTP ${res.status}`);
  return (await res.json()) as User;
}

On the provider side, verification boots the real service and replays every interaction in the pact file against it:

// contract/provider/users.provider.test.ts
import { Verifier } from '@pact-foundation/pact';
import { resolve } from 'node:path';
import { describe, it } from 'vitest';
import { startProvider, stopProvider, seedUser } from './provider-harness';

describe('users-api provider verification', () => {
  it('honours the web-frontend contract', async () => {
    const port = 8992;
    await startProvider(port);

    const opts = {
      provider: 'users-api',
      providerBaseUrl: `http://127.0.0.1:${port}`,
      pactUrls: [resolve(process.cwd(), 'pacts', 'web-frontend-users-api.json')],
      // Map each provider state named in the consumer test to real setup.
      stateHandlers: {
        'a user with id 42 exists': async () => {
          await seedUser({ id: 42, username: 'ada.lovelace', role: 'editor' });
          return { description: 'user 42 seeded' };
        },
      },
    };

    try {
      await new Verifier(opts).verifyProvider();
    } finally {
      await stopProvider();
    }
  }, 30_000);
});

The pacts/ directory now holds a machine-checkable record of what the consumer needs. To share it across teams, publish to a Pact Broker — a small service that stores pact files and verification results and can answer “is it safe to deploy?”:

# docker-compose.pact.yml — a local broker for experimenting
services:
  pact-broker:
    image: pactfoundation/pact-broker:latest
    ports:
      - "9292:9292"
    environment:
      PACT_BROKER_DATABASE_URL: "postgres://pact:pact@pact-db/pact"
      PACT_BROKER_BASIC_AUTH_USERNAME: pact
      PACT_BROKER_BASIC_AUTH_PASSWORD: pact
    depends_on:
      - pact-db

  pact-db:
    image: postgres:16-alpine
    environment:
      POSTGRES_USER: pact
      POSTGRES_PASSWORD: pact
      POSTGRES_DB: pact

Publish the pact from the consumer build and let the provider pull it back:

# Consumer: publish after the consumer test passes
npx pact-broker publish ./pacts \
  --consumer-app-version "$GIT_SHA" \
  --broker-base-url http://localhost:9292 \
  --broker-username pact --broker-password pact

The full consumer-plus-provider walkthrough, including versioning tags and can-i-deploy checks, lives in consumer-driven contract testing with Pact.

Phase 3 — A CI gate that fails the build on drift

Validation and verification only prevent drift if they run on every change and block the merge. The workflow below runs three required checks in one pull-request pipeline: fixture validation (Phase 1), a breaking-change diff of the spec against the last released version, and Pact provider verification (Phase 2).

# .github/workflows/contract.yml
name: Contract checks

on:
  pull_request:
    branches: [main]

jobs:
  contract:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0            # oasdiff needs history to reach the base spec

      - uses: actions/setup-node@v4
        with:
          node-version: "20"
          cache: "npm"

      - run: npm ci

      # 1. Mocks obey the documented shapes.
      - name: Validate fixtures against OpenAPI
        run: npm run contract:fixtures

      # 2. The spec did not introduce a breaking change vs. the base branch.
      - name: Detect breaking spec changes
        run: |
          git show origin/main:specs/api.yaml > /tmp/base-api.yaml
          npx --yes oasdiff breaking /tmp/base-api.yaml specs/api.yaml \
            --fail-on ERR

      # 3. The provider still honours consumer expectations.
      - name: Verify provider against pacts
        run: npx vitest run contract/provider

      - name: Summarise
        if: always()
        run: echo "Contract gate complete — see step logs for any drift."

Mark the contract job as a required status check in branch protection so a red result blocks the merge button, not just the notification. This gate is the merge-blocking companion to the broader guidance on running mock servers in CI pipelines, where the same mock stack that serves your integration tests is booted, health-checked, and torn down. The dedicated breaking-change step is expanded in detecting OpenAPI contract drift in CI.

Verification steps

Run these locally after wiring the three phases to confirm the loop is closed.

  • Fixtures validate cleanly against the spec:
    npm run contract:fixtures
# Expected final line: All fixtures match the OpenAPI contract.
  • Deliberately break a fixture and confirm the check fails:
    node -e "const f='./mocks/generated/get__api-v1-users__200.json';const d=require('fs');let j=JSON.parse(d.readFileSync(f));j.role=42;d.writeFileSync(f,JSON.stringify(j))"
npm run contract:fixtures
# Expected: non-zero exit and "role must be string"
  • The consumer test emits a pact file:
    npx vitest run contract/consumer && ls pacts/
# Expected: web-frontend-users-api.json present
  • Provider verification passes against the pact:
    npx vitest run contract/provider
# Expected: "1 interaction, 0 failures"
  • The breaking-change diff exits non-zero on an incompatible edit:
    npx oasdiff breaking specs/api.yaml specs/api-with-removed-field.yaml --fail-on ERR
# Expected: exit code 1 and a list of ERR-level changes

Troubleshooting

strict mode: unknown keyword: "nullable" when compiling a schema

OpenAPI 3.0 uses nullable: true, which is not a JSON Schema keyword ajv recognises in strict mode. Construct the validator with new Ajv({ strict: false }) as the Phase 1 code does, or pre-process schemas to convert nullable: true into type: ["string", "null"]. OpenAPI 3.1 aligns with JSON Schema and drops nullable entirely, so upgrading the spec version removes the mismatch.

can't resolve reference #/components/schemas/User

ajv compiled a response schema before the referenced component was registered. Add every entry under components.schemas with ajv.addSchema(schema, "#/components/schemas/<Name>") before calling ajv.compile, exactly as buildValidators() does. The $id/key you register under must match the $ref string character-for-character.

Pact provider verification fails with state handler not found

The consumer test declared .given('a user with id 42 exists') but the provider’s stateHandlers map has no key with that exact string. Provider states are matched by literal text. Copy the state description verbatim from the consumer test into the stateHandlers object; a trailing space or different casing will not match.

oasdiff reports breaking changes on a rename you consider safe

Renaming a response field is a breaking change for any consumer that reads the old name — oasdiff is correct to flag it. If the old field is being retired deliberately, keep both fields for a deprecation window (mark the old one deprecated: true), ship consumers onto the new name, then remove the old field in a later release once no pact depends on it.

Fixtures validate locally but the CI job cannot find the spec

The OPENAPI_SPEC or FIXTURE_DIR env var resolves to a path that exists on your machine but not on the runner. Use repository-relative paths (./specs/api.yaml) rather than absolute ones, commit the generated fixtures or regenerate them in a prior CI step, and echo the resolved paths at the top of the job so a missing file is obvious in the log.

When to advance

The contract loop is correctly in place when:

  • npm run contract:fixtures runs green locally and in CI, and a hand-broken fixture reliably turns it red
  • A consumer test produces a pact file and the provider verification replays it without a state-handler mismatch
  • The oasdiff step blocks a pull request that removes or retypes a response field
  • All three checks are required status checks, so a red result disables the merge button rather than merely warning
  • The same mock stack your integration tests use is the one being validated, so there is no separate “test” contract that can drift on its own

Once these signals hold, extend the loop outward: publish pacts to a shared broker so multiple consumers negotiate against one provider, and connect fixture generation back to schema-driven data generation so every mock the gate inspects is derived from the spec rather than authored by hand.

FAQ

What is contract drift, and how is it different from a failing test?

Contract drift is a silent divergence between three artefacts that are supposed to describe the same API: the OpenAPI spec, the mocks your tests run against, and the deployed provider. A failing test tells you something broke now; drift accumulates unnoticed because each artefact is edited independently and nothing compares them until integration. Contract testing turns drift into a loud, early failure by making the artefacts continuously validate each other.

Do I need Pact if I already validate fixtures against OpenAPI?

They catch different classes of drift. Validating fixtures against OpenAPI proves your mocks obey the documented shapes, but it cannot prove the real provider still obeys them. Pact records the exact interactions a consumer depends on and replays them against the live provider, catching cases where the provider changed but the spec did not. Use schema validation for mock hygiene and Pact for provider guarantees — they are complementary, not alternatives.

Where should the contract checks run so they actually block a bad merge?

In the same pull-request pipeline that runs your unit and integration tests, before the merge is allowed. Schema validation and provider verification must be required status checks; if they only run on a nightly job, drift merges during the day and you lose the early-warning value. Fail the build on any mismatch and surface the diff in the job log so the author sees exactly which artefact diverged.

← Back to API Mocking Fundamentals & Architecture