Reliable CI depends on reliable test data. This guide gives you a repeatable checklist for choosing between fixtures, factories, database seeds, snapshots, and ephemeral environments, with setup and teardown templates you can adapt to browser, API, and integration tests.
Overview
Test data management in CI is the practice of creating, controlling, and removing the records that automated tests need. The goal is not simply to populate a database. It is to make every test run predictable, isolated, and easy to diagnose when it fails.
A local test may pass because a developer's database already contains users, products, permissions, or feature flags. In a CI pipeline, that assumption becomes a source of failure. A clean runner may have no data at all, while parallel jobs may accidentally share the same database and modify one another's records.
A dependable workflow normally follows four steps:
- Provision: create the database, service dependencies, or test environment.
- Seed: add the smallest data set required for the test or test group.
- Execute: run tests with unique identifiers and controlled configuration.
- Reset: roll back, truncate, delete, or discard the environment after the run.
Choose the narrowest data strategy that meets the test's needs. A unit test usually needs an in-memory factory, while a full end-to-end test may need a disposable database and a running application. Keeping those layers separate reduces runtime and makes failures easier to understand.
Checklist by scenario
Use this table before adding a new data setup method to your CI pipeline.
| Scenario | Recommended approach | Isolation requirement | Typical warning |
|---|---|---|---|
| Unit test for a function or class | Inline values, builders, or factories | In-memory; no shared database | Do not load a full production-like seed for a small test |
| API or integration test | Factories plus targeted database fixtures | Transaction rollback or per-test records | Check that background jobs cannot consume another test's records |
| End-to-end browser test | Stable seed data and API-based setup | Unique user, tenant, or order identifiers | Avoid relying on records created by a previous test |
| Regression suite needing a known baseline | Versioned database seed or snapshot | Restore the same baseline for every run | Review snapshots when the schema or permissions model changes |
| Parallel CI jobs | Per-worker schema, database, or environment | Isolation must exist below the test process | Random test IDs alone do not isolate shared configuration |
| Complex system with several services | Ephemeral environment with service-specific setup | Disposable network, credentials, and data stores | Include cleanup for queues, object storage, and caches |
Fixtures for stable, readable scenarios
Fixtures are predefined records with known values. They work well when the same scenario appears across many tests: an active account, a suspended account, or an administrator with a specific permission. Keep fixtures small and name them by behavior rather than by database ID.
Before using a fixture, confirm that it is safe to reuse. A shared fixture becomes risky when tests update it, send it through a workflow, or depend on its current state. In those cases, copy the fixture into a test-owned record or use a factory.
Factories for test-owned data
A factory creates a valid record with sensible defaults and allows individual tests to override only the fields that matter. This is often the most flexible choice for API and integration testing.
const user = await userFactory.create({
role: 'admin',
status: 'active'
});
const order = await orderFactory.create({
userId: user.id,
state: 'pending'
});
Factories should generate unique values for fields such as email addresses, external IDs, slugs, and usernames. Avoid hiding important business conditions in defaults. A test should make its unusual state visible in the test body.
Seeds and snapshots for repeatable baselines
Database seeds are useful when many tests need the same reference data, such as plans, countries, permissions, or tax categories. Keep operational seed data separate from test-only data where possible. The test seed should be deterministic and safe to run repeatedly.
Snapshots can restore a larger known state quickly, but they introduce maintenance work. Treat the snapshot as versioned test infrastructure: document how it was created, record the schema version it matches, and replace it when migrations or authorization rules make it inaccurate.
Ephemeral environments for high isolation
An ephemeral environment creates disposable infrastructure for a test run, such as a temporary database, application instance, queue, and object store. This approach is valuable when tests modify global state or when several services must be validated together.
Use environment-specific names and credentials. A temporary environment is not isolated if it still points to a shared cache, queue, bucket, or third-party sandbox. If a fully ephemeral stack is too expensive or slow, isolate the most stateful dependencies first and use targeted cleanup for the rest.
Copyable setup and teardown pattern
The exact commands depend on your framework and database, but the lifecycle should be explicit. This shell-style pattern can be adapted to a GitHub Actions, GitLab CI, Jenkins, or another CI job:
set -euo pipefail
export TEST_RUN_ID="${CI_PIPELINE_ID:-local}-${CI_JOB_ID:-$(date +%s)}"
export DATABASE_NAME="app_test_${TEST_RUN_ID}"
./scripts/db-create "$DATABASE_NAME"
./scripts/db-migrate "$DATABASE_NAME"
./scripts/db-seed --profile test "$DATABASE_NAME"
cleanup() {
./scripts/db-drop "$DATABASE_NAME" || true
}
trap cleanup EXIT
TEST_RUN_ID="$TEST_RUN_ID" npm run test:e2e
The cleanup command is deliberately registered before tests start. It should run on success, failure, cancellation where the CI platform permits it, and partial setup. For production-like services, add equivalent cleanup for queues, files, containers, and temporary credentials.
What to double-check
- Unique identity: Build a run or worker identifier into emails, tenant names, database names, and external references. Do not use a timestamp alone if concurrent jobs can begin within the same interval.
- Database reset behavior: Verify whether your framework rolls back transactions, truncates tables, recreates schemas, or does nothing between tests. Confirm that the chosen method also handles data created by application workers.
- Parallel execution: If tests are sharded, check whether each shard receives its own database or schema. A test-level naming convention cannot prevent collisions in shared reference tables or queues.
- Seed ordering: Apply migrations before seeds, and make dependencies explicit. A permission seed that runs before its roles exist may fail intermittently or produce incomplete data.
- Application caches: Clear or namespace caches after changing seeded records. A test may appear to use fresh data while the application is returning a previous response.
- Time-dependent data: Freeze the clock or create dates relative to a controlled test time. Expiration tests should not depend on the wall clock of the CI runner.
- Secrets and external services: Use test credentials and explicit mocks or sandboxes. Never let a seed script contact a production endpoint because an environment variable was missing.
- Failure artifacts: Preserve logs, database error output, screenshots, traces, and the test run identifier. This connects a failed browser test to the records it created.
For browser suites, perform setup through a direct API or database helper rather than through the user interface when the setup itself is not under test. This shortens the run and reduces unrelated failure points. Keep at least a small number of tests that validate the setup path itself.
Common mistakes
Using one shared account for every test
A shared account creates hidden ordering. One test may change its password, consume a quota, accept an invitation, or alter permissions for the next test. Create a fresh account for state-changing tests, or restore the account completely between cases.
Deleting only the main record
Deleting an order does not necessarily remove its events, files, audit entries, search index entry, or queued jobs. Define cleanup according to the data graph and verify it with a periodic integrity check.
Making seeds silently destructive
A reset-and-seed command may be appropriate inside a disposable CI database but unsafe in a shared development environment. Require an explicit test-mode flag and fail fast when the connection does not look like a test database.
Assuming retries fix data problems
Retries can hide leakage and make a suite appear stable while leaving the underlying race condition intact. When a retry passes, compare the failed and successful run IDs, inspect setup logs, and determine whether the test began with a clean state.
Coupling tests to implementation IDs
Hard-coded primary keys make tests fragile after a migration or a different seed order. Prefer labels, generated references, or factory return values. Reserve fixed IDs for deliberate reference data and document why they are fixed.
When to revisit
Review test data management before seasonal planning cycles and whenever the workflow or its tools change. A short review is worthwhile when:
- the database schema, migration process, or data-retention behavior changes;
- the test suite begins running in parallel or adds new CI runners;
- an application introduces queues, workers, caches, file storage, or additional services;
- the team changes its browser testing framework, test runner, container image, or CI platform;
- tests begin failing with duplicate records, stale data, timeouts, or order-dependent results;
- seed files or snapshots have not been reviewed since a major permissions or workflow change.
Before making a change, run the suite from an empty environment, then run it again with parallel workers. Confirm that both runs produce the same result and that cleanup leaves no unexpected records. Finally, update the setup scripts, environment variable documentation, and troubleshooting notes together.
Practical final checklist: identify the data owner for each test, give each run a unique namespace, seed only what the scenario needs, isolate stateful dependencies, register cleanup before execution, preserve failure artifacts, and test the reset process itself. That routine turns test data from an invisible CI dependency into a controlled part of the automated QA workflow.