DevOps, CI/CD & Containers

Continuous Integration: Practices, Build Pipeline and a Working Setup

Beginner8 min readReviewed by on Updated

Continuous integration means every developer merges into the shared main line at least daily and an automated build proves each merge compiles and passes its tests. This guide covers the practices behind CI, the anatomy of a build pipeline, a complete CI workflow for a Java project, how to keep builds fast and trustworthy, and the habits that separate a team that has CI from a team that merely has a CI server.

What you will learn

  • State the practices that define continuous integration and why each exists
  • Design the stages of a CI build: compile, unit tests, static checks, packaging
  • Write a CI workflow that runs the build on every push and pull request
  • Keep the build fast and reliable with caching, parallelism and flaky-test hygiene
  • Choose a merge policy — trunk-based or short-lived branches — that supports daily integration

Integration used to be a phase. Teams worked for weeks on separate branches, then spent days — sometimes weeks — merging, discovering that two people had changed the same interface in incompatible ways, and hunting bugs that only appeared when everything was combined. Continuous integration (CI) dissolves the phase by making integration so frequent that it is never painful: everyone merges small changes into a shared main line every day, and an automated build proves after every merge that the software still compiles and its tests still pass. The practice is simple to describe and surprisingly demanding to live up to. This guide covers what it requires, how a build pipeline is put together, and how to keep the whole thing fast enough that people trust it.

What continuous integration requires

CI is a practice, and a CI server is only one of its ingredients. The full set:

A single source repository that contains everything needed to build the system: code, tests, build scripts, database migrations, configuration templates. Someone with a fresh machine and a checkout should be able to build without asking a colleague what else to install.

An automated build that runs with one command. mvn verify, ./gradlew build, npm ci && npm test — the exact tool matters less than the property that no manual step is involved. The build must be able to run on any machine, not only the one where it was first configured.

A self-testing build. The build runs the automated tests and fails if any test fails. A build that only compiles proves very little; the tests are what make a green result meaningful.

Everyone commits to the main line every day. This is the rule teams most often skip. A branch that lives for two weeks is two weeks of unintegrated work, regardless of how green its own build is. Small, daily merges keep the divergence between developers small enough to resolve in minutes.

Every commit triggers a build on an integration machine. Not "works on my laptop": a clean environment, a fresh checkout, the same steps every time. That is the CI server's job.

Broken builds are fixed immediately. The main branch is the team's shared asset; when it is red, nobody can safely merge, so fixing it (or reverting) is the top priority.

The build is fast. Feedback that arrives an hour later is not feedback; it is an interruption. Ten minutes is the widely quoted ceiling for the primary build.

Everyone can see the result. A dashboard, a Slack message, a status badge on the pull request. Hiding the build status in a server nobody visits is how red builds go unnoticed.

Anatomy of a CI build

A build for a typical service moves through stages of increasing cost and decreasing frequency. Each stage acts as a gate; if it fails, the later ones do not run, which keeps feedback fast for the common failure modes.

StageWhat it doesTypical timeFails on
CheckoutFresh clone of the commit under testsecondsRepository unreachable
CompileCompile main and test sourcesseconds to a minuteSyntax and type errors
Unit testsFast, isolated tests with no I/Oone to five minutesLogic regressions
Static analysisLinters, style checks, dependency vulnerability scanone to three minutesCode smells, known CVEs
PackageBuild the jar or container imagea minutePackaging misconfiguration
Integration testsTests against a real database or broker (often in a container)five to fifteen minutesWiring, SQL, serialisation bugs
PublishPush the artifact to a repository, tagged with the versionsecondsRepository permissions

Maven encodes most of this in its lifecycle: compile, test (unit tests via Surefire), package, verify (integration tests via Failsafe, plus checks like coverage thresholds), install and deploy. Running mvn verify from CI therefore executes compile, unit tests, packaging and integration tests in the right order and stops at the first failure.

<!-- pom.xml: run *IT classes as integration tests during "verify" -->
<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-failsafe-plugin</artifactId>
  <version>3.5.2</version>
  <executions>
    <execution>
      <goals>
        <goal>integration-test</goal>
        <goal>verify</goal>
      </goals>
    </execution>
  </executions>
</plugin>

Separating unit tests (Surefire, *Test) from integration tests (Failsafe, *IT) matters for speed: developers can run mvn test locally in seconds, while the CI server runs the full verify.

A complete CI workflow

Here is a GitHub Actions workflow for a Java 21 Maven project. The same structure translates directly into a Jenkins declarative pipeline (covered in the Jenkinsfile guide) or GitLab CI.

# .github/workflows/ci.yml
name: CI

on:
  push:
    branches: [main]
  pull_request:

# Cancel an older run of the same branch when a new push arrives
concurrency:
  group: ci-${{ github.ref }}
  cancel-in-progress: true

jobs:
  build:
    runs-on: ubuntu-latest
    timeout-minutes: 20
    services:
      postgres:                     # real database for the integration tests
        image: postgres:16
        env:
          POSTGRES_PASSWORD: ci
        ports: ["5432:5432"]
        options: >-
          --health-cmd "pg_isready -U postgres"
          --health-interval 5s --health-timeout 5s --health-retries 10
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-java@v4
        with:
          distribution: temurin
          java-version: "21"
          cache: maven              # caches ~/.m2 keyed on pom.xml hashes

      - name: Build and test
        run: mvn -B --no-transfer-progress verify
        env:
          DB_URL: jdbc:postgresql://localhost:5432/postgres
          DB_PASSWORD: ci

      - name: Publish test report
        if: always()                # upload even when tests failed
        uses: actions/upload-artifact@v4
        with:
          name: surefire-reports
          path: |
            target/surefire-reports
            target/failsafe-reports

Points worth noticing. -B (batch mode) stops Maven printing progress bars and colour codes that clutter logs. concurrency with cancel-in-progress avoids queueing five builds for five rapid pushes to the same pull request. timeout-minutes stops a hung test from occupying a runner for six hours. The if: always() on the report upload means a failing build still gives you the reports you need to diagnose it. Caching the Maven repository keyed on pom.xml contents turns a five-minute dependency download into a few seconds on unchanged dependencies.

Keeping the build fast

A slow build is the most common reason CI decays. Strategies, roughly in order of payoff:

Cache dependencies. Maven's ~/.m2, Gradle's cache, npm's node_modules restored from a cache keyed on the lock file. This is the biggest single win on hosted runners that start from a clean machine.

Split fast and slow. Unit tests in the primary job; integration and end-to-end tests in a second job that runs after it, or in a nightly pipeline if they take an hour. Developers get the fast verdict first.

Run tests in parallel. Surefire and JUnit 5 can run test classes in parallel on multiple cores (<parallel> / junit.jupiter.execution.parallel.enabled), and a large suite can be sharded across several runners.

Build only what changed. In a monorepo, tools like Gradle's build cache, Nx or Bazel skip modules whose inputs are unchanged.

Do not test through the UI what a unit test can cover. A test pyramid with many fast unit tests, fewer integration tests and a handful of end-to-end tests is faster and less flaky than the inverse. The test automation strategy guide covers how to shape that.

Watch the trend. Record build duration and plot it. Builds get slower one test at a time; nobody notices until the ten-minute build takes forty.

Flaky tests and trust

A flaky test passes and fails on the same code. Its cost is far higher than the minutes it wastes: once developers learn that a red build might be "just the flaky one", they stop investigating red builds at all, and the real failures slip through with the false ones.

Typical causes are order-dependent tests sharing static state, tests that depend on wall-clock time or timezone, race conditions in asynchronous code tested with Thread.sleep, port collisions when tests run in parallel, and reliance on external services. The fix is never to add a retry loop around the whole build; it is to quarantine the test (move it to a separate, non-blocking job), open an issue, and fix or delete it within days. A test suite is only useful if a red result means something.

Merging strategy: trunk versus branches

CI's demand for daily integration puts a constraint on how you use branches.

Trunk-based development — everyone commits to main, either directly or through pull requests that live for hours — is the approach most aligned with CI. Unfinished features are hidden behind feature flags (a configuration switch that keeps the code path off in production) rather than kept on a branch, so the code is integrated and compiled every day while the feature is invisible to users.

Short-lived feature branches with pull requests work well when each branch lasts a day or two and the branch build must be green before merging. Require the branch to be up to date with main before merging (or use a merge queue) so the build result actually reflects the combination.

Long-lived branches — a develop branch weeks behind main, release branches that receive parallel fixes, per-developer branches merged monthly — are where integration pain returns. They are sometimes necessary (supporting an old release) but should be the exception. The trade-offs among these models are explored in Git branching workflows.

Pull request checks

Modern CI runs on pull requests, not only on merges, and platforms let you make the merge button depend on checks. Sensible required checks for a service:

  • The build and unit tests pass.
  • Static analysis reports no new critical issues (SonarQube quality gate, ESLint, Checkstyle).
  • Dependency scanning reports no new high-severity vulnerabilities (OWASP Dependency-Check, Dependabot, Trivy).
  • Code coverage on changed lines has not dropped below the agreed threshold — a rule on new code, since imposing a global number on a legacy codebase leads to gaming.
  • At least one reviewer approved.

Keep the list short enough that a routine change passes in minutes; every check you add is a check people will look for ways around if it is slow or noisy.

Common mistakes

  • Committing to main once a week from a long branch. The CI server runs, but integration is not continuous. Merge daily; use feature flags for incomplete work.
  • Only building on the CI server. If mvn verify does not run locally, developers push to find out whether their change compiles and the pipeline becomes a slow compiler. The build must work on a laptop.
  • Letting the build stay red. After a day, everyone assumes someone else is on it. Assign it explicitly, or revert.
  • Tests that hit shared environments. A test suite that needs the staging database can only run one at a time and breaks when someone else uses staging. Start a database in a container per build instead.
  • Secrets in the pipeline file. A database password in ci.yml is in Git history forever. Use the CI system's secret store and inject it as an environment variable at run time.
  • Ignoring warnings. A build that prints three hundred deprecation warnings hides the one that matters. Fail on new warnings, or at least track the count.
  • Building a different artifact for every environment. The CI build should produce one artifact that later stages promote; rebuilding per environment means the thing you tested is not the thing you deployed. This is the bridge to continuous delivery.

Key takeaways

  • Continuous integration is a practice — daily merges into a shared main line verified by an automated, self-testing build — and a CI server on its own does not provide it.
  • Structure the build in stages of increasing cost so the common failures (compile errors, unit test regressions) are reported within minutes.
  • Keep the primary build under ten minutes with dependency caching, parallel tests and a sensible test pyramid; move slow suites to a later stage.
  • Flaky tests destroy trust faster than slow builds; quarantine and fix them rather than retrying.
  • Trunk-based development or short-lived branches with required checks support daily integration; long-lived branches recreate the integration phase CI exists to remove.
  • A red main branch is the team's top priority: fix within minutes or revert.

Sources and further reading

Frequently asked questions

We have a Jenkins server that builds every night. Is that continuous integration?
Not quite. A nightly build tells you the next morning that something merged yesterday broke the code, after everyone has gone home and possibly after more changes have piled on top. CI means the build runs on every push, finishes in minutes, and the person who broke it finds out while the change is still fresh. The server is necessary but the frequency and the response are what make it CI.
How long should a CI build take?
Aim for under ten minutes from push to result for the main feedback loop. Beyond that, developers stop waiting for the result, start the next task, and treat failures as interruptions instead of feedback. Longer suites — integration, end-to-end, performance — belong in a later pipeline stage that runs after the fast one has passed.
Should CI run on feature branches or only on the main branch?
Both. Running on every branch and pull request catches problems before merge and keeps main green. But branch builds are not a substitute for merging often: a branch that has been green for three weeks can still conflict badly with main. Keep branches short-lived and rebase or merge from main daily.
What do we do when the build is red?
Fixing it becomes the highest priority for whoever broke it, and nobody else merges on top of a red build. If the fix is not obvious within ten minutes, revert the offending commit and investigate on a branch. A team that tolerates a red main branch for days has lost the benefit of CI entirely.