DevOps, CI/CD & Containers
Jenkins in Practice: Test Reports, Quality Gates, Artifacts and Images
Intermediate8 min readUpdated
A pipeline that only compiles tells you almost nothing. This guide adds the parts that make a build trustworthy — JUnit and coverage reporting, static analysis, a SonarQube quality gate that can stop the build, fingerprinted artifacts, a Maven release to a repository manager, and container images built and pushed with proper tags — then wires it all into a multibranch job.
What you will learn
- Publish JUnit results, code coverage and static-analysis findings so failures are readable
- Wire SonarQube into a pipeline and block promotion on the quality gate
- Archive and fingerprint artifacts, and deploy releases to a repository manager
- Build, tag, scan and push Docker images from a pipeline without leaking registry credentials
- Configure a multibranch pipeline so branches and pull requests build automatically
Getting a pipeline to compile the code is the first ten per cent of the work. The rest is turning a build into a source of evidence: which tests ran and which failed, whether coverage moved, whether new code introduced defects, exactly which binary came out and where it went. Without that evidence, a green build means only that the compiler was happy, and teams end up trusting a manual checklist instead.
This guide picks up where the Jenkinsfile syntax guide leaves off. Every example is a fragment you can paste into a declarative pipeline, and the running example is a Maven service that ends up in a repository manager and a container registry.
Test results: making failures readable
The junit step reads any JUnit-format XML — which almost every test framework in every language can emit — and turns it into the test trend graph, the per-test history and the failure summary on the build page.
stage('Test') {
steps {
sh 'mvn -B -ntp verify'
}
post {
always {
junit testResults: '**/target/surefire-reports/*.xml, **/target/failsafe-reports/*.xml',
skipPublishingChecks: false,
keepLongStdio: true
}
}
}
Three details matter. Put the step in post { always }, not in steps, or a failing test suite aborts the stage before the report is collected and you get a red build with no visible reason. Include both the Surefire (unit) and Failsafe (integration) patterns. And leave allowEmptyResults at its default of false on projects that definitely have tests — a pattern that silently matches nothing is how a pipeline ends up reporting success for a build whose tests never ran.
A published report changes the failure workflow: instead of scrolling a console log, a developer opens the build, sees "3 failed, 1 new" and clicks straight to the stack trace and the history of that individual test. The Test Results Analyzer and the flaky-test reporting in newer plugin versions build on the same data, letting you find the tests that fail intermittently rather than deterministically.
Coverage and static analysis
Coverage is produced by the build tool and published by Jenkins. For Maven and JaCoCo, bind the agent and the report goal:
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.8.12</version>
<executions>
<execution><id>prepare</id><goals><goal>prepare-agent</goal></goals></execution>
<execution><id>report</id><phase>verify</phase><goals><goal>report</goal></goals></execution>
</executions>
</plugin>
Then publish with the Coverage plugin, which supersedes the older JaCoCo plugin and understands Cobertura, JaCoCo and JUnit-style formats through a common parser:
recordCoverage(
tools: [[parser: 'JACOCO', pattern: '**/target/site/jacoco/jacoco.xml']],
sourceCodeRetention: 'EVERY_BUILD',
qualityGates: [
[threshold: 70.0, metric: 'LINE', baseline: 'PROJECT', criticality: 'UNSTABLE'],
[threshold: 80.0, metric: 'LINE', baseline: 'MODIFIED_LINES', criticality: 'FAILURE']
]
)
The second gate is the interesting one: it demands 80 % coverage of the lines this change touched, while tolerating whatever the project average happens to be. That is the only coverage rule that works on an existing codebase, because it asks for improvement at the margin instead of a retroactive rewrite.
Compiler warnings, Checkstyle, PMD, SpotBugs and dozens of other analysers are collected by the Warnings Next Generation plugin into a single view with new-issue detection against a reference build:
recordIssues(
tools: [java(), checkStyle(pattern: '**/target/checkstyle-result.xml'),
spotBugs(pattern: '**/target/spotbugsXml.xml')],
qualityGates: [[threshold: 1, type: 'NEW', criticality: 'UNSTABLE']],
referenceJob: 'shop-api/main'
)
Setting the gate on new issues rather than total issues is what makes static analysis adoptable in a codebase that already has 4,000 warnings.
The SonarQube quality gate
SonarQube aggregates issues, coverage, duplication and security hotspots, and applies a configurable quality gate. Integrating it properly means two pipeline pieces: running the analysis, and waiting for the verdict.
Configure the server once under Manage Jenkins → System (name, URL, and an authentication token stored as a secret-text credential). Then in the pipeline:
stage('Analyse') {
steps {
withSonarQubeEnv('sonarqube-main') {
sh '''mvn -B -ntp sonar:sonar \
-Dsonar.projectKey=shop-api \
-Dsonar.qualitygate.wait=false'''
}
}
}
stage('Quality gate') {
steps {
timeout(time: 10, unit: 'MINUTES') {
waitForQualityGate abortPipeline: true
}
}
}
withSonarQubeEnv injects the server URL and token, so no credential appears in the command. Analysis is asynchronous: the scanner uploads data and returns, and SonarQube computes the result afterwards. waitForQualityGate blocks until the server calls back — which requires a webhook configured in SonarQube pointing at https://jenkins.example.com/sonarqube-webhook/. Without that webhook the step waits until the timeout and people conclude the integration is broken.
Splitting analysis and gate into separate stages lets other work run in between, and putting the gate stage on agent none releases the executor while waiting.
Configure the gate itself on the new code definition: no new blocker or critical issues, coverage on new code above a threshold, duplication on new code below one, no new security hotspots left unreviewed. A gate defined that way can be enforced on main from the first day without a cleanup project.
Artifacts: archive, stash, fingerprint
Three mechanisms move files around, and they are not interchangeable.
stash and unstash pass files between stages within one build. They are stored on the controller, so keep them small — a jar and a set of manifests, not a workspace.
stage('Build') { steps { sh 'mvn -B -ntp package -DskipTests'; stash name: 'app', includes: 'target/*.jar' } }
stage('Deploy') { agent { label 'deploy' }; steps { unstash 'app'; sh './deploy.sh' } }
archiveArtifacts attaches files to the build permanently, downloadable from the build page and via the REST API. Add fingerprint: true so Jenkins records the file's checksum and can later tell you which build produced a given binary and which builds consumed it.
archiveArtifacts artifacts: 'target/*.jar, target/classes/META-INF/build-info.properties',
fingerprint: true,
onlyIfSuccessful: true
A repository manager — Nexus, Artifactory, a cloud package registry — is where releases actually belong. Jenkins archives are for debugging and short-term retention; a repository manager is the system of record with retention policies, immutability and access control.
Releasing to a repository manager
Give the pipeline a settings.xml whose server credentials come from environment variables, so the file itself contains no secret:
<settings>
<servers>
<server>
<id>nexus-releases</id>
<username>${env.NEXUS_USR}</username>
<password>${env.NEXUS_PSW}</password>
</server>
</servers>
</settings>
Then bind the credential and deploy. Avoid the classic maven-release-plugin inside CI — it makes two commits, a tag and two full builds, and it fails halfway often enough to leave a repository in a strange state. The CI-friendly alternative sets the version from outside:
environment { NEXUS = credentials('nexus-deploy') }
stage('Release') {
when { buildingTag() }
steps {
sh '''
VERSION="${TAG_NAME#v}"
mvn -B -ntp -s ci/settings.xml \
-Drevision="$VERSION" \
-DskipTests \
deploy
'''
}
}
With <version>${revision}</version> in the POM, the pipeline supplies the version, the tag defines it, and one build produces one immutable release. Snapshots from main go to the snapshot repository; only tag builds reach the release repository, which most repository managers can configure to reject re-deployment of an existing version.
Building and pushing container images
The Docker Pipeline plugin wraps the daemon in convenient steps, and handles registry authentication from the credential store:
stage('Image') {
agent { label 'docker' }
environment {
REGISTRY = 'registry.example.com'
IMAGE = "${REGISTRY}/acme/shop-api"
TAG = "${env.GIT_COMMIT.take(12)}"
}
steps {
unstash 'app'
script {
def image = docker.build("${IMAGE}:${TAG}", "--build-arg JAR_FILE=target/app.jar .")
docker.withRegistry("https://${REGISTRY}", 'registry-credentials') {
image.push()
if (env.BRANCH_NAME == 'main') { image.push('latest') }
}
}
}
}
For multi-architecture images, or to build without a Docker daemon on the agent, use buildx directly:
docker buildx build \
--platform linux/amd64,linux/arm64 \
--tag "$IMAGE:$TAG" \
--label "org.opencontainers.image.revision=$GIT_COMMIT" \
--label "org.opencontainers.image.source=https://git.example.com/acme/shop-api" \
--provenance=true \
--push .
Those OCI labels are worth adding: they let anyone inspecting a running image find the commit that produced it. The Docker fundamentals guide covers the Dockerfile side, including multi-stage builds that keep the pushed image small.
Add a scan before the push, and fail on serious findings:
sh 'trivy image --exit-code 1 --severity HIGH,CRITICAL --ignore-unfixed "$IMAGE:$TAG"'
--ignore-unfixed keeps the gate actionable: it fails only on vulnerabilities that have a patched version available, which is something the team can actually do something about today.
Multibranch pipelines and pull requests
A multibranch pipeline scans a repository, creates a job for every branch containing a Jenkinsfile, builds it on push, and removes the job when the branch is deleted. Add the organisation folder variant and it does the same for every repository in a Git organisation. This is what replaces the habit of copying a job and editing the branch name.
Point it at the repository, give it credentials, and configure:
- Discover branches — usually "exclude branches that are also filed as PRs", so a pull request is not built twice.
- Discover pull requests from origin and forks — merging the PR with the target branch is the useful strategy, because it tests what would be on
mainafter merge, not what is on the branch now. - Orphaned item strategy — delete jobs a few days after the branch disappears.
- Scan triggers — periodic scanning as a fallback, with a webhook doing the real work.
Inside the Jenkinsfile, env.BRANCH_NAME, env.CHANGE_ID, env.CHANGE_TARGET and env.TAG_NAME let one file serve every context:
stage('Publish snapshot') {
when { allOf { branch 'main'; not { changeRequest() } } }
steps { sh 'mvn -B -ntp -s ci/settings.xml deploy -DskipTests' }
}
Pull-request builds should be fast and thorough on correctness; branch builds on main add the slower work — full integration suites, image build and push, deployment to a shared environment. Keep the pull-request feedback loop under ten minutes or people stop waiting for it.
Putting it together
A realistic pipeline for a service ends up looking like this in outline: build and unit test on a container agent; publish results and coverage in post; run static analysis and Sonar in parallel with integration tests; wait on the quality gate; build and scan the image; push it with an immutable tag; deploy to staging automatically; deploy to production behind an input gate or from a tag build.
Each of those steps produces evidence attached to the build — test trends, coverage delta, Sonar verdict, fingerprinted artifacts, the image digest. When someone asks "what is running in production and where did it come from", the answer is a link rather than an investigation.
Common mistakes
- Reports collected in
stepsrather thanpost. A failing stage skips them, and the build shows a failure with no detail. - Coverage gates on the whole project. They are unachievable on legacy code and get switched off. Gate on new or modified lines.
waitForQualityGatewithout the SonarQube webhook. The step blocks until timeout and the team blames Jenkins.- Docker tags that move. Deploying
:latestmeans the image running in production is unidentifiable. Push an immutable tag and deploy that. - Credentials interpolated into Groovy strings. Use single-quoted shell strings so the shell expands the variable and the value never enters the command Jenkins assembles.
- Archiving whole workspaces. Multi-gigabyte builds fill the controller's disk. Archive the artifact and the reports, nothing else.
- Releasing from a branch build. Two builds of the same branch can produce two different binaries with the same version. Release from tags, deploy the artifact you already built.
Key takeaways
- Publish
junitresults in apost { always }block, and make an empty report pattern a failure rather than a silent pass. - Record coverage and static-analysis findings with gates on new or modified code, which is the only formulation that works on an existing codebase.
- Wire SonarQube with
withSonarQubeEnvplus a separatewaitForQualityGatestage, and configure the webhook that makes the wait terminate. - Use
stashwithin a build,archiveArtifacts ... fingerprint: truefor provenance, and a repository manager as the system of record for releases. - Supply the version from the pipeline rather than letting a release plugin rewrite the POM, and deploy releases only from tag builds.
- Build images on a dedicated Docker agent or a daemonless builder, tag them by commit, add OCI labels, scan with
--ignore-unfixedand push before deploying. - A multibranch pipeline gives every branch and pull request a build automatically and deletes the job when the branch goes away.
Sources and further reading
Frequently asked questions
- Why does my build stay green when tests fail?
- Almost always because the build command swallowed the failure — a shell step ending in a pipe, a Maven run with -Dmaven.test.failure.ignore=true, or a junit step whose pattern matches nothing while allowEmptyResults is true. Check that the test tool returns a non-zero exit code and that the junit step reports the expected number of tests. A report pattern that matches nothing should fail the build, not pass it.
- Should the quality gate break the build?
- On the main branch, yes, once the gate is configured against new code rather than the whole codebase. A gate on overall coverage in a legacy project fails forever and gets disabled within a month. A gate on new code — coverage of lines added, no new blocker issues, no new duplicated blocks — is achievable from day one and keeps the codebase from getting worse.
- What does fingerprinting an artifact actually do?
- Jenkins records the MD5 of the file and remembers which build produced it and which builds later used it. Given a jar found on a server you can ask Jenkins which build made it, and from a build you can see everywhere its output travelled. It costs nothing to enable on archiveArtifacts and is the cheapest form of provenance an internal pipeline can have.
- How should container images be tagged from CI?
- Tag every image with something immutable and traceable — the short commit hash or the build's semantic version — and push that. Add a moving tag such as the branch name or latest only as a convenience for humans. Deployments must reference the immutable tag or, better, the digest, so that redeploying the same reference always yields the same bits.