Software Testing & Quality
Code Quality and Static Analysis: Error Prone, SpotBugs, PMD and SonarQube
Intermediate10 min readUpdated
Static analysis reads code without running it and finds a class of defect no test suite will reach — the unreachable branch, the resource never closed, the equals that compares an array. This guide separates the tool families, sets out how to configure a quality gate that developers respect, and describes adopting analysis on a codebase with ten thousand existing findings.
What you will learn
- Explain what static analysis detects that dynamic tests structurally cannot
- Distinguish formatters, style checkers, bug finders, SAST and dependency scanners
- Configure Error Prone, SpotBugs, PMD and Checkstyle so their findings are worth reading
- Set a SonarQube quality gate on new code rather than on the whole codebase
- Interpret complexity, duplication and technical debt metrics without over-trusting them
- Introduce analysis on a legacy codebase without generating an unusable backlog
Tests tell you whether the code did the right thing on the inputs you chose. Static analysis reads the code without running it and reasons about every path, including the ones no test has ever taken. That difference is the whole argument for using both: a test suite with excellent coverage can still miss a stream that is never closed on the exception path, a equals called on two arrays, a format string with the wrong number of placeholders, or a comparison of boxed integers with == that works up to 127 and fails at 128.
The other reason to run analysis is economic. A finding produced by a tool costs nothing to detect, arrives seconds after the code was written, and does not need anyone to agree it is worth a comment in review. Human review time is scarce and should be spent on whether the design is right, not on whether a resource leaks — which is the same argument the coding standards guide makes for automating formatting.
What analysis sees, and what it cannot
Analysers work on the abstract syntax tree, the control flow graph and, in the stronger tools, an approximation of the data flow. That gives them abilities no test has: they can consider a branch that has never executed, prove a null can reach a dereference regardless of input, or trace tainted user data from a request parameter to a SQL string.
It also gives them permanent limits. They know nothing about intent, so no analyser will ever tell you a VAT rate is wrong. They must approximate, because the underlying questions are undecidable, and approximation means either false positives (reporting something safe) or false negatives (missing something real). Every tool picks a point on that spectrum, and knowing where a tool sits tells you how to treat its output: Error Prone is tuned for near-zero false positives and can therefore fail a build, while a security scanner deliberately errs towards reporting and needs triage.
The tool families
Lumping everything under "static analysis" causes teams to adopt overlapping tools and drown. Five distinct jobs:
| Family | Question it answers | Examples | Build behaviour |
|---|---|---|---|
| Formatter | Is the layout canonical? | Spotless, google-java-format | Rewrites, then verifies |
| Style checker | Does it follow our conventions? | Checkstyle | Fails on violations |
| Bug finder | Is this code probably wrong? | Error Prone, SpotBugs, PMD | Fails on high-confidence findings |
| SAST | Is this exploitable? | SonarQube security rules, Semgrep | Reports, triaged |
| Dependency scanner | Do our libraries have known CVEs? | OWASP Dependency-Check, Trivy | Fails on new high severity |
Adopt one from each row, not three from the same one. Two style checkers with different opinions produce contradictory findings and an argument.
Start with the compiler
Before any tool, the JDK already reports a great deal that most projects ignore. Turning warnings on and treating them as errors is the cheapest quality improvement available:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<release>21</release>
<compilerArgs>
<arg>-Xlint:all,-serial,-processing</arg>
<arg>-Werror</arg>
</compilerArgs>
</configuration>
</plugin>
-Xlint:all covers unchecked generic operations, deprecation, fall-through in switches, missing @Override, and more. -Werror makes them fail the build, which is the only setting that works — a build printing four hundred warnings is a build where nobody reads warnings. On an existing project, enable categories one at a time.
Bug finders worth running
Error Prone hooks into javac and analyses with full type information during compilation. Its rules are curated for confidence, which is why many are errors by default: calling Object.equals on incompatible types, a @FormatMethod argument mismatch, a returned value that is ignored when it must not be, a Duration compared with ==. Because it runs in the compiler it is fast and it can offer automatic fixes.
<compilerArgs>
<arg>-XDcompilePolicy=simple</arg>
<arg>--should-stop=ifError=FLOW</arg>
<arg>-Xplugin:ErrorProne -Xep:MissingOverride:ERROR -Xep:UnusedVariable:WARN</arg>
</compilerArgs>
SpotBugs analyses compiled bytecode and finds a different set: null dereference on a path, resource leaks, exposing internal mutable state through a getter, inconsistent synchronisation, Serializable classes with non-serialisable fields. The find-sec-bugs plugin adds injection and cryptography patterns. Bytecode analysis sees what the compiler produced rather than what you wrote, which occasionally makes findings surprising until you look at the generated code.
PMD works on source and specialises in design and complexity rules — long methods, too many parameters, deeply nested conditionals, empty catch blocks — plus its CPD copy-paste detector, which finds duplicated blocks across files that no other tool in this list notices.
Checkstyle enforces conventions rather than correctness: naming, import order, Javadoc on public API, file structure. It overlaps least with the others and is the one whose rule set most needs pruning to fit the team, because a default configuration will complain about a great deal that nobody has decided to care about.
Two configuration rules apply to all of them. Fail the build on the rules you have decided matter, because a report nobody opens is a report nobody reads. And remove rules you disagree with from the shared configuration rather than suppressing them individually in code — a suppression comment is a per-file decision that quietly becomes policy.
SonarQube and the new-code gate
SonarQube aggregates its own analysers with imported reports from the tools above, stores history, and applies a quality gate: a set of conditions that a build passes or fails.
The single most important configuration decision is that the gate should apply to new code — the lines added or changed since a chosen baseline — rather than to the whole project. This is what makes analysis adoptable on a real codebase. A gate demanding 80% coverage across a fifteen-year-old system is unachievable and will be disabled within a month; a gate demanding 80% coverage on the lines you touched this week is a reasonable request, and the codebase improves wherever work is happening.
A workable gate on new code:
- Coverage on new lines above 80%
- Duplicated lines on new code below 3%
- No new issues rated blocker or critical
- No new security hotspots left unreviewed
- The overall rating for new code at A
SonarQube's newer model classifies issues by whether they threaten code being consistent, intentional, adaptable or responsible, and maps them to software qualities — security, reliability, maintainability — rather than the older bug/vulnerability/code-smell triad. The mapping matters less than the discipline of triaging every new blocker before merge.
Wire it into the pipeline so the gate is a merge condition, not a dashboard someone visits:
mvn -B verify sonar:sonar \
-Dsonar.projectKey=shop-service \
-Dsonar.qualitygate.wait=true
sonar.qualitygate.wait=true makes the command block until the gate result is known and exit non-zero on failure, which is what turns it into an actual gate. The continuous integration guide covers where in a pipeline that belongs relative to the test stages.
Metrics, assessed honestly
Static analysis produces numbers, and numbers get put on dashboards whether or not they mean anything.
Cyclomatic complexity counts independent paths through a method. It is a decent screening tool — sort methods by it and the worst offenders in a codebase are usually near the top — and a poor target. A flat switch over twenty enum values scores high and reads perfectly; a method with three nested loops and a continue scores lower and is far harder to follow. Cognitive complexity was designed to address exactly that by penalising nesting and ignoring shorthand structures, and it tracks human difficulty noticeably better.
Duplication percentage is useful when it points at copy-pasted business logic, which is where a bug gets fixed in two of three places. It is noise when it flags generated code, tests that are deliberately explicit, or DTO boilerplate.
Technical debt expressed as a remediation time — "4 days 2 hours" — is a rough sum of per-rule estimates. Treat it as an ordering device, not an estimate; nobody has ever fixed a codebase in the time the tool predicted. Its trend is more meaningful than its value, and its value per module is more meaningful than the total.
Coverage is imported rather than computed by the analyser, and carries the caveat from the test automation strategy guide: it measures execution, not assertion, and behaves badly as a target.
Maintainability, reliability and security ratings compress issue counts into a letter. They communicate well to people outside the team and hide everything a specialist needs, so use them for reporting and the issue list for work.
Security-focused analysis
Two distinct scans belong in every pipeline.
SAST looks for exploitable patterns in your own code: SQL built by string concatenation, a command executed from user input, path traversal, weak hashing, a hard-coded credential, a disabled certificate check, XML parsing with external entities enabled. The strongest engines do taint tracking — following data from an untrusted source to a dangerous sink — which is why they can distinguish concatenated SQL that uses a constant from concatenated SQL that uses a request parameter.
SCA (dependency scanning) checks your libraries against vulnerability databases. In practice this finds more real risk than SAST does, because most applications ship far more third-party code than their own. The operational rules that make it bearable: gate on new high-severity findings so the queue does not block every build, track transitive dependencies as well as direct ones, and record accepted risks with an expiry date rather than a permanent suppression.
Secret scanning is a third, cheap and worth adding: a scan of the diff for private keys, tokens and connection strings. It has to run in a pre-receive hook or an early pipeline stage, because once a secret reaches shared history the only remedy is rotating the credential.
None of these replace security review or penetration testing, and a clean scan is a floor rather than an assurance. What they do reliably is stop the well-known patterns from reaching main.
Adopting analysis on a legacy codebase
The first full run on an established codebase reports thousands of findings. That number is why most adoption attempts fail: a backlog that large is not actionable, so it gets ignored, and once ignored the tool is noise.
A sequence that works:
Freeze the baseline
Run the analysis, record the current state, and configure the gate to apply only to new code. The immediate objective is that the number stops growing, not that it shrinks.Turn on a handful of certain rules everywhere
Pick ten to fifteen high-confidence correctness rules — resource leaks, ignored return values, comparison mistakes, empty catch blocks — and make them build failures across the whole codebase. Fix the existing violations of just those. It is usually a day or two of work and it removes real defects.Automate formatting in one commit
Apply the formatter across the codebase in a single formatting-only commit, add its hash to the ignore-revs file so history stays readable, and enforce it thereafter. This removes an entire category of finding permanently.Fix opportunistically
When someone changes a file, they fix the findings in the part they touched. Over a year the files that matter — the ones under active change — get clean, and the dormant ones stay as they are, which is the correct allocation of effort.Add rules gradually
One new rule category per iteration, discussed in advance and agreed by the team. A rule imposed without discussion gets suppressed rather than followed.Review the suppressions
Once a quarter, list every suppression and exclusion. Some encode a decision worth keeping; some were a workaround for a version-specific false positive that has since been fixed.
The underlying principle is that analysis has to be part of the loop the developer is already in. A finding that appears in the IDE while typing costs seconds; the same finding in a nightly report costs a context switch, and the same finding in a quarterly audit costs a meeting. Configure the IDE plugins from the same shared configuration files the build uses, so that local and pipeline results agree — disagreement between the two is the fastest way to make people distrust the whole thing.
Key takeaways
- Static analysis reasons about paths no test exercises, and tests verify intent no analyser can know; the two cover different defect classes and neither substitutes for the other.
- Pick one tool per family — formatter, style checker, bug finder, SAST, dependency scanner — and configure the shared rule set deliberately instead of accepting defaults.
- Start with the compiler:
-Xlint:all -Werrorcosts nothing and catches a surprising amount. - Gate on new code, not on the whole codebase; that single choice is what makes analysis adoptable on a legacy system.
- Treat complexity, duplication and technical-debt figures as screening tools and trends, never as targets, and prefer cognitive complexity for readability questions.
- Dependency scanning usually finds more real risk than analysis of your own code, so gate on new high-severity findings and give accepted risks an expiry date.
Sources and further reading
Frequently asked questions
- Does static analysis replace testing?
- No, and the two find different things. Analysis reasons about all paths through the code but knows nothing about intent, so it can prove a stream is never closed while being entirely blind to a discount calculated with the wrong rate. Tests verify intent on the paths they exercise. A team needs both; treating either as a substitute leaves a large category of defect uncovered.
- How do we handle false positives without disabling the rule?
- Suppress at the narrowest possible scope with a reason attached — an annotation on the one method, not a class-wide or project-wide exclusion — so the next reader can judge whether it still applies. If a rule produces false positives repeatedly across the codebase, that is a signal to disable the rule deliberately in the shared configuration rather than to scatter suppressions. Both decisions should be visible in version control.
- Is high cyclomatic complexity always bad?
- Not always. A method that switches over twenty enum constants with one line each has high complexity and is perfectly readable, while a method with complexity six and four levels of nesting can be far worse. Complexity is a screening tool: sort by it, look at the top of the list, and use judgement. Cognitive complexity, which weights nesting more heavily and ignores flat switches, correlates better with what people find hard to read.
- We have 12,000 existing findings. Where do we start?
- Do not start by fixing them. Set the gate on new code only, so the number stops growing, and accept the existing findings as a known baseline. Then enable a small set of high-confidence correctness rules as build failures across the whole codebase and fix only those. Everything else gets fixed opportunistically when someone touches the file. Trying to clear the backlog first is how adoption efforts die.