# How to reduce false positives in AI code review

> Why AI reviewers post wrong or unwanted comments, and the settings, rules and feedback loops that cut the noise without hiding real bugs.

An AI reviewer that is right most of the time but posts a steady stream of wrong or pointless comments will be ignored within a month. Engineers learn to scroll past the bot, and then they scroll past the comment that would have caught the outage.

Reducing false positives is therefore not polish. It decides whether the tool is used at all. This guide separates the different kinds of noise, explains where each one comes from, and then goes through the levers MergeHarbor gives you to remove it, in roughly the order you should reach for them.

## Three kinds of noise

"False positive" is used loosely for anything an engineer did not want to read. It helps to split it up, because each kind has a different fix.

- **Wrong findings.** The reviewer reports a bug that is not there. Usually it did not see the code that handles the case: a validation in the caller, a guard in middleware, a constraint in the database.
- **Correct but unwanted findings.** The observation is true, but your team does it on purpose. Floats for display-only values, a broad `catch` at a process boundary, a missing docstring in generated code.
- **Low-value or repeated findings.** Naming and style notes on a change where nothing is wrong, the same issue reported in five places, or a point raised again after someone already answered it.

Wrong findings are a context and verification problem. Unwanted findings are a conventions problem. Low-value findings are a posting-bar problem. Tuning only the last one by raising thresholds hides real bugs along with the noise.

## Start with what is already on

Before changing anything, know the defaults, because several of the strongest levers are on out of the box:

- **Verification.** `verify_findings: true` runs a second pass that tries to disprove each finding, citing the code it checked. Disproved findings are never posted. See [how verification works](https://mergeharbor.com/product#verify).
- **The posting bar.** Notes about missing tests, naming, documentation or style are held back unless strictness is high. Style is not in the default `categories`.
- **Merging repeated findings.** One comment for an issue found in several places, and no repeats of issues raised on earlier commits.
- **Learned preferences.** Kinds of comments your team keeps passing on are held back where they keep being ignored.
- **Bot pull requests skipped.** Pull requests from Dependabot, Renovate, pre-commit.ci, GitHub Actions and All Contributors are not reviewed automatically.

Then measure. **Insights → Quality** shows, for the last 7, 30 or 90 days, the share of comments **acted on**, **fixed before merge** and **passed over**, the number of draft findings **stopped by verifier**, comments per review, and a breakdown by category. Look at the breakdown first: noise is rarely spread evenly, and one category or one folder usually accounts for most of it.

## Give the reviewer the context it is missing

Most wrong findings come from missing context, so fix that before you suppress anything.

**Let it see past the diff.** Two review-quality settings, **Trace what depends on the change** and **Bring in related code**, are off by default for Standard reviews and always on for Deep and Max. Turning them on lets the reviewer follow callers and pull in related code before it decides something is unhandled. If a repository produces many "this input is not validated" findings that turn out to be validated upstream, this is the setting to try.

**Point it at the documents that explain your design.** A `context.yml` in a `.mergeharbor/` folder lists files the reviewer always reads for changes under that folder (up to 20), such as an API contract or a design note, and background notes scoped to paths. Team documents in the knowledge base are searched when relevant. See [Repository configuration](https://mergeharbor.com/docs/repository-config).

**Group repositories that change together.** If findings keep claiming a function does not exist because it lives in a shared library, add both repositories to a cluster under **Memory → Cross-repo context**, so each can search the other.

## Write down your conventions as rules

Unwanted-but-correct findings are the reviewer applying general advice where your team has made a different decision. Tell it the decision.

Rules are short, plain-English statements, and MergeHarbor applies the rules that match the files a pull request changes. You can create them in the console, in the repository, or straight from a pull request:

```
@mergeharbor remember for services/billing/**: Amounts are integers in cents; never flag integer division on amounts.
```

In the repository, a `rules.md` in a `.mergeharbor/` folder applies to everything below that folder, and is only sent to the reviewer when a pull request changes files there. That keeps rules for one service from affecting another.

**Memory → Custom rules** shows, for each rule, how many comments it produced and the share that later code fixed. A rule that produces many comments nobody fixes is itself a source of noise; rewrite or remove it. The full workflow is in [Rules and learning](https://mergeharbor.com/docs/rules).

## Tune what gets posted

Once context and conventions are in place, adjust the posting settings. These live in `.mergeharbor.yml`, in the console, or in a folder's `.mergeharbor/config.yml`.

| Setting        | What it does                                                | When to change it                                                      |
| -------------- | ----------------------------------------------------------- | ---------------------------------------------------------------------- |
| `strictness`   | `low`, `medium` or `high`                                   | Lower it for prototypes; high brings back test, naming and style notes |
| `min_severity` | Lowest severity posted: `critical`, `major`, `minor`, `nit` | Raise to `major` for folders where minor notes are rarely acted on     |
| `categories`   | Which categories are reviewed                               | Remove `maintainability` or `testing` where they are mostly ignored    |
| `max_comments` | Inline comment limit; the rest go to the summary            | Lower it to keep the conversation readable on large changes            |
| `ignore_paths` | Paths left out of the review diff                           | Generated code, fixtures, snapshots                                    |

`min_severity` and `categories` apply per file, with the closest `.mergeharbor/` folder winning, so you can be strict in `services/payments/` and relaxed in `tools/scripts/` in the same repository.

A starting configuration for a repository with generated code and a noisy test folder:

```yaml
# .mergeharbor.yml
review:
  strictness: medium
  min_severity: minor
  categories: [correctness, security, concurrency, performance, error_handling, api_contract, data_integrity]
  max_comments: 15
  ignore_paths: ["gen/**", "**/*.pb.go", "**/__snapshots__/**"]
  verify_findings: true
```

```yaml
# test/e2e/.mergeharbor/config.yml
min_severity: major
```

Configuration is read from the pull request's merge base, so a pull request cannot loosen its own review; changes apply after they merge. Problems in configuration files never block a review: the invalid key is skipped and listed in the summary's **Configuration check** section.

## Keep paths out of the index, not just the diff

`ignore_paths` removes files from the review diff, and the reviewer's tools never return paths that match it. It does not stop those files from being indexed. If generated or vendored code is polluting search results, so that the reviewer keeps comparing your code with machine-written code, add a `.mergeharborignore` file at the repository root with gitignore-style patterns. Paths in it are kept out of indexing and search entirely.

## Use evidence-based confidence and a precision target

Two settings under **Settings → Review behaviour → Review quality** act on confidence rather than category:

- **Score confidence from evidence** (off by default) recomputes each finding's confidence from what the verifier checked, whether a deterministic check agrees, and your team's past responses to that kind of comment.
- **Precision target** (0, meaning off) then posts only the findings that together meet the expected precision you set. Critical findings are always posted.

This is the most direct way to trade a little recall for a lot less noise. Turn on evidence-based scoring first, watch the confidence values for a couple of weeks, and only then set a target.

## Let feedback do the rest

The cheapest noise reduction is the one your team does without thinking about it. MergeHarbor treats a comment whose lines changed before merge as acted on, and one unchanged at merge as passed over. It also reads reactions, dismissals in the console with their reason, and replies.

When the last three or more comments of one kind in a repository or folder were passed over and almost none were acted on, or when few were acted on across at least six responses, MergeHarbor stops posting lower-priority comments of that kind there. Each review summary says how many findings were held back, and **Insights → Quality** lists every learned preference with its evidence and an **Undo** button.

Some things are never held back, however often they are ignored: **security, correctness, concurrency and data-integrity findings, and every critical finding.** That is deliberate. A team that ignores a correctness warning may be wrong.

Replies that explain a convention become proposed rules, and nothing learned becomes a rule without an administrator's approval. Rules are only suggested once an organization has about ten reviewed pull requests in the last month, so a single comment cannot set policy. Reactions are collected on GitHub, GitLab, Gitea and Azure DevOps; on Bitbucket, rate comments in the console. See [Learning and review quality](https://mergeharbor.com/docs/learning).

## What not to do

- **Do not raise `min_severity` to `critical` everywhere.** You will remove noise and most of the real findings with it.
- **Do not turn off verification to save time.** It is the main defence against wrong findings.
- **Do not suppress a category because of one folder.** Scope the change to that folder with a `.mergeharbor/config.yml`.
- **Do not judge the tool in week one.** Learning needs responses to learn from. Compare the acted-on rate after a month.

## A rollout order that works

1. Leave the defaults on and connect a few active repositories.
2. After a week, read **Insights → Quality** and find the noisiest category and folder.
3. Fix missing context first: related code, context files, clusters.
4. Write rules for the conventions behind unwanted findings.
5. Scope severity and category changes to folders, and add `.mergeharborignore` for generated code.
6. Turn on evidence-based scoring, then consider a precision target.
7. Only then make the review a required check or turn on an enforced [merge gate](https://mergeharbor.com/guides/merge-gates).

## Try it on your repositories

Noise depends on your code and your conventions, so the honest test is your own pull requests. [Start a 14-day trial of the Team plan](https://app.mergeharbor.com/signup), with no card required, and watch the Quality page over the first few weeks.

## Further reading

- [How AI code review works, step by step](https://mergeharbor.com/guides/how-ai-code-review-works)
- [Learning and review quality](https://mergeharbor.com/docs/learning)
- [Configure reviews](https://mergeharbor.com/docs/configuration) and [Repository configuration](https://mergeharbor.com/docs/repository-config)
- [AI code review for monorepos and large repositories](https://mergeharbor.com/guides/monorepos-and-large-repositories)
- [A code review checklist](https://mergeharbor.com/guides/code-review-checklist) for human and AI-assisted review