Skip to main content
Per-repo customAgents let each repository add its own review agents. Org custom agents invert that: an organization admin defines an agent once in the dashboard, and it runs across the org’s repositories whether or not each repo asks for it. Use them for standards that are not a per-repo choice — a compliance rule, a security requirement, a house pattern every service must follow. Find them under Settings → Custom agents. Available on both managed SaaS and self-hosted.

​
Who can edit

Only organization admins can create or edit org agents. Other members see a read-only view — they can tell what is enforced and why a finding appeared, without being able to change it.

​
Defining an agent

Each agent also records who last edited it and when, so an unexpected finding can be traced to a change rather than argued about.

​
Enforcement: advisory vs blocking

This is the field to think hardest about.

Advisory

Findings appear in the review like any other. They feed the merge-readiness score but do not independently gate anything.

Blocking

A critical finding from this agent fails the check run and requests changes, whatever the overall merge score, unless the verifier refuted it.

​
What the verifier does to a blocking finding

Before a blocking agent’s critical can fail a check, MergeWatch sends it, with the file it cites, to a verifier model:
  • Refuted: the verifier read the code and found the claim false. The finding is still shown, marked unverified, and does not block. The review is posted as a comment and never approves.
  • Could not be confirmed: the verifier errored, gave no usable verdict, or dismissed the finding only because the code claims to be intentional. The finding blocks. A verifier failure never lets a blocking finding through.
  • Confirmed, or not verifiable (the file could not be read): the finding blocks.

​
When an agent fails

An agent can fail to answer: the model call errors, the reply is not the JSON it was asked for, or the reply has no usable findings in it. MergeWatch does not read a failed agent as “nothing found”:
  • Blocking agent failed: its policy was not evaluated, so the check fails, titled Custom agent failed: <name>. The review comment says which agent failed. The review is posted as a comment, never an approval (a low merge score still requests changes). Re-running the check retries.
  • Advisory agent failed: a quiet line in the comment says so; the check is unaffected.
  • Rate limited: the review is queued and retried, as for the built-in agents.
  • The org agents could not be loaded: the review fails and can be re-run. It does not run without them.
Findings a failed agent raised on an earlier commit are not treated as fixed, and their inline threads stay open until the agent answers again.
Blocking enforcement bypasses the normal scoring path. A PR that MergeWatch would otherwise consider ready to merge is still blocked by a single critical finding from a blocking agent. That is the point — but it also means a poorly-worded blocking prompt can stop every PR in the organization.Start new agents as advisory, watch what they actually flag for a week, then promote to blocking once the finding rate looks right.

​
Scoping and targeting

Scope decides which repositories the agent applies to: all, or an explicit allowlist. Targeting narrows further, within those repositories — the agent runs only when the PR actually touches relevant code:
  • Path globs — run only when a changed file matches one of these patterns.
  • Languages — run only when the PR touches one of these languages.
Targeting is what keeps an org-wide agent cheap. A Terraform security agent scoped to all repositories but targeted at **/*.tf costs nothing on the PRs that do not touch Terraform, and there is no reason to pay for an LLM call that has nothing to examine.

​
How org agents combine with repo agents

Org agents run in addition to each repository’s own .mergewatch.yml customAgents — the two sets are unioned. On a name collision, the org agent wins. A repository cannot disable or override an org agent by defining one with the same name. Repositories can add agents; they cannot remove org ones. That asymmetry is deliberate: an org standard that any repo could opt out of by naming a local agent the same thing would not be a standard.

​
Cost

Each enabled agent adds one LLM call per review it runs on. Ten org agents scoped to all repositories means up to ten extra calls on every pull request across the organization. The dashboard warns past 10 active agents. It is a soft cap — nothing is blocked — but treat the warning as real: past that point, targeting is no longer optional if you care about review cost and latency. Two levers keep this under control:
  1. Target aggressively. Most org concerns apply to a subset of files, not every PR.
  2. Consolidate prompts. Three narrow agents that always run together are usually better as one agent with a broader prompt.

​
Next steps

Per-repo custom agents

The customAgents block org agents union with.

Review behavior

How findings become a merge-readiness score.