Azure Boards

When a pull request mentions an Azure Boards work item, MergeHarbor reads the work item and checks the change against its acceptance criteria. The same connection can comment on linked work items, turn findings into work items, triage new work items and feed automations. Jira works the same way.

There are two ways to connect:

Integration	Connects with
Azure DevOps Services	Sign in with Microsoft (Entra ID), or a personal access token
Azure DevOps Server	A personal access token and your collection URL (2020 or later)

Your repositories do not have to be on Azure DevOps: work items linked from GitHub, GitLab or Bitbucket pull requests are read too.

What it does in reviews

• Finds linked work items. MergeHarbor looks for AB#123 references and work item links such as https://dev.azure.com/acme/Web/_workitems/edit/123 in the branch name, the pull request title, its description and its commit messages, in that order. On repositories hosted in Azure DevOps it also reads #123 mentions, looked up in the connected organization. When the repository is in the same Azure DevOps organization as the connection, the work items Azure DevOps links to the pull request (its Development / Work items section) are read too. Up to three work items are used per pull request. References inside code blocks are ignored.
• Adds a requirements check to the review summary. Each criterion is marked Met, Not met or Unclear, with file:line evidence from the pull request. If the work item has no criteria, MergeHarbor lists up to three requirements it inferred from the description, labelled “(inferred)”.
• Costs no extra model call. The check is part of the review itself. The work item text is bounded (by default it adds at most 3,000 input tokens), and nothing is added when no work item is linked.
• Looks for related work when nothing is linked. If the pull request names no work item, MergeHarbor searches the allowed projects for up to two work items that match the title, and gives them to the review as background only. They are never checked as requirements and never commented on. Turn this off with Look for related work.
• Cites what it used. A comment that relied on a work item ends with a “Source used:” line linking to it.
• Lets the review look things up. During a review the agent can read another work item by id (AB#123) or search for related ones, within the same project and repository limits.
• Shows linked work items in the console. The pull request page lists linked work items with links to Azure Boards and the latest requirements check.

Acceptance criteria

Criteria come from the work item’s Acceptance Criteria field first, then from an “Acceptance criteria” or “Definition of done” section, a checklist or Given/When/Then scenarios in the Description. For bugs, Repro Steps are read as part of the description. Azure Boards stores these fields as HTML: MergeHarbor converts them to plain text (lists and headings kept, scripts, styles and images dropped) and removes secrets such as tokens and keys before any text reaches a model.

Connect

Azure DevOps Services: sign in with Microsoft

1. In the console, open Integrations and choose Connect on Azure Boards. You need the settings.manage permission.
2. Type your organization name, the part after https://dev.azure.com/ (for example acme), or paste your organization URL. To let MergeHarbor comment on work items and create them from findings, turn on Allow comments and new issues.
3. Choose Sign in with Microsoft and approve access with your work account. MergeHarbor reads as that account, so it sees what that account can see.
4. Choose Test connection to list the projects the connection can see.

Your organization must be connected to Microsoft Entra ID and allow third-party application access through OAuth (Organization settings → Policies).

Personal access token (Services or Server)

1. In Azure DevOps, open User settings → Personal access tokens and create a token for the organization (or collection) with the Work Items scope: Read to read work items only, or Read & write (vso.work_write) to also comment and create work items. On repositories hosted in Azure DevOps, add Code: Read so MergeHarbor can read the work items linked to pull requests.
2. In Integrations, choose Use a personal access token on Azure Boards, and enter the organization URL (https://dev.azure.com/acme) or, for Azure DevOps Server, the collection URL (https://tfs.acme.com/tfs/DefaultCollection), and the token.

MergeHarbor checks the token before saving it. An Azure DevOps Server URL must be reachable from MergeHarbor; private and internal addresses are refused unless your installation allows them.

Settings

Open Manage on the card:

• Azure DevOps projects: project names as they appear in Azure DevOps (case and spaces kept). Empty means every project the connection can see.
• Repositories: only reviews of these repositories use the integration, and only people who can read them see what it returned.
• Look for related work: on by default.
• Use in pull request conversations: off by default. When on, people who can read an allowed repository can get answers that draw on the allowed projects when they ask MergeHarbor about a pull request. Turning it off hides earlier answers that relied on it.
• Comment on linked issues: off by default. See below.

The switch on the card turns the integration off without disconnecting it: reviews, chat, triage, automations and comments skip it, and its settings are kept.

Comments on linked work items

When you turn on Comment on linked issues, MergeHarbor comments on each linked work item when a review completes, for example:

MergeHarbor reviewed PR #12 in acme/api: 2 findings, confidence 4/5. Requirements: 2 of 3 met. View the review

It keeps one comment per work item and updates it on later reviews. This needs write access to work items: if you signed in without it, the console asks you to reconnect with comment permission.

Create work items from findings

On a pull request, choose Create issue on a finding, then pick the project, the work item type (for example Bug, Task, User Story or Product Backlog Item) and, optionally, the area path and iteration path (the project’s defaults otherwise). The work item gets the finding’s title, explanation, file and line, a link to the pull request and a link to the review in MergeHarbor; for a pull request in the same Azure DevOps organization it is also linked to the pull request natively. Secrets are removed first. The work item id (AB#123) appears on the finding, and asking again for the same finding returns the same work item. You need the pulls.review permission, and the connection needs write access.

Issue triage

Issue triage reads new work items in the projects you turn on and keeps one comment on each with possible duplicates, questions, acceptance criteria and suggested tags, priority and area path. For instant triage, create a webhook in Triage settings, then in Azure DevOps open Project settings → Service hooks, add a Web Hooks subscription for Work item created with that URL, and under basic authentication enter any user name and the secret as the password. Without one, MergeHarbor checks for new work items every few minutes.

Automations

Automations can start on a new work item (through an Azure DevOps service hook with basic authentication, password = the automation’s secret) and can post a review summary as a comment on the linked work item after an approval.

PR chat and lookup tools

With Use in pull request conversations on, PR chat and chat apps read the work items named in a question, the pull request title, description or branch. The review agent’s lookup_ticket and search_tickets tools read and search the same allowed projects.

What MergeHarbor keeps

• Work item id, title, type, state, tags, description, repro steps and acceptance criteria.
• Work item text is cached for a short time (10 minutes by default) and deleted within 24 hours. It is never written to logs. If an administrator turns on Keep linked documents in the knowledge base, text that issue triage reads is kept for the days they choose (1–365) after last use instead.
• A pull request keeps the ids and links of its work items and the requirements check from its review.
• The connection is named after your organization or collection. No user names, email addresses or account ids are stored.
• Tokens are encrypted at rest and never shown again.

Turn off or disconnect

Use the switch on the card to pause the integration. To remove it, choose Disconnect from the card’s menu. MergeHarbor deletes its tokens and cached work item text. Then remove access at the service:

• Signed in with Microsoft: remove MergeHarbor under User settings → Authorizations in Azure DevOps, or at myapps.microsoft.com.
• Personal access token: revoke the token under User settings → Personal access tokens.

Self-hosted installations

Sign in with Microsoft uses the same Microsoft Entra app registration as Azure DevOps repository connections (AZURE_DEVOPS_OAUTH_CLIENT_ID, AZURE_DEVOPS_OAUTH_CLIENT_SECRET, AZURE_DEVOPS_OAUTH_TENANT). On that registration:

1. Under API permissions → Azure DevOps → Delegated permissions, add vso.work_write (Work items, read and write). Grant admin consent if your tenant requires it.
2. Under Authentication, add the redirect URI {PUBLIC_URL}/integrations/connect/azure_boards/callback (platform Web).

Without the app, Azure Boards still connects with a personal access token. Optional server settings are the same as for Jira: INTEGRATIONS_CACHE_TTL, INTEGRATIONS_MAX_TICKETS, INTEGRATIONS_CONTEXT_TOKENS and INTEGRATIONS_FETCH_TIMEOUT.