Building a monitoring workflow

Sanctions screening cannot be treated as a one-off check at onboarding. Regulatory expectations increasingly require continuous monitoring: re-screening existing counterparties as watchlist data changes, so that newly designated individuals or companies are caught even if they passed the initial screen.

We do not offer a dedicated monitoring product with stored counterparty data and managed alerts. We are developing features to make monitoring easier, but as a data and screening provider, we will provide well-documented data and screening primitives, not a full case management tool. This guide explains how to build monitoring on the existing API and bulk data infrastructure.

Screening vs. monitoring#

Screening takes a counterparty and checks whether it matches any entity in the database right now. The /match endpoint does this.

Monitoring adds a time dimension: not "does this counterparty match today?" but "has anything changed since the last check that would affect screening results?" This requires:

1. Detecting relevant changes in the database: new designations, updated profiles, delistings, entity merges.
2. Re-screening counterparties against those changes.
3. Comparing new results to previous results to separate genuinely new alerts from matches already reviewed.

The challenge is doing this efficiently without generating excessive noise.

Building your alert state model#

Regardless of which implementation path you choose, you need to maintain state about screening matches over time. This section describes how that state is typically structured, how to keep routine data updates from overwhelming analyst review, and how to reason about when a change to an entity should raise a new alert.

Designing the alerting model#

Start from the compliance regime you operate under. Monitoring workflows differ in three ways: the regulatory exposure of the business (OFAC-only, EU FSF, UK OFSI, or a combination), its risk appetite, and the product the data is embedded in. Onboarding screening has different ergonomics than ongoing monitoring. The patterns below reflect what tends to work in practice; adapt them to where you sit on these axes.

Persist resolution decisions so they survive routine data updates. An entity you have resolved, whether as a match or a mismatch, should not re-enter triage every time a MOD arrives. Store the decision against the canonical ID; on each update, reconcile stored IDs against the incoming id and referents fields as described in the identifiers documentation. The /entities endpoint is free, which makes it cheap to re-check a stored ID's current risk picture without running a full re-screen.

Filter non-material MOD operations before escalating them. Distinguishing source-driven from pipeline-driven change is a shared problem: today the delta format does not tell you which of the two you are looking at. This is an area we are maturing on our side. In the meantime, the most effective filter is local. Diff the properties your matcher uses, track topics and programId changes, and watch for new Sanction objects. Most routine noise can be suppressed this way.

Alerting as two decision gates#

Split the alert decision into two gates: Resolution (is this really the right entity?) and Relevance (does it matter to your business?) A single alert-state field conflates two independent questions, and any data change then potentially re-opens the whole decision. Keeping them separate means each change can be routed to only the gate it affects.

Resolution. Is the counterparty in your portfolio actually the entity we returned?

• Confirmed match: you have enough evidence (identifier, date of birth, nationality, and alias convergence) to say these are the same real-world entity.
• Confirmed mismatch: you have enough evidence to say they are different entities. Persist this decision; it should not re-enter triage on entity-side change.
• Unresolved: the available data is not sufficient to decide. The match should re-enter triage when new data changes the resolution picture: a merge, a new source attribution, or a parser fix that surfaces a date of birth.

Some regulated contexts do not permit leaving a match unresolved; it must be treated as a confirmed match until disproven. Whether unresolved is a distinct state or collapses into confirmed match in your workflow is a policy question driven by your legal environment. Check your compliance requirements before designing around it.

Relevance. For confirmed matches (and for unresolved matches, under the policy above), does the entity's risk profile fall within the compliance scope you operate under? Three axes drive this gate:

• topics classify the kind of risk (sanction, role.pep, sanction.linked, reg.warn). A topic gain or escalation is a Relevance event.
• programId identifies the specific sanction programs the entity sits under. Evaluate each program against the programs and jurisdictions you monitor (see Re-alerting depends on your regulatory exposure below).
• Adjacent sanctions metadata records the authority, measures, start and end dates, and provisions of each designation via the nested Sanction objects returned by the nested entities API. A new Sanction on an already-matched entity is a new legal basis for the designation, even if the entity's own properties are unchanged.

Dataset membership serves as a coarser proxy for program-level tracking where finer granularity is overkill. Common Relevance outcomes include: in-scope, action required; in-scope, accepted with monitoring (a permitted counterparty that must still be watched for risk changes); out-of-scope (flagged by an authority the business is not subject to).

Persist the pair of outcomes, one per gate. Re-alerting rules then depend on which gate an incoming data change acts on.

What to store per match#

Re-alerting depends on your regulatory exposure#

A new designation only matters if it falls within the jurisdictions and programs you monitor. Relevance outcomes are relative to the compliance scope of the business using the data, not absolute properties of the entity.

A Person sanctioned only under OFAC is already a Relevance event for any business with OFAC in scope. When that person is later designated under EU FSF, a global bank that also monitors EU FSF records a new Relevance trigger; a US-only institution sees no change in exposure, since the entity was already flagged via OFAC. Store the programs and jurisdictions you monitor as configuration, and evaluate each incoming programId or dataset change against that configuration before escalating.

The axis cuts the other way too: a delisting in one jurisdiction does not close an alert if the entity is still designated under another program you monitor.

When to re-alert on previously-flagged entities#

Not every change to a matched entity warrants a new alert. Route each change by the gate it acts on:

• Changes that act on Resolution. A merge brings in a new identifier or date of birth that resolves or contradicts an earlier decision; a split separates a previously-merged entity. Re-run Resolution for unresolved matches and (if your policy allows) confirmed matches. Confirmed mismatches do not re-enter unless your counterparty data has changed.
• Changes that act on Relevance. A topic escalation (a role.pep entity gaining sanction, or a new sanction.linked), a new programId, new adjacent Sanction metadata, or a delisting. Evaluate each against the programs and jurisdictions you monitor. Only in-scope changes produce an alert.
• Changes that affect both gates. Merges can simultaneously resolve a previously-unresolved match and shift its Relevance picture (new source, new topics). Re-run both gates.
• Score drift. A material move in the match score, in either direction, is a signal to re-check Resolution even on a previously-confirmed match.

For confirmed matches, the /entities endpoint is the cheapest way to refresh the risk picture: polling the canonical IDs you have stored surfaces Relevance changes (new topics, new programs, new sanctions metadata) without running a full re-screen, and it does not count against your API quota.

Handling merges and deduplication#

Entity IDs are not permanent. Deduplication merges records from different sources into a single profile, and when it does, the entity's canonical ID can change; previously-merged entities can also be split. The identifiers documentation covers the mechanics: source IDs, canonical IDs, the referents list, and the six-month expiration window.

For a monitoring workflow, merges create three specific problems:

• Phantom new matches. A merge produces a new canonical ID. If you track matches by entity ID alone, the merged entity appears as a brand-new hit even though you already reviewed one of its constituents.
• Duplicate alert convergence. You may hold separate alerts for two entities that were later merged into one, and need to consolidate them.
• Score drift. A merge can add properties from a second source (a date of birth, an identifier) that shift the match score in either direction.

All three are detectable by checking each incoming match's referents against your stored entity IDs. If a referent matches a stored ID, the entity was merged rather than newly added. When you detect a merge:

1. Find all affected alerts. Search your alert store for matches referencing any of the merged IDs. There may be more than one if the entity was previously fragmented across sources.
2. Update the entity ID. Replace the old IDs with the new canonical ID so future lookups work correctly.
3. Consolidate duplicate alerts. If a single counterparty had separate alerts for two fragments, consolidate them. Keep the more conservative resolution: if one was a confirmed match and the other a confirmed mismatch, keep the confirmed match.
4. Re-run Resolution. The merged entity may have different properties than either fragment alone, which can shift the match score or flip a previously-unresolved match into confirmed territory. Re-evaluate the resolution decision.
5. Re-run Relevance. The merged entity carries topics, datasets, and programId values from both fragments. If the combination introduces new risk factors (one fragment was PEP-only, the other sanctioned), the risk picture has changed and this is a re-alert event.

Understanding how data changes#

Three sources of change#

Changes in the database originate from three distinct causes:

• Source-level changes. The upstream publisher (a government authority, for instance) added, modified, or removed an entity. A new OFAC designation, an EU delisting, a corrected date of birth. These are the changes most relevant to monitoring.
• Pipeline-driven changes. Our data pipeline was updated: an improved parser, better transliteration, additional contextual properties. The source data did not change, but our representation of it did. These can be large in volume. An update to our Wikidata crawler, for example, can touch hundreds of thousands of entities at once.
• Deduplication events. Two entity records were identified as the same real-world entity and merged into one profile, or a previous merge was corrected and an entity was split. Merges change entity IDs and can alter properties and match scores. See Handling merges and deduplication above for how to absorb these in an alert store.

Distinguishing source-driven from pipeline-driven changes is a problem we are still maturing. A MOD operation in a delta file looks the same whether the underlying government data changed or a parser was improved, and the last_change field does not separate these causes either. Reducing the noise this generates is a shared effort: we are refining the signals we expose; the workarounds below give you a pragmatic way to filter the majority of non-material MOD operations on your side today.

The noise problem#

Two patterns generate disproportionate noise:

• Large pipeline-driven updates. When we improve a crawler or data model, hundreds of thousands of entities can appear as "modified" even though nothing changed at source. You can reduce the impact by comparing the properties your matching system (names, aliases, dates of birth, nationalities, and identifiers) and relevance decisions (programId, topics, etc) use between the previous and current versions of a modified entity. If none of those changed, the modification is unlikely to affect screening results.
• last_change limitations. This field reflects any change to entity data, including pipeline-driven changes. It does not reflect deduplication events: merges do not bump last_change. At the dataset level, last_change can regress to an earlier date when entities are removed. Treat it as a rough filter, not an authoritative signal. See What does "last change" indicate?.

Choosing your approach#

Two strategies exist for monitoring, and you can combine them.

API-based re-screening calls the /match endpoint on a schedule and compares results to the previous run. It is simpler to implement and does not require managing bulk data updates locally. Best for customers using the hosted API or on-premise yente who want a straightforward integration. You still need to maintain identifiers along with Resolution and Relevance decisions.

Delta-based monitoring consumes incremental change files and re-screens only the entities that changed. It is more efficient at scale and gives finer control over which changes to act on. Best for bulk data customers who already manage a local copy of the dataset and have the engineering capacity for a delta processing pipeline.

A hybrid approach is often the most practical: use deltas to identify changed entities between runs, and do a periodic full re-screen (weekly or monthly) as a safety net.

API-based monitoring#

Scheduled re-screening workflow#

1. Screen your full portfolio using /match with appropriate scope filters.
2. Store the results in your own database: for each counterparty, record the matched entity IDs, scores, and the screening timestamp.
3. Re-screen on a schedule (daily, or more frequently for high-risk counterparties) and compare new results against stored results:
   • New matches: entities that now appear above your score threshold but did not before.
   • Changed scores: entities whose match score moved significantly in either direction.
   • Disappeared matches: entities that previously matched but no longer appear (possible delisting or data correction).
4. Make alerting decisions.

Reducing noise with scope filtering#

Use tight scope filtering. Every entity in scope is a candidate for re-screening on every run. The scope filtering guide covers all available mechanisms: collections, datasets, risk topics, and entity types. Narrowing the collection from default to sanctions alone can reduce the candidate set by an order of magnitude.

The changed_since filter. This parameter restricts results to entities whose last_change timestamp falls after a given date. Identify which entities changed since the last run, then re-screen only counterparties that previously matched or narrowly missed matching those entities. See the noise problem for caveats on last_change.

Now determine whether the changed fields imply alerting.

Delta-based monitoring#

Processing workflow#

The delta update mechanism provides incremental change files with ADD, MOD, and DEL operations between consecutive published versions. Read that documentation first for format details, access mechanism, and version tracking.

A typical workflow:

1. Load the full dataset into your screening system and run an initial full screen. Store the results.
2. Poll for new versions via the dataset's index.json metadata (see keeping data fresh).
3. Process each delta operation against your stored screening results:
   • ADD: Screen the new entity against your full counterparty portfolio.
   • MOD: Re-screen against counterparties that previously matched (or narrowly missed matching) this entity.
   • DEL: Mark any active alerts referencing this entity ID for review. The entity was removed or merged.
4. Detect merges: A merge could be between two entities, each with an original source ID, resulting in two DEL and one ADD operation for the newly minted canonical ID. Alternatively, one entity could be merged into another with a canonical ID, resulting in a DEL of the merged entity and a MOD of the receiving entity. In both cases, the IDs of the merged (DEL) entities end up as referents in the resulting (ADD or MOD) entity. Update stored entity IDs and consolidate duplicate alerts (see Handling merges and deduplication).
5. Make alerting decisions.

Reducing noise with property comparison#

When processing a MOD, compare the previous and current entity to assess whether the change is material:

• Extract the properties your matching system uses as scoring features: names, aliases, dates of birth, nationalities, identifiers (tax numbers, IMO numbers, and ISINs), and gender.
• If none of these changed, the modification is unlikely to produce different screening results. Skip re-screening for that entity.
• Also check topics and datasets. A change in risk classification or data source attribution is meaningful even when matching properties are identical.

This requires storing the previous version of each entity locally, which adds overhead but dramatically reduces re-screening volume during large pipeline-driven updates.

If you do not use FtM format#

Deltas are only available for the FollowTheMoney entity format. If you consume CSV or nested JSON, either adopt FtM for your delta pipeline (recommended) or diff successive full exports yourself. The latter works but is more resource-intensive and loses the explicit operation semantics.

Periodic full re-screening remains necessary#

Delta-based monitoring is more efficient, but it does not fully replace periodic full re-screening:

• Score drift from enrichment. Entity merges can add properties that change match scores. A full re-screen catches cases where a previously below-threshold entity crossed into alerting range, which an incremental check might miss if the merge produced a DEL/ADD pair for an entity you were not previously tracking.
• Threshold or scope changes. If you adjust your matching threshold or screening scope, a full re-screen catches matches that were previously filtered out.
• Safety net. A weekly or monthly full re-screen guards against edge cases in delta processing: a missed delta file, a bug in incremental update logic.

What we are building#

We want to be transparent about the direction of this work. Two principles guide our approach:

We do not plan to build a case management tool. Alert triage, analyst assignment, audit trails, and compliance workflow management are better served by dedicated tools. Our goal is to provide data primitives that make those tools effective, not to compete with them.

We strongly prefer not to store customer data. Most monitoring products require you to upload your counterparty portfolio, where it is stored and re-screened periodically. This creates data processor obligations, privacy risk, and operational complexity on both sides. We are exploring approaches that make monitoring viable without requiring us to hold your data, including clearer change metadata and higher-throughput screening endpoints designed for repeated use. This is an active area of work; please share what would be most useful for your use case.

We will update this guide as new capabilities ship. If you have specific monitoring requirements or encounter challenges with the workflows described here, reach out to our support team.