How do I audit a sanctions screening vendor's false-negative posture before I buy?

Ask the vendor to reproduce a screen it could not complete, then read the output word for word. An honest screening tool returns three states, hit, clear, and unable-to-verify, and reports the third as pending rather than folding it into clean. FATF Recommendation 10 already treats failure to complete customer due diligence as a distinct outcome with its own consequences, not as a pass, so a tool that returns only pass or fail is transferring its coverage gaps onto your risk register and, ultimately, onto the MLRO who signed the file.

Most vendor demonstrations are built around the true-positive story: an obvious name match against the OFAC Specially Designated Nationals and Blocked Persons List, a red banner, a satisfying screenshot. That is the easy half of the problem. The half that generates enforcement exposure is the counterparty, vessel, or beneficial owner the tool failed to resolve and returned as clear. Below is a procurement test you can run inside a thirty-minute demo slot, and the specific answers that should disqualify a vendor.

Why "clean" has to be a positive assertion, not a default

Screening produces three logically distinct results.

Hit. The tool matched a name, vessel identifier, or ownership link to a designated party on a list it covers, with an auditable match rationale.

Clear. The tool completed the check against a defined list set, at a defined time, with sufficient identifiers to be confident the subject is not the designated party. Clear is a claim about coverage plus data sufficiency.

Unable-to-verify. The tool could not complete the check. The name arrived transliterated with no date of birth or registration number. The corporate registry extract is years stale. The vessel has no recent AIS broadcast. The intermediary presenting the cargo will not name its principal.

The defect is not that state three exists. State three always exists in physical commodity trade, where mandate chains are long, documents circulate in copy form, and the party who actually controls the cargo may be several removes from the entity on the LOI or the ICPO. The defect is collapsing state three into state two. The moment "we could not resolve this" is rendered on screen as "no matches found," the uncertainty stops being the vendor's data problem and becomes your undocumented risk acceptance. Nobody made a decision. Nobody escalated. The file simply reads clean.

FATF Recommendation 10 is the useful reference point here because it does not permit the collapse. Where CDD cannot be completed, including identification and verification of the beneficial owner, the standard directs the institution not to open the relationship or to terminate it, and to consider a suspicious transaction report. Uncertainty is an actionable state under the standard. A screening product that has no field for it is not aligned with the framework its buyers are examined against.

The single most diagnostic procurement question

Not "how many lists do you cover?" That question invites a number and tells you almost nothing, since list count is a marketing metric that rewards padding with low-value sources.

Ask instead: show me a screen you could not complete.

Then watch the screen. You are looking for four things.

1. Does the result state say pending, unresolved, or unable-to-verify, in the primary result field, not buried in a footnote or a confidence score the reviewer can ignore?
2. Does it name the specific reason the check stalled? "Insufficient identifiers for name resolution" is useful. "Low confidence" is not.
3. Does it say what would resolve it? A registration number, a passport identifier, an IMO number, a current registry extract, the identity of the disclosed principal behind the mandate.
4. Does the unresolved state persist in the record, or does it silently age into clear when nobody actions it?

If the vendor cannot produce such a screen, either the product does not distinguish the state, or the sales engineer has never seen it do so. Both are findings.

Four scenarios worth bringing to the demo

Bring your own test cases. Sanitised versions of real onboarding files work best, but constructed cases are fine as long as they exercise the failure modes that matter in crude and products flow.

Transliterated or non-Latin-script name. A counterparty name rendered from Arabic, Cyrillic, or Mandarin, supplied without a date of birth, registration number, or address. Ask how the tool decides between a fuzzy match and a non-match, and what it does when there are no secondary identifiers to break the tie. There is no correct answer other than escalation.

Stale ownership chain. A trading entity whose last filed shareholder register predates the current designation environment. Ask whether the tool distinguishes "no blocked owner found" from "ownership data is older than the relevant designation date." Those are not the same sentence.

Vessel with no recent AIS. A tanker with a gap in broadcast history. AIS carriage is mandated for the relevant classes of vessel on international voyages, but transmission can lawfully be interrupted, and gaps also correlate with the dark fleet behaviour described in publicly issued maritime sanctions advisories from OFAC and partner authorities. Ask what the tool asserts during a gap. Silence is not a clean position report.

Opaque intermediary in the mandate chain. An EN590 gasoil offer arriving through a mandate holder who declines to identify the seller until a DLC MT700 is in place. Ask whether the tool can screen a party it has not been given, and what the record says about the layer cake sitting between your institution and the ultimate controller. If the output is clear because the unnamed principal was never submitted, the tool has screened your paperwork, not your risk.

List coverage, refresh cadence, and the gap between refreshes

Now ask about lists, but ask precisely. Name the regimes you actually need documented and verify each against its public source: the OFAC SDN List and OFAC's other published lists maintained by the US Treasury, the EU consolidated list of persons and entities subject to restrictive measures, the UK OFSI Consolidated List of Financial Sanctions Targets, and the UN Security Council Consolidated List. For each one, require three answers in writing.

Which specific list, by its published name. How often the vendor ingests it, and whether that cadence is contractual or best-effort. What the product does between refreshes, when a designation has been published by the authority but not yet loaded.

That third answer is the one that matters. Authorities publish designations when they publish them, not on a schedule convenient to a data pipeline. A vendor that returns clear against a snapshot without disclosing the snapshot date and time on the output is asserting something it cannot support. The result record should carry the as-of timestamp of every list it screened against. If you cannot see that timestamp, you cannot defend the screen to an examiner, and neither can your MLRO.

Ownership thresholds, aggregation, and control

OFAC's published guidance on the Fifty Percent Rule holds that an entity owned in the aggregate, directly or indirectly, fifty percent or more by one or more blocked persons is itself blocked, whether or not it appears by name on the SDN List. Two operational questions follow.

Does the tool aggregate indirect holdings across multiple blocked persons and across multiple layers, or does it test each shareholder individually and stop at the first level of the structure? Individual testing will pass a layer cake assembled precisely to sit under the threshold at every single node.

And how does it handle control without majority ownership? OFAC has publicly cautioned that dealings with entities controlled by blocked persons carry risk even where the ownership threshold is not met. A screening product cannot resolve control from a shareholder table alone. What it can do is flag the structure as unresolved and tell the analyst why. Ask to see that output.

The audit trail test

Finally, ask for an export. Not the hit report, which every vendor will show you. Ask for the record of a check that did not complete, containing the subject as submitted, the identifiers that were missing, the lists screened and their as-of timestamps, the reason the check stalled, the analyst who reviewed the pending state, and the disposition. Then ask whether negative and pending results are retained on the same retention schedule as hits.

If pending states are not exportable, they are not evidence. If they are not retained, they did not happen as far as any future examination is concerned. And if the platform overwrites a pending state with clear on the next refresh without preserving the interim record, the audit trail is describing a decision nobody actually made.

If you want a walkthrough of how a three-state screening output reads in practice, including the pending-state export, request a research demo or subscribe to the OilFlow Intelligence briefing for the weekly sanctions typology note.

What compliance teams should do

1. Add a three-state requirement to the RFP. Hit, clear, unable-to-verify, with the third rendered in the primary result field and reason-coded.
2. Bring your own unresolvable test cases to every demo. Transliterated name with no secondary identifiers, stale ownership chain, AIS gap, undisclosed principal behind a mandate holder.
3. Disqualify on the output language. A tool that says "no matches found" where the honest answer is "could not resolve" has told you how it will behave on your worst file.
4. Require as-of timestamps for every list screened, plus a written answer on what happens between refreshes.
5. Test ownership aggregation against OFAC's published Fifty Percent Rule guidance, including multi-layer and multi-blocked-person structures, and ask how control-based risk is surfaced when the threshold is not met.
6. Confirm that pending and negative results are exportable and retained on the same schedule as hits.
7. Set the escalation path before go-live. An unable-to-verify state that has no owner and no clock is functionally identical to a clean screen, which is the outcome you were trying to avoid.

The procurement decision is not about which vendor catches the obvious SDN match. They all will, in the demo. The decision is about which vendor tells you, in writing and on the record, what it could not see.