Rosetta Stone, British Museum — photograph by Cristian Bortes / CC BY 2.0
## The road not taken We’ve chosen to use a rule-driven approach so that results are explainable and consistent: the same input always yields the same output. While newer deep learning approaches (like the use of embeddings for name matching, explored in [project Eridu](https://github.com/Graphlet-AI/eridu)) are phenomenal at surfacing a wide set of match candidates, the technology is less suited to help discern false positives – the industry's main pain point. “A herd of 384-dimensional vectors told me so” is also an awkward explanation to share with a regulator who demands to understand a screening system. So if the system can’t have vibes, what can it have? We decided that the next best thing would be to provide the matcher with an expansive set of reference data – multi-lingual descriptions of many of the concepts in the domain of business names. ## How `logic-v2` works The core idea is to understand what each part of a name represents before we compare anything. We annotate tokens with **symbols** taken from curated reference data, such as organization types like `LLC`, generic business words like `Holding` or `Industries`, human names (`John`, `Smith`) or countries (`Palestine`), and so on. Each symbol collects multiple spellings and aliases across languages and scripts. This not only helps the engine understand that `business` and `бизнес` mean the same thing, but also allows us to de-prioritize this well-known term relative to more specific name parts (a human analyst would understand that `Sparkles Business Corporation` is really mostly `Sparkles`). Common filler terms in company names, such as `Holding` or `Industries`, are treated as soft stop words so that they can contribute less weight to the overall score compared to the distinctive core of the name. Once we’ve used symbols to explain and match most of a given name, we can be much stricter and precise in how we apply fuzzy string matching techniques to its remainder. For example: Comparing `Sparkles Business Corporation` vs. `Sporks Business Corporation` using basic Levenshtein scoring – the percentage of characters which need to be replaced to turn one text into another – produces an 89% match (alarm!). Tagging `Business` `Corporation` as generic symbols, however, the focus of the comparison reduces to `Sparkles` vs. `Sporks` – which errors out at a 50% match. Inversely, `Stripe LLC` and `Stripe, Limited Liability Corporation` is a hopeless challenge for text matching, but `Stripe [ORGTYPE:LLC]` produces a precise match. When it comes to individuals, name part awareness matters, too. First names only compare to first names, family names to family names, and middle initials can match with middle names. This reduces accidental near-matches between unrelated people. We've also decided to apply transliteration more selectively than in `logic-v1`. For language pairs where the romanization is reliable, it has proven very helpful. Other scripts tend to cause an unacceptable loss of precision from conversion, so we keep comparisons in the native script and rely on symbols to provide cross-script matches. ## What data powers the symbols It’s worth pointing out that while a lot of the reference data used in the new matcher is hand-curated (and open source, see [rigour data](https://rigour.followthemoney.tech/data/)) we have once again found invaluable help in [Wikidata](https://www.wikidata.org/wiki/Q4925477), the structured data twin of Wikipedia. Using various filters, we have picked over 161,000 items from its taxonomy of given and family names, with a total of 1.16 million different spellings. To illustrate the matching intelligence derived from this, it’s worth to show a single name symbol in full: `abd al-karim, abd al-karím, abd al-karīm, abd el-karím, abd-al-karim, abdalkarím, abdelkarím, abdu 'l-karím, abdul karim, abdulkarim, abdulkarím, abdülkerim, əbdülkərim, абд аль-карим, աբդլքարիմ, עבד אל-כרים, عبد الكريم, عبد الکریم, عبدالكريم, عبدالکریم, আবদুল করিম, アブドゥル・カリーム, 阿卜杜勒·卡里姆 => Q317304` We’re not in [SOUNDEX land](https://ofac.treasury.gov/faqs/249) any more. The same logic also applies to organization classes: `OAO`, `Joint Stock Company`, and `Aktiengesellschaft` are all normalized to a shared concept, which lets the matcher recognize them as potential synonyms regardless of language or abbreviation. ## Beyond Symbols: smarter name handling One of the biggest changes in `logic-v2` is how it treats name parts. In `logic-v1`, if you sent `firstName` and `lastName` separately, the system would simply concatenate them and run a full-string comparison. This often led to inflated similarity scores or missed matches. On top of symbolic matching, the new matcher also makes use of name part type annotations (a first and a family name will not be matched), supports middle initials (`Jesus H. Christ` matches `middleName:Helmut`). This means the matcher is *far more* precise when you provide structured inputs. For example: - Query: `{"firstName": "John", "middleName": "A", "lastName": "Smith"}` - Result: `John A. Smith` ranks much stronger than `John B. Smith` . We’ve also made name transliteration selective for languages where the result of romanization is of a suitable quality for producing strong matches. ## Explainability that travels well Besides the new screening logic, we’ve also re-worked the way in which we describe the results of each matching result. Until now, screening a person by name, date of birth and nationality would have returned an overall similarity score, and the individual weighted scores generated by the name matching, date matching and country matching functions. We’ve now expanded this further with small explanations, such as “Entities both linked to country X”, or “First names match, last names are a fuzzy 0.67 match.” The goal of these narrative results is to provide further transparency on how decisions are made, in particular in a manner that can be used as input to a generative AI in a case management system that will propose what steps an analyst needs to take to qualify a potential match. ## Using `logic-v2` in practice `logic-v2` ships with `yente 5.0`. You can enable the algorithm by adding `algorithm=logic-v2` to the `/match` API. Please note that the scores returned by `logic-v2` will naturally be different from those returned by `logic-v1`, and a different set of match alerts will result for adopting the new system. You can also supply a small set of configuration parameters by adding a `config` block to the request body (see `/algorithms` on the API for a description of the available parameters). This means you can fine-tune thresholds and weights to match your specific context, for instance if you want stricter cut-offs for sanctions screening or a broader tolerance for KYC. Using `logic-v2` , you should see fewer irrelevant hits on generic company names, cleaner cross-script person matches, and explanations that make it clear what to verify next. Because symbol tagging removes filler terms, fuzzy distance is only applied to the distinct core remaining part of the name, which is where precision matters most. ## Limitations and what’s next We believe that `logic-v2` will make for a significant improvement in the experience of API users: reduced false positives, better cross-alphabet matches, more explainability. It’s not [*revolutionary*](https://github.com/opensanctions/rigour/blob/759543820c96260685c8dfffd4c7b1885e0d2e24/resources/names/symbols.yml#L1606), but it is *much better*. What has us particularly excited: the reliance on extensive cultural reference data makes `logic-v2` a great target for continuous improvement. This is where we will come to rely on your kind cooperation: please share examples of false results – positive or negative – with our [support team](/support/) so that we can work together to fine-tune and evolve the system over time to make sure it meets the screening needs of a broad set of organizations. As always, we [welcome feedback and real-world cases](https://discuss.opensanctions.org/c/use-cases/7), as this is what will continue to allow us to make the matcher sharper for everyone.