Will your sanctions screening systems withstand FCA scrutiny?

Under its Modular Assessment Proactive Programme (MAPP), the UK’s Financial Conduct Authority has already assessed over 90 firms to determine whether they are able to correctly identify the individuals, entities and vessels contained in the Office of Financial Sanctions Implementation (OFSI) consolidated list.

This involves the FCA providing a synthetic data set of 100,000 records containing known sanctioned names, and variations thereof, as part of its in-house developed Sanctions Screening Tool (SST). Firms are required to pass the data through their screening systems and report to the FCA how many of the records matched correctly.

The FCA is continuing to test firms under this programme and, with it regulating more than 40,000, it is critical that all these organisations are ready for scrutiny.

In this post, we set out the steps firms must take to be ready to respond to testing. For tech-savvy financial institutions, in-house testing of sanctions screening is relatively straightforward and can help to ensure you’re ready to respond when the FCA call.

1 - Generate test data

Synthetic data sets used across the industry for sanctions testing purposes typically contain:

• Exact names: All names contained on the OFSI list. There are around 4,500 individuals, entities and vessels on this list and approximately 18,500 records which contain their primary names and aliases.

• Variations of these names: A random sample of 15–20 variations of the exact names. Variations include:

  • Alternative names – e.g. Ellie and Eleanor

  • Transliteration equivalencies – e.g. Andrei and Andrzej

  • Typographic errors – e.g. Michael and Micheal

  • Homoglyphs – e.g. Oliver and 0liver

  • Punctuation differences – e.g. Cho’n and Chon

Firms must generate data sets containing these records to enable them to test their screening systems, not only against records contained directly on the OFSI list, but also variations of these records. Variations help to simulate data quality issues that may exist in the OFSI list or firms’ internal data sets which the screening system must attempt to reconcile to correctly identify possible matches. For example, where names are abbreviated, misspelled or are otherwise incomplete.

Alternatively, such data sets can be provided by third parties to expedite testing.

Malverde offer a free data set to assist firms to get started with penetration testing of their screening systems. This can be downloaded here.

2 - Map the test data to your screening systems

The FCA tests both name (customer) screening systems and payment screening systems. The records generated in Step 1 must be mapped accurately to the data schema of these systems to enable accurate matching.

It is important to carefully consider the fields used by your screening system rules to ensure that they can be tested effectively. For example, most name screening systems will consider a selection of full name or components of a name (forename, middle name, surname), date of birth, country (e.g. nationality, country of residence, country of birth, country of incorporation) and gender.

Payment screening systems may screen multiple message types – e.g. SWIFT, SEPA, Faster Payments Originating Overseas – and have a number of fields embedded within them – e.g. originator name, beneficiary name, Bank Identifier Codes (BICs), International Bank Account Number, passport number.

It is important to ensure that each of these fields are embedded in these customer or payment fields correctly. For example, some name screening systems utilise year of birth rather than date of birth, so the correct field format must be provided to ensure you give the system the best chance of matching them correctly.

Mapping synthetic data fields correctly is also a critical exercise when your firm is subject to an FCA sanctions-MAPP exercise. The data provided in the FCA’s test file may not be representative of the data used, and the format of that data, in business as usual (BAU) processes. It is important to consider this to avoid under-reporting or over-reporting the accuracy of your system.

3 - Execute the tests and extract the results

Most screening systems have simulation functionality to enable you to run a test file of records – whether real or synthetic data – through the screening configuration and model the results. The test data should be uploaded in a ‘production mirror’ environment to avoid inadvertently impacting live operations.

Once the alerts have been generated, the results must be extracted to enable them to be analysed effectively. In some systems, this can be done through the front-end user interface, but others will require a back-end database extract.

The extract must then be mapped back on to the original test data set using a unique identifier in order to determine whether each test record was identified correctly. Care must be taken in mapping the results back to the original tests as, whilst a match may have been successfully identified against a given test record, it may not be the correct one particularly considering how similar some of the names on the OFSI list are.

4 - Analyse

Once the results have been successfully mapped back to the test file, they are ready to be analysed. Separate match rates should be calculated for the ‘exact’ records, all ‘variation’ records, and each of the different test types within the variation records. This will provide a transparent view of how your screening system performs against each of the test types, including detailed insights into how effective your screening configuration is and areas where it needs to be improved.

5 - Improve your system configuration

Based on your analysis, it will be possible to identify areas of the system configuration performing effectively, and areas requiring improvement. For example, if alternative names are not performing well, then it may be that synonym and/or other proprietary algorithms provided by your screening vendor need to be enabled, or the settings adjusted. Challenges with typographic errors may indicate that spelling distance is not being utilised or the thresholds for such algorithms are not configured correctly.

It is critical that synthetic data testing is repeated after making changes to your system configuration, to model the impact on the results before promotion to operational use. This is also an ideal opportunity to ensure that your system configuration is appropriately documented with clear rationale for why particular settings have been chosen, so that it can be readily explained to regulators. This evidence needs to include why particular test cases, such as those included in sanctions MAPP, do or do not match, as not all test cases provided by regulators may be appropriate for your organisation or system configuration.

6 - Don’t forget PEP configurations

Frequently, the sanctions and politically exposed persons (PEPs) screening configurations are different for name screening systems, according to the perceived risk associated with these different types of persons of interest.

Whilst carrying out synthetic data testing of your sanctions configuration, it is an opportune time to ensure that your configuration for PEPs is also appropriate. Despite the perceived differences in risk, the personal data on PEPs is often not readily available in the public domain, and therefore the quality of the data available for screening is highly dependent on your third-party list provider. Similar penetration tests should be performed using PEP records to ensure you are able to identify them and manage the risk associated with them appropriately.

Be prepared!

Pre-emptive testing of screening system configurations is fundamental to ensuring you can be confident you have appropriate controls in place and that, if you are subject to the FCA’s sanctions-MAPP programme, you are ready to explain the results.