Test List
This table lists all implemented tests. Note that not all tests have its own unique ID. Many are run over all columns in a table dynamically.
id |
table |
column |
description |
severity |
interpretation |
P049 |
All |
is_entity_deleted |
When is_entity_deleted = True, all non-key fields should be left unset or NULL. |
INFO |
When matching_rows > 0, it is good practice to avoid populating fields where is_entity_deleted = True to make it clear that these fields should not be populated. |
P050 |
All |
is_entity_deleted |
More than 40% of entities have is_entity_deleted = TRUE. |
WARN |
When count > 0, you should verify is_entity_deleted is correctly mapped. More than 40% of entities being deleted is a high number. |
P052 |
All |
All |
More than 10% of transactions have book_time with a round hour or half hour except 00:00:00. Common indicator of a datetime mapping from a source system in a different timezone e.g. transactions book_time=23:30:00 could be assigned to the wrong day. |
WARN |
When proportion > 0.10, you should verify timestamps are being correctly mapped. Why are there so many timestamps which are not at midnight but are at exactly a round hour or half hour? |
P057 |
All |
validity_start_time |
An entity has more than 500 validity_start_time changes. |
WARN |
When count > 0, you should verify the number of changes to an entity being passed to the API is correct. Why are there so many changes for these entities? |
P058 |
All |
validity_start_time |
An entity has more than 10000 validity_start_time changes. |
ERROR |
When count > 0, you should verify the number of changes to an entity being passed to the API is correct. Why are there so many changes for these entities? |
P001 |
All |
validity_start_time |
More than 1% of the fields in this table have the same validity_start_time. |
WARN |
When count > 0, you should verify the validity_start_time is correctly mapped. why is there a very common value? |
P002 |
Party |
source_system |
Excessive distinct values of source_system. Warns above 500. |
WARN |
When count >= 500, check source system is not populated using party data. Why are there so many distinct values of source_system? |
P003 |
Party |
birth_date |
WHEN type=’CONSUMER’, check for more than 1% of parties with the same birth_date. |
WARN |
When count > 0, verify birth_date is correctly mapped. Why are there so many consumer parties with the same value of birth_date? |
P004 |
Party |
establishment_date |
WHEN type=’COMPANY’, check for more than 1% of parties with the same establishment_date. |
WARN |
When count > 0, verify birth_date is correctly mapped. Why are there so many company parties with the same value of establishment_date? |
P005 |
Party |
occupation |
WHEN type=’CONSUMER’, check for more than 10% of parties with the same occupation. |
WARN |
When count > 0, verify birth_date is correctly mapped. Why are there so many consumer parties with the same value of occupation? |
P006 |
Party |
nationalities |
WHEN type=’CONSUMER’, more than 5% of parties have no nationality |
WARN |
When proportion >= 0.05, verify the intention not to populate nationality. Why do so many parties have no nationality? |
P007 |
Party |
nationalities.region_code |
Check for parties with more than 5 nationalities |
WARN |
When count >= 5, verify nationalities are populated as intended. More than 5 nationalities for a single party is unusual. |
P008 |
Party |
residencies |
WHEN type=’CONSUMER’, more than 5% of parties have no residencies |
WARN |
When proportion >= 0.05, verify the intention not to populate nationality |
P009 |
Party |
residencies.region_code |
A party has more than 5 residencies. |
WARN |
When count >= 5, verify residencies are populated as intended. More than 5 residencies for a single party is unusual. |
P010 |
Party |
exit_date |
Check for more than 5% of parties with the same exit_date |
WARN |
When proportion >= 0.05, verify exit_date is correctly mapped. |
P011 |
Party |
join_date |
Check for more than 5% of parties with the same join_date. Warning. |
WARN |
When proportion >= 0.05, verify exit_date is correctly mapped. |
P012 |
Party |
civil_status_code |
Check for more than 75% of parties with the same civil_status_code. Warning. |
WARN |
When count >= 1, check civil_status_code is correctly mapped as intended. |
P013 |
Party |
education_level_code |
Check for more than 75% of parties with the same education_level_code. Warning. |
WARN |
When count >= 1, check education_level_code is correctly mapped as intended. |
P015 |
AccountPartyLink |
account_id, party_id |
Check for accounts linked to a erroneously high number of parties. The API will not ingest more than 60000 links per account. |
ERROR |
When count >= 1, check the account_party_link table. Why are there so many links for a single account? |
P016 |
AccountPartyLink |
account_id, party_id |
Check for accounts linked to a erroneously high number of parties. Warning only. |
WARN |
When count >= 1, check the account_party_link table. Why are there so many links for a single account? |
P017 |
AccountPartyLink |
account_id, party_id |
Check for parties linked to a erroneously high number of accounts. The API will not ingest more than 60000 links per account. |
ERROR |
When count >= 1, check the account_party_link table. Why are there so many links for a single party? |
P018 |
AccountPartyLink |
account_id, party_id |
Check for parties linked to a erroneously high number of accounts. Warning only. |
WARN |
When count >= 1, check the account_party_link table. Why are there so many links for a single party? |
P019 |
AccountPartyLink |
source_system |
Excessive distinct values of source_system. Warns above 500. |
WARN |
When count >= 500, check source system is not populated using party data. Why are there so many distinct values of source_system? |
P021 |
Transaction |
source_system |
Excessive distinct values of source_system. Warns above 500. |
WARN |
When count >= 500, check source system is not populated using party data. Why are there so many distinct values of source_system? |
P022-CASH |
Transaction |
type |
CASH type is completely missing from the transactions. |
WARN |
When matching_rows = 0, verify the lack of CASH transactions. Why are there no CASH transactions? |
P022-WIRE |
Transaction |
type |
WIRE type is completely missing from the transactions. |
WARN |
When matching_rows = 0, verify the lack of WIRE transactions. Why are there no WIRE transactions? |
P022-CARD |
Transaction |
type |
CARD type is completely missing from the transactions. |
WARN |
When matching_rows = 0, verify the lack of CARD transactions. Why are there no CARD transactions? |
P022-CHECK |
Transaction |
type |
CHECK type is completely missing from the transactions. |
WARN |
When matching_rows = 0, verify the lack of CHECK transactions. Why are there no CHECK transactions? |
P023-CREDIT |
Transaction |
direction |
CREDIT transaction direction is completely missing across the entire dataset. |
WARN |
When matching_rows = 0, verify the lack of credit transactions. Why are there no credit transactions? |
P023-DEBIT |
Transaction |
direction |
DEBIT transaction direction is completely missing across the entire dataset. |
WARN |
When matching_rows = 0, verify the lack of credit transactions. Why are there no credit transactions? |
P024 |
Transaction |
direction |
Check the overall ratio of CREDITs to DEBITs by count of transaction_id in this dataset. Warns if the ratio of one direction is more than 80%. |
INFO |
When proportion > 0.8 or proportion < 0.2, verify the the imbalanced ratio. Why is there a significant imbalance? |
P025 |
Transaction |
direction |
Check the overall ratio of CREDITs to DEBITs by count of transaction_id in this dataset. ERRORs if the ratio of one direction is more than 95%. |
WARN |
When proportion > 0.95 or proportion < 0.05, verify the the imbalanced ratio. Why is there a significant imbalance? |
P026 |
Transaction |
account_id |
One or more accounts have a very high number of transactions. Errors at 50 million. |
WARN |
When count > 0, verify the account mapping. Why does a single account have so many transactions? |
P027 |
Transaction |
account_id |
One or more accounts have a very high number of transactions. Warn at 10 million. |
INFO |
When count > 0, verify the account mapping. Why does a single account have so many transactions? |
P028 |
Transaction |
account_id, counterparty_account |
A single account and counterparty combination have many transactions. Errors at 5 million. |
WARN |
When count > 0, verify the transaction mapping. Why does a single account-counterparty pair have so many transactions? |
P029 |
Transaction |
account_id, counterparty_account |
Account and Counterparty pairs have many transactions. Warns at 1 million. |
WARN |
When count > 0, verify the transaction mapping. Why does a single account-counterparty pair have so many transactions? |
P030 |
AccountPartyLink |
account_id |
More than 50% of accounts in the dataset have no transactions at all. |
WARN |
When proportion > 0.5, verify the accounts have all their transactions. Why are there so many accounts with no transactions? |
P031 |
AccountPartyLink |
account_id |
More than 20% of accounts in the dataset have no transactions at all. |
INFO |
When proportion > 0.2, verify the accounts have all their transactions. Why are there so many accounts with no transactions? |
P032 |
Transaction |
book_time |
>= 1 month has a transaction volume of less than 40% of the monthly average by volume. |
INFO |
When count > 0, verify transactions are complete for all months. Why is there a substantial difference in transaction volumes across months? |
P033 |
Transaction |
book_time |
>= 1 month has a transaction volume of less than 66% the monthly average by volume. |
WARN |
When count > 0, verify transactions are complete for all months. Why is there a substantial difference in transaction volumes between months? |
P034 |
Transaction |
book_time |
More than 1% of transactions have the same book_time. |
INFO |
When count > 0, verify book_time mapping. Why do so many transactions have the same book_time? |
P051 |
Transaction |
type |
5% of transactions have the same value across any transaction type |
INFO |
When count > 0, verify nanos and units mapping. Why do so many transactions have the same transaction value? |
P035 |
PartySupplementaryData |
party_supplementary_data_id |
Same number of party_supplementary_data_id for each customer |
ERROR |
|
P036 |
PartySupplementaryData |
party_supplementary_data_id |
LTE 100 supplementary_data_id for each customer |
ERROR |
When count > 0, verify there are less than 100 party_supplementary_data_ids for each customer. |
P037 |
PartySupplementaryData |
source_system |
Excessive distinct values of source_system. Warns above 500. |
WARN |
When count >= 500, check source system is not populated using party data. Why are there so many distinct values of source_system? |
P038 |
RiskCaseEvent |
event_time |
Many risk case events have the same event_time |
WARN |
When count > 0, why are there so many risk_case_events with the same event_time? |
P039 |
RiskCaseEvent |
type |
Check >=1 AML_PROCESS_START in table |
ERROR |
If matching_rows = 0, ensure there is at least one party with a AML_PROCESS_START for the model to learn from. |
P040 |
RiskCaseEvent |
type |
Check >=1 AML_PROCESS_END in table |
ERROR |
If matching_rows = 0, ensure there is at least one party with a AML_PROCESS_END for the model to learn from. |
P041 |
RiskCaseEvent |
type |
Check >=1 AML_EXIT in table |
ERROR |
If matching_rows = 0, ensure there is at least one party with a EXIT for the model to learn from. |
P042 |
RiskCaseEvent |
type |
Check not all parties have an AML_EXIT |
ERROR |
If proportion = 1, why do all parties have an AML_EXIT? |
P043 |
RiskCaseEvent |
type |
Check >=1 party with AML_SAR |
ERROR |
If matching_rows = 0, ensure there is at least one party with a SAR for the model to learn from. |
P044 |
RiskCaseEvent |
type |
Check not all parties have an AML_SAR |
ERROR |
If proportion = 1, why do all parties have an AML_SAR? |
P045 |
RiskCaseEvent |
type |
<=1 AML_PROCESS_START per risk_case_id and party_id |
ERROR |
If count > 0, there should only at most 1 AML_PROCESS_START for each risk_case_id and party_id |
P046 |
RiskCaseEvent |
type |
<=1 AML_PROCESS_END per risk_case_id and party_id |
ERROR |
If count > 0, there should only at most 1 AML_PROCESS_END for each risk_case_id and party_id |
P047 |
RiskCaseEvent |
type |
<=1 AML_EXIT per risk_case_id and party_id |
ERROR |
If count > 0, there should only at most 1 AML_EXIT for each risk_case_id and party_id |
P048 |
RiskCaseEvent |
type |
All AML_EXIT events have AML_PROCESS_START for same risk_case_id and party_id |
ERROR |
If count > 0, ensure all cases with an AML_EXIT have an AML_PROCESS_START record for the same party_id and risk_case_id. |
P068 |
RiskCaseEvent |
type |
All AML_SAR events have AML_PROCESS_START for same risk_case_id and party_id |
ERROR |
If count > 0, ensure all cases with an AML_SAR have an AML_PROCESS_START record |
P067 |
RiskCaseEvent |
type |
All AML_SAR and AML_EXIT events have AML_PROCESS_START for same risk_case_id and party_id |
ERROR |
If proportion < 1, ensure that all AML_SAR and AML_EXITS have a AML_PROCESS_START for the model to learn from |
P053 |
RiskCaseEvent |
risk_case_event_id |
More than 5000 risk case events for a single case. |
WARN |
When count > 5000, why are there so many risk case events for a single case? |
P054 |
RiskCaseEvent |
risk_case_event_id |
More than 1000 risk case events for a single case. |
INFO |
When count > 1000, why are there so many risk case events for a single case? |
P055 |
RiskCaseEvent |
risk_case_event_id |
More than 1000 risk case events for a single party. |
INFO |
When count > 1000, why are there so many risk case events for a single case? |
P056 |
RiskCaseEvent |
risk_case_event_id |
More than 5000 risk case events for a single party. |
WARN |
When count > 5000, why are there so many risk case events for a single case? |
P059 |
Transaction, RiskCaseEvent |
book_time,type |
Finds parties with risk events (AML_EXIT or AML_SAR) with no transactions within the specified activity period or for 365 days prior to AML PROCESS START |
ERROR |
When count > 0, why are there AML_EXIT or AML_SAR events without any associated transactional activity prior to the event for the party? |
P060 |
RiskCaseEvent |
type |
>1 suspicious activity period start for party in risk case |
ERROR |
When count > 1, fix for only one suspicious activity period start per case |
P061 |
RiskCaseEvent |
type |
>1 suspicious activity period end for party in risk case |
ERROR |
When count > 1, fix for only one suspicious activity period end per case |
P062 |
RiskCaseEvent |
type |
Suspicious activity period end but no start for party in risk case |
WARN |
|
P066 |
RiskCaseEvent |
type |
Party with multiple AML_EXIT events |
WARN |
|
V002 |
Party |
nationalities |
WHEN type=’COMPANY’, check no nationalities specified |
TODO |
|
V003 |
Party |
residencies |
WHEN type=’COMPANY’, check no residencies specified |
TODO |
|
V004 |
Party |
assets_value_range.start_amount.units |
Check all are positive or zero |
ERROR |
|
V005 |
Party |
assets_value_range.start_amount.nanos |
All values between 0 and +999,999,999 inclusive |
ERROR |
|
V006 |
Party |
assets_value_range.end_amount.units |
Check all are positive or zero |
ERROR |
|
V007 |
Party |
assets_value_range.end_amount.nanos |
All values between 0 and +999,999,999 inclusive |
ERROR |
|
V008 |
Party |
birth_date |
WHEN type=’COMPANY’, check no birth_date specified |
ERROR |
If count > 0, why are the companies with a birth_date specified? |
V009 |
Party |
establishment_date |
WHEN type=’CONSUMER’, check no establishment_date specified |
ERROR |
If count > 0, why are the consumers with an establishment_date specified? |
V010 |
Party |
occupation |
WHEN type=’COMPANY’, check no occupation specified |
ERROR |
If count > 0, why are there companies with an occupation specified? |
V011 |
Party |
gender |
WHEN type=’COMPANY’, check no gender specified |
ERROR |
If count > 0, why are there companies with a gender specified? |
V012 |
Party |
assets_value_range.start_amount |
If non-empty check currency code, units and nanos all not null |
TODO |
|
V013 |
Party |
assets_value_range.end_amount |
If non-empty check currency code, units and nanos all not null |
TODO |
|
V014 |
Transaction |
normalized_booked_amount |
If non-empty check currency code, units and nanos all not null |
TODO |
|
V015 |
Transaction |
normalized_booked_amount.units |
Check all are positive or zero |
ERROR |
Both nanos and units must be positive or zero |
V016 |
Transaction |
normalized_booked_amount.nanos |
All values between 0 and +999,999,999 inclusive |
ERROR |
|
V017 |
Transaction |
normalized_booked_amount.currency_code |
All currency codes must be a single normalized currency across the entire dataset |
ERROR |
|
DT001 |
All |
validity_start_time |
All values are before the interval end date (if provided) and before today’s date |
ERROR |
If matching_rows > 0, fix validity_start_time after today |
DT002 |
Party |
birth_date |
All values are before the interval end date (if provided) and before today’s date |
ERROR |
If matching_rows > 0, fix validity_start_time after today |
DT003 |
Party |
establishment_date |
All values are before the interval end date (if provided) and before today’s date |
ERROR |
If matching_rows > 0, fix validity_start_time after today |
DT004 |
Party |
exit_date |
All values are before the interval end date (if provided) |
ERROR |
If matching_rows > 0, fix validity_start_time after today |
DT005 |
Party |
join_date |
All values are before the interval end date (if provided) and before today’s date |
ERROR |
If matching_rows > 0, fix validity_start_time after today |
DT008 |
Transaction |
book_time |
All values are before the interval end date (if provided) and before today’s date |
ERROR |
If matching_rows > 0, fix validity_start_time after today |
DT011 |
RiskCaseEvent |
event_time |
All values are before the interval end date (if provided) and before today’s date |
ERROR |
If matching_rows > 0, fix validity_start_time after today |
DT012 |
Party |
join_date |
Join date before establishment date for party |
WARN |
If matching_rows > 0, review matching parties. Why are there parties who joined the bank prior to their establishment? |
DT013 |
Party |
exit_date |
Exit date after birth date for party |
WARN |
If matching_rows > 0, review matching parties. Why are there parties who joined the bank prior to their birth? |
DT014 |
RiskCaseEvent |
event_time |
Validates order of risk case events: AML_PROCESS_START, AML_EXIT, AML_PROCESS_END occur in exactly this order for each risk_case_id, party_id |
ERROR |
if matching_rows > 0, review matching cases. The API expects events to exactly match AML_PROCESS_START, AML_EXIT, AML_PROCESS_END. There must be an AML_PROCESS_START/END, but AML_EXIT is not required, but must happen before or at the same time as AML_PROCESS_END |
DT017 |
Party |
validity_start_time |
Earliest validity_start_time for party is after the join_date |
WARN |
If matching_rows > 0, review matching parties. Why are there parties with a validity_start_time before they joined the bank? |
DT018 |
Party |
exit_date |
Join date after exit date for party |
WARN |
If matching_rows > 0, review matching parties. Why are there parties who joined the bank prior to their exit? |
C001 |
All |
All non-nullable |
No nulls for non-nullable columns |
ERROR |
|
C002 |
All |
All non-nullable strings |
No empty strings in non-nullable fields |
WARN |
|
RI001 |
AccountPartyLink |
party_id |
All party_id in AccountPartyLink exist in Party |
ERROR |
|
RI002 |
Party |
party_id |
All party_id in Party exist in AccountPartyLink |
ERROR |
|
RI003 |
AccountPartyLink |
account_id |
All account_id in AccountPartyLink exist in Transaction |
ERROR |
|
RI004 |
Transaction |
account_id |
All account_id in Transaction exist in AccountPartyLink |
ERROR |
|
RI005 |
RiskCaseEvent |
party_id |
All party_id in RiskCaseEvent exist in Party |
ERROR |
|
RI006 |
PartySupplementaryData |
party_id |
All party_id in PartySupplementaryData (if supplied) exist in Party |
ERROR |
|
RI007 |
Party |
party_id |
All party_id in Party exist in PartySupplementaryData (if supplied) |
ERROR |
|
RI009 |
Party |
party_id |
All party_id in Party exist in AccountPartyLink with valid time ranges. No inconsistencies with validity_start_time or is_entity_deleted detected |
ERROR |
|
RI011 |
Transaction |
account_id |
All account_id in Transaction exist in AccountPartyLink with valid time ranges. No inconsistencies with validity_start_time or is_entity_deleted detected |
ERROR |
|
RI012 |
RiskCaseEvent |
party_id |
All party_id in RiskCaseEvent exist in Party with valid time ranges. No inconsistencies with validity_start_time or is_entity_deleted detected |
ERROR |
|
RI013 |
Party |
party_id |
All party_id in Party exist in PartySupplementaryData with valid time ranges. No inconsistencies with validity_start_time or is_entity_deleted detected |
ERROR |
|
RI014 |
PartySupplementaryData |
party_id |
All party_id in PartySupplementaryData exist in Party with valid time ranges. No inconsistencies with validity_start_time or is_entity_deleted detected |
ERROR |
|
E001 |
Party |
type |
Field is constrained to permitted enumerations |
ERROR |
|
E002 |
Party |
civil_status_code |
Field is constrained to permitted enumerations |
ERROR |
|
E003 |
Party |
education_level_code |
Field is constrained to permitted enumerations |
ERROR |
|
E004 |
AccountPartyLink |
role |
Field is constrained to permitted enumerations |
ERROR |
|
E005 |
Transaction |
type |
Field is constrained to permitted enumerations |
ERROR |
|
E006 |
Transaction |
direction |
Field is constrained to permitted enumerations |
ERROR |
|
E007 |
RiskCaseEvent |
type |
Field is constrained to permitted enumerations |
ERROR |
|
FMT001 |
All |
*.currency_code |
All entries are valid three-letter currency code defined in ISO 4217 |
ERROR |
|
FMT003 |
Party |
residencies.region_code |
Valid two-letter unicode CIDR format |
ERROR |
|
FMT004 |
Party |
nationalities.region_code |
Valid two-letter unicode CIDR format |
ERROR |
|
FMT006 |
Transaction |
counterparty_account.region_code |
Valid two-letter unicode CIDR format |
ERROR |
|
PK001 |
Party |
party_id, validity_start_time |
check columns defined are a primary key on the table |
ERROR |
|
PK002 |
AccountPartyLink |
account_id, party_id, validity_start_time |
check columns defined are a primary key on the table |
ERROR |
|
PK003 |
Transaction |
transaction_id, validity_start_time |
check columns defined are a primary key on the table |
ERROR |
|
PK004 |
RiskCaseEvent |
risk_case_event_id |
check columns defined are a primary key on the table |
ERROR |
|
PK005 |
PartySupplementaryData |
party_supplementary_data_id, party_id, validity_start_time |
check columns defined are a primary key on the table |
ERROR |
|
F001 |
All |
check no additional fields exist on the table |
INFO |
||
F002 |
All except RiskCaseEvent |
is_entity_deleted |
check for consecutive is_entity_deleted values for the same PK |
WARN |
|
F003 |
All |
All non-nullable |
check column exists on table and is the correct type |
INFO |
|
F004 |
All |
All |
check column is the correct type on table. See https://amlaidatatests.readthedocs.io/en/latest/usage/interpreting-schematests.html |
ERROR |
|
F005 |
All |
check for entities which are immediately deleted |
WARN |
||
T001 |
All |
Check all tables are present and not empty, and contains less rows than threshold |
ERROR |
Note that not all tables are required. |