I. Why AI Tax Enforcement Systems Are “High-Risk” Systems

In EU Member States, tax administrations are the public organs that make most use of artificial intelligence (AI) and machine learning (ML) systems to perform State prerogatives. Publicly-available data alone reveals at least 70 AI systems leveraged by national tax administrations, unequally spread over 18 EU Member States.¹ Even the EU itself, through Eurofisc members, has developed its own ML model: Transaction Network Analysis – a data matching model meant to detect missing trader intra-Community fraud.² Accordingly, in certain areas of taxation AI and ML are already used throughout the EU for the enforcement of taxation rules.

These AI tax enforcement systems perform a relatively broad range of tasks, reflecting the wide array of prerogatives of the administration itself. Generally, these different systems can be categorised into two archetypes. Some AI systems are leveraged by EU tax administrations for non-coercive purposes, including chatbots³, nudging systems⁴, and jurisprudence analysis⁵. These non-coercive systems constitute a minority of the models used by tax administrations in the EU, albeit a significant one.⁶ The remainder AI systems are leveraged for coercive purposes, i.e. for tax enforcement tasks such as web scraping⁷, the detection of statistical risk indicators⁸⁹, and risk scoring to screen and select taxpayers for audit.¹⁰ In a little more than a decade, predictive analysis has radically transformed tax enforcement and tax administrations in the EU. Currently, the use of statistics and ML underpins all coercive prerogatives when selecting a taxpayer for audit. Data is collected and processed through ML and taxpayers are algorithmically selected on the basis of risk indicators inferred from ML predictions. The transformative power of AI is also reflected in the human resources of tax administrations, increasingly composed of data scientists and increasingly less of tax law experts.¹¹

Some of these models were used by tax administrations in the EU as far back as 2004. This is for instance the case of XENON, a web scraping model leveraged by the Dutch tax administration (Belastingdienst).¹² This means that tax administrations were pioneering public algorithmic governance long before debates over other popular buzzwords in predictive policing, such as facial recognition, biometric surveillance, social scoring, etc. The primary reason for the prominence of the use of AI systems by tax administrations is the immense documentary burden placed on tax officials. Each year, tax administrations must process billions of documents¹³, answer millions of queries, and spend several millions of minutes on the phone.¹⁴ Processing such volumes of data manually with the human resources of national tax administrations is simply impossible. Accordingly, long before the advent of AI, tax administrations were already using traditional statistical approaches and heuristics to perform their fiscal prerogatives. The transition from traditional statistics to automated statistics and machine learning did thus not constitute a major scale-up.

1. The risks of AI tax enforcement systems

The EU AI Act follows a risk-based approach, meant to strike a proportional balance between the two policy goals of the instrument, namely: the promotion of innovation and the protection of citizens fundamental rights. Accordingly, the Regulation outlines four levels of risk ranging from prohibited to minimal risk. Minimal risk systems (level 1) generally escape the scope of the instrument aside from the invitation to self-regulation through codes of conduct and limited risk systems (level 2) are only bound to minimum transparency requirements in specific use cases, particularly chatbots and deep fakes. Models deemed as bearing unacceptable risk (level 4) are prohibited. By sheer number of articles, the majority of obligations in the instrument are imposed on high-risk systems (level 3). According to the current draft proposal, organisations with high-risk systems must comply with strict requirements such as certification, data governance, transparency, human oversight, record-keeping and cybersecurity. Comparatively to the other levels of risk, the obligations imposed on high-risk systems are numerous and substantively detailed, often requiring granular control of specific externalities. Hence, the risk-based approach seeks to ensure that obligations imposed on an AI system are proportional to the risks it generates.

In that regard, AI tax enforcement systems should be viewed as “high-risk” because these systems have been shown to contain various sources of conflict with EU citizens’ rights, documented in jurisprudence and doctrine. This is less true for non-coercive AI tax systems, in fact, some of these models are truly a net plus both for the administration and for taxpayers. Chatbots, for example, enable taxpayers to request information from the administration at any time of the day and year. Processing little to no taxpayer personal data¹⁵,these systems have opened up a new channel of communication with tax officials, while alleviating the substantial administrative burden of tax officials. Reports indicate that chatbots reduce the number of queries directly sent to the administration by a margin of up to 90%, with very high satisfaction rates amongst taxpayers.¹⁶ The same can be said of nudging, simply by adapting the language of default letters sent to taxpayers, e.g. referring to a taxpayer by his or her first name or by adding references to the benevolent purpose of tax collection, the speed and rate of compliance increase in noteworthy ways.¹⁷

Conversely, coercive AI tax systems used for tax enforcement bring about serious risk of conflict with taxpayers’ fundamental rights and tax procedure as a whole. These risks have already materialised in a number of cases. Coercive AI systems can conflict with the principle of legality because they disrupt procedures to such an extent that these no longer reflect procedural codes. For instance in eKasa¹⁸, the Slovak Constitutional Court ruled that machine-learning bolstered surveillance to such an extent that it required a specific framework and tailored safeguards to negate the risks of abuses. Currently, the majority of tax administrations in the EU use coercive AI systems without a specific legal basis to that effect and without safeguards to negate demonstrated risks of such systems.¹⁹ This is problematic in terms of legality as the different externalities these systems generate cannot be systematically captured by existing procedural rules.²⁰ Most notably, these systems entail risks of conflict with the right to a private life and right to data protection, as seen for example in SyRI²¹ or the State Council (Conseil d’Etat) on the use of web scraping²². The primary source of friction lies in the fact that tax administrations have adopted tools that increase their surveillance capability based on procedural rules that pre-date the internet. Through web scraping, tax administrations are capable of surveilling the internet, e-commerce platforms, social media, or satellite images without differentiation between compliant and non-compliant taxpayers. As these data processing activities are generally regarded as an administrative process, tax officials do not have to secure any form of prosecutorial assent to use web scraping systems and collect taxpayer personal data.²³ These tools collect bulks of data and match the data to the different taxpayers at a speed unrivaled by any human tax official, drastically increasing the scope of data collected and number of taxpayers surveilled by the administration. In spite of the apparent interferences with privacy, the use of web scraping by tax administrations in the EU, the scope of data collected, the sources of data collection, the limits and safeguards, etc. remain largely unregulated.²⁴

Moreover, predictive models such as risk detection and risk scoring tools are prone to errors, statistical biases and discrimination. These models are predictive, hence these systems only forecast a probable outcome based on what is statistically likely. Such a process by nature involves a great deal of uncertainty, errors, and deviations from objective reality. For these reasons, predictive models have already resulted in serious scandals such as Robodebt²⁵ in Australia and the toeslagenaffaire²⁶ in the Netherlands. The latter is perhaps the best illustration of the devastating consequences that AI tax enforcement systems may occasion, particularly when these are not sufficiently regulated.

2. The toeslagenaffaire, stark example of the risks of AI tax enforcement

In the toeslagenaffaire, the Dutch tax administration (Belastingdienst) attempted to automate the assessment of childcare allowance (kinderopvangtoeslag) fraud with a predictive model. The model had the power to, without any human input, discontinue the allowances of welfare recipients and request the reimbursement of all aids ever received. Parents labelled as fraudsters by the AI system were made to pay back large sums of money (€35,000 on average – up to €250,000), testimony to the high childcare costs in the Netherlands, among the highest in the OECD.²⁷ As the label was disclosed to other public and private actors, following so-called “linkage of records”²⁸, parents were denied credit cards, bank accounts, loans, other means of public assistance, etc. In some cases, child protective services paid visits to their children’s school or homes to forcibly separate them from their parents.²⁹ Later inquiries by the State Secretary revealed that the predictions of the models were erroneous in 94% of cases.³⁰ A substantial part of these errors were the result of discrimination induced by the historical biases in data of the administration, data inaccuracies, and the processing of data on nationality and ethnicity by the risk scoring model.³¹ A central element of the scandal was the fact that the model contained a feature “Dutch/non-Dutch” (Nederlander/niet-Nederlander) whereby the predicted risk of fraud of non-Dutch individuals was systematically increased. The application of such a model meant that foreign residents and dual nationals would be excessively targeted by the model, and thus disproportionately became the victim of unlawful reimbursement requests. Upon revelation of the scandal, the entire Dutch cabinet resigned. Estimations suggest that the cost may be totaling €5.5 billion in compensation for the estimated 40,000 victims.³² Although the affair was revealed more than two and a half years ago, over 1,500 children have not yet been returned to their parents³³, and testimonies suggest that compensations could last until the year 2030.³⁴ The scandal perfectly illustrates the potential risks of AI tax enforcement to data protection, privacy, non-discrimination, fair trial, and good governance. The models of the tax administration target a wide and highly heterogeneous population, often based on inaccurate data sources³⁵, using opaque and potentially biased features. Leveraging statistics to profile taxpayers under such conditions significantly increases the risk of disparity and discrimination.

II.  AI Tax Enforcement Systems and the Notion of “Law Enforcement” in the EU AI Act

Despite widespread use and empirically demonstrated risks, substantial confusion remains around the treatment of AI tax enforcement systems in the upcoming EU AI Act. Tax enforcement systems are conspicuously absent from the draft proposal despite AI tax enforcement systems having given rise to the most unsettling case of automation bias to date. The notion that such systems would not constitute a priority in an instrument meant to regulate the externalities of AI is astonishing. Yet, unlike justice, education or law enforcement, tax enforcement is not singled out as a specific area in Annex III of the proposal, where sectors with high-risk systems are listed. The absence of AI tax enforcement from the draft raises questions, particularly as the initial proposal was published in April 2021, a couple of months after the revelations around the toeslagenaffaire. To be qualified as high-risk, the only alternative is thus for tax enforcement systems to be allocated to another category listed in Annex III. By elimination, law enforcement appears as the likeliest candidate given that tax enforcement is, in part at least, a form of law enforcement. Tax officials enforce taxation rules, investigate tax crimes, and are viewed as a competent authority in the Law Enforcement Directive (LED).³⁶ However, Recital 37 of the Preamble of the initial draft proposal specified that AI systems used by tax administrations should not be regarded as systems used for the purpose of law enforcement. In a move completely at odds with the lessons learned in toeslagenaffaire, the draft proposal seemed to create an exemption for tax administrations whereby AI tax enforcement systems would not be regarded as high-risk. This position was striking as it was in direct conflict with the LED, of which the AI Act will be lex specialis.³⁷ The proposal was later amended by the common position of the Council.³⁸ Recital 38 (formerly 37) now prescribes that AI systems specifically intended for administrative purposes should not be regarded as high-risk systems used by law enforcement, establishing a strict dichotomy between AI used either for administrative or law enforcement purposes. A prima facie distinction between criminal and administrative processes seems to make sense under a risk-based approach. Crimes typically result in harsher sentences compared to administrative offences. In the Recital, the severity of sanctions for criminal offences is explicitly mentioned as a factor that should be taken into account. Yet, upon closer analysis, it appears that this dichotomy will generate additional confusion around the treatment of AI tax enforcement systems and whether these qualify as high-risk systems.

In the context of taxation, distinguishing between administrative and criminal offences is a complex and arbitrary exercise. Rare exceptions aside, what distinguishes administrative from criminal offences in taxation is the subjective intention of the perpetrator. Simply put, a tax crime is a fiscal administrative offence committed intentionally. Hence, the salient feature is the mens rea. However, AI tax enforcement systems are not used to predicting the subjective intention of a perpetrator. These tools merely predict a risk of non-compliance based on objective material factors. This risk is forecast by examining the gradient between what is declared by a taxpayer, and what level of wealth is stochastically and comparatively probable. In other words, AI tax enforcement systems detect actus reus, not mens rea. As a result, AI tax enforcement systems are all used interchangeably both for administrative and criminal tax offences, none are used specifically for administrative purposes. Predictive policing tools of the tax administration are exclusively used in the audit phase, when subjective intentions have not yet been determined. Taxpayers are subjected to the same AI tax enforcement scrutiny whether they are subsequently suspected of fraud or cleared of any suspicion. The fact that the AI system is used to detect what is later qualified as an administrative or criminal offence has little bearings on the model itself and the risks resulting from its use. By the time the offence has been qualified, the model has generated all its potential risks. Yet, the obligations imposed on high-risk systems in the EU AI Act are not retroactive; in fact, most of these are pre-emptive and should be performed prior to using the model. In such a context, it is hard to see how the dichotomy of Recital 38 could be correctly applied. Furthermore, the definitions of fiscal crimes have not been harmonised in the EU, hence some offences may be of administrative nature in one Member State, while being a crime in another.³⁹ Based on the Engel⁴⁰ criteria, the dichotomy would rest primarily on the national law qualification of the offence, and whether it is viewed as a crime or an administrative offence in the respective jurisdiction. Since that qualification has not been harmonised, two identical tools may be categorised differently under the AI Act.

Seemingly, Recital 38 attempts to uphold a binary and obsolete notion of “law enforcement” in an era where policing is increasingly integrated. Law enforcement is an organic process involving a multitude of stakeholders, including the traditional police, administrative authorities, and even external corporate actors. This is particularly true for tax administrations that must, by virtue of the wide array of prerogatives performed, involve numerous public and private actors. Tax evasion and tax fraud are umbrella terms meant to qualify an enormous number of offences. Importing an excessive amount of cigarettes or liquor, illegal species of fauna and flora, counterfeited goods, not declaring workers, employing migrant workers, under-valuing an asset, and hiding financial assets are all considered forms of tax evasion and fraud. To detect these offences, the tax administrations continuously collaborate with other agencies, such as food safety administrations, asylum authorities, financial administrations, labour inspectorates, corporate brands, etc. In such a context, distinguishing between different actors and whether their role was incremental to administrative or punitive aspects of a procedure, is so complex that it is bound to be arbitrary. Tax enforcement is becoming increasingly integrated, precisely because of the integration of AI, as the use of certain models relies on the know-how of specific stakeholders. Corporations provide support to police forces and tax administrations, online, in public spaces, through proprietary models, etc.⁴¹ NGOs and investigative journalists use web scraping to detect fraudulent schemes and tax evaders using offshore entities.⁴² These actors are neither administrative nor criminal, yet play an integral role in the law enforcement apparatus.

III.  Conclusion

Overall, the treatment of AI tax enforcement systems in the upcoming EU AI Act is riddled with uncertainty and confusion. Despite several amendments to the draft proposal, Recital 38 seems to raise more questions than it provides answers. Distinguishing between administrative and criminal processes is bound to be an arbitrary, impractical, and reductionist exercise. Moreover, given the state of harmonisation of fiscal crimes in the EU, a literal application of Recital 38 is likely to result in the fragmentation of EU law. While AI is upending pre-existing notions of tax enforcement and law enforcement as reflected in tax and criminal codes, EU legislators remain attached to an anachronistic vestige of public law. As such, this dichotomy is not novel, with this issue also reflected in the GDPR and the LED.⁴³ Yet, by resting a crucial part of the EU AI Act on this very distinction, the AI Act not only replicates that confusion but strongly exacerbates its effects.

The treatment of AI tax enforcement systems reveals a certain arbitrariness inherent to the risk-based approach in the EU AI Act. Discussions on the potential inclusion of ChatGPT and generative AI as high-risk systems in the proposal⁴⁴ indicate that the risk-based approach is excessively focused on buzzwords and may not be the product of a consistent methodology. Conversely, despite the empirically demonstrated prejudicial effects of these systems, tax enforcement is not viewed as warranting its own risk category in Annex III. Factually, the AI systems used by tax administrations are quite unique and do not always correspond to traditional predictive policing. Tax administrations perpetually oscillate between administrative and law enforcement in ways that are hard to capture in a binary legal construct. This is perhaps indicative of AI tax enforcement systems requiring their own sui generis category with specific rules and limits, different from “law enforcement” as intended in the instrument. With its wide array of AI systems, both coercive and non-coercive, it is clear that AI tax enforcement escapes traditional dichotomies and legal qualifications. Attempting to fit tax enforcement within a pre-existing mold may thus not be the best strategy. The uniquely ambivalent nature of the tax administration and diversity of AI systems should warrant a dedicated sectorial instrument or specific area of attention in Annex III of the AI Act. In that regard, a risk-based approach should distinguish between non-coercive AI tax systems and coercive systems as suggested in this article.