Dutch Judges Stop Automated Welfare Fraud Detection

ARTIFICIAL INTELIGENCE - A Dutch court has ruled that an automated surveillance system using artificial intelligence (AI) to detect welfare fraud violates the European Convention on Human Rights, and has ordered the government to cease using it immediately. The judgement comes as governments around the world are ramping up use of AI in administering welfare benefits and other core services, and its implications are likely to be felt far beyond the Netherlands.

The Dutch government’s risk indication system (SyRI) is a risk calculation model used by the social affairs and employment ministry. It gathers government data previously held in separate silos – such as housing, employment, personal debt and benefit records – and analyses it using an algorithm to identify which individuals might be at a higher risk of committing benefit or tax fraud. It is deployed primarily in neighbourhoods with a high proportion of low-income and minority residents.

The case against the government was brought by a number of civil society organisations, including the Netherlands Law Committee on Human Rights, and two citizens. They argued that poor neighbourhoods and their inhabitants were being spied on digitally without concrete suspicion of individual wrongdoing.  

The court found that the SyRI legislation fails a balancing test in Article 8 of the European Convention on Human Rights (ECHR), which requires that any social interest – in this case to prevent and combat fraud in the interest of economic wellbeing – be weighed against the violation of individuals’ privacy, and is therefore unlawful.

ABC Comment, have your say: