AI: experts call for testing and federal digital agency

Mandatory testing of Algorithmic Decision Making (ADM) systems and certifications as a consumer protection measure, a digital agency for government competence building and regulation, and further intensive research. These are the proposals and demands of an interdisciplinary group of experts consisting of lawyers and computer scientists of the Gesellschaft für Informatik e.V. (German Informatics Society). It has carried out a study on algorithmic decision-making on behalf of the German Council of Consumer Experts SVRV. The study on the technical and legal considerations of algorithmic decision-making procedures was submitted to Dr. Katarina Barley, Federal Minister of Justice and Consumer Protection (BMJV).

The research looks at how discrimination can be prevented in an increasingly ADM-driven economy, in which many decisions are no longer made by people but by computer programs. For example, algorithmic decision-making procedures are used to assess the creditworthiness of individuals (credit scoring) and to determine prices in online trading (dynamic pricing). “Consumer protection in the digital sector is playing an increasingly important role,” says Daniel Krupka, Managing Director of Gesellschaft für Informatik e.V.

According to Krupka, one challenge consisted of bringing together the different worlds of computer scientists and lawyers. Nevertheless, the expert group first had to agree on a common nomenclature. “Lawyers define the term ‘error’ differently from computer scientists,” says Krupka. Nevertheless, a central issue is whether special regulation is necessary in many cases. “Many laws are already in place and may simply have to be adapted in individual cases,” he explains. Asia was omitted from the legal comparison for lack of transparency. “All other countries are already working on the subject, albeit at different speeds,” Krupka says. Not all of them are comparable, however, because in some cases the legal systems are simply too different.

Dr. Erich Schweighofer from the University of Vienna and spokesperson of the “Legal Informatics” expert group points to the long path ahead to comprehensive regulation: “It is important that we regulate algorithmic decision-making procedures legally, but this is still in its infancy. On the one hand, the risks associated with these so-called ADM processes is not yet comprehensively known. On the other hand, decisive legal issues need to be clarified to effectively address discrimination that violates the law. However, the report shows that testing, auditing and certification procedures are effective tools for creating greater transparency and, ultimately, better protection for consumers.”

Erich Schweighofer sees great potential for consumer protection in testing procedures in particular: “Testing of ADM systems is an essential element of protection against incorrect algorithmic decisions. To this end, we need legal frameworks that provide certainty for both the performance and the evaluation of such testing procedures. Once the legal framework for suitable testing procedures has been established, a legal obligation to carry out sufficient testing should be introduced. After all, it is important that ADM systems are checked for errors, especially discrimination, before they are used.”

In order to comply with transparency and information obligations as well as to implement efficient and effective testing and auditing procedures, the experts recommend establishing governmental competence for algorithmic decisions. The essential task of a possible “digital agency” must also be to increase transparency by advising and informing decision-makers in companies, administration and politics and by educating society as a whole.

“The biggest challenge is to find the right people who are really familiar with the topics,” explains Managing Director Daniel Krupka. “There are currently too few people with the necessary expertise.” And while he believes that the special areas of sensor and actuator technology provide particularly good opportunities in terms of the state of research, he particularly sees Germany as lagging behind in this regard. “We have missed the wave here,” said the spokesperson of the Gesellschaft für Informatik e.V.

According to the study, additional (primarily international and interdisciplinary) research efforts are required at the interfaces between computer science and law (and beyond) to address the manifold unresolved issues. Specific topics must be examined through well-defined and concrete research projects and initiatives, such as scholarships, doctorates or other expert opinions, and prepared for discourse.

The bottom line:

The German Council of Consumer Experts SVRV made an important step in convening this expert group for initial classification of ADM and AI for this study. No time should be wasted, as the speed of technological development is constantly increasing. It will be interesting to see how governments and the parties they represent will continue to address this issue in the future. It is already clear that the exceptionally important discussion on the subject of “privacy” will continue. Society and politics as its representative, are actively called upon to engage in discourse. This also applies to companies. From the outset, they should review the ethical guidelines of their application, for example, and be transparent in their approach to the market. Consumers are sure to reward such initiatives.