Automated Disadvantage, Personalized Persuasion, and the Societal Ramifications of the Commercial Use of Personal Information.
WORKING PAPER BY CRACKED LABS Vienna, October 2017 Author: Wolfie Christl Contributors: Katharina Kopp, Patrick Urs Riechert
This working paper examines how the corporate use of personal information can affect individuals, groups of people, and society at large, particularly in the context of automated decisions, personalization and data-driven persuasion. After briefly reviewing our previous research’s findings and key developments in recent years, this paper explores their potential to be used against people in detail. Systems that make decisions about people based on their data produce substantial adverse effects that can massively limit their choices, opportunities, and life-chances. These systems are largely opaque, nontransparent, arbitrary, biased, unfair, and unaccountable – even in areas such as credit rating that have long been regulated in some way. Through data-driven personalization, companies and other institutions can easily utilize information asymmetries in order to exploit personal weaknesses with calculated efficiency. Personalized persuasion strategies provide the means to effectively influence behavior at scale. As companies increasingly and unilaterally shape the networked environments and experiences that underlie and determine everyday life, manipulative, misleading, deceptive, or even coercive strategies can be automated and customized down to the individual level. PDF